Software Quality Metrics:The complete picture

Few weeks back we just finished a release and the way we used to evaluate the software quality was solely based on defects.Then I had this eye-opening discussion with a colleague and he explained how defect metrics are just tip of the iceberg and there is a whole lot that should be done to actually determine the Software Quality Metrics.I am trying to recollect what he said and put it down here.

First of all according to him traditional approach has been to use “defect metrics” but it only gives an idea on how less buggy the software is.Depending on what metrics are used it can also give some idea into testing effectiveness or code quality(defects/KLOC) but knowing all of this will still not give us a way to accurately determine the quality of software.So what are the other ingredients that should go into the mix.Here are few that I will write down to see how others feels about and then probably go in more detail with examples on each of them later.

Coverage based metrics

Identify ways to measure requirements coverage by testcases and actual testcase execution coverage.Some examples are

Num of TestCases written for a module/Num of Requirements for a module

Num of TestCases executed for a release/Num of TestCases written for a release

Process Effectiveness Metrics

Identify metrics to measure process stats and identify improvement opportunities.Some examples include

Total time taken for # of testcases for every build.

Resource Efficiency Metrics

Metrics to measure how effectively we are employing resources.One example being

How many testcases were developed by a particular resource?

How many defects were logged by a particular resource?

Optimization Metrics

Metrics to track and give us feedback into how to optimize testing activity around a particular module.E.g

How much time was spent in testing a particular module?.Using this metrics we can see if a particular module is getting huge and can creep into regression time and take corrective action.

Automation ROI Metrics

Metrics to see how effective the automation effort is.Some simple metrics include tracking time taken by manual testcases against time taken to execute it when those testcases are automated.

In addition performance testing metrics can also be included in this set.Solution specific metrics(e.g if different models of mobile are tested.Coverage of each model can be included as part of this metrics.)

Other stories you may enjoy...

The Secrets to Five Star App Reviews

Let's face it. Every app developer wants five star reviews. But the process of generating five star reviews needs to be a little more sophisticated than releasing it and crossing...

Is the ALM Reboot Resulting in the Downfall of QA?

Everyone will become developers during sprint — and quality will be everyone’s responsibility Lately we have all been hearing terms like Agile lifecycle, Continuous...

Everything You Need to Know About Shift Left Testing

It’s becoming increasingly important for organizations to “Shift Left.” Shift Left refers to the practice of starting testing earlier in the software development lifecycle,...