What is a Software Testing Metric?

Software Testing metrics are the attributes or elements with the help of which we, can detect the quality of the software. Basically, there has to be something which could measure the quality of the software. And this unit is someway termed as Software Testing metric.

Software testing metric is the measurement by which the quality of the software is analyzed. In recent days numerous testing has been developed.

Not all of them are used in all the softwares, but each of the testing metrics has its individual importance in determining the quality, efficiency, and the productivity of the software.

Since there are different aspects and fields, the software can be used, therefore the software testing metrics are also divided into different categories based on different levels. The scope of testing metrics is huge, that is, the metric can be influenced to produce different or influence the results itself.

Below we shall see different levels of testing metrics:

Time Metric:

This metric is specifically created with the notion that time plays an important role, in analyzing the performance of the software. Generally, we analyze the quality of software, by the time taken to cover certain test cases, or the durability or the load balancing capability, which are all related to time.

Therefore, there should be a metric which can give a current or point in time status of the test. The significance of the time metric is that it is related to all other metrics in some way. This metric helps the organization or the team to analyze where they stood in terms of testing.

Quality Assurance Metric:

This metric along with the quality control metric is used to assure the quality and control the quality of the product. This metric analyses or measures whether all the objectives are fulfilled during the software development phase. At the same time whether the quality of the requirements is controlled.

This metric is used by the software organization and the quality control testing, which is somewhat different from the Quality assurance metric which is for the team of developers only.

It is seen that there are different levels of creating or developing software testing metric. Let us discuss some of the metrics which are in use in the software industry:

  • Defect Density: This is a metric where the density of the defects is counted. The objective is to identify defects in a particular area of the software within a given period of time. The defect density is measured as errors per KLOC (Kilo Lines of Code). [DD=Defect Count/module size].
  • Defect Leakage: This is a testing metric which helps us to find, the leakage for the defect. That is the number of defects seen before testing the number of defects seen after testing. The defect leakage efficiency can be calculated as: [no_of_defects_before_test/ no_of_defects_after_test] * 100.
  • Defect Elimination: This is a testing metric, which measures the efficiency of the team to eliminate the errors as far as possible before the release of the software. The defect elimination efficiency can be calculated as: [No_of_defects eliminated/no_of_defects that were present]* 100
  • Test Case validity: This is a testing metric which is to check whether the test case we really able to check the errors. The efficiency of which can be calculated by [no_of-errors/ number_of_testcases_run]/100.
  • Review metric: This is a testing metric which is the measure of the defects which are not seen in the review. This metric can be very important to filter the product very efficiently and reduce the cost, time and effort of the future tests. Review efficiency= [no_of_review defects(RD)/{RD+number of defects in testing}]*100.
  • Test Case efficiency: This metrics is about the efficiency of the test cases. It is the ratio of the passed test case to the total number of test cases deployed or input to the system. The efficiency can be calculated in terms of percentage as: [no-of-test case failed/number-of-total-test-passed]*100.
  • Failed test case percentage: These metrics determine the failing efficiency of the test cases. It is the ratio of the number of failed test cases to the number of total test cases applied to the system. The deficiency can be calculated as: [number-of-test-case-failed/total-no-of-testcase-applied]*100.
  • Barred or blocked test case: This is a metric measuring the total number of a test case that has been blocked by the system or is not accepted. It is the ratio of the number of barred test cases to the total number of test cases applied to the system. The block percentage can be calculated as:

[no-of-test cases blocked/ total-no-of-test case deployed to the system]*100.

  • Executed Test Case: This is a testing metric which gives the number of a test case executed with respect to the total number of test cases applied to the system. It is the ratio of the number of test cases executed to the number of test case which was actually written or was applied to the system.

The efficiency of the executed test case can be calculated as:

[no-of-testcase-executed/no-of-test-case applied to the system]*100.


Importance of Software Testing metric

As we have seen that there are different types of software testing metric, which are used to measure the quality of the software and the validity of the tests carried out.

Some of the importance of the software testing metrics are listed below:

  • Software testing metrics help us to identify the progress of our work and the tests conducted.
  • Software testing metrics are unbiased and thus helps us to improve the quality of the product.
  • Software testing metric helps us to derive different other attributes which can be easily referenced.

The importance of using the software testing metric lies in the fact that there has to be some unit of measurements which can assure that, what is the degree of the quality of the test that is carried out or is being carried out.

Since the metrics are the real-time parameters which can be traced during the software development or the test, therefore the metrics can be easily interpreted and the process of testing can be upgraded.

There are different levels of software testing metrics. For different levels of the organization, there are different types of software testing metrics that are used.

Used in the project level:
  • Requirements Coverage: This defines what are the different workflow which is covered during the test.
  • Distribution of Defects: It is the measure of the number of defects detected in the software in unit time.
  • Defect open and close time: This is the time when the defect is opened is detected and resolved.
  • Test execution trend: This is the metric which calculates the test performed by the team
  • Burn Down Chart: This is a chart showing the progress of the work with the amount of test covered.
Used in the departmental level:
  • MTTD: Mean Time To Detect a Defect, is the average of time it requires to detect a defect. This is: calculated by aggregating all the individual times it takes to detect a particular defect.
  • MTTR: Mean Time to Recover, is the average of time it requires to recover from a defect after it has been detected. It is calculated by aggregating all the individual times of recovering from the defect before the defect has harmed the users of the software severely.
Note: The MTTD and MTTD both can be interpreted together to get the average time for detecting the problem in the test and the time to recover from that error.
  • Defect Removal: This is the metric which gives the number of defects and the number which are detected and resolved actually. The efficiency can be calculated as the ratio of the number of defects fixed to the number of defects available in total.
  • Testing trend: This is the matric which is helpful to understand, the rate at which we are able to carry out our tests.
  • Defect solution trend: The defect solution trend, is the number of defects for which the solution has been found with respect to a particular time.
Note: This has to be mentioned that, the testing trend and the defect solution trend is can be interpreted to get the overall efficiency or the rate at which the defects are calculated and the solution for them is found out.

Below we shall see the metrics which are not for the top level persons in organizations or companies. This is important that these metrics can be itself, can be derived from the basic software testing metric or the metrics which are discussed earlier.

Used in the Company Level:
  • Issues reported : This metric measure the number of issues reported by the customers or the user of the software. This is very important to know whether the software is really able to satisfy it's targetted, customers. And at the same time, the users and customers proving positive feedbacks for the software.
  • MTTR and MTTD: We have already studied about the MTTR and MTTD metrics in the previous section, these two can be applied here too. The company top head can use these metrics directly in the top level to see whether the defect is addressable by the company and how much the company takes to recover from it.
  • System Interrupt: This metric measures the time, how much the system is in the state of interruption. This may be due to software maintenance, the server connection error or any other related issues.
  • System Downtime: This metric measures the time for which the system is out of order. This is almost similar to the system interrupt metric, just a little difference lies in the fact that the interrupt is caused suddenly due to vulnerability or any simple error. On the other hand, system down means a sever error must have occurred.
  • Cost to detect: This metric measures the cost required to detect the bugs or errors. This is done earlier or even much just/much after the software release.
  • Cost to recover: This metric measures the cost to recover from the bugs or defects of the software.

Backend Testing

Types of Environment

Basically, in regard to testing, we are discussing two environments only. One is the WaterFall environment and the other is the Agile environment.

Before we start to describe which are the metrics that are used in Walter fall environment and Agile Environment, we need to understand, what is a Waterfall environment and what is an Agile environment.

In the Waterfall model environment in which, the phases of the software development is designed in such a way, that the top level is the starting phase and the ending phase is the testing phase.

In the Waterfall Model, we believe that after the completion of the phase only, the other phase started then the level which is just down to that level.

On the in agile environment, the tasks are divided into shorter and small subtasks so that, the defects or the problems can be sorted out quickly and progress can be made within a shorter period of time.

Now, the software testing metrics for these two kinds of environments is a little bit different.

In the Waterfall environment model, the metrics which are of concern are:

  • Quality of Product
  • Effectiveness of Tests
  • Status of Test
  • Test Resources
  • Test Time
  • Test efficiency
  • Test Quality

On the other hand, in an agile environment, the metrics of concern are a bit different:

  • Speed of Test
  • Cumulative Flow
  • Earned Analysis
  • No of test cycles completed.

Comment / Suggestion Section
Point our Mistakes and Post Your Suggestions

Protractor Training

new tutorial I am starting new Protractor training batch on Weekends. Course starts from 24-Aug-2019 [10am IST to 1 pm IST] based on online.

Interested people can confirm the seat by calling to +91 8971673487 or whatsapp to +91 9003381224. You can attend first class(3 hours) for free. Only 10 Seats in Total

Find the course content : View Content