When to Stop Testing Or Exit Criteria?

Stop Testing or Exit Criteria: Every tester is very passionate at the beginning of a project. Create test documents like test cases, plans, and strategy documents. Then, once the application is developed, we test it, develop some interesting defects in the early stage of the project, and follow up until those defects are closed.

In the first iteration, we got lots of defects and fixed those. After that, we started one more iteration. We get fewer defects, and the same process runs for some iterations. But after some repetition, we are thinking about, “When we stop the Testing of the application?”.

So, how will we conclude that the testing is enough? We will try to figure this out by taking some real-time examples. But the fact is that testing can not be considered complete because we can not prove scientifically that our application is free from errors.

Test Criteria

But there are some common criteria for the software industry, and those criteria are:

• We Can stop the testing when the planned testing deadlines are over.
• We Can stop the testing when we cannot find any more defects in the application by following the predefined test cases.

But we can say that both of the above statements are contradictory because we can satisfy the statement even by doing nothing. Also, the second statement is meaningless because it does not ensure the quality of our test cases.

Exit Criteria In Testing

There are a few other common factors that can help us decide when to stop testing:

• Stop the testing of the application when the release deadlines or testing deadlines have been reached.
• If there are some predefined passed test case percentages and you can achieve that pass percentage number, then you can stop the testing.
• When the allocated budget for testing the application comes to an end,
• Stop the Testing when the code coverage and functionality requirements reach the desired level.
• When the bug level of the application comes below the prescribed level.
• We can stop the testing of the application when the Alpha and beta testing of the application is over.

But to find the above things, we have to keep track of the progress of our testing because those metrics can help us make better and more accurate decisions regarding when to stop testing or when the application is ready to release.

So, to find out the above things, we must have strong and thoroughly covered test cases that can include various scenarios before the beginning of the test execution cycle. As we progress, we can find out the number of test cases executed and, out of that, how many test cases got passed. We can easily determine the software product’s quality based on these metrics.

When To Stop Testing?

By comparing the total test cases and the number of passed test cases, we can come up with some of the metrics like:

• Percentage Completion: We can determine this by dividing the number of executed test cases by the total number of test cases. [ (Number of executed test cases) / (Total number of test cases) ]
• Percentage of Test Cases Passed: We can also calculate by dividing the number of passed test cases by the number of executed test cases. [ (Number of passed test cases) / (Number of executed test cases) ].
• Percentage Test Cases Failed: We can also find this by the number of failed test cases / Number of executed test cases. [ (Number of failed test cases) / (Number of executed test cases) ]
  Note: If you have found any defect in the test cases, you must mention that they have failed; otherwise, it will be calculated as a pass.

Exit Criteria To Be Considered

• All test documents are prepared, reviewed, and published.
• Passing all the critical test cases
• Testing of all the complete functional coverage
• Successful execution of major functional or business flows
• Fixing of high-priority defects is done and retested.
• Fixing “Show Stopper defects” or” Blockers.”
• Meetings with stakeholders have been conducted. This is where a decision is made to either go to production or not.

Final Words

It is very hard to decide to stop doing the software tests because we can not deny that errors are the never-ending problems that occur in software. For instance, a tester has just finished doing tests on software. Upon further investigation, it has been discovered that a new bug exists. Even though tests have been done, previous bugs and errors have been resolved. This will require more tests.

The cycle continues. Does that mean software tests can be stopped?