Last week I was in a little discussion with fellow Twits @michaelbolton, @testingqa and @thatqaguy about the handover date and end date of testing. Especially the end date was an issue. Because Twitter only accepts 140 characters so I couldn’t get my point across. In this post I’ll try to do this better. I will describe three options when testing can stop. If you have more, please give me your feedback in the comments.

The good thing about a project is that it has a start date and an end date. Testing usually resides at the end date; with Agile development this is different. So testing is executed near the end of the project. This makes it that testers always need to live with pressure from the client (business), project manager, development team, users and other stakeholders to finish the testing process. But when is it a good decision to stop testing?

“The end of the testing process is stated in the assignment”

As I am one of the authors of the book TMap NEXT® – Business Driven Test Management I need to address the option that the client sets the time for testing in the assignment at the start of the project. In the preconditions and assumptions (part of the assignment) a test manager can state how many test cycles there are going to be executed before testing stops. It is needed to know this because of the budget and planning of the test process. If not, the test manager can’t give a proper planning for the testing process. So, the client decides when testing stops and gets insight the quality of the product at that moment in time. When quality lacks the preferred result a new testing process can start after the development team has fixed the lag in expected quality (bugs or defects). The test manager can consult the client is how many test cycles there are expected to be needed to stop testing.
 

“The end of the testing process is stated in the exit/release criteria”

Another option to state when to stop testing can be set in the exit or release criteria in the master testplan. In the exit criteria I always state when testing stops (this is subordinate to the assignment agreed with the client). These criteria can be all test cases are executed, no more defects found (of a certain priority), budget runs out or risks are covered at a certain level. All these criteria can support the decision to stop testing.
 

“The end of the testing process depends on the quality of the product”

Another option is that the quality of the product is at a certain level. This means several test cycles and defect solving cycles before the testing process stops. Advantage with this approach is that the client gets a product with as best as possible quality, most bugs (not all) are found and solved. Disadvantage is that you cannot know when the testing process will end beforehand. It can take 2 hours, 2 weeks or two years, depending on the quality of the total development process.

For myself I state that testing stops when a certain amount of test cycles has been executed including the regression test. After this I write a release advice for the client to release or follow up on testing. The regression test should always be executed flawlessly!

1. software testing tool Says:
   January 17th, 2012 at 14:52

   I suppose if you can define a number of testing cycles and then deliver the product to your client who will then test it on his own and let you know if he has spotted any bugs ?

2. Ewald Roodenrijs Says:
   January 19th, 2012 at 05:45

   Julie,

   Depends on how the contract is made up. But yeas, that’s a good economic option. You say we test x amount of cycles and then you’re finished.

   -Ewald