Sometimes testers make some assumptions, but hopefully they avoid them. This series of posts will show you some well-known pitfalls and assumptions that tester make. This third post describes assumptions about testing during lifetime. (post 1, post 2)

The application react the same in production as in the test environment
Some people assume that if you finished testing in the test environment everything is ready to go into production. Some times this is possible if they use for example the hardware of the test environment as a production machine. This works fine with new projects. But what about releases? What if they move it to another machine?

Very often the application is copied or configured again on a different machine. This machine has other characteristics, like hardware and other configuration. As you can imagine, the application reacts different to this environment. It can happen that new defects are caused by the configuration.

To test this difference it’s useful to have a regression test set. A set with test cases you used earlier during testing. Most of the time it isn’t needed to test all the functionality again. But a representative test set can cover these risks. If there are parts of the application that needs more configurations you have to test them with more depth.

How many of you testers also plan a retest in production? Are you in the office, the weekend of the big bang, when they go live with the new application? Sometimes we underestimate the change of the environment at the software quality!

Once an application is tested it doesn’t mean it’s tested for his lifetime
Are you really finished with testing when the application is production? Regression testing is still needed during the lifetime of the application. The quality attribute Security is an example of aspect that needs a lifetime testing.

Each day new threats occur; most of them don’t have much impact at each application. But the bigger threats have more impact and retesting must be done. Think about, Cross-site Request Forgery (CSRF), Click Jacking and the latest one beginning this year Cross-site History Manipulation (XSHM). These vulnerabilities must be checked in the application. A regression test each half year for security reasons can be very useful. Also a check of for example, the web server is updated with the newest version the last half year.

A couple of months ago I tested an application that has an Apache web server latest update June 2004. The site http://www.milw0rm.com/search.php gives the vulnerabilities in this version (and a lot of other software).

A test that often works fine is
- Go to the highest level in the domain: “www.mysite.net/”
- Add “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.bbbbbbbbbbbbbbbbbbbbbbbbba.aaaaaaaaa
aaaaaaaat”,
- Send this URL to the server: “www.mysite.net/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.bbb
bbbbbbbbbbbbbbbbbbbbbba.aaaaaaaaaaaaaaaaat“.

Tip: Testing is needed throughout the total lifetime of the application!