This post is a translation of an original post I made in a Dutch IT magazine: Computable, see the original article here (in Dutch).

Organisations see quality assurance (QA) as a cost, however, targeted use decreases development costs! How can an organization keep the costs for quality as low as possible and still meet the highest quality standards? One answer is Evaluations! Evaluations are quality measures which can be used to improve the quality from the start of a project and have a final product with as high as possible quality

From a cost perspective, a client has only one question: What’s in evaluations for me? Evaluations seem an expensive measure. The client has to invest in more work, more activities, ensuring the work (and thus the deadline) only seems to hold. To answer this question, the benefits of the investment needed to prove conclusively before they can be realized. Over the years, much research has been done on the effect of evaluations on the overall development process. With this is a good basis for the benefits are given.

Already in 1979, Boehm [Boehm, 1981] investigated repair costs of software. His results were that the repair costs rise as they are made later in the development process. Thus, solving an error early in the development process is cheaper than later. Boehm (and later Gilb) states that finding a problem in a design is 16 times cheaper compared to repairing the same mistake in the testing phase [Boehm, 1981].

The prevention or correction of errors found early in the development process benefits the client. This way to the failure and repair costs are kept low. Meanwhile the quality and prevention costs are less than the projected cost for recovery (without these measures) on overall cost savings. Executing a serious error detection process with one evaulautions round cuts 88% of the significant errors found in a document. Errors which are not found during the testing process and do not need to be restored! Thus, there are lees defects in the software and every defect costs about 30 minutes testing time when testing manually. Results are faster test execution per test round and less test rounds of software testing.

With evaluations to costs do rise early in the project, without the prospect of any short-term results to be obtained. In the long-term these costs will be recovered during testing. Applying different evaluations and reading techniques enables the short-term costs to be cut down to a minimum. In a evaluations strategy for each risk area it´s determined what type of document needs to be reviewed. Using the right techniques and a evaluations strategy is the possible evaluations process can be optimalised. This leads not only to better results, but even a more efficient (thus lower cost) evaluations process.

Bibliography
[Boehm, 1981]
Boehm, BW (1981), Software Engineering Economics, Prentice-Hall Inc., Englewood Cliffs, ISBN 0-13-822122-7
[Gilb, 1993]
Gilb, T. (1993), Software Inspection, Addison-Wesley Longman, ISBN 978-0201631814

1. Timi Stoop-Alcala Says:
   August 24th, 2009 at 11:09

   It still surprises me how evaluation and quality assurance gets deprioritised in budget decisions. Totally agree that an inegrated evaluation approach right from the start of a project will be more efficient. Won’t necessarily translate to less testing time per se, but definitely to less testing for defects.

   In practice, having an integrated evaluation approach has a lot to do with both the project methodology and internal processes in place. The process should help the project run smoother, not hinder the dynamics. Orgs should be clear with what exactly review processes involve in terms of time and resources and overall project development. And a step back would actually be to determine what is quality in the firs place. A clear set of accpetance criteria should be identified and agreed by the parties involved. There are lots of cases when too many stakeholders or reviewers get involved in the whole evauation process that it deters progress rather than facilitate it.

   I personally like the agile methodology as testing/evauatio/acceptance criteria is a basic principle and approach to developing and running the project. Evaluation is not a by-product but a required function not just by a tester but by other team members, esp the product owner / business analyst and developers.