This post is a translation of an original post I made in a Dutch IT magazine: Computable, see the original article here (in Dutch).

Organisations see quality assurance (QA) as a cost, however, targeted use decreases development costs! How can an organization keep the costs for quality as low as possible and still meet the highest quality standards? One answer is Evaluations! Evaluations are quality measures which can be used to improve the quality from the start of a project and have a final product with as high as possible quality

From a cost perspective, a client has only one question: What’s in evaluations for me? Evaluations seem an expensive measure. The client has to invest in more work, more activities, ensuring the work (and thus the deadline) only seems to hold. To answer this question, the benefits of the investment needed to prove conclusively before they can be realized. Over the years, much research has been done on the effect of evaluations on the overall development process. With this is a good basis for the benefits are given.

Already in 1979, Boehm [Boehm, 1981] investigated repair costs of software. His results were that the repair costs rise as they are made later in the development process. Thus, solving an error early in the development process is cheaper than later. Boehm (and later Gilb) states that finding a problem in a design is 16 times cheaper compared to repairing the same mistake in the testing phase [Boehm, 1981].

The prevention or correction of errors found early in the development process benefits the client. This way to the failure and repair costs are kept low. Meanwhile the quality and prevention costs are less than the projected cost for recovery (without these measures) on overall cost savings. Executing a serious error detection process with one evaulautions round cuts 88% of the significant errors found in a document. Errors which are not found during the testing process and do not need to be restored! Thus, there are lees defects in the software and every defect costs about 30 minutes testing time when testing manually. Results are faster test execution per test round and less test rounds of software testing.

With evaluations to costs do rise early in the project, without the prospect of any short-term results to be obtained. In the long-term these costs will be recovered during testing. Applying different evaluations and reading techniques enables the short-term costs to be cut down to a minimum. In a evaluations strategy for each risk area it´s determined what type of document needs to be reviewed. Using the right techniques and a evaluations strategy is the possible evaluations process can be optimalised. This leads not only to better results, but even a more efficient (thus lower cost) evaluations process.

Bibliography
[Boehm, 1981]
Boehm, BW (1981), Software Engineering Economics, Prentice-Hall Inc., Englewood Cliffs, ISBN 0-13-822122-7
[Gilb, 1993]
Gilb, T. (1993), Software Inspection, Addison-Wesley Longman, ISBN 978-0201631814

This post is about the opportunities given by a test strategy on the evaluation process. The use of an evaluation strategy.

Finding defects in an early stage of the product is the cheapest manner of increasing the quality of a product. Everybody in ‘quality land’ knows this! The way to do this is by using evaluations, like reviews. But this isn’t done by many. Why?

Well first of all it is expensive to do. And you don’t get much in return in a project. A projectleader will only see it at the end of the project, when there is less testing done. It also is difficult to do right. When do you use which evaluation technique on which product? The easiest way to do this is by using an evaluation strategy. This is much like a test strategy. A test strategy is created by using risk classes, test levels and combine these with test design techniques. The result is a strategy of testing the high risk pieces of software most thorough. Because “No risk, no test!”

How can this be done when evaluating intermediairy products? The same to be exact. A QA Manager should do a risk analysis for the product of the project. The risk analysis should include the evaluation goals of the project, the delivered products and the risks of these combines. Now you have an risk analysis for the intermediairy products. So you get a list like underneath.

When combining these risk classes and products with evaluation techniques you get a clear view of the importance of the risks. A high risk product should be thoroughly reviewed. See the next table.

Now, how do you get the effectiveness in the evaluation strategy? The answer is: by using reading techniques! With a reading technique you get insight in the document without overdoing the review or inspection. For example. The risk of documenttype Funtional Design is ‘A’ (the highest) and you evaluate it with a (Fagan) inspection the process is quite expensive. When using PBR (Perspective Bases Reading) reviewers only look at a docuement from their perspective. Thus, getting better results in finding defects in less time.

When using all these options into the evaluation strategy you get this result.

An other benefit is that with using a evaluation strategy, like a test strategy, you always stick by the business case while doing quality assurance in your project. This generates the best results and at the end a cheaper project with less risks to fall behind on planning…