A hot item in the development life cycle is software versioning and/or revision control. According to Wikipedia software versioning is “the process of assigning either unique version names or unique version numbers to unique states of computer software”. And revision control is “the management of changes to documents, programs, and other information stored as computer files. It is most commonly used in software development, where a team of people may change the same files”. With software versioning and revision control tracks and provides control over changes in the documentation and source code. How do testers do this and what is their take on it?
Revision control is not only done in ICT. It is also done in engineering, business and law. Examples like “contract redline” and “legal blackline” are some of the earliest forms of revision control, and are still employed with varying degrees of sophistication.
Defects hide in different versions
As teams design, develop, test and deploy software, it is extremely common for multiple versions of the same software to be deployed in different environments. This is also true for developers to be working simultaneously on updates of the same software. Bugs and other issues (defects) with software are often present in certain versions. Therefore, for the purposes of locating and fixing defects, it is vitally important to be able to retrieve and run different versions of the software to determine in which version(s) the defect occurs.
At the simplest level, designers and developers could simply retain multiple copies of the different versions of the document or program, and label them appropriately. This simple approach has been used on many large software projects. This requires a lot of self-discipline on the part of author of the document, and unfortunately often leads to mistakes. Consequently, applications that automate some or all of the revision control process have been developed.
‘Practice what you preach’
Revision control also has multiple effects on testing. As said before defects can be ‘hidden’ between versions and testers need a stable test base for creating there test cases. Therefore testers are highly aware of revision control of their test base and source code. As a test manager I’m always indulged in the revision control of a project and when there is none I explain the risks it implies. But do we testers also take this so highly when it comes to our own revision control of our test cases? Most test plans we create we see as a document so we follow the control of this, but with test cases we should ‘practice what we preach’!
So why is it so important for testers to be involved in the revision control? TMap® NEXT tells us it is because testers need to know:
- Which version of the test scripts is required. As a tester it is important to know which version of the various components is installed. In practice this is most important on the version of the software, usually the test object. Depending on the version, certain requirements or not built thus resulting that some scripts could not be needed;
- Which release is shipped to the customer. The test takes place at a particular release version, and therefore the release advice should cover that release.
But testers should also get involved in their own revision control. How do I think this should be done? At first there is a difference in approach between development testing and system and acceptance testing.
Document a change in test cases
With system and acceptance testing test cases cover certain versions of the documentation (requirements or designs). In a perfect (testing) world the this test base should be frozen, but unfortunately this is more exemption rather than common. And of course there is also Agile. Test cases are based on a version of the documentation/requirements and these test cases can vary. This because test cases aren’t right the first time, documentation changes, but also progressive insight. Therefore it is needed to link test case versions with documentation/requirement versions.
With Agile, and thus iterations, the same applies. Within iterations there are pieces functionality (and non-functionals). The pieces of functionality will be yielded for testing in certain iterations. Thus revision control is needed on the iterations and the test cases. When certain functionality is passed to another iteration this should also be changed in the version of the test case.
Unfortunately many tools don’t support versioning standard. However, there are many add-ons to obtain the same result.
Source code and unit test case together
With development testing revision control is a little bit more complex. You could continue like with system and acceptance testing and link every version of source code with a version of a test cases. Unfortunately you are dependent on the approach of revision control of the source code, this usually differs from the revision control of documentation. Ideal is to create a direct link with the source code and the unit tests. When the source code is revised you directly change the test case. This is the simplest when it’s in the same version all together.
Some tooling is available for this.
Meta information in tags
The thing I use for my test projects is a lot like the post I wrote about test ware management. I use tagging on the test cases my team creates. We tag them with test case version, documentation version, requirement (version), date, author, quality attribute, system part, iteration, etc. As a result I have the advantage that all the meta information is clear and stored with the test cases. This hads always been done on system and acceptance testing.
Other options to include revision control in the testing process are also available. You could think of always using ‘track & trace’ in the documentation. This way the changes compared to the other version are always vissible to others. And make test revision control part of overall revision control from a QA point of view.
Tip: When a new version of the software or application is released a new version of the test documentation should also be released.