Software development should become cheaper! And with this, testing should become cheaper! That’s a line I’ve been hearing a lot the last few years, but even more now. But How can this be achieved?
I’ve been blogging quite a few posts (like here, here, here and here) about finding defects in an earlier stage in the development process. One of the most important activities that can be done to find these defects, is to integrate evaluations (like testing) in the quality strategy of software development projects. But as all systems are becoming more and more complex, this emphasis on finding defects early is also growing. And as Boehm told us: the earlier errors are found, the cheaper it is to fix them. But are there other options than evaluations to do this?
Yes, there are. Advancing technologies make this possible, even now. How? By using models! Models can be used to generate test cases automatically (MBT), evaluate the software and can help with understanding the functionality of the software. But it’s also possible to use it for other practices. In embedded software it’s a common use to use models for testing the software in an early stage of the project. This way errors are solved early before the software is implemented in the appliance, like a VCR, refrigerator, car or even an airplane. But how can we do this with non-embedded software?
With Model-Based Design (MBD)! The next figure is an example of an embedded system. But it’s also to create a figure like this for a non-embedded system. With using Object Oriented (OO) languages a model can be create, which can be executed.
Within the embedded world, various hardware components are processed in the design process. OO can also do this, by using different functions as the components. These (standard) functions can be incorporated into a model and connected with each other. As a result a model of a component arises which then can be simulated. And the functional correctness of the component can be determined.
Using models gives us (testers and developers) insight into the dynamics of the development process and algorithmic aspects of a system by simulation them. Therefore models can be used:
- as executable and testable specifications;
- to communicate (system) requirements and interface definitions;
- as a model of the entire system when provided with virtual prototypes;
- automatic code generation for software development (algorithm or logic), and
- automatic test case generation for testing the software.
Execute the functional design
As a tester I find the first option particularly interesting; ‘as executable and testable specifications’. With embedded software this is maybe easier to understand. There models are used to simulate the hardware or real world, in order to test the software of the embedded system. But with OO ‘normal’ software applications, such as administrative systems, can also be executed as a testable specification!
When using models it is now possible to simulate the (functional) design and visualise and debug the specification. In this simulation it is possible to integrate both the initial input of the test as well as the expected result, thus creating a functional test. Within the embedded world this is called model-in-the-loop testing (MIL). A term we can use also in non-embedded testing.
Development testing
A full exploration of a (functional) design based on testing scenarios by using simulations is a way to test requirements. Within these testing scenarios diverse subjects can be included, such as input parameters and environmental factors. And therefore covering all the possible input and parameters of a component or unit and this , subsequently, leads to an increased coverage of the development tests.
Best results can be achieved when the testing of the models (this is called simulation testing or virtual testing) is done parallel with the design process. When the models are created, the tests are focused on the design aspects and are simulated in the model. Since the design is still in a draft phase, you are not only able to adjust the (functional) design, but also the tests. And, as a result, a design model (at unit level) is created that enjoys the confidence of analysts, developers and testers.
Integration of the components
With MBD and simulation testing it is possible to continually test and validate the design. The next step is to integrate the validated models with other validated models and check the integration of the different components. The previously developed testing scenarios at an unit level can also be integrated with each other to use as a comprehensive integration test.
This can be done the same way as an end-2-end test of a built system. But when integrating various subsystems (models), it is important to have a good overview of the various requirements of each individual component. Or else the complexity will only get harder.
Reuse test cases
In the end a fully tested system can be built with good confidence. All system components and subsystems can be built, (re)tested and integrated in larger systems.
And with tooling it’s possible that the tests of the simulation can be reused during the testing of the actual code and software. The use of identical test cases allows a comparison between the model and reality simpler (eg. Test Driven Development).
Simulation Testing
By using Simulation Testing (modelling and executing these models), it is possible to visualise and debug designs without any (newly) written code. Design errors can be removed quickly from the documentation. Moreover, a large part of the (development) testing can be performed based on simulations. By using simulations for checking the development process, models offer a different technique to find defects in the design at the earliest possible moment.
