Since we first posted about Augmented Reality and testing, we’ve been doing some further research about this topic. Augmented Reality (AR) is still a relatively new industry but has people talking about it on the web. The word Augmented Reality is twittered 5 times per hour, and more and more people are getting involved. In our research we’ve contacted a few companies that are creating Augmented Reality and how they would advise to test the software.

I’ll be presenting on this topic at my company, internally and externally. But here I would like to set out what I’ve found so far. And so far is: “Testing Augmented Reality consist of an Integration approach, Task-Based testing and hardware testing”. In this post I’ll try to explain the integration approach.

“An integration approach is about how to integrate the different modules into the system”

Augmented Reality software has the same testing life cycle as with testing normal software. Normally there is development testing and system and acceptance testing. A system integration test (SIT) should be performed, many defects are related to the integration of the different modules or system parts. Incorrect interaction between modules is often caused by wrong interpretations of different developers of the different modules. The best instrument to detect these defects is by executing an integration test. The system integration test relies on the integration approach. An integration approach is a decision about how to integrate the different modules into the complete system. The integration approach depends on:

• Availability of the integration parts;
• Size of the system;
• Whether it is a new system or an existing system with added/changed functionality;
• Architecture of the system.

“Testing AR compares a lot too testing embedded software”
Even though the testing of Augmented Reality compares a lot too embedded software, there are also differences. Especially when it comes to the integration approach. In testing embedded software there are a lot of approaches possible. These are bottom-up integration, top-down integration, layer integration and client/server integration, big bang integration, centralized integration or collaboration integration. The first four are less suited for AR because of the use of many stubs and drivers that is why the other three can be used. I will explain these three a little.

“Big bang integration is the integration of the whole system at once”
The big bang integration approach is basically quite simple; all modules are integrated and the system is tested as a whole. The main advantage of this approach is that no stubs and drivers have to be used.

This strategy can only be successful if a large part of the system is stable before the integration and only a few new modules are added. Also to make the strategy successful the system should be rather small and the modules are so tightly coupled that it is too difficult to integrate the different modules stepwise.

With this approach, it is difficult to find causes of defects and the integration can only start if all the modules are available.

“With centralized integration the central part of the system is tested first”
The centralized integration is used when a central part of the system is necessary for the rest of the system or function (e.g. the kernel of an operating system). This central part is necessary to run tests and it is too difficult to substitute this part with a stub. Also the architecture of the system is such that first the central part of the system is developed and made ready for production and after that new modules or subsystems are released to upgrade the system or add complete new functionality.

The central part of the system is tested first. The next step is the integration of the control structure. The coupled subsystems are tested. The bottleneck of this approach is the integration of the central part of the system. Sometimes the only usable integration strategy is the big bang strategy because the modules of this part are often very tightly connected. This strategy can only be successful if the centralized components are tested very thoroughly. A very strong point of this approach is the combination of the different strategies and the ability to choose the most effective strategy for every system part.

“Collaboration integration is only usable for object oriented systems”
A collaboration is a collection of objects collaborating to achieve a common purpose, e.g. the realisation of a use case. The system supports many collaborations because it has to realise many use cases. Many objects are part of more than one collaboration. This means that with a clever choice of collaborations the complete system can be covered. Collaboration integration is only usable for object oriented systems. This strategy is only usable if the collaborations are clearly defined and cover all components and interfaces. The last is also a disadvantage because the subtle dependencies between different collaborations can be left out of the collaboration model. This strategy is partly a big-bang implementation because all the components of one collaboration are put together and only if the collaboration is complete integration testing starts. Only a few tests are necessary because the focus is on end-to-end functionality. Because of the overlap of collaborations it is often not necessary to test every collaboration. Because a group of related components are tested none or only a few stubs and drivers are necessary to execute the test.