A few weeks ago I had a conversation with a colleague of mine and fellow Twit @bartvrenegoor. We had a conversation about what to test with standard software product. I came to the statement: “With standard software products functional and usability testing don’t exist anymore. You buy them with the product.” Now it’s needed for me to support that statement!

An assumption here is that I think standard software products are chosen. Chosen for the standard functionality the product has. And, as it is standard software, with standard usability. The word “chosen” is the thing here. A standard software product is, well, standard. What You See Is What You Get! How it functions in your environment is what the question that needs to be asked. The software should function properly standalone or with the directly supported. But how does it function in relation to other programs, in your environment? These things should be found in a product risk analysis of the system. And these results is what I should be testing!

So if I get a standard software package I don’t need to test the functionality and usability? Yes! It’s not needed, from a risk based view. You can’t change it, because you chose it that way. It’s much more important to test for security, performance, maintainability, portability, suitability, connectivity, continuity, interoperability and all other non-functional quality characteristics.

This idea I use at my client. We are implementing 7 standard software packages within the software environment. Our main target is interoperability, performance and connectivity. Why these? Because the client wants them to be tested. It’s stated in the test goals and elaborated in the product risk analysis. I was asked why don’t you look at the functionality of the software. And I asked them why did you choose this software? The answer was: “it was the software that fitted our need”. In which I replied that the functionality now was given. In the test strategy I stated this all and everyone agreed. So, as our test cases are derived from the test strategy, we created test cases to support is interoperability, performance and connectivity.

Can’t I test the functionality and usability of standard software packages. Yes, you could ‘test’ the software when you decide which software package you are going to use, but that’s something else.

1. Michael Bolton Says:
   October 16th, 2009 at 16:25

   I have no idea what you mean by a “standard” software product. Can you help me understand?

   —Michael B.

2. Ewald Roodenrijs Says:
   October 19th, 2009 at 07:20

   @Michael,

   Maybe in English the incorrect name. But I mean “Commercial off-the-shelf (COTS) or simply off the shelf (OTS)”.

   - Ewald