The last few weeks I get a lot of requests to tell something about testing and cloud, like this interview for the Australian newspaper Sydney Morning Herald. And one of the most asked questions of course is what’s different in testing on the cloud and where do you need to pay extra attention at.

Cloud applications are still few compared to traditional applications, but they are the future. But what are cloud applications? When the question is asked “can you name a few cloud applications?” most people answer Salesforce.com, Facebook, Google Apps and even Microsoft Azure. Four hits (where one isn’t actually a cloud application), as the best known examples. Are there more? Yes and they are growing in numbers!

Note: Microsoft Azure is not a cloud application, but an infrastructure and platform.

But how do we test these cloud applications? What’s so special about them that they need a different type of testing than traditional applications? Cloud applications are applications that are created to leverage the opportunities the cloud gives them, but they also work with the disadvantages the cloud offers, like, for example, standardization.

The cloud is defined by its service model, deployment model and usage

Mostly cloud applications are based in the third cloud layer: SaaS. They are Software as a Service solutions that run completely on cloud infrastructure and platforms. And that is exactly the reason the testing of SaaS applications is different from traditional applications. When they are integrated in the current architecture they need to be tested on three levels: namely the infrastructure, the platform and the application itself, see figure. The usage of standard services of applications also means a change for system testing. Functional testing will be executed at a minimum, as the standard applications are already tested and approved by the supplier. But that doesn’t say anything about how it integrates into the client’s cloud.

But that’s my idea. How do you see this?

Do you know what Insanity means? As Einstein once said it’s “Doing the same thing over and over again and expecting different results”.  I already said this in an earlier post, but it also fits here perfectly. And we’re being insane in IT quality today. Every year, money is thrown away by implementing bad software. It’s either lacks quality or the quality use is not practiced to get the highest quality against the least costs and still be flexible enough.

Sounds familiar isn’t it? Are you also struggling with the right balance in cost and quality?

Quality IT supports the Business in achieving business goals. Aspects in realizing shorter lead times, more flexibility, higher quality and lower cost are crucial. Quality Assurance and structured testing have been contributing to determining quality for years. Now we can increase this value and move to ‘Business Quality’.

With a new process around Business Quality to see three major movements in quality. By using technological opportunities, like TMap and TPI and Lean thinking we can industrialize quality activities.

When trying to start the quality process before the project even has started the business case is the driver of the IT project. All activities are evaluated against the business case, because by paying attention to quality as early as possible the greatest savings are achieved.

Integrating (future) SMART innovations into the whole process will help being quality driven from the very beginning of the project; once the business case is made, quality is part of the solution!

This all helps the Business to determine to what extent the described process is complete and accurate and effective solution to an identified business need; Business Quality is created, with PointZERO!

With this we van stop the insanity and proving Murphy wrong!

If you have any other ideas or the answer just let me know…

I’m writing this again as a response to Andréas’ post (who wrote this as a response to mine).

As I’ve been looking at the ROI of testing the last few weeks I found out that the most used numbers are still based on the initial study of Boehm from 1979. He calculated the cost of change of a waterfall method and found out that ‘the earlier you fix the problem, the cheaper it is’. That idea is still the most relevant of all. If you look at the implementation of LEAN you get the same. Fix the issues when it occurs, instead of waiting till the end. It improves the lead time and reduces the costs.

But the numbers from Boehm are based on the waterfall method, as said. Now what is the cost of change for Agile? Of course you cannot really compare them both directly, but what I found out surprised me a little bit.

Both of them are based on studies, waterfall by Barry Boehm and STBC, and the Agile by STBC again. Why am I looking at these studies? Well people in QA always use Boehm to elaborate why they want to start testing as soon as possible and both of them are used in the IV&V method for the Public Sector in the US.

When you look at both you see a rising amount of cost after the requirements phase of the Waterfall method and after the test phase these costs just go sky high. When you look at agile there is a slight rise in costs after the coding phase in the Agile cycle, but when going into Production the costs will go sky high.

Why are the costs of Agile lower after the business case? Well, one reason is that with Agile documents are created as needed for the development process. So during the full Agile cycle there is not so much need in changing documentation, only the code when an error is found. But when the product is shipped for production, that documentation still has tob e delivered for the Management & Control team of the organization.

Note: This something that is often forgotten in Agile development teams not doing a good usage of the method. Speed is generated by producing as less as possible documentation, but it still has to be delivered at the end of the project. And it’s only human to ‘forget’ this!

But statements from Agile devotees like ‘collaboration between the various partners in the development process is what makes Agile better compared with Waterfall’ are bullshit. In Waterfall collaboration is also possible! That doesn’t have to be a difference. It’s only true that quite a lot of times this is not done in Waterfall. But maybe you have some ideas around this

But one thing is the same in Agile and Waterfall: issues found in Production are always very expensive to fix!