Dear tester, what makes you feel satisfied? A hundred bugs or hundred passed test cases without finding a bug? What are your feelings when you’re testing a couple of days, making real progress in terms of functions that are covered, without finding bugs? Are you doubting about your qualities at that moment? Let me share you a concrete situation that gives me mixed feelings. And please help me to give an answer on some questions.

Difficult situation of the project
For the System under Test (SuT) we planned to execute a System Test (ST); based on the history of the project (2 years of development already) we decided to execute an intake test before we start the real ST. It so happens that all the releases from 0.1 until now the ST couldn’t be executed.

Last Monday we started with release 4.0. After one day we concluded that we cannot execute the complete system so there was a blocking issue in the intake test. Result: no System Test.Two days later, on Wednesday, the developers delivered a 4.1 release in the test environment. After half a day of testing we found another blocking issue in the system. And again the result was: no System Test. Friday we got another new release, not a complete new build but only a couple of small changes; a 4.1.1. We started enthusiastic with high expectations, but again after two hours there was a blocking issue. Except it was earlier in the process than release 4.1 and even earlier than release 4.0. I think something was changed that wasn’t aloud to change.
Working on this project for more than a month now but I still don’t have a clue what’s going on, this happens time after time. These things gives me sometimes some bad feelings.

Finding blocking issue and the feeling
These things doesn’t give me a good feeling at the moment. I’m worried about a couple of things.
First of all: the quality of the software, there are some laws that needs to be implemented next year and the software should support these laws. We really have a hard deadline, however we started early enough (2 years ago) and we still can’t even execute the intake of the ST successfully.

Second: our budget. Yes we found some other bugs in the parts of the software that is in operational state! So we deliver added value, but the goal was to pass the intake in just one day and start with the ST. The complete test team isn’t effective at this moment so this costs the customer a lot of money.
And what about the planning, having said the things above you can imagine we’re out of planning for months.

What would you do as a test manager in this concrete situation? Reporting? What steps would you take? Stop testing? Go on with testing? Go back to an earlier phase? Are you as a test manager struggling with these situations?

Finding bugs and the good feeling
In this real situation the test team is busy with executing other tests in functions of the software that can be used. In these parts they find issues that must be solved before they can ship the software. So they deliver added value, and hopefully these issues don’t appear again in a later release (but no guarantees at this point).

Finding these defects can give you a good feeling for two reasons:
- If these issues appear in production the owner of the application has a huge problem with the law at the background
- With finding issues you have the feeling that you deliver added value to the organization.

My lesson learned
The result from this situation is a lesson learned with an open question.

“Feeling good”  for a tester is created by a balance between making progress and finding bugs.

Having said that several questions comes up in my mind, please help me to answer them:
- Does this balance exist?
- If yes, can you define this balance? Are there parameters who determine the balance?
- How can you measure the situation and determine in which state the balance is?
- Are there others who have influence at this balance
- Is this type of balance a measurement for the quality of the software?

Or is the question “When do you feel good” not aloud for a professional tester? Are feelings not in scope?

If you can answer these question, you know how to report the progress, you know how to act in a project meeting and how to inform stakeholders. Managing these feeling is part of the “Mastering the test profession“. Feel free to share your feelings about this type of progress.

Last week they published a new issue of the Security Acts magazine. You can download and read this magazine for free via this link. A couple of months ago I wrote a blog about 20 ways to test the login function. A lot of people have read this post so the idea comes up to write an article about this topic.

The result is the article in the 4^th issue of the Security Acts magazine. You can find this article at page 34. If you want to read only my article, click at the image below.

Enjoy reading, I can tell you that there are quit some good articles in this magazine, so it’s worth to register and download the complete magazine.

The fun is, after writing the article I found out that there are only 19 ways described in the article and the blog. I had 2 choices, or writing an extra test case, or change the title. I choose the simple one.

Worth reading in this context:

• Hidden treasures for everyone, About vulnerabilities in applications
• A Letter to a friend, About common mistakes by building applications
• Threats are caused by a combination of defects!, about how threats occur by different defects together in one application.

Cloud computing is an emerging phenomenon that offers enormous advantages, such as shorter time to market, flexible computing capabilities and limitless power, but the cloud market, still in a very early stage, continues to grow and evolve.

But cloud computing is much more than technology, cutting costs or getting more agile. The cloud model is a new business model, a new way of thinking and doing IT as a business instead of a technology! The cloud business model. More modular, incremental, selective, collaborative and with a value proposition based on financial liquidity and operational flexibility [Forrester, 2010]. This has a lot of (side) effects, also on software testing…

New model for testing

The way we do business around testing will change also. Not the way we test, but how we deliver it. The market will be more focused on value driven, agile, modular and flexible solutions from test service providers. These providers need to create more ‘building blocks’ around how they deliver services: testing. Testing will also become a service, Software Testing as a Service (has a nice buzz ring to it).

Software Testing as a Service

Software Testing as a Service (STaaS) was coined by my colleague Leo van der Aalst in 2008, but his ideas didn’t go as far as I would think of today. In Leo’s proposal all testing was done by a service provider on demand at the scale as needed by the client. My idea goes further.

Software testing needs to be even more flexible, from the test manager to the test engineer, from the system test environment to the performance test environment, from the security scan to the full usability test and from audits to end-2-end tests. And everything can be done separate and fully integrated with one other. This creates a full cloud model for testing. With the possibility to be fully standardized, agile and elastic to help the client when needed; a full service consisting of several building blocks.