Dear tester, what makes you feel satisfied? A hundred bugs or hundred passed test cases without finding a bug? What are your feelings when you’re testing a couple of days, making real progress in terms of functions that are covered, without finding bugs? Are you doubting about your qualities at that moment? Let me share you a concrete situation that gives me mixed feelings. And please help me to give an answer on some questions.

Difficult situation of the project
For the System under Test (SuT) we planned to execute a System Test (ST); based on the history of the project (2 years of development already) we decided to execute an intake test before we start the real ST. It so happens that all the releases from 0.1 until now the ST couldn’t be executed.

Last Monday we started with release 4.0. After one day we concluded that we cannot execute the complete system so there was a blocking issue in the intake test. Result: no System Test.Two days later, on Wednesday, the developers delivered a 4.1 release in the test environment. After half a day of testing we found another blocking issue in the system. And again the result was: no System Test. Friday we got another new release, not a complete new build but only a couple of small changes; a 4.1.1. We started enthusiastic with high expectations, but again after two hours there was a blocking issue. Except it was earlier in the process than release 4.1 and even earlier than release 4.0. I think something was changed that wasn’t aloud to change.
Working on this project for more than a month now but I still don’t have a clue what’s going on, this happens time after time. These things gives me sometimes some bad feelings.

Finding blocking issue and the feeling
These things doesn’t give me a good feeling at the moment. I’m worried about a couple of things.
First of all: the quality of the software, there are some laws that needs to be implemented next year and the software should support these laws. We really have a hard deadline, however we started early enough (2 years ago) and we still can’t even execute the intake of the ST successfully.

Second: our budget. Yes we found some other bugs in the parts of the software that is in operational state! So we deliver added value, but the goal was to pass the intake in just one day and start with the ST. The complete test team isn’t effective at this moment so this costs the customer a lot of money.
And what about the planning, having said the things above you can imagine we’re out of planning for months.

What would you do as a test manager in this concrete situation? Reporting? What steps would you take? Stop testing? Go on with testing? Go back to an earlier phase? Are you as a test manager struggling with these situations?

Finding bugs and the good feeling
In this real situation the test team is busy with executing other tests in functions of the software that can be used. In these parts they find issues that must be solved before they can ship the software. So they deliver added value, and hopefully these issues don’t appear again in a later release (but no guarantees at this point).

Finding these defects can give you a good feeling for two reasons:
- If these issues appear in production the owner of the application has a huge problem with the law at the background
- With finding issues you have the feeling that you deliver added value to the organization.

My lesson learned
The result from this situation is a lesson learned with an open question.

“Feeling good”  for a tester is created by a balance between making progress and finding bugs.

Having said that several questions comes up in my mind, please help me to answer them:
- Does this balance exist?
- If yes, can you define this balance? Are there parameters who determine the balance?
- How can you measure the situation and determine in which state the balance is?
- Are there others who have influence at this balance
- Is this type of balance a measurement for the quality of the software?

Or is the question “When do you feel good” not aloud for a professional tester? Are feelings not in scope?

If you can answer these question, you know how to report the progress, you know how to act in a project meeting and how to inform stakeholders. Managing these feeling is part of the “Mastering the test profession“. Feel free to share your feelings about this type of progress.

1. Richard Enthoven Says:
   August 11th, 2010 at 16:09

   Great post with some recognizable situations.

   Yes, I think the balance between making progress and the amount of findings exists.
   I think every tester wants to discover findings (especially in an early stage). But as a tester you’re also part of a project team with a shared goal: bringing a quality product in time and budget. Those two goals have some dependencies, because with finding (1st goal) the qaulity of the product (2nd goal) will decrease.

   But when the project planning is done well and there is some time for a second test round, both goals can be achieved.

   But in some projects (like the one you described in your post) it’s impossible to achieve both goals.

   In my opinion the test manager should think about the project progress in stead of his test team performance. If the project is successful, the testing is successful too.

2. Michel Kraaij Says:
   August 11th, 2010 at 17:25

   @Richard:
   “Those two goals have some dependencies, because with finding (1st goal) the qaulity of the product (2nd goal) will decrease.”

   The quality of the product will not decrease. You just exposed part of the *real* quality, instead of some *made up* quality.

   “In my opinion the test manager should think about the project progress in stead of his test team performance”

   Don’t mix the responsibilities of a test manager with those of a project manager. There is a difference between feeling responsible and being responsible. I might care for project progress, but it’s the project managers responsibility to manage it. Goal for testing is questioning the product in order to evaluate it. The responsibility of the testing roles (disregarding the actual roles defined for testing) are doing this task as efficient and effective as possible.

   @Andréas:
   “Does this balance exist?”
   “Feeling good” for me comes with customer satisfaction. If i can deliver *on-time* information which helps deciding what to next, i get that fuzzy feeling. If this happens by the first problem during an intake test, excellent! Wouldn’t you be happy if you were investigating a murder and found the murder weapon just after 10 seconds of searching?

   So the balance for me might be between effective testing (valuable information) and efficient testing (as soon as possible).

3. Andréas Prins Says:
   August 12th, 2010 at 07:37

   @Richard and @ Michel,

   Thanks for your comments.
   @Michel, I hope that these bugs don’t appear in the future, but testing the most basic process with all these findings give some feeling for the future.
   Thanks for sharing how you measure your satisfaction / what your satisfaction is. The strange thing is that there is no real customer… and yes that’s a separate discussion

   @Richard,
   We have to take these steps to finish the project succesful, so we have added value the thing is based on the things happened I can’t make any prediction or planning for the future.

4. vishnu Says:
   March 10th, 2011 at 12:20

   i feel very good .and lif is like fun……..

5. Trackbacks Says:
   February 5th, 2012 at 22:42
   • Tweets that mention An “I feel good” moment :: Software Testing and more — Topsy.com