Tag Archives: test strategy

I’m not the only one that is responsible for the quality

Posted on by
100

Sometimes testers make some assumptions, but hopefully they avoid them. This series of posts will show you some well-known pitfalls and assumptions that tester make. This second post describes is about responsibilities.

Assumption: The developer has performed all the technical checks
In my opinion it’s the task of the developers to perform all the technical checks. Unfortunately  daily business shows us that many defects aren’t found during development testing. (Clemens Reijnen has several post about testing for developers) The developers goal is to prove the software works properly.
The goal of a tester is give insight in the quality of the application. If needed a tester has to prove the software doesn`t work as expected. The goal of a test expert is not to break the software, however this is the motivation most testers have.

Talk with the developers in your project team to understand them and to know who they are and what they do. After knowing who a developer  is you can discuss what is in his scope and what`s in yours. (I think you can write a separate post about this topic, maybe in the future)

Assumption: The business will test the complete application
This isn’t the truth, there are reasons the business will not test the total application.

They test what they need, not every person needs the complete application. That means they only test the part of the application that will affect their daily processes.

Most of the time during an user acceptance test they will accept the software, the result from this is that they test the happy flow. The happy flow is mostly a short path through the application without. Not all the possible exceptions are part of their test cases.

Assumption: They business knows all the exceptions and they’ll test them.
As you gueseds this assumption isn’t true for several reasons. Of course often does the business know the exceptions that are in the application, for example in the business rules or constraints. But the people that will test during the User Acceptance Test are not always a correct representation of the organization. For example, not all roles in the process are selected, or sometimes people are just available in the organization so they can do the UAT.

It is also possible that they forget some exceptions. What do you do with these forgotten things?

Assumption: The test strategy is accepted, that’s the only thing we do
If you want to be out of business very quickly I would suggest this approach. If the project is constantly changing and you’re rigid in the things you do, you don`t have an added value that is needed to successfully accomplish the project.

Be aware for the changes in the infrastructure, think about the scope of the project. But also the role stakeholders will have in the project or the always changing risks that are caused by unexpected incidents.

In a lot of projects is the lead time for test executing is under pressure. With this pressure some test cases can`t be executed. You have to redefine what you will test and what not.

Tips: How to handle in this situation
What to do with these things, some practical tips:

  • Be focused at changes of scope or responsibilities, if needed change your test strategy during the project.
  • Ask for an overview of the test cases the developers did. With this overview you get insight in what is tested and what`s not.
  • Do some coordination of the UAT now you can trace what test cases are done and what part of the application is covered.
  • Be sure all exception are covered, you can avoid this in an early stage with evaluating the requirements for example.

This is not an extensive list but a few tips you can use to help you cope with these falsely made assumptions

Standard software products vs. Testing

Posted on by
2

A few weeks ago I had a conversation with a colleague of mine and fellow Twit @bartvrenegoor. We had a conversation about what to test with standard software product. I came to the statement: “With standard software products functional and usability testing don’t exist anymore. You buy them with the product.” Now it’s needed for me to support that statement!

An assumption here is that I think standard software products are chosen. Chosen for the standard functionality the product has. And, as it is standard software, with standard usability. The word “chosen” is the thing here. A standard software product is, well, standard. What You See Is What You Get! How it functions in your environment is what the question that needs to be asked. The software should function properly standalone or with the directly supported. But how does it function in relation to other programs, in your environment? These things should be found in a product risk analysis of the system. And these results is what I should be testing!

So if I get a standard software package I don’t need to test the functionality and usability? Yes! It’s not needed, from a risk based view. You can’t change it, because you chose it that way. It’s much more important to test for security, performance, maintainability, portability, suitability, connectivity, continuity, interoperability and all other non-functional quality characteristics.

This idea I use at my client. We are implementing 7 standard software packages within the software environment. Our main target is interoperability, performance and connectivity. Why these? Because the client wants them to be tested. It’s stated in the test goals and elaborated in the product risk analysis. I was asked why don’t you look at the functionality of the software. And I asked them why did you choose this software? The answer was: “it was the software that fitted our need”. In which I replied that the functionality now was given. In the test strategy I stated this all and everyone agreed. So, as our test cases are derived from the test strategy, we created test cases to support is interoperability, performance and connectivity.

Can’t I test the functionality and usability of standard software packages. Yes, you could ‘test’ the software when you decide which software package you are going to use, but that’s something else.

One definition, one goal but two perspectives at testing

Posted on by
Reply

My assignment at this moment is to define some test goals and goals from the test community for my client. This is an interesting job to do, because you have to go back to the start and define why we’re doing our work. During this task I had a nice discussion with a colleague and I will share this with you.

This discussion brought me to the conclusion that there are many things that have an impact at the perspective of software testing. An outcome for me is also, be very clear to each other (in this kind of discussions). If possible draw a picture or something to explain your thoughts.

The following picture is my conclusion and if you see this picture I hope it’s very simple.

2 perspectives of testing

But first let me tell you the discussion. I started with some definitions of testing and it doesn’t matter which one you take in general they have the same message.

Here are three of them:

TMap®: Testing is a process that provides insight into, and advice on, quality and the related risks. (source)

ISTQB: The process consisting of all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products 39 to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects. (source)

ISO/IEC: Technical operation that consists of the determination of one or more characteristics of a given product, process or service according to a specified procedure [ISO/IEC Guide 2, 1991]. (source)

I chose the TMap® definition, because in my opinion this was the best my client. After describing them I send an overview to my colleague. He send me this comment (free translation Dutch to Englisch). What he did is to split the definition into two parts and described, in my opinion, a combination of definitions and goals of software testing.

Goal of testing from the perspective of the test engineer: testing is doing research to give an answer to the question: Do we deliver the products with the correct quality to our customer?

Goal of testing from the customer perspective: Testing gives an answer at the question: Do we get the right quality of services caused by the product, as a result that it is fitting to the end-user.

These two definitions are different! That conclusion we made together when we read these statements.

I think these two statements are not definitions but “roadmaps” or “routes” of how we’re doing our work. In the picture this is the twisted line. And as you can see in the picture there are two lines with different colours.

Each line is different and this is caused by the background of different people. The tester has other knowledge; another role and other interest as for example our customer. The tester for example knows a lot about test approaches,  test design techniques and more While the customer has the interest to deliver the product in a specific time, to overwhelm the market.

I think that in what kind of role you’re having, the definition is equal. But the goal is shared, in terms of delivering an information system that fits the use and meets the requirements. Because of a background the route (twisted line) to reach this goal is different.

So realize that if you’re doing your work, there will be differences. But what more important is, there are agreements and there’s always a common goal. Try to figure out how this drawing fit in your situation.