Tag Archives: development testing

Why developers don’t find defects

Posted on by
8

Fellow Twit @testingqa recommended this article to me. It’s about how your brain hides facts from you. Even if these facts are the thruth. This also has its effect of the software development process. On how testers should look at the SuT, but even more important why developers don’t find defects.

As a test manager I try to help developers test their created code and software. Most of the time they tend to check their code quite thorough but don’t see the errors in the software itself. They tend to look only at the mistakes in their own code, but don’t see errors everywhere else in the system. It’s like they don’t ‘see’ the defects.

When I was reading the article I learned that scientists have the same problem. They ignore failures. Kevin Dunbar states “Although we pretend we’re empiricists — our views dictated by nothing but the facts — we’re actually blinkered, especially when it comes to information that contradicts our theories.” I think this goes for a lot of ‘logical’ engineers. I state engineers because not only do software developers have this problem, but also analysts and testers. But why is this? Why is all data is created equal in our mind’s eye, but some data is created more equal than others?

This is because a part in the brain. This part, the dorsolateral prefrontal cortex, or DLPFC, is editing our reality. If you would like to know where it is. It’s located just behind the forehead. This DLPFC is one of the main reasons people ignore or suppress unwanted representations. It deletes unwanted information.

When a developer is checking the software for defects the software produces failures. And because the failures are unwanted to the developer they are ignored. Maybe they are not ignored on purpose, but because of the developer’s brain. Their DLPFC’s kick into gear and delete the ‘image’ of the failure from their consciousness. In most contexts, this act of editing is an essential cognitive skill. However, when it comes to noticing anomalies (defects), an efficient DLPFC can actually be a serious liability. The DLPFC is constantly censoring the world, erasing facts from our experience.

How can we help developers overcome this disability? We know it’s in their brain, but there can be action on taken on this. The article states four steps.

First of all check your assumptions. This is a skill a tester sometimes lacks. Run the test again and check if the failure is still there. Then look at your test and see if the test is correct. But also developers need this skill. They run checks at the system and when these checks fail, the system can be correct and the check can be false. Seek the correct failure.

Next is to seek out the ignorant. Use people from the outside to look at your software. Explain to them how it works and what is wrong. They can have a different view at the problem and can help you fix the problem in another manner.

Too often, developers assume that a failed check is a wasted effort, but testers or users think otherwise. Even analysts can see in differently. Let this diversity help enhance the software and make it work better. More different eyes on a system creates more helpful views on it! When everybody ‘sees’ the system the same way they all have the same DLPFC that kicks in.

The DLPFC shall kick in ‘delete’ the unwanted information. But if you are aware of this you can help yourself to take actions on it. You can try to be very methodical at what you see and act on this instead of your brain.

What I get out of this is that diverse teams help to develop better software. These teams can help each other to look at problems from different viewpoints. Agile teams try to use this diversity create a product faster, but also better. With this in mind agile seems to be a good option to use to help developers not to use their brain J!

The future comes later, but what now?

Posted on by
Reply

And here I am again. I just came back from vacation and already have the urge post an article on this beautiful blog that we have together. We already discussed the near future of testing. One of the things that are changing is the change to standard software packages. These systems should of course be properly tested when installing at the client. I have experience with this package testing when I tested an implementation of PeopleSoft! But we also see a shift where the testing takes place.

More and more clients require that their suppliers provide well-tested software. I state well tested, because they already wanted quality software! By proving the needed prove that the software is properly tested a supplier can count on sales to the client and the client does not have to test it by itself. I think there are some contractual snags, but I am a test manager and no lawyer. For companies that provide testing services to customers they lose clientele. The client is no longer testing by itself. Something that in the current economy is really popular (for the client). A test service provider also use it to an advantage.

With less purely functional testing to offer, but more development testing, a newly created market can be approached. Suppliers often have no idea how to test and then they look to the experts. Only are these suppliers not looking for functional testers. As a result there is a shift in focus on testing software, there still is testing that needs to be done, but at a different level. Development testing can be done on many different levels. It also makes the use of test automation easier. Applications for reading code or simple errors can be applied immediately, but also things like model-based testing can be better deployed during development testing.

Since testing is a form of quality there will also be an important focus on other measures. By determining whether the design of an application really good pick it is a good first step to review or evaluate a document. In this way, a document is ‘tested’ and has defects are found in an early stage. Actually evaluating is just testing! And another good thing; it’s cheaper compared to testing! This is because the errors are removed early and are not build into the application and while testing are newly discovered, which in turn leads to adjustments. This makes the job as a test manager actually simpler.

What should a test manager do? First, a test manager will work even more closely with the Quality Manager. Together they will develop the best strategy for how a project and suppliers can be checked on results. A test manager will therefore have to look into the evaluation and testing strategy. Checking for risks, time, cost and result. A test manager will also require the supplier to establish in what form the communicate on how well (or how bad) there is tested.

Also a end-to-end test will be needed to determine if all the software applications work well together. As posted earlier, this may be in large chains, something most test managers are not prepared for. In the end the client still needs to execute an acceptance test, because that is required by law (at least in the Netherlands). Maybe something of a lawyer in a test manager after all?

The field of a test service provider has changed, but particularly for a test manager. Be prepared!