Today (March 17, 2010) I gave a presentation at the Swiss Testing Day in Zürich, Switzerland. The talk was about 25 minutes long and I had a great questioning section at the end.
The presentation was about that testing is approving, accepting and exploring. Most testers only approve software and some explore it. But how is software accepted? Software can be accepted when we know we have tested what we needed to test. To know this we have to get information on what we have tested and what we haven’t tested. A tester interprets different information on what needs to be tested. This information should be documented in requirements, but there are more requirements that are not documented that needs to be taken into account to accept a software product. Good information on all requirements is needed to get to know what is tested! Accepting the software is a task for the users, but how can the users be helped. Crowds can help accepting software.
Last year I’ve written a post about the use of crowds while testing application: crowdtesting. And now I’m invited to present about this at the Swiss Testing Day on March, 17 of this year. I would like to share some more thoughts with you about this subject
Alan Whitney Brown (American comedian) once said: “There are a billion people in China. It’s not easy to be an individual in a crowd of more than a billion people. Think of it. More than a billion people. That means even if you’re a one-in-a-million type of guy, there are still a thousand guys exactly like you.” That means that when we have enough people, we have a thousand special people. These people can also look at a application from a different side and have an opinion about it. They can test the application! Wouldn’t this make an applications better?
I was looking for a demonstration video of a crowdtest en found this one of the Millennium Bridge in London. This bridge was designed for pedestrians.
In this test it’s shown that the bridge can support all these pedestrians in one time.
Crowdtesting is dependent on the quality of the crowd
Crowdtesting depends on a crowd that is composed out of a large group of divers people. This is maybe the most important part of crowdtesting; to create a crowd! A crowd should consist out of test experts, users, specialty testers, novices and everybody else that wants to test. A small group of 10 people with the same background gives crowdtesting no added value. But they should not only be of various test knowledge, but preferably also from different backgrounds and even different languages. The more different views there are on an application, the more different vies that can help get a better product.
The goal of testing: getting a better product!
That’s the goal of testing, if I’m correct. To get a better product! At least a better insight into the risks of the product. So if your product depends on the diversity of the crowd it is important to know what the people in the crowd can or cannot do. But also why they would be interested in testing software and what their knowledge or background is. This information is needed and needs to be available. This information should be added to something like a crowd Atlas.
The most difficult task of crowdtesting is determining a good enough crowd
But how can you determine what the most beneficial structure of the crowd it when it is needed? That is the most difficult task of crowdtesting, determining a good enough crowd. The use of the crowd itself is decided by the client that owns the work that needs to be tested. And beside these needed people there are people that can join voluntarily and most of these people will do this in addition to their daily work.
Crowdtesting is complementary to ‘normal’ testing
As I said in earlier posts is that crowdtesting aims on the acceptance of software. The crowd tests the software and can use the application further or even make others enthusiastic about it. But crowdtesting complements ‘normal’ or traditional testing. Normal test runs are still needed in previous test levels. Crowdtesting can use the creativity and diversity of various testers around the world and not just a small user group to accept the software.
Crowdtesting can do system tests for usability and performance testing
Most of the time this is true. Executing system tests or development tests is much more difficult to be done by a crowd. However it is always possible to work with a crowd for development or system tests. But keep in mind that you kind of release a product that can still be full of bugs. When a product is still full of bugs acceptation by the crowd will be less and these users may spread the word that the application is bad because it’s full of bugs. That no good marketing! However you can test the usability and other things, like the performance, load and stress ratio’s and security testing.
Crowdtesting can be used for testing software. Not only for mobile applications, but for all kinds of applications. This crowd consist of a diverse and talented group of people. People that are looking to accept software for you. With crowdtesting you can tap into the skills and knowledge of almost 7 billion people. How special is that?
As the end of 2009 comes closer and closer it’s time for a little reflection on 2009. This my first full year as a business developer at the company Andréas and I work at. Last year I started in April and we had one assignment: ‘write a book about BDTM’. But this year started with us doing more various things. We also started this blog (actually it was December 2008) and started to write about the things we come across in our work. Although the financial crisis didn’t really help I did a lot of nice things.
I thought of crowdtesting. Not totally new, but I tried to grasp it with all its potential and weaknesses. I thought of it while I was consulted by a game manufacturer on how their test process was. During the year I did 2 assignments for clients to implement it. One was successful, the other a little less (at least for me). The fun part was that also other people started upon it and we both posted several articles in magazines about this phenomenon. And I am invited to present on this subject on the Swiss Testing Day in 2010.
Evaluations became much more important in software development. Documents need to be checked more and more thoroughly before it can be used in later phases of the development lifecycle. I try to use reading techniques to help streamline this method of checking the documentation. A process that needs to developed even further next year.
This because the new urge for industrialisation of software testing in Europe tends to outsource checking to India or software. Like the opportunities Model-Based Testing adds to automatically create test cases for checking an application. These test cases can from there also be executed automatically to decrease the amount of work testers need to do to check a software program.
Another big topic for me this year was Augmented Reality. This fabulous new technology also needs to be tested and we tried to create a method to test it. But most important was to think about the use of this technology for testing. This became Augmented Testing; a subject I will share with you in 2010.
2009 was a good year, but 2010 has a lot of potential to get even better.
