What You See is Not What You Get
Dec18
Sometimes testers make some assumptions, but hopefully they avoid them. This series of posts will show you some well-known pitfalls and assumptions that tester make. This first post describes two things you can`t see if you only read the documentation or click throughout the application. The next one is about responsibilities.
Galileo Galilei once said: “All truths are easy to understand once they are discovered; the point is to discover them.”
In my opinion this also applicable at software testing. Change the word truth into defects and this quotation fits for these days. All defects are easy to understand once they are discovered, the point is to discover them. But to find and discover them you have to be sure the complete application is in scope.
Assumption: The requirements and designs describe the total application
If the requirements and design represent all the needs the business and users have, you have a good starting point to execute the test. What I see in daily business is that this documentation doesn’t describe the complete application. If you trust this as the complete description of the application you`re not complete.
If you’re working with a pyramid model with the business requirements in the top, the user requirement in the middle and the system requirements on the bottom 
not everything will be specified. Most of the time the high-level business requirements clear enough. These requirements describe the “why”, the goal of the project. But sometimes it happens that the users doesn’t know from the beginning what they really want, they don`t tell all the wishes they have or they even don’t know what they want.
If the developer builds the system and he is missing expected functions he will build them with his best effort. If you as a tester have to test such an application and you only specify your test cases with the documentation you can miss some vital elements. Think about for example some field validations about length or format. Or thing about the side the developer will implement the validations. Does this happen client side or server side? In the first situation your application is very insecure.
Besides this people can forget some elementary elements in, for example the architecture of the application. These things will not lead to bugs in the software but to ’flaws‘. These flaws are not expected “features” you can misuse. The following video shows you how the McDrive of the McDonalds has this type of flaw. I guess when people designed this McDrive they do not expect that people would take the food of somebody behind them for free at the second window.
And if you don`t trust the application ask the business or developers or any other oracles to give you more information. Gather as much information you need, from existing processes in the organization, do some interviews, ask for some prototypes if they are available.
Assumption: The System under Test I see is the total application
It happens to many testers that sometimes you do not know exactly what the scope of the application which you should test is. Sometimes you know how many pages there are in the documentation but you’re not sure how many functions or pages there are in the application. Most of the time are there more pages at the background of the application you can`t see from the GUI (Graphical User Interface). The page phpinfo.php is such an example, or other default pages. As a end user this page hasn`t any value but for a hacker is shows you for example the release of the web server and a lot of configuration settings.
One of the goals of software testing is to compare the software that is build with the design. Those two should be equal to each other. If a part is described but not implemented this is a defect, but if there are parts implemented but not described this is an unexpected risk and can be the origin of defects.
You can discover the total scope of the application with a spider tool. WebScarab for example or BurpSuite offer this functionality. You can install one these tools on your PC and configure it that the request that goes to the internet will go first to your tool. You can intercept the request and response.
The function you have to use to get the real scope is the spider function. You start this functionality at a certain page. The tool crawls all hyperlinks and will open them. Secondly the tool will open all the hyperlinks on those pages. In seconds you get an insight in the complete application. WebScarab for example sorts all those hyperlinks and other information in a kind of a tree navigation.
Be aware for the “delete account”, “change password” and “logout function” in the application. If the spider opens them he will execute them. If the spider automatically change your password, you can’t use this test/production account. If there are a lot of pages behind the login function and the spider hit the “logout button” you can’t spider the complete application.
So you have to use a combination of knowledge and tools to get a good scope.
Spidering is also very useful if you haven`t any documentation of the application.
My advice is:
- Install a spider tool to be sure you cover the complete application, because as a tester you’re often responsible to cover the total application
- Evaluate the documentation and ask to yourself by each functionality or requirement how can I use this feature in another way? And check of all these exceptions are already in the documentation.
Sources:
WebScarab: www.owasp.org
BurbSuite: http://portswigger.net/
February 1st, 2010 at 15:07
Thanks for your article.I’m planning to learn many methods to make money with supporting outsourcing services. Outsourcing Services, such as IT Outsourcing,Web Development Outsourcing Services. I got some information about the Outsourcing from this article and is useful to me.Tks again.
September 10th, 2010 at 22:03