Do not turn a blind eye to website optimization (part two)

In the posts about website optimizing, Thomas Veltman, explains how A/B testing can deliver added value to website optimizing. This post is his second.

In part one of this blog we discussed the problem many companies face with their web presence: How do you increase the effectiveness of your website as much as possible? In this second part of “Do not turn a blind eye to website optimization” we describe one technique for improving conversion rates of your website.

The technique is called A/B or multivariate testing.

A/B Testing tests the effectiveness of many different versions of your website against each other.

How does is it work? The main principle is simple. Instead of keeping just one version of your website online, you keep two or even more versions online. One version is the main version; most of the websites’ visitors are guided to this site. A smaller percentage of visitors is guided to another version of the website.

AB testing

The conversion and other performance indicators are measured for both versions of the website and compared. After that, the version with the best results becomes the main website.

And then the process starts over: another change is implemented in another new version of the website and that version is put alongside the main version the compare.

Dependent on the number of visitors to a website, it’s also possible to implement several changes in several other versions of the website. That way, more versions of the website can be compared. Since we are working with statistical figures here, the group of visitors has to be of a considerable size to have a dependable measurement. This limits the number of website version that one can have online concurrently

This approach is effective for two main reasons:

  • * Other (usability) test methods always measure some kind of simulation, weather in a laboratory or elsewhere. This measurement is taken on the real visitors, the real costumers, while they are using the product in real life. Very dependable, and without having to worry about factors that might alter the results of a test.
  • * If you make a change in your site and you have only one version, and it has a dramatic effect on the performance of the site, the conversion of the site and with that the turnover of the company can decrease dramatically (in the video which was linked in a response in my last blog a drop of 90% conversion rate was reported because of the introduction of a coupon code). By testing the change in this way, the performance will drop only for a smaller number of visitors and the risk is decreased.

There are several tools to implement such a test. Even free ones, such as Google website optimizer.

In the first post I talked about the challenge of website optimization being similar to that of a blind man climbing a mountain. The A/B testing method helps companies with their website optimization in the same way a mountain climber can be helped by being tied with a rope to someone else. If one falls into a gorge, the other person is there to haul them out of it. It helps companies gradually and structurally improve their websites!

Klimmen

Do not turn a blind eye to website optimization! (part one)

In the posts about website optimizing, Thomas Veltman, explains how the testing community can deliver added value to this process. Since functionality is not the only part of a good application, usability becomes more and more popular and needed. This post is the first one in a series.

The world of websites is a very competitive one. If you type the Dutch word “hypotheek” (meaning mortgage) in google the results are clear. Every big Dutch mortgage bank is trying to beat each other in Sponsored Links and search results. Every major party is present.

The internet is a tank filled with piranhas, typing a search term in google is like dropping a piece of meat in this tank.

Once you follow a link to a bank website you’ll see that here more effort has been put in to attract customers. They are all very fancy looking sites aimed at creating leads for mortgage offers. This is understandable: although the competition is not visible on the site itself, it is always just a mouse click away.

Underneath these seemingly smooth looking websites there is a problem for the sponsors of the websites. How can they make sure that their website will be able to keep up with the competition? How can they keep track of the changing market and wishes of their clients? To show how big this challenge is and to show a beginning of a solution, I have to explain something else first.

On May 25th 2001, Erik Weihenmayer became the first blind man to climb to the top of Mount Everest. Erik, blind since he was 13, had to overcome quite a challenge. Imagine going up such a mountain without seeing anything: you would have to go on uncertain triggers like touch and hearing to find your way up and the risks of stepping in a canyon or a ravine seem overwhelming. And besides that, how do you know which way you have to go to the top? And once you get there, how do you know that you are there? How do you know that there is not just a little bit going down and than an even higher summit after that?

The challenges that Erik faced are similar to the challenges that the owners of a website in a competitive market face. The sponsors for the websites are interested in being on top: on top of the results page for the google search term in which they are interested  and the top site in website conversion.

Conversion is a measure of the effectiveness of a website. It measures the ratio of the people who come to the website and the people who reach the goals that have been set for the website by the sponsor of the website. For example, the conversion of Amazon.com could be the number of people that purchase an item divided by the number of visitors for the website. The conversion ratio is the industry standard for websites and most e-business websites are benchmarking themselves by this number.

Now back to Erik. How do you climb Everest when your blind? Let’s look at it step by step. Erik will start somewhere, and he will decide where to put his foot down for his first step. He will decide where to put his foot by taking into account several things: touch, of course: the feeling of the ground beneath him; but also the feeling of the wind blowing in a certain direction. Maybe also hearing: the sound the wind makes or just simply someone telling him where to go. Maybe some other things as well.

Once he decided to put his foot down, he will start measuring: am I falling? No? then I have not stepped in a ravine. Is this foot higher than my previous foot? Then I’m probably closer to the top. Have I gone down, then maybe I’m going in the wrong direction and I have to go back. But it could also be temporary, the next step could go up. To be sure that he reaches the top, Erik has to try all options to put down his foot, and then every option from there, etc. A long and tedious, not to mention dangerous process.

Of course this description is exaggerated. In real life Erik didn’t have to try every route because he had some help in maps and he knew reasonably well where he was going. I have included this description because it describes the challenge of website optimization.

Website optimization is like a blind person that is climbing to the top of a mountain.

If you want to improve the search engine result or the conversion of a website, the same applies: if you make a change in your website you have no sure way of knowing if this change is going to improve these things. There are guidelines from marketing research and usability, but there is always a chance that the effectiveness of your site plummets: even small changes can have a big impact.

The first step in keeping your search engine results and conversion rate at a high level and improve them is to start optimizing your website in a structured way. This means that the effectiveness of your website is closely monitored. Once you make a change, this change is well documented to make sure that you can always go back. Furthermore, the search engine results and the conversion rate of the new and (hopefully) improved website are well monitored. Measuring the behavior of your users is the most accurate measure of the quality of your website. Once the result is measured, you have to decide what to do next: what change to implement next? (This could also include going back to the previous web site). Once you know what the next change is going to be, the whole process starts all over again.

Once you have done this, you have the capacity to decide if one version of our website is better than the other: this will increase the manageability of your website and enable you to do take a first step towards optimizing it. In the next post I will discuss some other techniques to improve website conversion. This will appear later on.

Non-functional don’t exist!

Perhaps the title is a little strange. You might not feel happy with the statement, but non-functionals don’t exist! Of course this isn’t completely true. If we believe Google a search for “non-functional” delivers nearly 2 million hits. The first hit refers to http://en.wikipedia.org/wiki/Non-functional_requirement. Something I noticed was that there are many blogs writing about non-functionals. This could mean several things:

  • There is an ongoing development and the early adopters are working to convince the world.
  • Some people think that non-functionals exists, but they do not exist.

Whatever way you look at it if you read a (functional) design document you will notice the lack of non-functionals. They are hardly mentioned. The design describes what the application needs to do to work and why this is needed, but nothing further… For instance take a web shop. What would we like to do in a web shop. We order product, shop around and before buying logging in. This is a simple view but also true. As a client we order the build of the web shop and expect that it is properly constructed. A testers will use the design document and test is the product meets the specifications. At the end a tester can give his advice about the product. This can be positive when the specifications are met. But even after a positive advice from a tester a customer could still not want to shop. Why? The reason is that some things, like security, usability or performance are not be regulated. What if you have a product in the cart and when you click on it, to see it, it takes 30 seconds to show up? What if you have to click an item 5 times to put that your product into your cart because you have to go through to 4 screens? What if you can order products for free? Good for you but less for the store owner and maybe you are even punishable? A lot of people call these things non-functionals. But is it a non-functional? Do you agree to reverse it? Does a shop function when customers stand in a line for the cashier for 3 hours? If products are for free? If they have to look for creamer at the toothpaste? No! The shop will close if this is for real. So, part of the function of a shop is how the shop operates. Why don’t we view software development not just like that? Let’s think a little bit more about the operation when developing software. As a testers you have a great an fun new role to play. Testers are involved in testing that web shop and look at it from a user perspective! But first look at the design documentation. Are there specifications about security and the loading times of screens? If “yes” then create test cases for them and cover them. But if nothing is mentioned about these kind of things; indicate it to your client! Initially as a recommendation, but perhaps you could get involved at the beginning of a project and give these recommendations at the start. The quality of the web shop will greatly be enhanced because the specifications are already in the documentation. Perhaps ask yourself the following questions during a review of the documentation:

  • What requirements are listed in the field of security, performance and usability?
  • What are the requirements translated into the design documentation?
  • Are the requirements created SMART, for example “screen should appear within 5 seconds” instead of “the screen must be displayed within a reasonable time”?
  • Are the requirements in only 1 way to explain?

A checklist and a real-life test are excellent test design techniques for this. I think that in many organisations non-functionals hardly exist. However we as testers can have a major contribution to make this more and more noticeable. So together we make the software faster, safer and more user friendly!