In the posts about website optimizing, Thomas Veltman, explains how A/B testing can deliver added value to website optimizing. This post is his second.

In part one of this blog we discussed the problem many companies face with their web presence: How do you increase the effectiveness of your website as much as possible? In this second part of “Do not turn a blind eye to website optimization” we describe one technique for improving conversion rates of your website.

The technique is called A/B or multivariate testing.

A/B Testing tests the effectiveness of many different versions of your website against each other.

How does is it work? The main principle is simple. Instead of keeping just one version of your website online, you keep two or even more versions online. One version is the main version; most of the websites’ visitors are guided to this site. A smaller percentage of visitors is guided to another version of the website.

The conversion and other performance indicators are measured for both versions of the website and compared. After that, the version with the best results becomes the main website.

And then the process starts over: another change is implemented in another new version of the website and that version is put alongside the main version the compare.

Dependent on the number of visitors to a website, it’s also possible to implement several changes in several other versions of the website. That way, more versions of the website can be compared. Since we are working with statistical figures here, the group of visitors has to be of a considerable size to have a dependable measurement. This limits the number of website version that one can have online concurrently

This approach is effective for two main reasons:

• * Other (usability) test methods always measure some kind of simulation, weather in a laboratory or elsewhere. This measurement is taken on the real visitors, the real costumers, while they are using the product in real life. Very dependable, and without having to worry about factors that might alter the results of a test.
• * If you make a change in your site and you have only one version, and it has a dramatic effect on the performance of the site, the conversion of the site and with that the turnover of the company can decrease dramatically (in the video which was linked in a response in my last blog a drop of 90% conversion rate was reported because of the introduction of a coupon code). By testing the change in this way, the performance will drop only for a smaller number of visitors and the risk is decreased.

There are several tools to implement such a test. Even free ones, such as Google website optimizer.

In the first post I talked about the challenge of website optimization being similar to that of a blind man climbing a mountain. The A/B testing method helps companies with their website optimization in the same way a mountain climber can be helped by being tied with a rope to someone else. If one falls into a gorge, the other person is there to haul them out of it. It helps companies gradually and structurally improve their websites!

In the posts about website optimizing, Thomas Veltman, explains how the testing community can deliver added value to this process. Since functionality is not the only part of a good application, usability becomes more and more popular and needed. This post is the first one in a series.

The world of websites is a very competitive one. If you type the Dutch word “hypotheek” (meaning mortgage) in google the results are clear. Every big Dutch mortgage bank is trying to beat each other in Sponsored Links and search results. Every major party is present.

The internet is a tank filled with piranhas, typing a search term in google is like dropping a piece of meat in this tank.

Once you follow a link to a bank website you’ll see that here more effort has been put in to attract customers. They are all very fancy looking sites aimed at creating leads for mortgage offers. This is understandable: although the competition is not visible on the site itself, it is always just a mouse click away.

Underneath these seemingly smooth looking websites there is a problem for the sponsors of the websites. How can they make sure that their website will be able to keep up with the competition? How can they keep track of the changing market and wishes of their clients? To show how big this challenge is and to show a beginning of a solution, I have to explain something else first.

On May 25^th 2001, Erik Weihenmayer became the first blind man to climb to the top of Mount Everest. Erik, blind since he was 13, had to overcome quite a challenge. Imagine going up such a mountain without seeing anything: you would have to go on uncertain triggers like touch and hearing to find your way up and the risks of stepping in a canyon or a ravine seem overwhelming. And besides that, how do you know which way you have to go to the top? And once you get there, how do you know that you are there? How do you know that there is not just a little bit going down and than an even higher summit after that?

The challenges that Erik faced are similar to the challenges that the owners of a website in a competitive market face. The sponsors for the websites are interested in being on top: on top of the results page for the google search term in which they are interested  and the top site in website conversion.

Conversion is a measure of the effectiveness of a website. It measures the ratio of the people who come to the website and the people who reach the goals that have been set for the website by the sponsor of the website. For example, the conversion of Amazon.com could be the number of people that purchase an item divided by the number of visitors for the website. The conversion ratio is the industry standard for websites and most e-business websites are benchmarking themselves by this number.

Now back to Erik. How do you climb Everest when your blind? Let’s look at it step by step. Erik will start somewhere, and he will decide where to put his foot down for his first step. He will decide where to put his foot by taking into account several things: touch, of course: the feeling of the ground beneath him; but also the feeling of the wind blowing in a certain direction. Maybe also hearing: the sound the wind makes or just simply someone telling him where to go. Maybe some other things as well.

Once he decided to put his foot down, he will start measuring: am I falling? No? then I have not stepped in a ravine. Is this foot higher than my previous foot? Then I’m probably closer to the top. Have I gone down, then maybe I’m going in the wrong direction and I have to go back. But it could also be temporary, the next step could go up. To be sure that he reaches the top, Erik has to try all options to put down his foot, and then every option from there, etc. A long and tedious, not to mention dangerous process.

Of course this description is exaggerated. In real life Erik didn’t have to try every route because he had some help in maps and he knew reasonably well where he was going. I have included this description because it describes the challenge of website optimization.

Website optimization is like a blind person that is climbing to the top of a mountain.

If you want to improve the search engine result or the conversion of a website, the same applies: if you make a change in your website you have no sure way of knowing if this change is going to improve these things. There are guidelines from marketing research and usability, but there is always a chance that the effectiveness of your site plummets: even small changes can have a big impact.

The first step in keeping your search engine results and conversion rate at a high level and improve them is to start optimizing your website in a structured way. This means that the effectiveness of your website is closely monitored. Once you make a change, this change is well documented to make sure that you can always go back. Furthermore, the search engine results and the conversion rate of the new and (hopefully) improved website are well monitored. Measuring the behavior of your users is the most accurate measure of the quality of your website. Once the result is measured, you have to decide what to do next: what change to implement next? (This could also include going back to the previous web site). Once you know what the next change is going to be, the whole process starts all over again.

Once you have done this, you have the capacity to decide if one version of our website is better than the other: this will increase the manageability of your website and enable you to do take a first step towards optimizing it. In the next post I will discuss some other techniques to improve website conversion. This will appear later on.