In the posts about website optimizing, Thomas Veltman, explains how the testing community can deliver added value to this process. Since functionality is not the only part of a good application, usability becomes more and more popular and needed. This post is the first one in a series.

The world of websites is a very competitive one. If you type the Dutch word “hypotheek” (meaning mortgage) in google the results are clear. Every big Dutch mortgage bank is trying to beat each other in Sponsored Links and search results. Every major party is present.

The internet is a tank filled with piranhas, typing a search term in google is like dropping a piece of meat in this tank.

Once you follow a link to a bank website you’ll see that here more effort has been put in to attract customers. They are all very fancy looking sites aimed at creating leads for mortgage offers. This is understandable: although the competition is not visible on the site itself, it is always just a mouse click away.

Underneath these seemingly smooth looking websites there is a problem for the sponsors of the websites. How can they make sure that their website will be able to keep up with the competition? How can they keep track of the changing market and wishes of their clients? To show how big this challenge is and to show a beginning of a solution, I have to explain something else first.

On May 25^th 2001, Erik Weihenmayer became the first blind man to climb to the top of Mount Everest. Erik, blind since he was 13, had to overcome quite a challenge. Imagine going up such a mountain without seeing anything: you would have to go on uncertain triggers like touch and hearing to find your way up and the risks of stepping in a canyon or a ravine seem overwhelming. And besides that, how do you know which way you have to go to the top? And once you get there, how do you know that you are there? How do you know that there is not just a little bit going down and than an even higher summit after that?

The challenges that Erik faced are similar to the challenges that the owners of a website in a competitive market face. The sponsors for the websites are interested in being on top: on top of the results page for the google search term in which they are interested  and the top site in website conversion.

Conversion is a measure of the effectiveness of a website. It measures the ratio of the people who come to the website and the people who reach the goals that have been set for the website by the sponsor of the website. For example, the conversion of Amazon.com could be the number of people that purchase an item divided by the number of visitors for the website. The conversion ratio is the industry standard for websites and most e-business websites are benchmarking themselves by this number.

Now back to Erik. How do you climb Everest when your blind? Let’s look at it step by step. Erik will start somewhere, and he will decide where to put his foot down for his first step. He will decide where to put his foot by taking into account several things: touch, of course: the feeling of the ground beneath him; but also the feeling of the wind blowing in a certain direction. Maybe also hearing: the sound the wind makes or just simply someone telling him where to go. Maybe some other things as well.

Once he decided to put his foot down, he will start measuring: am I falling? No? then I have not stepped in a ravine. Is this foot higher than my previous foot? Then I’m probably closer to the top. Have I gone down, then maybe I’m going in the wrong direction and I have to go back. But it could also be temporary, the next step could go up. To be sure that he reaches the top, Erik has to try all options to put down his foot, and then every option from there, etc. A long and tedious, not to mention dangerous process.

Of course this description is exaggerated. In real life Erik didn’t have to try every route because he had some help in maps and he knew reasonably well where he was going. I have included this description because it describes the challenge of website optimization.

Website optimization is like a blind person that is climbing to the top of a mountain.

If you want to improve the search engine result or the conversion of a website, the same applies: if you make a change in your website you have no sure way of knowing if this change is going to improve these things. There are guidelines from marketing research and usability, but there is always a chance that the effectiveness of your site plummets: even small changes can have a big impact.

The first step in keeping your search engine results and conversion rate at a high level and improve them is to start optimizing your website in a structured way. This means that the effectiveness of your website is closely monitored. Once you make a change, this change is well documented to make sure that you can always go back. Furthermore, the search engine results and the conversion rate of the new and (hopefully) improved website are well monitored. Measuring the behavior of your users is the most accurate measure of the quality of your website. Once the result is measured, you have to decide what to do next: what change to implement next? (This could also include going back to the previous web site). Once you know what the next change is going to be, the whole process starts all over again.

Once you have done this, you have the capacity to decide if one version of our website is better than the other: this will increase the manageability of your website and enable you to do take a first step towards optimizing it. In the next post I will discuss some other techniques to improve website conversion. This will appear later on.

1. Justin Hunter Says:
   August 18th, 2009 at 20:34

   Good post.

   There is a phenomenally good 22-minute talk on related topics here: http://bit.ly/ZbL0k

   I firmly believe that applied statistics-based experiments are under-appreciated by businesses (and, for that matter, business schools). Few people who understand them are as articulate and concise as Kohavi, the presenter in this video.

   Kohavi’s presentation provides several excellent examples that underscore the point that data trumps intuition, particularly when innovative ideas are under consideration. The presentation covers using Design of Experiments optimization methods at Amazon, Microsoft, etc. The examples show how the optimal combination of web page features, colors, options, prices, etc. are often counter-intuitive.

   - Justin Hunter

2. Thomas Veltman Says:
   August 21st, 2009 at 20:14

   Justin,

   Thanks for the hint about the presentation. It has some excellent examples of how blind we really are when we are optimizing a website. A 90% drop in revenue by just introducing a coupon code.

   I hope that people will watch it after I have posted part two, because Kohavi talks about some things about which I was going to blog as well