Marissa Mayer, vice president of search products and user experience at Google while speaking at Google I/O conference explained how they improve search results by using A/B split testing.
Using split A/B testing Google shows different versions of the pages to users and measures how they respond. Multivariate or split A/B testing is a statistical term which means testing hypotheses on complex multi-variable systems, especially used in testing market perceptions.
It is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous split tests or A/B tests performed on one page at the same time. Split tests and A/B tests are usually performed to determine the better of two content variations, multivariate testing can theoretically test the effectiveness of limitless combinations.
One of the example Mayer gave was when company wanted to find out how many search results to show users–the customary 10, or 20, 25, or 30?
For this they showed two different search results, one with 10 results and other with 30 results to different users, which revealed that when the results increased to 30 per page, people searched 20 percent less overall. After analysis of server logs, the company found it was because it took about twice as long to display the longer results list for the user, and speed matters.
It should be interesting to note that before the start of this test when Google asked users directly as to how many search results they want on page, most of them replied they want more results but the test proves it otherwise.
According to Mayer:
As Google gets faster, people search more, and as it gets slower, people search less.
Split A/B testing also led Google to refine exactly how much white space to pad around its logo and other elements on the search results page. And it changed from the industry practice of a pale blue background behind ads to a pale yellow background. People not only clicked on ads more, they also searched more in general, she said.
She also gave the example of Google Maps when they trimmed the 120KB page size down by about 30 percent and it started getting about 30 percent more map requests.
It was almost proportional. If you make a product faster, you get that back in terms of increased usage.
she said.
0 responses so far
There are no comments yet...Kick things off by filling out the form below.
You must log in to post a comment.