Introduction To Multivariate Testing

In Uncategorized on 1 April 2010 tagged Google website optimizer, multivariance testing, Split Testing with no comments

In split testing, we test one element of your sales page or website. For example, we can test headline A vs headline B (or vs C, D and so on), and we see which is the winner.

Split testing is pretty easy and straightforward to set up and implement.

But there are several drawbacks.

It’s very time consuming and costly. That’s because you can only test 1 element at a time.

Say we want to also test different images, location of image, sub headlines and so on…we can only do so one element at a time – one after another.

Because of this, by the time you are finished with all the elements, it could be weeks or months later!

You waste more time. And money as well as you incur more cost when the duration of the test drags.

There is another important point to consider. With split testing, it doesn’t tell us how each elements affect one another.

You might independently test the headline, and find one that performs really badly – so you don’t use it. Then you test your image, and find one that does really bad – so again, you don’t use it.

But what if the combination of the seemingly bad headline and image can send conversion through the roof? Well…you won’t know! Because you have already rejected the bad headline!

The Solution – Multivariate Testing

As the name suggest, multivariate testing allows you to test multiple elements (e.g. headline, image, offer, guarantee etc) – simultaneously - all in 1 go.

So say you have the following elements to test:

3 versions of headlines

2 versions of images

2 versions of offer

You will have a total of 12 (3 x 2 x 2) combinations of these elements.

When you carry out a multivariate test, traffic will then be split equally to each of these combination.

Sounds like a lot of work? Not to worry.

You can automate the whole process by using Google’s free Google Website Optimizer.

Benefits: