MultiVariate Testing

Multivariate testing allows you to test different content to see the effect on conversion rates and other key performance indicators.

Multivariate testing also supports A/B testing (also known as split testing), which is a method of testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve conversion rates.  A classic direct mail tactic, this method has been adopted within the online space to test content such as banner ads, emails and landing pages.

Significant improvements in conversion rates (and other KPIs) can be seen through testing elements like copy text, layouts, images and colors. However, not all elements produce the same improvements, and by looking at the results from different tests, it is possible to identify those elements that consistently tend to produce the greatest improvements.

Users of A/B testing method create multiple copies of content, including the control, to see which single variable is most effective in increasing the conversion rate or other desired outcome.  The test, in order to be effective, must reach enough visitors that there is a reasonable chance of detecting a meaningful difference between the control and other content.

A/B testing is different than multivariate testing which allows a test to contain multiple variables within the same content.  However, our system supports both A/B and multivariate testing with the same interface.

Our technology enables multivariate testing in conjunction with dynamic segmentation.  The MultiVariate Report has the standard segmentation filter.

With this version, users are responsible for managing their own content and assigning multivariate keys within the tracking script.

To start tracking, perform the following steps:
1) Create the various landing pages or content you wish to test
2) Place the new tracking script on the page with the content. The tracking script has a new option called 'Include MultiVariate Testing'.  When you check this box on, the tracking script exposes a new variable:

wa_MultivariateKey = ''; // Set this variable to perform multivariate testing

3) While rendering your page content, fill in this variable with whatever you want. The value you use will be attached to the visit. From this, conversion rates and other KPIs can be tracked.
4) For example, to set a baseline page:

wa_MultivariateKey = 'Baseline';

5) There are two automatically generated groupings in this report:

a. --- Overall ---: This represents all activity for the account (if a segmentation filter is set, this includes activity only within that segment).
b. --- Not Assigned ---: This refers to activity that does was not assigned a multivariate key in the tracking script.

The report displays improvement from ‘Overall’.

For websites that do not engage in ecommerce, it is possible to optimize for other values, such as time on site.  Several KPIs are supported to optimize for.  To display the other KPIs on the report, click on the ‘Column Editor’ icon.

For example, to view the results optimized for time on site, select the ‘Minutes per Visit’ and ‘Minutes per Visit Improvement’ columns.  Then sort the report on the ‘Minutes per Visit’ column.

Terms Used on Report:

Visits

Also referred to as a 'Unique Visitor'.  By default, this value represents a visitor that comes to your site for the first time in a day.  All subsequest visits during the day by the visitor are not considered unique.

The definition of a visitor can be customized.  You can set a custom period of inactivity that will cause the next access of the visitor to be considered a new visit.  The visit definition screen can be found on the 'Setup' menu.

This differs from a page view or hit in that a single visitor can have one visit per day but many page views during a visit.

Conversion Rate

The percentage of transactions generated per click-through.