Create an A/B Test

An A/B test compares two or more versions of your Web site content to see which best lifts your conversions, sales, registrations, or any other success metric.

A/B tests are well-suited for large changes that might involve new layouts or drastically different treatments of the elements. If your test design does not easily break down into individual page elements, you should run an A/B test before a multivariate test. Also, if you anticipate large interactions between elements, an A/B test is the best choice. Like any other campaign A/B campaigns can span multiple pages, can be used to reinforce messaging or content throughout your site, and are often used to target different content to different types of visitors.

Note: For detailed information about determining the optimum sample size for an A/B test, see Sample Size Determination for A/B Testing in Target Overview and Reference.

Before setting up the test, complete the procedures in Set Up Your Site.

  1. Select Campaigns > Create > A/B…N.
  2. Name your campaign.

    Name the campaign something descriptive that you will recognize months later.

    Example: Home Page AB Test November 2013

  3. Set the start and end dates, including time of day.

    The time zone is determined when you create your account. To change your time zone, ask your Account representative.

    By default, start and end dates are set to When Approved and When Deactivated.

  4. Set the campaign priority.
  5. Target the campaign to certain visitors.

    By default, campaigns allow all visitors to view them. To target your campaign, set up Target a Campaign or Test.

  6. Select the location where content in your campaign will appear by choosing your mbox.

    Your campaign can deliver content to many locations. Click Add Location to add mboxes.

  7. In Experience A, select content for each location by clicking the change offer from the actions menu next to the offer name.

    Note that Default Content is assigned to each location initially. After you click select/create offer, you can select offers that you've created from the offer selection pop-up. Alternatively, you can create a new offer by writing or pasting offer content and naming the offer.

    An effective technique for creating new offers while setting up a campaign is to edit an existing offer. Click the Edit link in the offer selection pop-up, make changes to the offer content, give the offer a new name, and click Save. Note that you will only be able to edit HTML offers from the offer select pop-up. Redirect offers must be edited from the main offer edit page. Refer to Offers for more information.

  8. Click Add Experience to add more experiences or alternatives to your campaign.

    Select content for each location in each experience.

    Note: When there is no targeting on your experiences, each experience is randomly shown an equal number of times over time. Initially, however, the distribution of traffic is slightly uneven. For example, in an A/B test with four experiences, each experience is shown to 25% of visitors. Note too that you may override the random equalization by adding targeting to each experience. Please refer to Target a Campaign or Test for more information.
  9. Choose your conversion or other success metric by assigning an mbox to your conversion success metric, which appears by default.

    You might rename the conversion metric, but the first/default metric always appears at the bottom of the list of success metrics because it is often the final and primary event in your conversion funnel. Refer to Success Metrics for more information.

  10. (Conditional) If you have no conversion activity in mind, select display mboxes as the conversion activity.

    Anyone viewing any display mbox is counted as a converted visitor. You will also be able to see impressions and visits to all display mboxes. This is especially useful for display ad tracking.

    Select click from display mbox to count all clicks to display mboxes.

    Note: A conversion is only counted if the click leads to a page that has an mbox.js and is in the same domain. A click to another page without an mbox.js or in a different domain is only counted if the visitor returns to a page that has an mbox.js and is in the original domain.
  11. Add segment filtering orTarget a Campaign or Test as needed for your testing strategy.
  12. If this campaign shares mboxes with other approved campaigns, set campaign priority.
  13. Click Save.
  14. Complete quality assurance.