Test Mutually Exclusive Campaigns

You can use profile attributes to set up tests that compare two or more campaigns but do not let the same visitors participate in each campaign.

Testing mutually exclusive campaigns prevents a visitor in one campaign from affecting the test results for the other campaigns. When a visitor participates in multiple campaigns, it can be difficult to determine whether positive or negative lift resulted from the visitor's experience with one campaign, or if interactions between multiple campaigns affected the results of one or more of the campaigns.

For example, you can test two areas of your ecommerce system. You might want to test making your Add to Cart button red instead of blue. You might also test a new checkout process that reduces the number of steps from five to two. If both campaigns have the same success event (a completed purchase), it can be hard to determine whether the red button improves conversions, or whether those same conversions were also increased because of the improved checkout process. By separating the tests into mutually exclusive campaigns, you can independently test each change.

Setting Up Two Campaigns

To sort visitors into groups that each see a different campaign, you must create a profile attribute. A profile attribute can sort a visitor into one of two or more groups. To set up a profile attribute called "twogroups," create the following script:

if (!user.get('twogroups')) {

var ran_number=Math.floor(Math.random() *99);

if (ran_number <= 49) {

return 'GroupA';

} else {

return 'GroupB';

}

}

if (!user.get('twogroups')) determines whether the twogroups profile attribute is set for the current visitor. If they do, no further action is required.

var ran_number=Math.floor(Math.random() *99) declares a new variable called ran_number, sets its value to a random decimal between 0 and 1, then multiplies it by 99 and rounds it down to create a range of 100 (0-99), useful for specifying a percentage of visitors who see the campaign.

if (ran_number <= 49) begins a routine that determines which group the visitor belongs to. If the number returned is 0-49, the visitor is assigned to GroupA. If the number is 50-99, the visitor is assigned to GroupB. The group determines which campaign the visitor sees.

After you create the profile attribute, set up the first campaign to target the desired population by requiring that the user profile parameter user.twogroups match the value specified for GroupA.

Note: Choose an mbox early on the page. This code determines whether a visitor experiences the campaign. As long as an mbox is encountered first by the browser, it can be used to set this value.

Set up the second campaign so the user profile parameter user.twogroups matches the value specified for GroupB.

Setting Up Three or More Campaigns

Setting up three or more mutually exclusive campaigns is similar to setting up two, but you must change the profile attribute JavaScript to create a separate group for each campaign and determine who sees each one. The random number generation is different, depending on whether you create an odd or even number of groups.

For example, to create four groups, use the following JavaScript:

if (!user.get('fourgroups')) {

var ran_number=Math.floor​(Math.random()*99);

if(ran_number <= 24) {

return 'GroupA';

} else if(ran_number <= 49) {

return 'GroupB';

} else if(ran_number <= 74) {

return 'GroupC';

} else {

return 'GroupD';

}

} 

In this example, the math used to generate the random number that assigns a visitor to a group is the same as it is with only two groups. A random decimal is generated, then rounded down to create an integer.

If you create an odd number of groups, or any number that 100 does not divide evenly into, you should not round the decimal down to an integer. Not rounding the decimal enables you to specify non-integer ranges. You do this by changing this line:

var ran_number=Math.floor(Math.random()*99);

to:

var ran_number=Math.random()*99;

For example, to place visitors in three equal groups, use the following code:

if (!user.get('threegroups')) {

var ran_number=Math.random()*99;

if(ran_number <= 32.33) {

return 'GroupA';

} else if(ran_number <= 65.66) {

return 'GroupB';

} else { return 'GroupC';

}

}