Compare Models in Audience Lab

You can use several different types and sources of models in Audience Manager. Audience Lab offers an easy way to compare your customers' conversion rates, across your active models.

In this use case, you are comparing different models. You can either use models created via an in-house data warehouse and import them in Audience Manager as Onboarded Traits or you can use the Algorithmic Models feature in Audience Manager.

  1. Create two models, either in the Model Builder, or via an outside platform.
  2. Create algorithmic traits from the algorithmic model or import your own models as onboarded traits.
  3. Create mutually exclusive segments so users in both models don't overlap:
    • Create a Model 1 Segment and a Model 2 Segment.
    • Have the segment rule for Model 1 Segment be model 1 trait AND NOT model 2 trait, and vice-versa for Model 2 Segment.
  4. Create two segment test groups in Audience Lab, one with Model 1 Segment as the baseline, the other with Model 2 Segment as the baseline.
    • Keep the variables the same for both test groups: same destinations, creative, conversion traits.
    • Make sure the test segments have similar numbers of users (e.g. 1.6 million and 1.8 million is alright, 1.6 million and 16 million is not).
    • Reserve a control segment in each test segment test group. This way, you can set aside a small part of each segment and not target them explicitly in the test.
  5. Examine the results:
    • The Audience Lab reporting view will show the number of conversions each model is driving. For conversion based campaigns, the test segment that drives the most conversions will signify the model that is performing best.
    • Because you have control segments, you can also evaluate how the model did against "standard targeting." You are not only just testing one model versus the other, but testing the question of "did this model do better than normal practices?"