Bot Rules

Bot Rules let you remove traffic that is generated by known spiders and bots from your report suite. Removing bot traffic can provide a more accurate measurement of user activity on your website.

After bot rules are defined, all incoming traffic is compared against the defined rules. Traffic that matches any of these rules is not collected in the report suite and is not included in traffic metrics.

To update or upload bot rules, navigate to Analytics > Admin > Report Suites. Select the correct Report Suite, and then go to Edit Settings > General > Bot Rules.

Removing bot traffic typically reduces the volume of traffic and conversion metrics. Many customers find that removing bot traffic results in increased conversion rates and increases in other usability metrics. Before removing bot traffic, communicate with stakeholders to make sure they can make the necessary adjustments to key performance indicators as a result of this change. If possible, it is recommended to first remove bot traffic from a small report suite to estimate the potential impact.

We recommend defining no more than 500 bot rules per report suite. The user interface allows for 500 rules to be manually defined. After this limit is reached, rules must be managed in bulk through the Import File and Export Bot Rules options.

Bot traffic data is stored in a separate repository for display in the Bots and Bot Pages reports.

Rule Type

Description

IAB

Selecting Include IAB uses the IAB/ABCe International Spiders & Bots List to remove bot traffic. This list is updated monthly by the IAB.

To submit a bot to the IAB list, visit http://www.iab.net/sites/spiders/form.php.

Adobe is unable to provide the detailed IAB bot list to customers, though you can use the Bots Report to view a list of bots that have accessed your site.

Custom Bot Rules

See Create a custom bot rule.

Impact of Bot Rules on Data Collection

Bot Rules are applied to all analytics data. Data removed by Bot Rules is visible only in the Bots and Bot Pages Reports. ASI Slots honor bot rules on historical data if they are enabled at the time of reprocessing.

VISTA rules are applied after Bot Rules (see Processing Order).

High-Hit Visit Processing: If more than 100 hits occur in a visit, reporting determines if the time of the visit in seconds is less than or equal to the number of hits in the visit. In this situation, due to the cost of processing long, intense visits, reporting starts over with a new visit. High-hit visits are typically caused by bot attacks and are not considered normal visitor browsing.

Note: Hits marked as bots are billed as server calls.

Impact of IP Obfuscation on Bot Filtering

The IAB bot list is based solely on user agent, so filtering based on that list is not impacted by IP obfuscation settings. For non-IAB bot filtering (custom rules), IP may be part of the filtering criteria. If filtering bots using IP, bot filtering happens after the last octet has been removed if that setting is enabled, but before the other IP obfuscation options, such as deleting the entire IP or replacing it with some unique ID.?

If IP obfuscation is enabled, IP exclusion happens before the IP address is obfuscated, so customers do not need to change anything when they enable IP obfuscation.

If the last octet is removed, that is done before IP filtering. As such, the last octet is replaced with a 0, and IP exclusion rules should be updated to match IP addresses with a zero on the end. Matching * should match 0.