Data Feed Service History

Cumulative release notes for Clickstream data feeds.

January 19, 2017 - New User Interface and Service Name

There is now a user interface for data feeds and job management. Until now, Data Feed administrators had to rely on Adobe Customer Care to get a status update on their Data Feed jobs. This is no longer the case. The Data Feed user interface is now available to all Adobe Analytics administrators.

With this release, the name of the service has changed from Clickstream Data Feeds to Analytics Data Feed.

Clickstream Data Feeds History

The following sections contain legacy history information from the Clickstream Data Feeds service.

Data Feed Column Expansion

The following data feed column sizes will increase in a future release:

  • geo_zip field - column size will increase from 16 to 50 bytes
  • geo_region field - column size will increase from 32 to 255 bytes
  • geo_city field - column size will increase from 32 to 255 bytes

New OS Types Lookup File

Clickstream data feed customers that are currently receiving the os column will automatically receive an additional lookup file for OS types starting June 19, 2014.

Mobile Carrier Column Now Available

Mobile carrier data is now available in clickstream data feeds (this data is currently displayed on the Visitor Profile > Technology > Mobile Carrier report in marketing reports & analytics). Contact customer care to have this column added.

Data Feed Column Expansion

Adobe is currently evaluating an increase to the size of many data feed fields. Before these field sizes are increased, you need to expand the data fields in your ETL system to accommodate these new sizes. You also need to expand the fields used to store lookup file keys, as these are increasing in size as well. Note that the column expansions that were previously scheduled for April 2014 have been postponed, and an announcement will be included in the release notes when these expansions are rescheduled.

Note: You can download a sample data feed with expanded columns and lookup files to test your ETL process. We recommend all customers test processing of this sample feed to prepare for the upcoming expansion.

Lookup file key expansion

The column used to store each lookup file key value should be expanded to store a 32-bit unsigned integer.

Note that additional lookup files could be added to the data feed when additional columns added, If possible your ETL system should be designed to accommodate the addition of new lookup files without impacting processing.

String expansion

Expand the following columns to store a 255 character string:

ip2 click_tag click_action
page_event_var2 post_page_event_var2 pagename
post_pagename post_pagename_no_url channel
post_channel user_server t_time_info
post_t_time_info domain first_hit_pagename
visit_start_pagename transactionid post_transactionid
prop1-75 post_prop1-75 ref_domain
code_ver geo_zip plugins
accept_language c_color ct_connect_type
ip page_type post_ page_type
purchaseid post_purchaseid s_resolution
ua_color ua_pixels visit_keywords
post_keywords stats_server geo_city
geo_region j_jscript state
post_state zip post_zip
namespace currency post_currency
ua_os

Integer expansion

Expand the following columns to store a 32-bit unsigned integer:

browser_height browser_width search_engine
post_browser_height post_browser_width browser (Feb 20, 2014)
color country language
os (Feb 20, 2014) resolution visit_search_engine
geo_dma post_search_engine search_page_num

October 17, 2013

Social, Mobile, and Video solution variables are now available

You can now receive solution variables in data feeds, which currently includes Social, Mobile, and Video metrics.

To update your data feed definition to add solution variables, contact Customer Care.

Event lookup file now included in data feed download

Starting October 17, 2013, an additional event lookup file will be added to all data feeds. Your ETL might require a modification to consider this file.

This new lookup file provides a map for all events, including shopping cart events, instance events, custom events, mobile-specific events, social-specific events, and so on. Be aware that the map for Mobile, Social, and Video solution event numbers are different for each report suite. The event lookup file delivered with each report suite should be considered specific to that report suite.

May 23, 2013

Hourly Data Feeds

Hourly data feeds provide customers raw Analytics data in hourly increments instead of the normal daily increments. By splitting the data into hourly increments, the data can be delivered in a more timely manner with less peak load on both origination and destination servers. See Feed Information.

February 14, 2013

Format of post_product_list column when events are present but product is empty

In post_product_list, If events are contained on the event list for a given hit but there is no product in the product list, the post_product_list contains four semicolons:

;;;;

or in rare cases as semicolon, semicolon, zero, semicolon, zero, semicolon:

;;0;0;

If you are performing a check for an empty product list you should update your ETL to handle both formats.

January 22, 2013

Expansion of page_url and page_event_var1 Columns

These columns were expanded to store URLs that are longer that 255 characters. This change enables you to receive the entire URL of each page and tracked link, which might contain long query strings or other values that were previously truncated at 255 characters.

After this change, the page_url and page_event_var1 length will increase from 255 bytes to 64 kilobytes. Update any applications or scripts that you are using to process the data feed to use a 64Kb variable to store these strings. If you are storing the data feed in a database, make sure that the page_url and page_event_var1 columns are large enough to contain a string up to 64Kb. Alternatively, if your systems require the truncated versions of these columns, please make certain your ETL handles truncation appropriately.

This change does not impact the post_page_url or post_page_event_var1 columns, which remain at 255 characters.

June 21, 2012

  • You may now opt to receive clickstream data feed data as a once-daily package of multiple files rather than a single large monolithic file. The multiple file package is particularly useful for report suites with very high traffic volumes and has several advantages:
    • Improved delivery consistency, especially during days with traffic spikes.
    • Better parallel data processing. Using the multiple file package can significantly reduce the time required to compress and extract large data sets.
  • The following additional optional columns are now available for inclusion in your data feed. In order to maintain compatibility with your current ETL processes, existing data feeds will not be altered unless requested to Customer Care through your Supported User.
    • post_event_list

      post_ page_event

      post_page_event_var1

      post_page_event_var2

      post_ page_type

      post_page_url

      post_pagename

      post_pagename_no_url

      post_product_list

      post_channel

      post_mvvar1 – post_mvvar3

      post_purchaseid

      post_referrer

      post_state

      post_hier1-post_hier5

      post_currency

      post_partner_plugins

      visit_keywords

      post_transactionid

      post_page_event_var3

      va_instance_event

      va_new_engagement

      post_prop1-post_prop75

      tnt_post_vista