Status Updates for Metadata Files

The S3 status directory holds a .info file with success and failure information about your uploaded files. The file contains JSON-formatted data with status results in an array.

The contents of your .info file will look similar to this example.

//sample file path
/log_ingestion/pid=1234/dpid=567/status/20150827.info

//sample contents
{
    "Files": [
        {
            "FileByteSize": 488900,
            "FileChecksumMD5": "94b821082daff242e452c0d8796b08f0",
            "FileName": "20141112_4_2",
            "MetadataType": "Creative",
            "Parent": "Site",
            "Status": "SUCCESS",
            "Description": ""
        },
        {
            "FileByteSize": 58812,
            "FileChecksumMD5": "db79f148e6a635629701c13a7bcc8db0",
            "FileName": "20141112_0_4",
            "MetadataType": "Site",
            "Parent": "None",
            "Status": "FAILURE",
            "Description": "Invalid format."
        }
    ],
    "Summary": {
        "Day": "2014-11-12",
        "ProcessingTimeRFC2822": "Wed, 10 Dec 2014 21:07:58 -0000",
        "ProcessingTimestampPOSIX": 1418263678,
        "TotalByteSize": 547712,
        "TotalNumberFiles": 2,
        "NumberSuccess": 1,
        "NumberFailure": 1,
        "GlobalStatus": "FAILURE"
    }
}

Metadata Key-Value Pairs Defined

The following tables list and define the keys in the Files and Summary sections of a metadata status file.

Keys in the Files Array

Key Description

Description

Contains a brief description of why processing failed. This field is empty when processing is successful.

FileByteSize

File size in bytes.

FileChecksumMD5

The MD 5 checksum for the metadata file uploaded to your meta directory.

FileName

The name of the metadata file uploaded to your meta directory.

MetadataType

The human-readable name for the type of data your file contains. It is based on the child ID in your file name.

See Naming Conventions for Metadata Files.

Parent

The human-readable name for the type of data your file contains. It is based on the parent ID in your file name.

See Naming Conventions for Metadata Files.

Status

Returns 2 text values that describe the processing status of your metadata file:

  • SUCCESS
  • FAILURE

Keys in the Summary Object

Key Description

Day

File processing date in yyyy-mm-dd format.

GlobalStatus

Returns 2 text values that describe the processing status for all your files for an entire day:

  • SUCCESS
  • FAILURE

NumberFailure

The number of files that were processed unsuccessfully.

NumberSuccess

The number of files processed successfully.

ProcessingTimeRFC2822

Returns a human-readable time stamp for processing start times.

ProcessingTimePOSIX

A UNIX time stamp for processing start times.

TotalByteSize

Total number of bytes for all your metadata files for the day.

TotalNumberFiles

Total number of all your files processed for the day.