Event Ingestion

Foxtrot supports both single event ingestion and bulk event ingestion. Please go through the event structure to understand what to send.

##Single event ingestion API The single event ingestion API will consume single event encoded in JSON format. The Content-type header must be set to application/json for the event to successfully propagate to the system. The API will return only after successfully saving the document to both the query store and the data store.

Endpoint: /foxtrot/v1/document/{table}

HTTP Method: POST

Returns: HTTP 201/Created on having successfully saved the document in both stores.

###Request body

{
    "id": "569b3782-255b-48d7-a53f-7897d605db0b",
    "timestamp": 1401650819000,
    "data": {
        "header": {
            "eventType": "APP_LOAD",
            "os": "android",
            "device": "XperiaZ"
        },
        "data": {
            "appVersion": {
                "major": "1",
                "minor": "01"
            },
            "deviceTime": 1401650817000
        }
    }
}

##Bulk event ingestion API Although single event ingestion is good for small and mostly test use-cases, a more real life use-case would involve persisting events to your own queue and then batching and pushing them to foxtrot. The bulk ingestion API will be used to do this. We have benchmarked this API to be able to take more than a billion events a day on only 3 ES boxes and a kerberos authenticated HBase cluster. Please run benchmarks to understand the optimal batch size. For our tests we used a batch size of 1500. Like the single ingestion API, the Content-type header must be set to application/json for the events to successfully propagate in the system. This API returns only when it has saved all events to both the stores.

Endpoint: /foxtrot/v1/document/{table}/bulk

HTTP Method: POST

Returns: HTTP 201/Created on having successfully saved the document in both stores.

###Request body

[
    {
        "id": "569b3782-255b-48d7-a53f-7897d605db0b",
        "timestamp": 1401650819000,
        "data": {
            "event": "APP_LOAD",
            "os": "android",
            "device": "XperiaZ"
        }
    },
    {
        "id": "569b3782-255b-48d7-a53f-7897d605ab99",
        "timestamp": 1401650829000,
        "data": {
            "event": "APP_LOAD",
            "os": "android",
            "device": "SamsungGalaxy5"
        }
    }
]