[RFC] Clickstream Health for Observation V2

[radityagumay]

Preface

This document aims to cover the next steps in Clickstream health observation. This document would revisit the current health events being tracked and propose changes along with the new ones that need to be added. Also, propose the platform with which events would be tracked with.

Prior Art

The Clickstream health events currently are being tracked via both Clickstream and CleverTap. The events are broken two kinds the first being error events and the rest others. The error events are tracked via CleverTap and others via Clickstream. Following are the health events being tracked for Clickstream’s health.

Event Name	Tracked Via	Purpose
“ClickStream Failed Init”	CleverTap	To track instances of the clickstream failing to initialise
“Clickstream Event Received”	Clickstream	Tracks the instances where the event is received by the Clickstream library
“Clickstream Event Object Created”	Clickstream	Tracks the instances when the clickstream event object is created within the library.
“Clickstream Event Cached”	Clickstream	Tracks the instances when the clickstream event object is cached.
“Clickstream Event Batch Trigger Failed”	CleverTap	Tracks the instances when the batch fails to get triggered.
“Clickstream Event Batch Created”	Clickstream	Tracks the instances when the clickstream event batch is created.
“ClickStream Write to Socket Failed”	CleverTap	Tracks the instances when the clickstream event batch fails to get written on the socket.
“ClickStream Connection Failed”	CleverTap	Tracks the instances when the clickstream fails to connect to the socket.
“ClickStream Batch Sent”	Clickstream	Tracks the instances when the clickstream batch gets successfully sent to raccoon.
“Clickstream Event Batch Success Ack”	Clickstream	Tracks the instances when raccoon acks the event request
“Clickstream Event Batch Error response”	CleverTap	Tracks the instances when the clickstream request results in a error response
“Clickstream Event Batch Timeout”	CleverTap	Tracks the instances when the clickstream event batch gets timed out
“ClickStream Flush On Background”	Clickstream	Tracks the instances when the clickstream batches are flushed on background
“Clickstream Event Object Creation Failed”	CleverTap	Tracks the instances when the clickstream event object creation failed.

Traces

No changes to the traces being sent to firebase.

Proto Structure

message HealthDetails {
  // Array of event guids.
  repeated string event_guids = 1;
  // Array of event batch guids.
  repeated string event_batch_guids = 2;
}
message ErrorDetails {

string reason = 1;

}
message TraceDetails {

string time_to_connection = 1;

ErrorDetails error_details = 2;

}
message Health {
// Name of the health event.

string event_name = 1;

// Health details, captured by the SDK when the mode is verbose.

HealthDetails healthDetails = 2;

// Number of events tracked.

int64 number_of_events = 3;

// Number of event batches tracked.

int64 number_of_batches = 4;

// Tracks the error details for the event

ErrorDetails error_details = 5;

// Tracks the traces

TraceDetails trace_details = 6;
// Note: Auto-filled by the ClickStream SDK, need not be set by the products for every event! If set, will be overridden.

google.protobuf.Timestamp event_timestamp = 101;

// Note: Auto-filled by the ClickStream SDK, need not be set by the products for every event! If set, will be overridden.

// Deprecated, please use healthEventMeta.

gojek.clickstream.common.EventMeta meta = 102 [deprecated = true];

// Tracks Clickstream health meta.

gojek.clickstream.internal.HealthMeta healthMeta = 103;

// Note: Auto-filled by the ClickStream SDK, need not be set by the products for every event! If set, will be overridden.

google.protobuf.Timestamp device_timestamp = 104;

}

Health.proto

message HealthMeta {

message App {

string version = 1;

}
message Customer {

string signed_up_country = 1;

string current_country = 2;

int32 identity = 3;

string email = 4;

}
message Device {

string operating_system = 1;

string operating_system_version = 2;

string device_make = 3;

string device_model = 4;

}
message Location {

double latitude = 1;

double longitude = 2;

}
message Session {

string session_id = 1;

}
enum NetworkType {

NETWORK_TYPE_UNSPECIFIED = 0;

NETWORK_TYPE_NO_CONNECTION = 1;

NETWORK_TYPE_WIFI = 2;

NETWORK_TYPE_WWAN2G = 3;

NETWORK_TYPE_WWAN3G = 4;

NETWORK_TYPE_WWAN4G = 5;

}
message Network {

NetworkType type = 1;

}
string event_guid = 1;
Location location = 4;

Customer customer = 5;

Device device = 6;

Session session = 7;

App app = 8;

Network network = 9;

}

HealthMeta.proto

All events have a Quality of Service as 1.

Tracking the events

There can be two ways to make track the events:

1. Clickstream generates a stream of events which the client listens to and triggers the health event tracking

2. Clickstream tracks the health events internally as it currently does

3. Hybrid approach where Clickstream tracks events internally and also generates the health events be tracked by the client

Clickstream generates a stream of events which the client listens to sends the protos back

In this process, clickstream on being provided by the dependency for the health tracker will start generating the health events. The health events can be generated in a dictionary form or as a struct. This can be listened at the clients end and be forwarded to any place.

Pros:

1. Makes health tracking generic

2. Can be tracked with any provider

3. Can be added to the open source repo as such

4. No need for health meta, as the meta can be enriched on the client side

5. Easily expandable to cover custom tracker as in CleverTap, Firebase, etc.

Cons:

1. Adds additional task to client

2. Needs to be replicated for all clients

3. The implementation would undergo changes and hence revalidation

Clickstream tracks the health events internally as it currently does

Makes use of the current process of health event tracking. Does not change how the internal events are tracked.

Pros:

1. No changes to the implementation

2. Health tracking stays internal

Cons:

1. Cannot be open sourced as such an option to track with alternate option needs to be provided

2. Too rigid

3. HealthMeta duplication

Clickstream tracks the health events internally as it currently does and exposes health events too

Makes use of the current process of health event tracking. Does not change how the internal events are tracked but introduces another approach where the health events will be streamed for the client to subscribe to and track via whatever medium they wish to track.

enum TrackedVia {

case internal // Tracked internally via clickstream. Use health proto for it.

case external // Tracked via client. Generates health events in the ClickstreamHealthEvent Contract.

}

struct ClickstreamHealthEvent {

// health props

}

Pros:

1. No changes to the implementation

2. Health tracking stays internal

3. Get best of both worlds

Cons:

1. Internal tracking remains the part of the code even if the client does not want it (can be controlled via compiler flags)

2. HealthMeta duplication

Decision

Chosen: Clickstream tracks the health events internally as it currently does and exposes health events too