You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When using the PSC Stream API, it requires either:
A timepoint pointer of the last record seen through the API, which will then return records seen since here
Nothing sent through, which will obtain any new records from now
stream_client.read_stream(timepoint:)
This timepoint is important to ensure that if there is any restart of requests (eg after a timeout or failure), that no records were lost in the meantime.
This needs to be:
Retrieved when starting up, to read latest timepoint
Stored after a new record has been processed
A simple key/value pair is sufficient for storage and retrieval of this, so potentially Redis would be a good choice if that is available. However, note that if this is on a single EC2 instance it is a single point of failure.
Estimate: 6 hours
The text was updated successfully, but these errors were encountered:
When using the PSC Stream API, it requires either:
timepoint
pointer of the last record seen through the API, which will then return records seen since hereThis timepoint is important to ensure that if there is any restart of requests (eg after a timeout or failure), that no records were lost in the meantime.
This needs to be:
A simple key/value pair is sufficient for storage and retrieval of this, so potentially Redis would be a good choice if that is available. However, note that if this is on a single EC2 instance it is a single point of failure.
Estimate: 6 hours
The text was updated successfully, but these errors were encountered: