Consuming messages from the past (Event Hub stream) #8341
Unanswered
petr-kostelansky
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi,
How start consuming messages from the past on a GrainService from the persistent stream (Event Hub stream)?
How should I get the SequenceToken if I start a new silo (for example if I scale my app in k8s and add more pods=silos)?
I tried to subscribe with: new EventHubSequenceTokenV2("0", 0, 0); hoped I will get data from the past based on Checkpointer or at least the oldest data from IQueueCache (in this case EventHubQueueCache), but this will throw Orleans.Streams.QueueCacheMissException: Item not found in cache... Because I used older SequenceToken than PurgedToken (logic in PooledQueueCache.SetCursor(..,..) line 215.
How else should I start receiving events from the oldest messages in the QueueCache?
Where should I get relevant SequenceToken pointing at the oldest message in the QueueCache?
Why I need this:
I need the GrainService that will subscribe to the stream on each silo. I need to start receiving messages from the past.
The GrainService keeps some data that will be accessed during the lifetime.
The GrainService will load data from DB on Start() and also subscribe to the stream, but there is a gap between loading from DB and starting a subscription and that is why I need to start receiving data from the past (to synchronize and keep loaded data up to date).
Orleans version: 7.0.0.0
Beta Was this translation helpful? Give feedback.
All reactions