You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have not understood this part:
It includes a Storage Exception Handler plugin that will post messages back to a Kafka queue if writing to storage fails
Failed writes (to HBase or whatnot) will post back to the "TSDB_Requeue" topic in Kafka. We want a separate topic so that the values don't circulate in the main data feed.
Instead we write to another topic, then create another consumer group (usually with fewer threads) that are dedicated to consuming just those requeued messages. So you do need KafkaRpcPlugin.TsdbRequeueConsumer.topics = TSDB_Requeue along with other KafkaRpcPlugin.TsdbRequeueConsumer..... settings.
Understood.
Also the writes of failed messages to re-queue topic would work only if at opentsdb level we set:
tsd.core.storage_exception_handler.enable=true (default is false)
and add
tsd.core.storage_exception_handler.plugin=net.opentsdb.tsd.KafkaStorageExceptionHandler
If these two are not set and even if we set the TsdbRequeueConsumer properties it will still not work. Correct?
I have not understood this part:
It includes a Storage Exception Handler plugin that will post messages back to a Kafka queue if writing to storage fails
The config specifies
So does this mean that second group is created to post the failed messages back to the requeue topic?
Do I also need to specify this:
KafkaRpcPlugin.TsdbRequeueConsumer.topics = TSDB_Requeue
The text was updated successfully, but these errors were encountered: