Customer reports missing slots

[dancamarg0]

Problem

1. A*** from Sol****** reported some missing slots in his kafka instances (Ex: 168391349, 168391352, 168391375)

1.1 - The slots weren't skipped or forked off - I see them available on explorer for example.

1.2 - I don't see errors in our logs that could indicate what was the problem - But I also am not sure if Kafka plugin has such logs at all

1.3 - At least prometheus dont have logs for connectivity errors / or other kind of errors that can be useful to debug https://github.com/rpcpool/solana-accountsdb-plugin-kafka/blob/v1.13-triton/src/prom.rs#L19-L46

Solution

Build more metrics for debugging? For example connectivity issues, updates dropped or anything relevant.

Does kafka plugin have a buffer? So in the event of an issue in the update flow, it'll hold the update in the buffer and retry until the problem is fixed.