Fix Coordinator Provisioning Status Errors

[darunrs]

Coordinator seems to continuously fail to set provisioning status while running, for some indexers. It's possible this is due to hard coded provisioning flags in Runner. Needs to be investigated and then fixed.

[darunrs]

The error messages no longer appear although no targetted fix was made. It's possible this was fixed by other changes.

[darunrs]

My guess was it was related to the historical process failing to start due to no such key since a fix to that seems to have resolved the error messages too. In addition, I noticed the messages would begin after the failure occurs.

Can reopen if problem appears again.

[darunrs]

I found the underlying issue which is the likely culprit of why it keeps reappearing. Reopening issue to track the fix for it.

Explanation in linked PR. Will copy here too.

[darunrs]

"Coordinator pulls indexer config from registry upon initialization. While it runs, it continues to check each block for registry update calls. When the registry is updated, it kicks off a historical process if necessary, and updates its local indexer configurations to reflect the new config for that particular indexer.

It turns out that the local indexer config was in fact not correctly updating. It was setting the wrong key for storing the config. As a result, existing indexer updates were not being captured while new indexers would have the wrong config. Coordinator would attempt to set various status such as provisioning but the incorrect keys would lead to errors around that. And, failing to capture indexer config updates leads to runner never seeing those updates and continuing to run stale code. Restarting coordinator resolved these issues since the config is read freshly from registry, hiding these issues for some time, before they would once again reappear.

In addition to the above problem, the configuration is only updated in redis when the next matching block appears. So, if a new indexer is published without a start block, its redis config and stream will not be set until a new matching block appears. This can be problematic for developers who are the producers of blocks matching their contract filter. In that case, matching blocks may be rare, leading to a long delay before updated code is read and executed. Setting the config immediately after an update is seen eliminates such a problem.

I've fixed the particular line which was in correctly setting the key wrong, as well as set the redis config immediately after an update, to resolve the above issues."