Distributed Restate preview

[tillrohrmann]

This is the umbrella issue for the first distributed version of Restate.

The goal is to support a static 2f+1 node Restate cluster where killing f nodes won't cause downtime for the processing of invocations or loss of data. The cluster won't support membership changes. The metadata store and the cluster controller can run on a separate node with durable storage. Bonus points if the metadata store would be HA as well. It is ok if every node runs all partition processors.

Performance-wise, this version should be in the same ballpark as the single node Restate variant.

Tasks

Preview Give feedback

1. Implement replicated loglet #1805
   1 of 2
   +0
   
2. Implement partition processor scheduler #1799
   6 of 6
   +0
   
3. Bifrost #1830
   0 of 1
   +0
   
4. Only trim logs to min persisted lsn across all known nodes #1781
   +0
   
5. Snapshot partition processor state to object store #1807
   7 of 9
   distributed-architecture snapshots umbrella +3
   
6. Known problems for correctness of distributed Restate #1801
   3 of 3
   +0
7. Miscelleanous follow-ups for distributed Restate #1800
   9 of 13
   +0
8. Update Datafusion data access to work in distributed setup #1806
   +0
   
9. Testing correctness of distributed Restate #1810
   2 of 7
   +0
10. Make cluster controller highly-available #1815
    2 of 7
    +0
    
11. Implement a highly available metadata store #1784
    3 of 6
    +0
    
12. Make Kafka work with distributed architecture #2188
    +0
    
13. Implement location-aware logs configuration #2198
    0 of 3
    +0
14. Update deployment documentation for users
15. Configuration of initial cluster state #2328
    +0