DKV write fast-acknowledge mode

[kingster]

By default, a write involves persisting the data on a majority of replicas’ Paxos logs and writing the data to RocksDB on the primary before acknowledging the write to the client. With the default write mode, a read on primary will always see the most recent write. Some applications cannot tolerate cross-region latencies for every write, so ZippyDB supports a fast-acknowledge mode, where writes are acknowledged as soon as they are enqueued on the primary for replication. The durability and consistency guarantees for this mode are obviously lower, which is the trade-off for higher performance.

From https://engineering.fb.com/2021/08/06/core-data/zippydb/

IMO we should have something similar for DKV. This would help for latency-sensitive writes with the tradeoff of possible data loss in the case of a leader crash