You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The write path involves combining a bunch of writes into a single batch and placing the batch on a queue to be flushed to the database. Each batch is flushed by a single thread. This has a couple of performance problems:
If the batch size is large, it can take a long time for a single thread to flush a batch, even if other flush threads are idle.
If multiple batches are writing to the same target, then we miss an opportunity to combine those writes into a single write. For use cases where values are written frequently, combining these writes could speed things up by >100%.
The alternative I have in mind is to have a queue of pending writes (single value writes, not batches). But this would be more than a queue; it would also combine all items having the same key. So, as new writes arrive in the queue, they would be combined with existing writes to the same destination.
The text was updated successfully, but these errors were encountered:
The write path involves combining a bunch of writes into a single batch and placing the batch on a queue to be flushed to the database. Each batch is flushed by a single thread. This has a couple of performance problems:
The alternative I have in mind is to have a queue of pending writes (single value writes, not batches). But this would be more than a queue; it would also combine all items having the same key. So, as new writes arrive in the queue, they would be combined with existing writes to the same destination.
The text was updated successfully, but these errors were encountered: