Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple Producers Single Consumer Batching Concurrent Queue #10

Open
epeshk opened this issue Aug 3, 2023 · 0 comments
Open

Multiple Producers Single Consumer Batching Concurrent Queue #10

epeshk opened this issue Aug 3, 2023 · 0 comments

Comments

@epeshk
Copy link
Owner

epeshk commented Aug 3, 2023

Add a new concurrency primitive - MPSC (Multiple Producers Single Consumer) Batching Concurrent Queue. The main purpose of this queue is to optimize logging libraries.

The new queue should be faster (throughput-first, then latency) than Channel<T>, and BlockingCollection<T>.

  • Java Disruptor, used in the log4j2
  • dpdk ring with burst dequeues
  • Infinite array? One fetch-and-add for most enqueues, segments could be recycled in the single consumer scenario.
  • Linked list queue (non batching).
  • ConcurrentBoundedQueue. A simple implementation that must be compared to a complex one (to argue that a complex implementation is needed at all)
  • HellBrick.AsyncCollections - implementation for comparison, and becnhmarks
  • DataflowChannel - unordered, for comparison
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant