You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should try parallelizing single-threaded reads over multiple files. Single-threaded reads which use update_reader would allow for more granular progress bars, and might wind up being more efficient overall.
The text was updated successfully, but these errors were encountered:
I like this idea. Spin up $CORE_COUNT number of threads, and show that many progress bars. Would be cool to see 8 or 16 progress bars whipping by. Since there's already a transitive dependency on Rayon, could experiment with their thread pool rather than rolling something from scratch (using Thread::spawn and mpsc channels)
casey
changed the title
Try parallelizing hashing over multiple file
Try parallelizing hashing over multiple files
Sep 10, 2024
This is apropos of the discussion in BLAKE3-team/BLAKE3#205.
We should try parallelizing single-threaded reads over multiple files. Single-threaded reads which use
update_reader
would allow for more granular progress bars, and might wind up being more efficient overall.The text was updated successfully, but these errors were encountered: