npm install pareach
const parEach = require('pareach');
await parEach(work, args[, options]);
work
- an async function that accepts the args providedargs
- an array of arguments to call thework
function (if you pass more than one then use a nested array[['foo', 'bar'], ...]
)options
- a configuration objectconcurrencyLimit
- which is5
by default, controls concurrency per threadparallel
- process work with multiple threads
await parEach(work, args, { concurrencyLimit: 10, parallel: true });
This was will spawn a thread for every logical CPU and split the work.
Sequential took 155.45 seconds
Batched took 16.04 seconds
Parallelized took 3.55 seconds
This was made for the article:
Kicking Node into high gear for data processing or how to hire 100 cats to catch a mouse