Unoptimized implementation of pMapIterable preserveOrder option
#79
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…fer and `promiseEmitter` - move `promise` inner `try-catch` logic to `promise.catch` (now also catches when `iterator.next()` throws) - when `returnValue === pMapSkip`, `popRandomPromise` unconditionally (skip `popNextPromise` in main `while` loop instead) - add tests for `preserveOrder: false`, sync throwing mappers/iterables, more intricate `pMapSkip` scenarios
Richienb
suggested changes
Sep 1, 2024
| } | ||
|
|
||
| trySpawn(); | ||
|
|
||
| while (promises.length > 0) { | ||
| const {error, done, value} = await promises[0]; // eslint-disable-line no-await-in-loop | ||
| const {promise, result: {error, done, value}} = await nextPromise(); // eslint-disable-line no-await-in-loop |
|
|
||
| promises.push(promise); | ||
| promise.then(p => { |
There was a problem hiding this comment.
Use try/catch/async/await here. Also, would be better to use a deferred promise and we call it after anything resolves, instead of racing? https://github.com/sindresorhus/p-defer
Richienb
reviewed
Sep 1, 2024
| // This occurs when `concurrency > 1`: the first `promise` will `trySpawn` and `promises.push` another promise before `await`ing the mapper, | ||
| // but the ongoing `Promise.race(promises)` call from `nextPromise` is oblivious to this new promise as it was not present in `promises` | ||
| // when the race began. | ||
| new Promise(resolve => { |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #72.
Spun off from #74 in light of
pMapIterablepreserveOrderoption #74 (comment)pMapIterablepreserveOrderoption #74 (comment)Summary
Adds a
preserveOrderoption topMapIterable, which indicatesImplementation considerations
promisemust return reference to itself sopopRandomPromisecan findindexOf.preserveOrder: false, racing thepromisespool is not enough, sincetrySpawnmay be add additional promises to the buffer after the race has begun: so we race an event listener alongside (promisespool is still needed to buffer multiple promises finishing simultaneously.returnValue === pMapSkip && preserveOrder: false, unlike whenpreserveOrder: falsewe have no way to know if returning will cause ourpromiseto win an ongoing race, and so have no way of knowing whether to skip popping ourselves (to allow the main loop to pop us instead). So, whenreturnValue === pMapSkip, we insteadpopRandomPromiseourselves unconditionally and instead skippopNextPromisein mainwhileloop.promiseinnertry-catchlogic topromise.catch(which now also catches wheniterator.next()throws)preserveOrder: false && result.done, some promises in the pool may still be pending, socontinue(as a matter of fact, we could probablycontinuein all cases since the current implementation does not run concurrentiterator.nextcalls, so ifpreserveOrder: true && result.donethere should be no remainingpromisesin the queue - but hopefully that will change soon as we think about concurrentiterator.nextcalls!