Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes stack overflow possibility with merge operators #2616

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

geoffmacd
Copy link

@geoffmacd geoffmacd commented Jul 26, 2024

Fixes #2615 See issue for details

when completing Merge operator iterations (concat(), concatMap() and merge(maxConcurrent:), fixes subscribing immediately to the next in the queue, which can produce values immediately which can re-enter and cause stack overflows. ultimately uses CurrentThreadScheduler and the isScheduleRequired prop to know if it needs to schedule or not. This is similar to Producer.

All tests pass

@geoffmacd geoffmacd changed the title fixes stackoverflows in MergeLimitedSink fixes stack overflow possibility with merge operators Jul 26, 2024
@danielt1263
Copy link
Collaborator

This looks like a good candidate for a dot release @freak4pc. What do you think?

@@ -170,7 +170,10 @@ private final class MergeLimitedSinkIter<SourceElement, SourceSequence: Observab
case .completed:
self.parent.group.remove(for: self.disposeKey)
if let next = self.parent.queue.dequeue() {
self.parent.subscribe(next, group: self.parent.group)
_ = CurrentThreadScheduler.instance.schedule(()) { _ in
self.parent.subscribe(next, group: self.parent.group)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how important this is, but scheduling the subscribe call will take the corresponding modification of self.parent.group out of the lock that is held while synchronized_on is executing.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, I think you are right that there is an issue and I need to ensure the NSRecursiveLock is re-locking inside this block. I think for the current locking cycle from synchronized_on, there is no more mutation after this line to worry about. The lock will recursively lock on the next inner subscribe.

There is a separate issue here which is that the the disposable from _ = CurrentThreadScheduler.instance.schedule(()) needs to be added to the group otherwise this will not dispose if the CompositeDisposable is disposed. This is minor though since the parent.subscribe would have no impact.

@freak4pc
Copy link
Member

freak4pc commented Oct 3, 2024

Hey @geoffmacd, sorry for the long delay on this.

  1. Did you use your own fix in prod by now? Is it stable?
  2. Is the code ready from your perspective ?
  3. Let's rebase / merge with main and have CI run all the tests to make sure we're good.

Thanks!

fix subscribing immediately in merge operators can produce values immediately which can re-enter and cause stack overflows
@freak4pc freak4pc closed this Oct 4, 2024
@freak4pc freak4pc reopened this Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Stack overflows caused by MergeLimitedSink operators
4 participants