Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TOREE-546] Mitigate ZeroMQSocketRunnableSpec hang #212

Merged
merged 12 commits into from
Aug 10, 2023

Conversation

pan3793
Copy link
Member

@pan3793 pan3793 commented Aug 10, 2023

This is the second try to fix the ZeroMQSocketRunnableSpec hang. Unfortunately, the previous try #205 seems not help.

The stacktrace is

[info]   - should set the linger option when provided *** FAILED *** (124 milliseconds)
[info]     java.lang.AssertionError: assertion failed: Runnable is not processing or is closed!
[info]     at scala.Predef$.assert(Predef.scala:223)
[info]     at org.apache.toree.communication.socket.ZeroMQSocketRunnable.close(ZeroMQSocketRunnable.scala:187)
[info]     at org.apache.toree.communication.socket.ZeroMQSocketRunnableSpec.$anonfun$new$12(ZeroMQSocketRunnableSpec.scala:138)
...

This time I follow the guidance of ZMQ.Context#close()'s JavaDoc to set ZMQ_LINGER to 0, hope this help.

Warning

As ZMQ_LINGER defaults to "infinite", by default this method will block indefinitely if there are any pending connects or sends. We strongly recommend to

  • set ZMQ_LINGER to zero on all sockets
  • close all sockets, before calling this method

@pan3793
Copy link
Member Author

pan3793 commented Aug 10, 2023

Seems to help but still has a chance to hang, open TOREE-548 to address it in another approach.

Note: this PR does not conflict with TOREE-548, the CI shows it reduces the hang rate compared to the master branch, the reviewer could decide to merge it or ignore it.

@pan3793 pan3793 changed the title [TOREE-546] Fix ZeroMQSocketRunnableSpec hang [TOREE-546] Mitigate ZeroMQSocketRunnableSpec hang Aug 10, 2023
@lresende lresende merged commit 7243d3e into apache:master Aug 10, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants