Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

streamingProcess does not work with non-threaded runtime #40

Open
nh2 opened this issue Jun 28, 2017 · 1 comment
Open

streamingProcess does not work with non-threaded runtime #40

nh2 opened this issue Jun 28, 2017 · 1 comment

Comments

@nh2
Copy link
Member

nh2 commented Jun 28, 2017

In the non-threaded runtime, the waitForProcess (waitpid()) call used in streamingProcess blocks the entire Haskell process, thus making the program block forever if the child process needs to be fed input by Haskell in order to terminate.

This is because waitpid() is not Fd-based and thus cannot use the non-blocking IO of the IO manager; see the waitForProcess docs here:

GHC Note: in order to call waitForProcess without blocking all the other threads in the system, you must compile the program with -threaded.

Detail repro with explanation: https://github.com/nh2/sourceProcessWithStreams-nonthreaded-problem

I use the nonthreaded runtime a lot, and I also use streamingProcess and conduits on top of it, so I'd like those to work with each other.

@nh2
Copy link
Member Author

nh2 commented Jun 28, 2017

I have a proposed solution in the fix branch of my repro:

nh2/sourceProcessWithStreams-nonthreaded-problem@3709139...fix

The solution turns out to be a 3-line change, but I need some advice of backward compatibility in that package:

I'm currently using createPipeFd which according to https://www.stackage.org/lts-6.17/package/process-1.2.3.0 was added in 1.4.2.0 January 2016.

So I'm not sure I can use that function.

An alternative would be to use createPipe that works on Handle and then handleToFd, but I don't know if this function works on Windows.

I'll create a tentative PR using createPipe to see what the AppVeyor Windows CI thinks about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant