Free attached threads

[mboes]

From @alpmestan on February 29, 2016 15:46

When Haskell threads die, we should tell Java about it so that it frees the corresponding Thread object and removes it from the ThreadGroup. Otherwise we're leaking memory.

Copied from original issue: tweag/sparkle#21

[robinbb]

What is the multi-threading model that underlies this issue? Do we think that Haskell (GHC RTS) threads are controlled by Java Thread objects? I suspect that they are not. Is there an observed memory leak that gives rise to this issue? Maybe we should close this issue?

[mboes]

All good questions. I believe @alpmestan originally opened this issue. @alpmestan care to elaborate?

[alpmestan]

If I remember correctly, that's something we thought about during a discussion and that I quickly opened an issue about, to serve as a reminder. I'm pretty sure we have never investigated anything like this, and neither @mboes nor myself remember where this comes from now. I agree that this issue can be closed until 1/ we remember what we meant and 2/ we have a concrete example that exhibits some problematic behaviour.

However, it is indeed not entirely obvious to figure out how the two runtimes interact as far as exceptions, threading and memory management are concerned. But without a concrete problem to solve, there's not much we can do right now.

[mboes]

I'm guessing this issue refers to the AttachCurrentThread() call in the jni package, https://github.com/tweag/inline-java/blob/master/jni/src/Foreign/JNI.hs#L186. Normally, threads attached by native code should have a correspond Detach call. Attaching associates a new Java Thread object. So if we're not detaching then we're leaking Thread objects.

Now, this issue talks about "Haskell threads". But to clarify we're only really leaking as many Thread objects as there are OS threads. That's because if multiple RTS threads multiplexed over a single OS thread call Attach(), then apparently the JVM is smart enough not te create redundant Thread objects. But there can be many OS threads over the lifetime of a program. Even if there might be fewer than RTS threads. And the lifetimes of these threads may or may not match the lifetime of the Haskell program.

So yes, I think we're leaking memory. (But for most applications, that don't use forkOS, this is likely an irrelevant issue.)

BTW,

However, it is indeed not entirely obvious to figure out how the two runtimes interact as far as exceptions, threading and memory management are concerned.

I wouldn't put that way. I think the interactions are all fairly well understood actually. Which is not to say all the problems have been solved.

Synchronous exceptions are checked for using the throwIfException wrapper that all the JNI bindings use internally upon return of each JNI call. Synchronous Java exceptions can be raised on the Haskell side at that point and that point only. Uncaught Haskell exceptions that bubble all the way up to the Java boundary kill the whole application, which is okay for most people, and matches the behaviour of the C FFI (so no surprises there). Users are always free to set an exception handler first thing at the point of entry if need be.

Java remains oblivious to RTS threads. This is fine so long as threads don't share references to Java objects (and threads that interact with the JVM, just like with OpenGL, are bound to avoid having them be moved mid-execution to another OS thread, a fact that we should document better). Note that the main thread is always bound. See #7 for more discussion. We know "how the runtimes interact" in this regard, but some design questions are still open as to a solution. #7 also discusses memory management.

[mboes]

Another note: code entered via a foreign export 'ed function always runs on a bound thread. IOW threading works out-of-the-box in the sparkle use case, modulo the limitations raised in #7.

[facundominguez]

This task is not longer relevant after the updates to address tweag/sparkle#72. The user is responsible for making JNI calls from attached threads now. JNI no longer attaches threads automatically.