-
Notifications
You must be signed in to change notification settings - Fork 485
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement IPC with java 16 unix domain sockets #967
Conversation
A multi-release jar seemed like the right approach for this but IDE support when targeting multiple versions in the same module isn't great. Happy to refactor if there are any suggestions for a better way to organize things. No issues building with maven on command line though. I was able to successfully use this branch to bind a set of ipc server sockets that pyzmq can connect to. This is the motivation for implementing this feature (used for a jupyter kernel implementation on the jvm). |
I started a work on providing network protocols as services, but never had time to finish it. It should be a good starting point. |
@fbacchella Referring to networkprotocolserviceprovider? Could you elaborate a bit on what you mean? There is still the problem of picking the ipc implementation at runtime and building part of jeromq targeting java 16 from what I can tell. i.e. if working off that branch I would still take the same approach here just in a different place. I don't think I could pick up where you left off on that branch but I also don't think these changes necessarily conflict either. That work would still be useful regardless of this PR. |
It could be a simple jar dropped in the classpath. So no problem with java version and the need to use the module trick. |
Thanks, ok so you'd prefer the new ipc implementation to be a separate artifact? Leave the existing one there and put both behind an SPI? This would be an extra maven dependency that consumers would need right? |
yes, that’s exactly what I had in mind. |
@fbacchella Switched to loading The new module reruns the ipc tests I could find against the new implementation. The old fake ipc implementation is in Please take a look when you get a chance. |
This PR is why I released 0.6.0 and launched 0.7.0-SNAPSHOT. Protocol will be provided as services. I need to investigate how to resolved the multi-release jar problem, but I think the way it’s handled in this PR is the good way. |
@fbacchella I should update the description again because I got rid of the multi-release jar with the service oriented refactor but I'm kind of on the fence if it should be added again. i.e. still keeping the separate maven module and artifact but package that artifact as the multi-release jar, no idea how a service provider is declared with that packaging though. For my use case I always want to use this new ipc implementation if running in jre >=16 and just not use ipc at all otherwise. So I would want to always include I also see you revived the protocol as a service branch (#974), I can try and rebase this PR based on that if the plan is to merge that PR as well. Please let me know and I can try and find some time this weekend :) |
The multi release jar is a good solution to provide a simple way to use IPC. Once the PR is merged, I will create another PR to implement modulorisation. Then we will be able to merge your implementation. |
The new multi module organisation has just been merged. If you agree I will create a branch with empty IPC socket module to demonstrate how I think this should be handled. Or I can implement it myself, stealing the ideas, but you will miss the glory of attribution |
I have a working branch that could do the job that stole some of your ideas, see https://github.com/fbacchella/jeromq/tree/ipc_unix_socket. |
This PR added unix domain socket support: #998 |
Compatibility is a bit tricky because the current "ipc" implementation isn't ipc, so anyone relying on the jeromq tcp loopback implementation intentionally will break. I think this would only really be a problem if mixing an older and newer jeromq client and server. If both are updated then this should just start using unix domain sockets underneath with the same interface and the consumer wouldn't need to be aware of that difference.
Since this ipc implementation relies on a newer feature, I wanted to leave the existing tcp loopback implementation when running on a jvm that is missing the unix domain sockets. Thought a multi-release jar was the best approach for that targeting java 16 for the real ipc implementation.
To check at runtime if using the tcp loopback or unix domain socket implementation, you can query
IpcAddress.isImplementedViaTcpLoopback()
.The approach for the imlementation was to just pull all the non-tcp specific code out of
TcpListener
andTcpConnector
into an abstract base class leaving abstract methods for the protocol specific details. The unix domain socket interface is stilljava.net.{ServerSocketChannel,SocketChannel}
so a lot of that implementation can be shared.Removed
zmq.io.net.{ProtocolFamily,StandardProtocolFamily}
android replacements because they are in 24 (looks like jeromq targets 28 currently?) and I needed to use thejava.net.
ones.Fixes #953 and fixes #927