building static libraries with shared library wrapper #105
Conversation
CMakeLists.txt
Outdated
| ############ | ||
| # Boost | ||
| ############ | ||
|
|
||
| set(Boost_NO_BOOST_CMAKE 1) | ||
| set(Boost_USE_MULTITHREADED ON) | ||
| set(Boost_USE_STATIC_LIBS ON) |
There was a problem hiding this comment.
On Ubuntu 20.0 and 22.04, I get this error with this field set to ON:
[100%] Linking CXX shared library libmonero-java.so
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_serialization.a(basic_iarchive.o): warning: relocation against `_ZTVN5boost7archive6detail14basic_iarchiveE' in read-only section `.text'
/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libboost_chrono.a(chrono.o): relocation R_X86_64_PC32 against symbol `_ZN5boost6system6detail10cat_holderIvE24system_category_instanceE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
But it works when set to OFF. Anything wrong with leaving it off?
There was a problem hiding this comment.
Hm, if I set it to OFF, it builds successfully, but the libraries built on Ubuntu 20.04 still can't be run on Ubuntu 22.04, showing an error from libboost_chrono.so.
There was a problem hiding this comment.
I think boost is compiled without -fPIC on ubuntu. I will look into libboost_chrono.so error.
There was a problem hiding this comment.
Yeah, maybe it's not possible to build libs compatible with both versions then.
There was a problem hiding this comment.
@woodser , does the actual build only run on Ubuntu 20.04?
Shared libs have different versions on Ubuntu 20.04 and Ubuntu 22.04.
Ex:
ubuntu-20-04:~$ ls /lib/x86_64-linux-gnu/libssl.*
/lib/x86_64-linux-gnu/libssl.a /lib/x86_64-linux-gnu/libssl.so /lib/x86_64-linux-gnu/libssl.so.1.1
ubuntu-22-04:~$ ls /lib/x86_64-linux-gnu/libssl.*
/lib/x86_64-linux-gnu/libssl.a /lib/x86_64-linux-gnu/libssl.so /lib/x86_64-linux-gnu/libssl.so.3
There was a problem hiding this comment.
I don't know that the same build has ever worked on both platforms.
Was hoping the static libs with a shared library wrapper (for JNI) would be cross compatible.
There was a problem hiding this comment.
Hi @woodser .
I got haveno working with native wallet on ubuntu 20 and 22 but I have had to:
- use openssl and unbound static lib
- clone boost (1.71.0) and build it static with -fPIC (maybe it will become a module in monero-java)
- disable Trezor hardware wallet support on monero build (USE_DEVICE_TREZOR=OFF). Because I have had problems to run haveno with protobuf built static with -fPIC. Is it a problem?
OBS: I built monero/monero-cpp/monero-java on ubuntu 20.
There was a problem hiding this comment.
Its ok to go in this way?
Yes I think we may have to go this way if we want this static/shared library wrapper idea to work, since the goal is to make the libraries standalone and supported in JNI. Maybe this is the best way?
There was a problem hiding this comment.
Yes I think we may have to go this way if we want this static/shared library wrapper idea to work, since the goal is to make the libraries standalone and supported in JNI. Maybe this is the best way?
I think so. This is done with the last commit.
PS. I only tested this on linux. I dont have how testing on others platforms.
|
I've applied this PR and woodser/monero-cpp#64. Looks promising, but I'm having trouble building it. On Ubuntu 20.04, I first get this error: Setting Then the build fails during linking: On macOS, I get this error: Changing |
|
Hoping the errors can be resolved because it would be really great to build to portable, static libraries. :) |
This is done in bin/build_libmonero_java.sh
OpenSSL will have to become a module built with -fPIC. I will look into this.
I will look into this |
@woodser , could you try to build again on Ubuntu 20.04 aarch64-linux-gnu? |
|
I was able to build successfully on Ubuntu 20.04 aarch64-linux-gnu, but I'm still having trouble building on macOS, first failing on this error: I'm confused why the build should work at all, including on Ubuntu, because By modifying CMakeLists.txt to |
|
I see that monero-cpp is using boost and openssl with these paths instead of the new submodules, wondering if that could be a problem: Perhaps boost and openssl should be included within ./external/monero-cpp/external instead and used to build monero-cpp as well? |
It works because you have boost installed. You are write, I have fixed boost include path. |
Whats the boost version installed on macOS? |
I have boost 1.84 installed through brew.
Wondering again then if boost and openssl should be a submodule within external/monero-cpp/external, which both monero-java and monero-cpp build from? This would be consistent with how monero-ts works, where boost [1] and openssl [2] are downloaded to external/monero-cpp/external. |
Ok. I will try. But monero-project also use boost and openssl and maybe we dont have control of it. |
Maybe it's not necessary, and the boost versions merely need adjusted to be compatible, but I would think monero-cpp should be built with the same version as monero-java.
Don't know if the |
|
@woodser could you try to build on macos again? |
|
Hm, I still end up with these errors building monero-java with the new changes to this and monero-cpp: I confirmed that the build succeeds from the master branches. |
|
Could you past here or send me the full build output? I want to know if build is using boost lib from submodule or system. |
|
|
@woodser could you try again on macos and put the log here please? |
|
It's failing to build boost with the latest changes to monero-java and monero-cpp: |
|
This is the error: |
|
Its related with the newer macOS plus boost 1.7x. See bambulab/BambuStudio#3957 |
|
Yeah I saw that. I just updated my OS to the latest version, but it didn't help. Using 1.8x sounds good, it's what I've been using in the libraries anyway. I don't have any experience emulating mac in linux unfortunately. |
|
Build failed on ubuntu-20.04 with boost 1.86. We need to update monero-project. See monero-project/monero#9313 |
|
Ok, I'm in the process of updating to monero-project v0.18.3.4 now. |
|
Ok it's all updated to monero-project v0.18.3.4. |
|
@woodser , build ok on ubuntu-20.04. Could you try again on macos? |
|
Sure, please resolve the merge conflict with master. :) |
|
With the PRs applied to monero-java and monero-cpp (and the merge conflict resolved), and using the latest monero-project v0.18.3.4 (pushed to my master at https://github.com/woodser/monero), I get these errors on building monero-project: Here's the full build log: monero-project succeeds to build if I run I confirmed that the build succeeds by switching to the latest master branch of all 3 projects. To ensure the build is completely fresh, I first deleted the |
|
Hey, happy to see more changes coming in. Please just let me know when it's ready for retesting. |
|
I finally figured out what was happening with sodium include path. Monero-project considers that include path for sodium is the same for boost and openssl. Now with this PR, it is not true. I have to modify monero-project in my fork. |
|
I'm hitting this error with your PR, which is probably related to your monero-project branch not having the latest fix for macOS: monero-cpp/external/libsodium is being used in the monero-ts project. Perhaps it's better to move your libsodium solution to monero-cpp/external so it can be used by monero-ts as well? |
|
Applying that fix from monero-project myself, I then get this error with your PRs:
|
|
These errors are about boost-1.85+. I just downgraded boost to 1.84 in this solution because this. |
|
I would confirm it with output log but for now, could you try to remove boost from homebrew and do a clean build again? |
|
I was able to finally successfully build libmonero-cpp.a and libmonero-java.dylib. :) By uninstallingn boost through brew and by making these changes to CMakeLists.txt to include hidapi: patch.txt However, basic tests to execute the libraries through JNI are hitting an error. To recreate the error, simply run the tests within TestMoneroUtils.java. They can easily be invoked by importing the project into Visual Studio Code (or Eclipse if you prefer): 
|
Cool!!
Hidapi is needed only for Trezor support but it is off on monero build. I didnt understand why it is needed on mac.
I will fix this. |
|
@woodser , any problem keep on without hardware wallet support? Because this has new dependencies that we will have to build with static and fPIC. |
|
Ideally we can keep existing hardware wallet dependencies, unless they are not already included? In which case, no need to add them as part of this PR. |
No. if we keep existing hardware wallet dependencies (libs linked to libmonero-java.so), the host running monero-java application must have dependencies installed on. For example, haveno running on host with no hidapi lib installed and using monero-java (this PR) build with hidapi linked (your patch) gets this error: At the moment there is just this problem, mac build needs hidapi. I didnt understand why because |
|
I've released updates to monero-java and monero-cpp, and I'd like to retest this PR with the new updates. Maybe it could have fixed the previous error we saw? But can you rebase your PRs on master, please? |
|
Ok, I will do this and move openssl, boost and sodium build to monero-cpp. |
nsec1
force-pushed
the
sharedLibraryWrapper
branch
from
99925e5
to
bca404b
Compare
|
Done.
|
pom.xml
Outdated
| @@ -4,7 +4,7 @@ | |||
| <modelVersion>4.0.0</modelVersion> | |||
| <groupId>io.github.woodser</groupId> | |||
| <artifactId>monero-java</artifactId> | |||
| <version>0.8.33</version> | |||
| <version>0.8.33x</version> | |||
There was a problem hiding this comment.
Any reason to change the version number now? Normally I bump it just before release.
nsec1
force-pushed
the
sharedLibraryWrapper
branch
from
1ad252e
to
1e5b6c9
Compare
|
I have your latest changes applied to monero-java, monero-cpp, and monero-project, but I'm getting this linker error at the end of the build process: Here's the full build.log: build.log I confirmed that the build is working from the master branches and my release-v0.18 branch in monero-project, which includes a couple of custom commits. Confirming this change should still be applied to monero-project? Also I was wondering, is it necessary to checkout all of boost's submodules? There are so many of them :) |
It's the include path conflict. Monero project is using
Yes. It resolves include path conflict for zmq and sodium Could you apply this patch to monero-project and build again? It is for printing INCLUDE_DIRECTORIES list.
Did you use my monero fork? I can fork yours and apply my change.
It is out of my control, comes from git module update. I would have to track each module with its version. it doesn't worth. |
Yes please, if you can please submit a PR to my release branch here: https://github.com/woodser/monero/tree/release-v0.18. That's now the default for the libraries, since that's what they're based on. |
|
I applied your patch manually (it wasn't included in your PR). Here's the resulting log file: build.log |
It is for debug only, can be ignored. I make a fix on monero-cpp build script. |
|
I pulled in your commit to monero-cpp. Looks like I'm getting the same error, here's the full build log: build.log |
|
I don't know what to do. Monero project is using boost from |
Ref. #74 haveno-dex/haveno#795
Now, there is only monero-java.so. It wraps monero, monero-cpp, boost etc.
Working with haveno on linux (--useNativeXmrWallet=true).
Depends woodser/monero-cpp#64