Fix windows support #99
Conversation
roblabla
force-pushed
the
windows-support
branch
from
b7da974
to
2afe5b8
Compare
|
Interesting, you can also update the action files to add windows target and prove it works. |
|
So as I said, I have a bit of a weird setup where I use a cross-compiling toolchain to build the crate, and that works. However, when I tried to add it in CI, but doing the naive thing of just running This might be fixable by entering a visual studio shell so the cl.exe program is in the PATH. I'll give it a try. |
|
I don't think space in path is the problem.
The line shows that full compiler path is used. However the target it tries doesn't match the compiler it uses. |
|
The space is absolutely the problem, but the logs don't show it clearly. If I run the build locally on a windows vm and check the config.log, it has an error to the tune of "C:\Program: no such file or directory", which stops the build. Autoconf does not support spaces in the compiler path. It's a rather well-known problem. I just had a pretty solid idea to avoid it though: I can use the short path syntax (c:\progra~2) to avoid the space 👀. The target is not wrong. As I explain in the post, the way you do an msvc build with jemalloc is by building for mingw/cygwin with an msvc compiler in CC. The autoconf script will detect that and set use_msvc to yes, at which point the msvc codepaths will be used. |
roblabla
force-pushed
the
windows-support
branch
23 times, most recently
from
4ec1604
to
ca6616a
Compare
roblabla
force-pushed
the
windows-support
branch
from
ca6616a
to
8c2b937
Compare
roblabla
force-pushed
the
windows-support
branch
3 times, most recently
from
5bd1578
to
4e874d2
Compare
ti-chi-bot
bot
added
dco-signoff: yes
dco-signoff: no
and removed
dco-signoff: no
dco-signoff: yes
labels
|
Finally found the time to push this across the finish line, this PR now builds correctly in CI: https://github.com/roblabla/jemallocator/actions/runs/10995602458 Had to fix a compilation issue in jemalloc when using MSVC 2022, which I submitted both:
I then had to update the submodule to point to the new commit, and regenerate the vendored configure script. With all this in place, windows support is now working properly. I also cleaned up the commit a little, removing the DBG commits and providing more description to some otherwise "bare" commits. |
To successfully build jemalloc for msvc-windows, we need to pretend we're doing a mingw build. The configure script will then check if the compiler is actually an MSVC compiler (by checking if it defines _MSC_VER), and if so switch to an MSVC build (which enables a bunch of compatibility hacks, such as removing the need for pthread). Signed-off-by: roblabla <[email protected]>
On windows, the generated library is called jemalloc_s instead of jemalloc_pic. Signed-off-by: roblabla <[email protected]>
Signed-off-by: roblabla <[email protected]>
It turns out, jemalloc doesn't support having unprefixed malloc on these targets. When --with-jemalloc-prefix is not specified, jemalloc will implicitly add a `je_` prefix regardless. Signed-off-by: roblabla <[email protected]>
Signed-off-by: roblabla <[email protected]>
When invoking MSVC CC, a handful of environment variables must be set for the compiler to find its supporting files (such as system includes, libraries, etc...). When invoking it through cc-rs, these environment variables are automatically detected and set. However, we currently don't set those environment variables when running `./configure` or `make`, leading to those commands failing due to the compiler not having access to any of the system include/libraries. To fix this, we ask cc-rs to give us the environment variables to set, and set them whenever running `configure` and `make`. Signed-off-by: roblabla <[email protected]>
Autoconf does not support spaces in the compiler path. This is not usually a problem, as on most platforms, the compiler will reside in a path without spaces (such as /usr/bin or /opt). However, on windows, the MSVC compiler is usually found under C:\Program Files, which causes all hell to break loose. Similarly, the jemalloc Makefile also assumes the shell (stored in the SHELL variable) is in a folder without spaces, and is used unquoted all throughout the makefile, causing issues if the shell is in C:\Program Files, such as when using Git Bash (which installs in C:\Program Files\Git\bin\bash). Thankfully, windows has something called "Short Paths", which we can use to turn C:\Program Files into C:\PROGRA~2, eliminating the path. If the build script detects the compiler path has a space, and is running on windows, it will attempt to turn the path into a short path and run with this. Signed-off-by: roblabla <[email protected]>
Signed-off-by: roblabla <[email protected]>
Signed-off-by: roblabla <[email protected]>
Background Threads are unsupported on windows in jemalloc, due to requiring pthreads (which aren't available on windows, at least on the msvc target). Trying to enable it will cause a runtime error causing the program to immediately abort. As such, to avoid problems, we set windows as a target that does not support background_threads. Signed-off-by: roblabla <[email protected]>
Profiling is not currently supported by jemalloc on windows, as it requires a profiling backend based on libraries available on Windows. As such, we disable the profiling-related tests when running on windows. Signed-off-by: roblabla <[email protected]>
Signed-off-by: roblabla <[email protected]>
roblabla
force-pushed
the
windows-support
branch
from
4e874d2
to
738cc67
Compare
|
Hi, I tested this branch and it does not work when targeting |
|
Giving it a quick look I would say the issue is build.rs assuming the target for itself is the actual build target. |
The windows targets weren't working at all for me:
i686-pc-win32doesn't exist as far as jemalloc's build system is concerned, onlyi686-pc-mingw32does (and if jemalloc detects the compiler is MSVC, it will automatically switch to an MSVC-based build).Finally, I also added support for the
win7target, which is a tier3 target supporting windows 7.Note that this is tested manually on my end using a bit of a bespoke cross-compilation toolchain. The crate doesn't successfully build on a more "normal" toolchain due to autoconf failing to build when the compiler path contains spaces. I'd like to add a CI, but I'd need to first need to find a way to fix that issue...