Fix building with Emscripten toolchain #111

volo-zyko · 2021-11-18T20:38:42Z

Proposed changes

From the commit message

The fix includes the following:

Fix "ambiguity found when searching for best transformation" error by overriding corresponding generator.
Change <link-optimization> setting to a proper boolean value. Current Emscripten doesn't support anything else.
Add detection of ar and ranlib which are necessary for creating static libraries. BTW, building with shared libraries is possible but it requires additionally setting MAIN_MODULE/SIDE_MODULE flags.
Add customization for detection of NodeJS.
Set NodeJS as <testing.launcher> which is necessary when during features/environment inspection boost needs to compile and run a test program.
Remove DEMANGLE_SUPPORT flag from <debug-symbols> setting which, on its own, is not enough for wasm code generator and is not supported when generating JavaScript.
Remove $(FINDLIBS-ST-PFX) and $(FINDLIBS-SA-PFX) from link action as those correspond to -Bstatic and -Bdynamic flags which are not supported by wasm-ld.
Add web as a new <target-os> value. It is necessary to disable appending of -pthread compiler flag.

The first change mentioned above was tested here.

Types of changes

What types of changes does your code introduce?

Put an x in the boxes that apply

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Code style update (formatting, renaming)
Refactoring (no functional changes, no api changes)
Documentation content changes
Other (please describe):

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

I searched the discussions
I searched the closed and open issues
I read the contribution guidelines
I added myself to the copyright attributions for significant changes
I checked that tests pass locally with my changes
I added tests that prove my fix is effective or that my feature works
I added necessary documentation (if appropriate)

Further comments

The most questionable part of this PR is adding the new web <target-os> value just to disable adding -pthread compiler flag. I thought that <threading>single must have done the job but it haven't.

The fix includes the following: - Fix "ambiguity found when searching for best transformation" error by overriding corresponding generator. - Change <link-optimization> setting to a proper boolean value. Current Emscripten doesn't support anything else. - Add detection of ar and ranlib which are necessary for creating static libraries. BTW, building with shared libraries is possible but it requires additionally setting MAIN_MODULE/SIDE_MODULE flags. - Add customization for detection of NodeJS. - Set NodeJS as <testing.launcher> which is necessary when during features/environment inspection boost needs to compile and run a test program. - Remove DEMANGLE_SUPPORT flag from <debug-symbols> setting which, on its own, is not enough for wasm code generator and is not supported when generating JavaScript. - Remove $(FINDLIBS-ST-PFX) and $(FINDLIBS-SA-PFX) from link action as those correspond to -Bstatic and -Bdynamic flags which are not supported by wasm-ld. - Add web as a new <target-os> value. It is necessary to disable appending of -pthread compiler flag.

Kojoley · 2021-11-18T21:01:48Z

src/tools/emscripten.jam

@@ -94,12 +120,13 @@ actions compile.c++

 actions archive
 {
-    "$(CONFIG_COMMAND)" $(AROPTIONS) -r -o "$(<)" "$(>)"
+    "$(.AR)" $(AROPTIONS) rc "$(<)" "$(>)"
+    "$(.RANLIB)" "$(<)"


Is ranlib really needed? AFAIK ar s does the same thing for a long time. See #56.

Indeed there is such option and according to Wikipedia it seems the overall processes is geared towards ranlib deprecation. I'll remove it.

Kojoley · 2021-11-18T21:08:05Z

src/tools/emscripten.jam

-toolset.flags emscripten.link OPTIONS <link-optimization>off : --llvm-lto 0 ;
-toolset.flags emscripten.link OPTIONS <link-optimization>on : --llvm-lto 1 ;
-toolset.flags emscripten.link OPTIONS <link-optimization>full : --llvm-lto 3 ;
+toolset.flags emscripten.link OPTIONS <link-optimization>on : -flto ;


FIY: link-optimization option is undocumented and presented only in emscripten, there is a documented lto feature that does the same thing. If emscripten like Clang supports -flto=thin/-flto=full you might be better to switch the toolset to inherit from Clang instead of GCC.

there is a documented lto feature that does the same thing

Should link-optimization be renamed to lto? The meaning now is the same.

If emscripten like Clang supports -flto=thin/-flto=full

No. Despite it's based on Clang Emscripten's LTO implementation is a bit more complicated. See its documentation. In other words lto-mode is not applicable here.

In general, renaming to lto seems like a right thing because according to the docs above -flto should be used during compilation as well.

No. Despite it's based on Clang Emscripten's LTO implementation is a bit more complicated. See its documentation. In other words lto-mode is not applicable here.

I think emscripten-core/emscripten#10868 tells an opposite thing.

Ok, the official Emscripten documentation is out of date. I found such custom processing of -flto in emcc.py. Do you think that inheriting from clang (which, BTW, on its own inherits from gcc) should be done in this PR? I tried to inherit from clang but it didn't work out of the box. So, there will be a lot more changes.

Did you inherit from clang or clang-linux? You should have been inheriting from clang-linux (that 'linux' suffix means clang with default/posix interface, not the os), sorry I did not tell you that. I would try doing it myself but I don't have the toolset and the CI doesn't tests it either. Don't bother with this if you are short on time.

I used clang-linux. clang is kind of wrapper around clang-win, clang-linux, etc. I will try to work it out but don't expect too much from me. My knowledge of bjam is rather scarce.

Kojoley · 2021-11-23T18:12:18Z

src/tools/emscripten.jam

 }

 toolset.flags emscripten.link USER_OPTIONS <linkflags> ;

 actions link bind LIBRARIES
 {
-    "$(CONFIG_COMMAND)" $(USER_OPTIONS) -L"$(LINKPATH)" -o "$(<)" "$(>)" "$(LIBRARIES)" $(START-GROUP) $(FINDLIBS-ST-PFX) -l$(FINDLIBS-ST) $(FINDLIBS-SA-PFX) -l$(FINDLIBS-SA) $(END-GROUP) $(OPTIONS)
+    "$(CONFIG_COMMAND)" $(USER_OPTIONS) -L"$(LINKPATH)" -o "$(<)" "$(>)" "$(LIBRARIES)" $(START-GROUP) -l$(FINDLIBS-ST) -l$(FINDLIBS-SA) $(END-GROUP) $(OPTIONS)


I think you need to modify GCC instead

b2/src/tools/gcc.jam

Lines 933 to 941 in 371b47a

# On *nix mixing shared libs with static runtime is not a good idea.

toolset.flags gcc.link FINDLIBS-ST-PFX <target-os>$(generic-os)/<runtime-link>shared : -Wl,-Bstatic ;

toolset.flags gcc.link FINDLIBS-SA-PFX <target-os>$(generic-os)/<runtime-link>shared : -Wl,-Bdynamic ;

# On windows allow mixing of static and dynamic libs with static

# runtime is not a good idea.

toolset.flags gcc.link FINDLIBS-ST-PFX <target-os>windows/<runtime-link>static : -Wl,-Bstatic ;

toolset.flags gcc.link FINDLIBS-SA-PFX <target-os>windows/<runtime-link>static : -Wl,-Bdynamic ;

toolset.flags gcc.link OPTIONS <target-os>windows/<runtime-link>static : -Wl,-Bstatic ;

Can these settings be overridden in emscripten.jam?

Will the following in emscripten.jam do the job?

toolset.flags emscripten.link FINDLIBS-ST-PFX <target-os>web/<runtime-link>static : ; toolset.flags emscripten.link FINDLIBS-SA-PFX <target-os>web/<runtime-link>static : ;

Anyway, I will try.

Nope, it does not override, it appends additional value.

Kojoley reviewed Nov 18, 2021

View reviewed changes

Kojoley reviewed Nov 23, 2021

View reviewed changes

jonathanpoelen mentioned this pull request Jun 26, 2022

emscripten toolset does not work #170

Closed

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111#discussion_r752622386

ce3f137

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

b427c8f

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

7c4f457

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

ac01647

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

183c6b6

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

c586efb

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

be68ae6

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

b127b72

ShuangLiu1992 added a commit to ShuangLiu1992/b2 that referenced this pull request Oct 19, 2022

https://github.com/bfgroup/b2/pull/111

4e72943

Pospelove mentioned this pull request Feb 8, 2023

[boost-filesystem] fix emscripten build microsoft/vcpkg#29540

Merged

grafikrobot assigned grafikrobot and unassigned grafikrobot Apr 3, 2023

grafikrobot mentioned this pull request Apr 25, 2023

Building static or shared boost libraries with Emscripten boostorg/build#381

Closed

Kojoley mentioned this pull request May 7, 2023

emscripten renovation #310

Merged

grafikrobot closed this in #310 Jul 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix building with Emscripten toolchain #111

Fix building with Emscripten toolchain #111

volo-zyko commented Nov 18, 2021

Kojoley Nov 18, 2021

volo-zyko Nov 19, 2021

Kojoley Nov 18, 2021

volo-zyko Nov 19, 2021 •

edited

Loading

Kojoley Nov 19, 2021

volo-zyko Nov 23, 2021

Kojoley Nov 23, 2021

volo-zyko Nov 24, 2021

Kojoley Nov 23, 2021

volo-zyko Nov 24, 2021

volo-zyko Nov 24, 2021

Kojoley Nov 24, 2021

	# On *nix mixing shared libs with static runtime is not a good idea.
	toolset.flags gcc.link FINDLIBS-ST-PFX <target-os>$(generic-os)/<runtime-link>shared : -Wl,-Bstatic ;
	toolset.flags gcc.link FINDLIBS-SA-PFX <target-os>$(generic-os)/<runtime-link>shared : -Wl,-Bdynamic ;

	# On windows allow mixing of static and dynamic libs with static
	# runtime is not a good idea.
	toolset.flags gcc.link FINDLIBS-ST-PFX <target-os>windows/<runtime-link>static : -Wl,-Bstatic ;
	toolset.flags gcc.link FINDLIBS-SA-PFX <target-os>windows/<runtime-link>static : -Wl,-Bdynamic ;
	toolset.flags gcc.link OPTIONS <target-os>windows/<runtime-link>static : -Wl,-Bstatic ;

Fix building with Emscripten toolchain #111

Fix building with Emscripten toolchain #111

Conversation

volo-zyko commented Nov 18, 2021

Proposed changes

From the commit message

Types of changes

Checklist

Further comments

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

volo-zyko Nov 19, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

volo-zyko Nov 19, 2021 •

edited

Loading