Ask @paperdave about it

[xnox]

TODO: This is out of date. TLDR is pass -DUSE_DEBUG_JSC=1 or -DWEBKIT_DIR=... to CMake. it will probably need more fiddling. ask @paperdave if you need this.

As per contributing - i have tried building bun multiple times, and the core issue is that Clang's that are available for bun and rust and webkit are not the same, and either everything fails to install build-dependencies, or fails to build, or fails at runtime.

So far it seems like I really need to build webkit from scratch with matching toolchain as the bun I am trying to build.

Any additional comments apart from the TLDR above? As in now to rebuild release-grade webkit like it is done for bun releases? but using my own distro toolchain. Any pointers as to which .github .docker .buildkit etc is compiling webkit for the WEBKIT_DIR setting would be appriciated.

I am interested in Linux x86-64 and arm64 builds only. No windows/macosx.

[xnox]

Is webkit build done at https://github.com/oven-sh/WebKit/blob/cabf6d445740d65603190cfaf4618bb7997d97b3/.github/workflows/build.yml#L114 ?

[paperclover]

Is webkit build done at https://github.com/oven-sh/WebKit/blob/cabf6d445740d65603190cfaf4618bb7997d97b3/.github/workflows/build.yml#L114 ?

yes, ci builds every commit with that

[paperclover]

use LLVM 16.0.6

try to bun's cmake -DUSE_SYSTEM_ICU=1
(option(USE_SYSTEM_ICU "Use the system-provided libicu. May fix startup crashes when building WebKit yourself." OFF))

ive had icu issues in the past on linux (i do most dev on mac os). i think that could fix your runtime crash

https://github.com/oven-sh/webkit i think we use the Dockerfile in that repo to build it, the primary thing is to run cmake with all of the flags passed. and then the final output of webkit dir is just copying a bunch of extra headers.

[xnox]

use LLVM 16.0.6

i have LLVM 16 available; I do not however have rust that uses LLVM 16, it is higher. That may also be causing me problems.

Where do you get rust that is using LLVM 16?

[xnox]

So i'm getting better and worse at the same time. Managed to do all the things, but now stuck at this:

2024/08/12 12:01:16 WARN CMake Warning:
2024/08/12 12:01:16 WARN   Manually-specified variables were not used by the project:
2024/08/12 12:01:16 WARN
2024/08/12 12:01:16 WARN     CMAKE_INSTALL_LIBDIR
2024/08/12 12:01:16 WARN     USE_CUSTOM_LIBDEFLATE
2024/08/12 12:01:16 WARN
2024/08/12 12:01:16 WARN
2024/08/12 12:01:16 INFO -- Build files have been written to: /home/build/build-release
2024/08/12 12:01:16 INFO [1/2] Re-checking globbed directories...
2024/08/12 12:01:16 WARN ninja: file is missing and not created by any action: '/home/build/build/bun-deps/libcrypto.a'

Thus not sure if it is, or is not possible to use sytem/prebuilt libdeflate, but that's just a warning.

the

2024/08/12 12:01:16 WARN ninja: file is missing and not created by any action: '/home/build/build/bun-deps/libcrypto.a'

Is a problem. I was hoping to either use system library or have it built from source. but it didn't do either of those things. Is that supposed to be boringcrypto or openssl? And where does it come from normally? My current actions are:

  - uses: git-checkout
    with:
      repository: https://github.com/oven-sh/bun
      expected-commit: ae19489250ab74ee3ea7d5b50fca5d7d2b4e6d66
      tag: bun-v${{package.version}}
      recurse-submodules: true

  # bun requires a specific zig to build
  - runs: |
      bun install -g @oven/zig
      # "Modifying file https://github.com/oven-sh/bun/blob/main/CMakeLists.txt, removing flags Werror=uninitialized and  -Werror."
      sed -i '/-Werror=uninitialized/d' ./CMakeLists.txt
      sed -i '/-Werror/d' ./CMakeLists.txt
  - runs: |
      export PATH="$PATH:/$HOME/.bun/bin"
      # Prefer wolfi libraries, when available
      cmake -B build-release -G Ninja \
        -DCMAKE_INSTALL_PREFIX=/usr \
        -DCMAKE_INSTALL_LIBDIR=lib \
        -DCMAKE_BUILD_TYPE=Release \
        -DUSE_STATIC_SQLITE=OFF \
        -DUSE_CUSTOM_ZLIB=OFF \
        -DUSE_CUSTOM_LIBDEFLATE=OFF \
        -DUSE_CUSTOM_LIBARCHIVE=OFF \
        -DUSE_CUSTOM_MIMALLOC=OFF \
        -DUSE_CUSTOM_ZSTD=OFF \
        -DUSE_CUSTOM_CARES=OFF \
        -DUSE_SYSTEM_ICU=ON
      cmake --build build-release

Trying to use cmake directly, without using bun build.... to avoid building things multiple times, but it seems like I am under-building things right now.

[xnox]

oooh let me run scripts/all-dependencies.sh

[xnox]

ok getting further now.

[xnox]

2024/08/12 12:14:36 INFO
2024/08/12 12:14:36 INFO [1.90s] Bundled "src/js" for production
2024/08/12 12:14:36 INFO   1634 kb
2024/08/12 12:14:36 INFO   66 internal modules
2024/08/12 12:14:36 INFO   340 internal functions across 29 files
2024/08/12 12:14:42 INFO [11/381] Building zig code
2024/08/12 12:14:47 WARN info: zig compiler v0.13.0
2024/08/12 12:15:16 WARN obj
2024/08/12 12:15:16 WARN +- zig build-obj bun ReleaseFast native-native-gnu.2.27 3 errors
2024/08/12 12:15:16 WARN src/runtime.zig:200:40: error: unable to open './runtime.out.js': FileNotFound
2024/08/12 12:15:16 WARN src/node_fallbacks.zig:8:40: error: unable to open './node-fallbacks/out/assert.js': FileNotFound
2024/08/12 12:15:16 WARN root.zig:13:38: error: unable to open 'packages/bun-error/dist/bun-error.css': FileNotFound
2024/08/12 12:15:16 WARN error: the following command failed with 3 compilation errors:
2024/08/12 12:15:16 WARN /home/build/.cache/zig/zig build-obj -freference-trace=16 -fno-strip -fno-stack-check -fno-stack-protector -fno-omit-frame-pointer -fPIC -OReleaseFast -target native-native-gnu.2.27 -mcpu haswell --dep async_io --dep zlib-internal --dep async --dep ZigGeneratedClasses --dep ResolvedSourceTag --dep build_options -Mroot=/home/build/root.zig -Masync_io=/home/build/src/io/io_linux.zig -Mzlib-internal=/home/build/src/deps/zlib.posix.zig -Masync=/home/build/src/async/posix_event_loop.zig -MZigGeneratedClasses=/home/build/build-release/codegen/ZigGeneratedClasses.zig -MResolvedSourceTag=/home/build/build-release/codegen/ResolvedSourceTag.zig -Mbuild_options=/home/build/.zig-cache/c/67365bdcdad3907c98a331c15639fabc/options.zig -lc++ -lc -fformatted-panics --eh-frame-hdr --emit-relocs -ffunction-sections --cache-dir /home/build/.zig-cache --global-cache-dir /home/build/.cache/zig --name bun -fno-compiler-rt --zig-lib-dir /home/build/src/deps/zig/lib --listen=- 
2024/08/12 12:15:16 WARN Build Summary: 1/4 steps succeeded; 1 failed (disable with --summary none)
2024/08/12 12:15:16 WARN obj transitive failure
2024/08/12 12:15:16 WARN +- zig build-obj bun ReleaseFast native-native-gnu.2.27 3 errors
2024/08/12 12:15:16 WARN +- install generated to bun-zig.o transitive failure
2024/08/12 12:15:16 WARN    +- zig build-obj bun ReleaseFast native-native-gnu.2.27 (+1 more reused dependencies)
2024/08/12 12:15:16 WARN error: the following build command failed with exit code 1:
2024/08/12 12:15:16 WARN /home/build/.zig-cache/o/97575821bf9284d4fac5f308510d016c/build /home/build/.cache/zig/zig /home/build /home/build/.zig-cache /home/build/.cache/zig --seed 0x9bdd8359 -Z66502e70a8ed3d51 obj --zig-lib-dir /home/build/src/deps/zig/lib --prefix /home/build/build-release/CMakeFiles -Dgenerated-code=/home/build/build-release/codegen -freference-trace=10 -Dversion=1.1.20 -Dcanary=0 -Doptimize=ReleaseFast -Dcpu=haswell -Dtarget=native -Denable_logs=false -Dreported_nodejs_version=22.3.0 -Dobj_format=obj
2024/08/12 12:15:16 WARN ninja: job failed: cd /home/build && /home/build/.cache/zig/zig build obj --zig-lib-dir /home/build/src/deps/zig/lib --prefix /home/build/build-release/CMakeFiles -Dgenerated-code=/home/build/build-release/codegen -freference-trace=10 -Dversion=1.1.20 -Dcanary=0 -Doptimize=ReleaseFast -Dcpu=haswell -Dtarget=native -Denable_logs=false -Dreported_nodejs_version=22.3.0 -Dobj_format=obj
2024/08/12 12:15:16 WARN ninja: subcommand failed
2024/08/12 12:15:16 ERRO ERROR: failed to build package. the build environment has been preserved:
2024/08/12 12:15:16 INFO   workspace dir: /home/xnox/.cache/melange-tmp/melange-workspace-619770269
2024/08/12 12:15:16 INFO   guest dir: /home/xnox/.cache/melange-tmp/melange-guest-356942660
2024/08/12 12:15:16 ERRO failed to build package: unable to run package bun pipeline: unable to run pipeline: exit status 1

sigh.

[xnox]

pondering if i need to build zig code first and separately.... as in

bun/.buildkite/scripts/build-zig.sh

Line 71 in 4447668

run_command cmake .. "${CMAKE_FLAGS[@]}" \

but that seems odd

[xnox]

more progress

2024/08/13 18:07:23 INFO /home/build/src/bun.js/bindings/helpers.h:187:82: warning: suggest braces around initialization of subobject [-Wmissing-braces]
2024/08/13 18:07:23 INFO static const BunString BunStringCwd = BunString { BunStringTag::StaticZigString, ZigStringCwd };
2024/08/13 18:07:23 INFO                                                                                  ^~~~~~~~~~~~
2024/08/13 18:07:23 INFO                                                                                  {           }
2024/08/13 18:07:23 INFO /home/build/src/bun.js/bindings/helpers.h:188:74: warning: suggest braces around initialization of subobject [-Wmissing-braces]
2024/08/13 18:07:23 INFO static const BunString BunStringEmpty = BunString { BunStringTag::Empty, nullptr };
2024/08/13 18:07:23 INFO                                                                          ^~~~~~~
2024/08/13 18:07:23 INFO                                                                          {      }
2024/08/13 18:07:23 INFO /home/build/src/bun.js/bindings/helpers.h:188:74: warning: suggest braces around initialization of subobject [-Wmissing-braces]
2024/08/13 18:07:23 INFO static const BunString BunStringEmpty = BunString { BunStringTag::Empty, nullptr };
2024/08/13 18:07:23 INFO                                                                          ^~~~~~~
2024/08/13 18:07:23 INFO                                                                          {      }
2024/08/13 18:07:23 INFO /home/build/src/bun.js/bindings/webcore/SerializedScriptValue.cpp:2576:61: error: non-const lvalue reference to type 'VM' cannot bind to a value of unrelated type 'JSGlobalObject *const'
2024/08/13 18:07:23 INFO             JSMapIterator* iterator = JSMapIterator::create(m_lexicalGlobalObject, m_lexicalGlobalObject->mapIteratorStructure(), inMap, IterationKind::Entries);
2024/08/13 18:07:23 INFO                                                             ^~~~~~~~~~~~~~~~~~~~~
2024/08/13 18:07:23 INFO /home/build/src/bun.js/WebKit/WebKitBuild/Release/JavaScriptCore/PrivateHeaders/JavaScriptCore/JSMapIterator.h:68:38: note: passing argument to parameter 'vm' here
2024/08/13 18:07:23 INFO     static JSMapIterator* create(VM& vm, Structure* structure, JSMap* iteratedObject, IterationKind kind)
2024/08/13 18:07:23 INFO                                      ^
2024/08/13 18:07:23 INFO /home/build/src/bun.js/bindings/webcore/SerializedScriptValue.cpp:2624:61: error: non-const lvalue reference to type 'VM' cannot bind to a value of unrelated type 'JSGlobalObject *const'
2024/08/13 18:07:23 INFO             JSSetIterator* iterator = JSSetIterator::create(m_lexicalGlobalObject, m_lexicalGlobalObject->setIteratorStructure(), inSet, IterationKind::Keys);
2024/08/13 18:07:23 INFO                                                             ^~~~~~~~~~~~~~~~~~~~~
2024/08/13 18:07:23 INFO /home/build/src/bun.js/WebKit/WebKitBuild/Release/JavaScriptCore/PrivateHeaders/JavaScriptCore/JSSetIterator.h:68:38: note: passing argument to parameter 'vm' here
2024/08/13 18:07:23 INFO     static JSSetIterator* create(VM& vm, Structure* structure, JSSet* iteratedObject, IterationKind kind)
2024/08/13 18:07:23 INFO                                      ^
2024/08/13 18:07:23 INFO /home/build/src/bun.js/bindings/webcore/SerializedScriptValue.cpp:2814:15: warning: field 'm_identifier' will be initialized after field 'm_string' [-Wreorder-ctor]
2024/08/13 18:07:23 INFO             : m_identifier(identifier)
2024/08/13 18:07:23 INFO               ^~~~~~~~~~~~~~~~~~~~~~~~
2024/08/13 18:07:23 INFO               m_string(identifier.string())
2024/08/13 18:07:23 INFO 4 warnings and 2 errors generated.