date-3.4.0/lib/date_core.bundle : symbol not found in flat namespace '_rb_str_format'

[tykonu]

Hey!

I wanted to experiment with transferring a large-ish Rails app to truffleruby, but it seems to fail every time when building the Nokogiri gem.

OS: MacOS Sequoia 15.1 (24B83)
Ruby: truffleruby 24.1.1, like ruby 3.2.4, Oracle GraalVM Native [arm64-darwin20] (installed with rbenv)
Rails: Tried 7.0, 7.1, 7.2 and 8.0

Tried different Nokogiri versions as well, still the same issue.
Also tried truffleruby+graalvm-24.1.1, same issue.

It also fails when I install nokogiri separately with just gem install nokogiri

Could you help?

current directory: /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/nokogiri-1.16.7/ext/nokogiri
/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/bin/ruby extconf.rb --disable-static
checking for whether -std=c99 is accepted as CFLAGS... yes
checking for whether -Wno-declaration-after-statement is accepted as CFLAGS... yes
checking for whether -O2 is accepted as CFLAGS... yes
checking for whether -g is accepted as CFLAGS... yes
checking for whether -Winline is accepted as CFLAGS... yes
checking for whether -Wmissing-noreturn is accepted as CFLAGS... yes
checking for whether -Wshorten-64-to-32 is accepted as CFLAGS... yes
checking for whether -Wno-error=unused-command-line-argument-hard-error-in-future is accepted as CFLAGS... no
checking for whether -Wno-unknown-warning-option is accepted as CFLAGS... yes
Building nokogiri using packaged libraries.
Static linking is disabled.
Cross build is disabled.
checking for iconv.h... yes
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/bin/ruby
	--help
	--clean
	--prevent-strip
	--enable-system-libraries
	--disable-system-libraries
	--use-system-libraries
	--enable-system-libraries
	--disable-system-libraries
	--use-system-libraries
	--enable-static
	--disable-static
	--enable-cross-build
	--disable-cross-build
	--enable-cross-build
	--disable-cross-build
	--with-zlib-dir
	--without-zlib-dir
	--with-zlib-include
	--without-zlib-include=${zlib-dir}/include
	--with-zlib-lib
	--without-zlib-lib=${zlib-dir}/lib
	--with-iconv-dir
	--without-iconv-dir
	--with-iconv-include
	--without-iconv-include=${iconv-dir}/include
	--with-iconv-lib
	--without-iconv-lib=${iconv-dir}/lib

Contents of mkmf.log:
block in append_cflags: checking for whether -std=c99 is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -o conftest -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args conftest.c  -L.  -L/opt/homebrew/opt/[email protected]/lib         -L/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext -Wl,-rpath,/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext -ltrufflerubytrampoline"
2024-11-11 14:56:07.441 xcodebuild[4900:43945] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionSentinelHostApplications for extension Xcode.DebuggerFoundation.AppExtensionHosts.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
2024-11-11 14:56:07.442 xcodebuild[4900:43945] Requested but did not find extension point with identifier Xcode.IDEKit.ExtensionPointIdentifierToBundleIdentifier for extension Xcode.DebuggerFoundation.AppExtensionToBundleIdentifierMap.watchOS of plug-in com.apple.dt.IDEWatchSupportCore
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args  -std=c99 -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wno-declaration-after-statement is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99  -Wno-declaration-after-statement -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -O2 is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement  -O2 -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -g is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2  -g -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Winline is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g  -Winline -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wmissing-noreturn is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline  -Wmissing-noreturn -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wshorten-64-to-32 is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn  -Wshorten-64-to-32 -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wno-error=unused-command-line-argument-hard-error-in-future is accepted as CFLAGS... -------------------- no

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn -Wshorten-64-to-32  -Wno-error=unused-command-line-argument-hard-error-in-future -Werror -c conftest.c"
error: unknown warning option '-Werror=unused-command-line-argument-hard-error-in-future'; did you mean '-Werror=unused-command-line-argument'? [-Werror,-Wunknown-warning-option]
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

block in append_cflags: checking for whether -Wno-unknown-warning-option is accepted as CFLAGS... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn -Wshorten-64-to-32  -Wno-unknown-warning-option -Werror -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

--------------------

have_header: checking for iconv.h... -------------------- yes

LD_LIBRARY_PATH=. "clang -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include/ruby/backward -I/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/cext/include -I.  -D_DARWIN_C_SOURCE   -I/opt/homebrew/opt/[email protected]/include -fdeclspec -O3 -fno-fast-math -ggdb3 -Werror=implicit-function-declaration -Wno-int-conversion -Wno-int-to-pointer-cast -Wno-incompatible-pointer-types -Wno-format-extra-args -std=c99 -Wno-declaration-after-statement -O2 -g -Winline -Wmissing-noreturn -Wshorten-64-to-32 -Wno-unknown-warning-option   -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <iconv.h>
/* end */

--------------------

<internal:core> core/kernel.rb:229:in `gem_original_require': dlopen(/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date_core.bundle, 0x0009): symbol not found in flat namespace '_rb_str_format' (RuntimeError)
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
	from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date.rb:4:in `<top (required)>'
	from <internal:core> core/kernel.rb:229:in `gem_original_require'
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
	from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/time-0.4.1/lib/time.rb:4:in `<top (required)>'
	from <internal:core> core/kernel.rb:229:in `gem_original_require'
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
	from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/open-uri.rb:4:in `<top (required)>'
	from <internal:core> core/kernel.rb:229:in `gem_original_require'
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:149:in `require'
	from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/mini_portile2-2.8.7/lib/mini_portile2/mini_portile.rb:7:in `<top (required)>'
	from <internal:core> core/kernel.rb:229:in `gem_original_require'
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
	from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/mini_portile2-2.8.7/lib/mini_portile2.rb:2:in `<top (required)>'
	from <internal:core> core/kernel.rb:229:in `gem_original_require'
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
	from extconf.rb:431:in `process_recipe'
	from extconf.rb:871:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/extensions/arm64-darwin/3.2.4.24.1.0.1/nokogiri-1.16.7/mkmf.log

extconf failed, exit code 1

[eregon]

I cannot reproduce on Linux (it works fine) with:

$ gem install date:3.4.0
$ ruby -e 'require "date"; puts $"'
$ gem install nokogiri

At the end of your output I can see you have date-3.4.0 installed:

<internal:core> core/kernel.rb:229:in `gem_original_require': dlopen(/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date_core.bundle, 0x0009): symbol not found in flat namespace '_rb_str_format' (RuntimeError)
	from <internal:/Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/mri/rubygems/core_ext/kernel_require.rb>:97:in `require'
	from /Users/rasmus/.rbenv/versions/truffleruby-24.1.1/lib/gems/gems/date-3.4.0/lib/date.rb:4:in `<top (required)>'

And this sound like macOS is not respecting RTLD_LAZY, as in #3390
Do you have XCode 14.2? If not which version do you have?

FWIW, TruffleRuby does not implement rb_str_format() yet, but that's only used in date_strftime() -> date_strftime_with_tmx(), which is not called on require 'date' as far as I can tell so it seems to indicate that macOS is not respecting dlopen(path, RTLD_LAZY) here, or made it less lazy.
In Init_date_core there is https://github.com/ruby/date/blob/v3.4.0/ext/date/date_core.c#L9811 which is not calling that but taking the reference of a function indirectly using rb_str_format(), maybe that's enough on macOS to trigger the error?

In any case, we should add TruffleRuby in ruby/date's CI and implement rb_str_format(), however the fact it works fine on Linux means macOS might have broken something in their linker/dynamic loader.

[eregon]

So just to clarify, this has nothing to do with nokogiri, it's about the date gem, how it uses rb_str_format() and how the dynamic loader handles undefined symbols.

[tykonu]

@eregon thank you for pointing me in the right direction. Not sure which Xcode version I had, but I updated it to 16.1 and the problem was solved.

Could you also confirm what's the latest version of Rails that is supported on 24.1.1? I noticed the bundle was installed fine on Rails 7.1+ but the app didn't start, it raised some NoMethodErrors related to ActionCable 7.1+. Downgraded to Rails 7.0 and ActionCable 7.0 and the app launched correctly.

[eregon]

it raised some NoMethodErrors related to ActionCable 7.1+

Could you share those NoMethodErrors as a separate issue?

Generally any version of Rails should work, currently known issues are #3683 (comment) for Zeitwerk (applies to multiple Rails versions) and rails/rails#53490 (comment) (Rails 8).

[andrykonchin]

Just for the record.

Now there are the following failures in the date gem tests on TruffleRuby:

`Failure: test_to_time__from_datetime(TestDateConv)
.../date/test/date/test_date_conv.rb:95:in `test_to_time__from_datetime'
     92:     if Time.allocate.respond_to?(:subsec)
     93:       d = DateTime.new(2004, 9, 19, 1, 2, 3, 0) + 456789123456789123.to_r/86400000000000000000000
     94:       t = d.to_time.utc
  => 95:       assert_equal([2004, 9, 19, 1, 2, 3, Rational(456789123456789123,1000000000000000000)],
     96: 		   [t.year, t.mon, t.mday, t.hour, t.min, t.sec, t.subsec])
     97:     end
     98:   end
<internal:core> core/throw_catch.rb:36:in `catch'
<internal:core> core/throw_catch.rb:36:in `catch'
<[2004, 9, 19, 1, 2, 3, 456789123456789123/1000000000000000000]> expected but was
<[2004, 9, 19, 1, 2, 3, 456789123/1000000000]>

Failure: test__parse_too_long_year(TestDateParse)
.../date/test/date/test_date_parse.rb:594:in `test__parse_too_long_year'
     591:   def test__parse_too_long_year
     592:     str = "Jan 1" + "0" * 100_000
     593:     h = EnvUtil.timeout(3) {Date._parse(str, limit: 100_010)}
  => 594:     assert_equal(100_000, Math.log10(h[:year]))
     595:     assert_equal(1, h[:mon])
     596:
     597:     str = "Jan - 1" + "0" * 100_000
<internal:core> core/throw_catch.rb:36:in `catch'
<internal:core> core/throw_catch.rb:36:in `catch'
<100000> expected but was
<Infinity>

The reason of the first failure is that Time.new (that is used by DateTime#to_time) on TruffleRuby truncates seconds fraction to nanoseconds, so only 9 digits are stored. That's why 456789123456789123/1000000000000000000 becomes 456789123/1000000000. On CRuby Time instance stores not truncated subseconds.

The second failure occurs because on TruffleRuby Math.log10 accepts only Java double argument. And 10^100_000 value exceeds double's range of 4.9e-324 to 1.7e+308.

So it seems to we can just disable these tests on TruffleRuby.