5.35.6 Cross compile issue, byteorder

[jedavies-dev]

This happens for me on 5.35.6, but I can't get most previous version to cross build either. The most recent version I can cross build successfully is 5.32.1.

Error:

syntax error at lib/Config_heavy.pl line 1468, near "x;"
Compilation failed in require at configpm line 1144.
written lib/Config.pod
updated lib/Config.pm
updated lib/Config_heavy.pl
make[1]: *** [Makefile:207: lib/Config.pm] Error 255
make[1]: Leaving directory '/home/pokey/.cache/kiss/proc/86046/build/perl'
make: *** [Makefile:80: all] Error 2

Build system: powerpc64le-linux-musl
Target system: aarch64-linux-musl

...but this happens when targetting other architectures also. Doing a native build is fine.

lib/Config_heavy.pl line 1468 is:

our $byteorder = '?'x;

I have tried setting -Dbyteorder=12345678 in my config, it makes no difference if present or not.

One difference on my system is that it is using clang as the compiler, and no gcc or binutils are present. Not sure if this will affect things?

[arsv]

Can you check config.log and config.sh? Specifically the values for byteorder, ivsize and ivtype.

I'm looking through the code that generates Config_heavy.pm and that '?'x makes me think ivsize is not being set.

[jedavies-dev]

From config.log:

Guessing byte order
| #include <stdint.h>
| #include <sys/types.h>
| unsigned long foo = 0x8877665544332211;
> cc -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -c -o try.o try.c
> objdump -j .data -j .sdata -s try.o
Setting byteorder=12345678
Result: 12345678

Choosing C types to be used for perl internal types
Setting ivtype=long
Setting uvtype=unsigned long
Setting ivsize=8
Setting uvsize=8
Setting nvtype=double
Setting nvsize=8
        IV will be long, 8 bytes
        UV will be unsigned long, 8 bytes
        NV will be double, 8 bytes
Guessing nv_preserves_uv_bits value
Setting nv_preserves_uv_bits=53
Result: 53

[jedavies-dev]

Here are my config settings:

CFLAGS="-march=armv8-a -mcpu=cortex-a53 -O3 -fPIC --target=aarch64-linux-musl --sysroot=/path/to/aarch64/root/usr -resource-dir=/path/to/aarch64/root/usr/lib/clang/13.0.0 -fPIC"
LDFLAGS="--sysroot=/path/to/aarch64/root/usr -L/path/to/aarch64/root/usr/lib"

./configure \
    --prefix=/usr \
    --build=powerpc64le-linux-musl \
    --target=aarch64-linux-musl \
    --sysroot=/path/to/aarch64/root \
    -Des \
    -Dvendorprefix=/usr \
    -Dusevendorprefix \
    -Duseshrplib \
    -Dusesoname \
    -Dusethreads \
    -Dccflags="$CFLAGS" \
    -Dccdlflags="$CFLAGS" \
    -Dcccdlflags="$CFLAGS" \
    -Dlddlflags="$CFLAGS -shared" \
    -Dshrpldflags="$CFLAGS" \
    -Dldflags="$CFLAGS $LDFLAGS" \
    -Dlkflags="$CFLAGS" \
    -Doptimize="-Wall" \
    -Dusecrosscompile \
    -Dar="ar" \
    -Dnm="$nm" \
    -Dranlib="ranlib" \
    -Dreadelf="readelf" \
    -Dobjdump="objdump" \
    -Dinc_version_list=none \
    -Dd_sockaddr_in6=define \
    -Dcccdlflags=-fPIC \
    -Dlddlflags="-shared $CFLAGS $LDFLAGS" \
    -Dperl_static_inline="static __inline__" \
    -Dd_static_inline

I also tried adding -Dbyteorder="1234", -Divtype="long", -Divsize="8" to the config, but got the same error.

[arsv]

There should be two relevant sections in config.log, one for the host and another one for the target. The quoted snippets are the host part, the target should have aarch64-linux-musl-gcc instead of cc, and it's going to be further down in the log. It is the target values from config.sh that go into Config_heavy.pl. The host values go into xconfig.sh and only affect miniperl.

./miniperl_top configpm is the command that writes Config_heavy.pl, you can run it at any time as long as you have miniperl and config.sh.

[jedavies-dev]

In this case there is no gcc present on the system. The same clang binary is used for both the host and the target.

Here are the values for the target from config.log:

Choosing C types to be used for perl internal types
Setting ivtype=long
Setting uvtype=unsigned long
Setting ivsize=8
Setting uvsize=8
Setting nvtype=double
Setting nvsize=8
        IV will be long, 8 bytes
        UV will be unsigned long, 8 bytes
        NV will be double, 8 bytes
Guessing nv_preserves_uv_bits value
Setting nv_preserves_uv_bits=53
Result: 53

Deciding whether nv preserves full uv
Setting d_nv_preserves_uv=undef
Result: no

Checking integer capacity of nv
Setting nv_overflows_integers_at=256.0*256.0*256.0*256.0*256.0*256.0*2.0*2.0*2.0*2.0*2.0
Result: double

Guessing byte order
| #include <stdint.h>
| #include <sys/types.h>
| unsigned long foo = 0x8877665544332211;
> clang -march=armv8-a -mcpu=cortex-a53 -O3 -fPIC --target=aarch64-linux-musl --sysroot=/path/to/aarch64/root/usr -resource-dir=/path/to/aarch64/root/usr/lib/clang/13.0.0 -fPIC -DNO_POSIX_2008_LOCALE -D_GNU_SOURCE -fno-lto -fPIC -c -o try.o try.c
> objdump -j .data -j .sdata -s try.o
Setting byteorder=12345678
Result: 12345678

From config.sh:

...
ivtype='long'
uvtype='unsigned long'
ivsize='8'
uvsize='8'
nvtype='double'
nvsize='8'
nv_preserves_uv_bits='53'
d_nv_preserves_uv='undef'
nv_overflows_integers_at='256.0*256.0*256.0*256.0*256.0*256.0*2.0*2.0*2.0*2.0*2.0'
byteorder='12345678'
...

[arsv]

So the values are correct. Then it's down to debugging miniperl configpm, trying to figure out why it's not generating Config_heavy.pl properly from valid inputs.

miniperl may be either misconfigured or miscompiled.
configpm is the second script miniperl runs, and the first script is much simpler.
So it all might be just an indication of a broken miniperl.

Side note, I have no idea whether clang works for perl. GCC needs -fwrapv -fno-strict-aliasing to produce usable code.

[lancethepants]

So I am also experiencing this issue with aarch64-musl, cross-compiling with GCC. From the OP's comments I tried 5.32.1 and that version is working for me.
Here is a backtrace where miniperl is segfaulting.

admin@RT-AX88U-6860:~/src/perl/perl-5.36.0# make
make crosspatch
make[1]: Entering directory '/mmc/src/perl/perl-5.36.0'
make[1]: Nothing to be done for 'crosspatch'.
make[1]: Leaving directory '/mmc/src/perl/perl-5.36.0'
make miniperl
make[1]: Entering directory '/mmc/src/perl/perl-5.36.0'
make[1]: 'miniperl' is up to date.
make[1]: Leaving directory '/mmc/src/perl/perl-5.36.0'
make dynaloader
make[1]: Entering directory '/mmc/src/perl/perl-5.36.0'
./miniperl_top make_patchnum.pl
GNU gdb (GDB) 11.2
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-openwrt-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /mmc/src/perl/perl-5.36.0/miniperl...
(gdb) run
Starting program: /mmc/src/perl/perl-5.36.0/miniperl -I/mmc/src/perl/perl-5.36.0/cnf/stub -I/mmc/src/perl/perl-5.36.0/cnf/cpan -I/mmc/src/perl/perl-5.36.0/cpan/AutoLoader/lib -I/mmc/src/perl/perl-5.36.0/dist/Exporter/lib -I/mmc/src/perl/perl-5.36.0/dist/Cwd -I/mmc/src/perl/perl-5.36.0/dist/Cwd/lib -I/mmc/src/perl/perl-5.36.0/cpan/ExtUtils-Command/lib -I/mmc/src/perl/perl-5.36.0/cpan/ExtUtils-Install/lib -I/mmc/src/perl/perl-5.36.0/cpan/ExtUtils-MakeMaker/lib -I/mmc/src/perl/perl-5.36.0/dist/ExtUtils-MakeMaker/lib -I/mmc/src/perl/perl-5.36.0/cpan/ExtUtils-Manifest/lib -I/mmc/src/perl/perl-5.36.0/cpan/File-Path/lib -I/mmc/src/perl/perl-5.36.0/cpan/Pod-Simple/lib -I/mmc/src/perl/perl-5.36.0/cpan/Pod-Escapes/lib -I/mmc/src/perl/perl-5.36.0/ext/File-Find/lib -I/mmc/src/perl/perl-5.36.0/ext/re -I/mmc/src/perl/perl-5.36.0/cpan/ExtUtils-Constant/lib -I/mmc/src/perl/perl-5.36.0/dist/ExtUtils-ParseXS/lib -I/mmc/src/perl/perl-5.36.0/dist/constant/lib -I/mmc/src/perl/perl-5.36.0/cpan/Getopt-Long/lib -I/mmc/src/perl/perl-5.36.0/cpan/Text-Tabs/lib -I/mmc/src/perl/perl-5.36.0/dist/Carp/lib -I/mmc/src/perl/perl-5.36.0/cpan/podlators/lib -I/mmc/src/perl/perl-5.36.0/cpan/parent/lib -I/mmc/src/perl/perl-5.36.0/cpan/version/lib -I/mmc/src/perl/perl-5.36.0/dist/Pod-Simple/lib -I/mmc/src/perl/perl-5.36.0/dist/if -I/mmc/src/perl/perl-5.36.0/lib make_patchnum.pl
warning: Unable to determine the number of hardware watchpoints available.
warning: Unable to determine the number of hardware breakpoints available.

Program received signal SIGSEGV, Segmentation fault.
0x00000055557d457c in Perl_scan_str (my_perl=0x7fb7f48020, start=0x7fb7f06d24 "() { $inline_all_bits };\n", keep_bracketed_quoted=0, keep_delims=0, re_reparse=0, delimp=0x0) at toke.c:11441
11441                              : * (U8 *) close_delim_str;
(gdb) backtrace
#0  0x00000055557d457c in Perl_scan_str (my_perl=0x7fb7f48020, start=0x7fb7f06d24 "() { $inline_all_bits };\n", keep_bracketed_quoted=0,
    keep_delims=0, re_reparse=0, delimp=0x0) at toke.c:11441
#1  0x00000055557b4f10 in yyl_sub (my_perl=0x7fb7f48020, s=0x7fb7f06d24 "() { $inline_all_bits };\n", key=218) at toke.c:5385
#2  0x00000055557c7304 in yyl_word_or_keyword (my_perl=0x7fb7f48020, s=0x7fb7f06d23 " () { $inline_all_bits };\n", len=3, key=218,
    orig_keyword=0, c=...) at toke.c:8520
#3  0x00000055557c8d0c in yyl_keylookup (my_perl=0x7fb7f48020, s=0x7fb7f06d23 " () { $inline_all_bits };\n", gv=0x0) at toke.c:8834
#4  0x00000055557cac34 in yyl_try (my_perl=0x7fb7f48020, s=0x7fb7f06d20 "sub () { $inline_all_bits };\n") at toke.c:9319
#5  0x00000055557cb9c8 in Perl_yylex (my_perl=0x7fb7f48020) at toke.c:9621
#6  0x0000005555634138 in Perl_yyparse (my_perl=0x7fb7f48020, gramtype=258) at perly.c:357
#7  0x00000055556942e4 in S_doeval_compile (my_perl=0x7fb7f48020, gimme=2 '\002', outside=0x0, seq=6, hh=0x0) at pp_ctl.c:3613
#8  0x0000005555697350 in S_require_file (my_perl=0x7fb7f48020, sv=0x7fb7f12e88) at pp_ctl.c:4424
#9  0x00000055556974c0 in Perl_pp_require (my_perl=0x7fb7f48020) at pp_ctl.c:4448
#10 0x00000055557fdcfc in Perl_runops_standard (my_perl=0x7fb7f48020) at run.c:41
#11 0x0000005555903b70 in Perl_call_sv (my_perl=0x7fb7f48020, sv=0x7fb7f12ed0, flags=13) at /mmc/src/perl/perl-5.36.0/perl.c:3057
#12 0x0000005555909068 in Perl_call_list (my_perl=0x7fb7f48020, oldscope=2, paramList=0x7fb7f42c20)
    at /mmc/src/perl/perl-5.36.0/perl.c:5136
#13 0x00000055558e4470 in S_process_special_blocks (my_perl=0x7fb7f48020, floor=42, fullname=0x7fb7f03130 "BEGIN", gv=0x7fb7f12f90,
    cv=0x7fb7f12ed0) at /mmc/src/perl/perl-5.36.0/op.c:12102
#14 0x00000055558e3ec4 in Perl_newATTRSUB_x (my_perl=0x7fb7f48020, floor=42, o=0x7fb7f0ee40, proto=0x0, attrs=0x0, block=0x7fb7f40e08,
    o_is_gv=false) at /mmc/src/perl/perl-5.36.0/op.c:12007
#15 0x00000055558da664 in Perl_utilize (my_perl=0x7fb7f48020, aver=1, floor=42, version=0x0, idop=0x7fb7f0ea18, arg=0x0)
    at /mmc/src/perl/perl-5.36.0/op.c:8960
#16 0x0000005555634d58 in Perl_yyparse (my_perl=0x7fb7f48020, gramtype=258) at /mmc/src/perl/perl-5.36.0/perly.y:370
#17 0x00000055559029e4 in S_parse_body (my_perl=0x7fb7f48020, env=0x0, xsinit=0x55555903e0 <xs_init>)
    at /mmc/src/perl/perl-5.36.0/perl.c:2536
#18 0x00000055559016c8 in perl_parse (my_perl=0x7fb7f48020, xsinit=0x55555903e0 <xs_init>, argc=30, argv=0x7fffffee98, env=0x0)
    at /mmc/src/perl/perl-5.36.0/perl.c:1827
#19 0x00000055555902fc in main (argc=30, argv=0x7fffffee98, env=0x7fffffef90) at miniperlmain.c:115

[arsv]

Noted. That particular function has been re-written between 5.32 and 5.36, however, the re-write was much closer to 5.36 than to 5.32, so that does not align with the failures.

Version 5.37 builds just fine on aarch64-glibc, no issues, which makes me think the failures might have something to do with musl. I don't have a musl build environment to test it right away, but at least aarch64 is something I can work with. Maybe I'll try to get a musl system running and investigate it a bit further.

[lancethepants]

I had just tried out 5.32 because of the OP's comment. I've now cross-compiled 5.35.9 and that seems to be working for me, so the breakage I'm experiencing seems to have happened between 5.35.9 and 5.36.0. Maybe my issue is a different one than the OP's.

[arsv]

Well I tried building 5.36.0 in a musl system. No issues, miniperl works, perl works. The system is OpenWRT 21.02.3 running on RPi3, native gcc 8.4.0, target aarch64-openwrt-linux-musl. No idea what's going on.

The new version of toke.c was introduced in 5.35.10. The old one did not use memmem, the new one does.
I don't think memmem would be the culprit, but there's frankly not much else going on in the code around that line.

Might be worth poking around with gdb when it segfaults, just to get a better idea of what exactly goes wrong. Like what's the value of close_delim_str, where does tmps point relative to legal_paired_opening_delims, maybe probe other variables there.

[yu-re-ka]

I am seeing the same issue as the OP when cross-compiling 5.36.0 or 5.34.1 from x86_64 to riscv64 with gcc. 5.32.1 works fine.

[yu-re-ka]

Bisected the first failure to Perl/perl5@2e8a5b7.

       > Manifying 1 pod document
       > make[2]: *** [Makefile:423: manifypods] Bus error (core dumped)

Or in some revisions after Perl/perl5@b33c3c1 the error is slightly different

       > Manifying 1 pod document
       > make[2]: *** [Makefile:423: manifypods] Segmentation fault (core dumped)

Then, in Perl/perl5@bb38256 and later, the OP's error appears:

syntax error at lib/Config_heavy.pl line 1468, near "x;"
Compilation failed in require at configpm line 1144.

[yu-re-ka]

Changes in commit Perl/perl5@63da814 also seem to break something. Weird that there are so many different breakages within a short time frame.

[predators46]

@arsv
do you usually create a perl makefile for openwrt? the perl in openwrt is very old