Cross built ILCompiler NuGet contains HostOS (Linux) ELFs not TargetOS (FreeBSD) ELFs

[Thefrank]

Overview:

When cross compiling runtime, the ILCompiler (and NuGet) that is produced contains HostOS (Linux) and not TargetOS (FreeBSD) ELFs.

Reproduction:

Using a docker container on Linux pulled from: https://raw.githubusercontent.com/dotnet/versions/master/build-info/docker/image-info.dotnet-dotnet-buildtools-prereqs-docker-main.json containing a FreeBSD ROOTFS and the most recent net9 preview tag from runtime: v9.0.0-preview.5.24306.7

docker run -e ROOTFS_DIR=/crossrootfs/$ROOTFSARCH -v ${BUILD_SOURCESDIRECTORY}/runtime:/runtime $DOTNETDOCKERCONTAINERUSED /runtime/build.sh -c ${{ parameters.buildType }} -cross -os freebsd -arch $ROOTFSARCH -ci /p:OfficialBuildId=$OFFICIALBUILDID --subset clr+mono+mono.manifests+tools+libs+host+packs

Expected behavior:

ELFs should be like other items generated for TargetOS:

$file crossgen2 
crossgen2: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.2, FreeBSD-style, BuildID[sha1]=54a7f1c2a4752435c2cffd15eeb959f609966907, stripped

Actual behavior:

The resulting runtime.freebsd-x64.Microsoft.DotNet.ILCompiler.9.0.0-preview.5.24306.7.nupkg contains Linux ELFs and FreeBSD libs.

$find ./ * | xargs file
./:                                           directory
./ILCompiler.RyuJit.pdb:                      Microsoft Roslyn C# debugging symbols version 1.0
./libclrjit_unix_x64_x64.so:                  ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=66177aebc4ab51f16fe1e6a5faa90a7ade09b674, stripped
./libclrjit_win_x86_x64.so:                   ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=79ecdf1053497bde0393928dee1a727bc6b6b6a1, stripped
./libclrjit_universal_arm_x64.so:             ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=b15d888e793cca18b6dd42b3b672f7144fbe45ec, stripped
./libjitinterface_x64.so:                     ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=02a2d4a17bcbd0ff35f3caca9252853f95529a3c, stripped
./ILCompiler.TypeSystem.pdb:                  Microsoft Roslyn C# debugging symbols version 1.0
./ILCompiler.DependencyAnalysisFramework.pdb: Microsoft Roslyn C# debugging symbols version 1.0
./ILCompiler.Compiler.pdb:                    Microsoft Roslyn C# debugging symbols version 1.0
./libclrjit_universal_arm64_x64.so:           ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=0b476dc684291af72ab673b77dccbbf7f386cbf8, stripped
./libclrjit_win_x64_x64.so:                   ELF 64-bit LSB shared object, x86-64, version 1 (FreeBSD), dynamically linked, for FreeBSD 13.2, BuildID[sha1]=a58827fd1b7ac68612408fa5e13c7db9091938a2, stripped
./ilc.pdb:                                    Microsoft Roslyn C# debugging symbols version 1.0
./ilc:                                        ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=41cb9e347020ad19b6402190528b968b6850c46f, stripped

Regression:

To the best of my knowledge, it has always been this way. This was uncaught until recently when I tried to use a cross built package to bootstrap a native build.

Known Workarounds:

None?

Other info:

TargetOS=linux only appears three places in the .binlog for the build that are after Evaluation. All three are from ILCompiler.cspoj : The first seems to come as a return from ResolveReadyToRunCompilers and the other two (_PrepareForReadyToRunCompilation) and (_CreateR2RImages) use it. NativeAotSupported is reassigned here:

runtime/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj

Line 17 in a5cc707

<NativeAotSupported Condition="'$(TargetOS)' == 'freebsd' and '$(CrossBuild)' == 'true'">false</NativeAotSupported>

but Crossgen2 is still used on the ilc binary and the process does not error from

runtime/src/tasks/Crossgen2Tasks/ResolveReadyToRunCompilers.cs

Lines 116 to 120 in a5cc707

	_targetPlatform != hostPlatform ||
	!GetCrossgenComponentsPaths())
	{
	Log.LogError(Strings.ReadyToRunTargetNotSupportedError);
	return false;

There is no "Property reassignment" note in the binlog when ResolveReadyToRunCompilers changes(?) the TargetOS=freebsd to linux

.binlog is not an allowed attachment type so hopefully a screenshot from the MSBuild Structured Log Viewer is enough to help explain what I am seeing.

The Crossgen2 project seems to avoid the ReadyToRun part during packaging:

runtime/src/installer/pkg/sfx/Microsoft.NETCore.App/Microsoft.NETCore.App.Crossgen2.sfxproj

Lines 52 to 53 in a5cc707

	<Target Name="RunPublishedCrossgen" AfterTargets="PublishCrossgen"
	Condition="'$(TargetOS)' == '$(HostOS)' and '$(TargetArchitecture)' == '$(BuildArchitecture)' and '$(CrossBuild)' != 'true'">

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[jkotas]

This was refactored in #103508 . Does the problem still exist in current main?

[Thefrank]

@jkotas

Just checked. It still exists as of e125e93

[jkotas]

Ok, here is the problem:

The regular build uses "last known good" toolchain (the one that comes from https://github.com/dotnet/runtime/blob/main/global.json#L3) for aot compilation of most managed tools and components. The one exception is crossgen2 that uses live built aot toolchain that leads to interesting problems every once in a while.

When cross-compiling from Linux to FreeBSD, there is no "last known good" aot toolchain that is able to cross-compile from Linux to FreeBSD. We end up doing Linux->Linux publish instead due to some incorrect build logic and produce binaries for wrong architecture as you have observed.

I can think about two options for addressing this:

• Enable use of live built aot toolchain for compilation of managed tools and components. The current prior art with crossgen2, is complicated and fragile. We had several attempts to extend it to other tools and components (e.g. Crossgen2 should publish with NativeAOT at the beginning of the build #71557) that we ended up not pursuing. I think that the proper solution would be a multi-stage build: We need to first build the compiler that is then going to be used to compile the shipping compiler. It should be opt-in option and we should keep using "last known good" to keep the default build simple. The source build bootstrapping does similar multi-stage build already, perhaps we can use some of the same controls.
• Disable aot compilation of the managed tools and components when there is no "last known good" toolchain. It may be simpler, but it would mean that the cross-compiling Linux to FreeBSD build does not have full fidelity with the native FreeBSD build.

[Thefrank]

Would disabling AOT for managed items still give FreeBSD (TargetOS) ELF's on components like ILCompiler?
If yes, that might be the simplest solution for the time being.

The multi-stage build sounds like the best solution for this issue. But, unless there is something I am missing, this issue seems limited to a niche case of cross compiling from Linux to FreeBSD/Illumos/Haiku/Other. If it helps other build scenarios for officially supported platforms then there is a better case for this but I am not familiar enough with intricacies here.

[jkotas]

unless there is something I am missing, this issue seems limited to a niche case of cross compiling from Linux to FreeBSD/Illumos/Haiku/Other

I think having the recipe for how to use live-built AOT compilers would be generally useful. We keep having long discussions about whether it is better to use live-built AOT compiler or last-known-good AOT compiler. So having both options and use the one that's more appropriate for given situation would be best.

Would disabling AOT for managed items still give FreeBSD (TargetOS) ELF's on components like ILCompiler?

It would require some extra build logic. Nothing impossible.

[jkotas]

cc @jkoritzinsky @lambdageek

[am11]

It would be nice to take #103375 (which is currently blocked on #104077) then tackle it in a single unified target. It is possible to decouple it from #104077, but not ideal; since cdac and future projects under src/native/managed will continue to require separate maintenance.

[Thefrank]

As there is no way currently to build an ILCompiler on Linux for FreeBSD is there anyway to build the ILCompiler without needing a prior version of it natively? I tried poking around the various --subset items but none of them wanted to build ILCompiler without a prior version. Am I missing something?

[jkotas]

Yes, that's correct. The ILCompiler is AOT-compiled using prior version ("last known good" version in my comment above). The potential ways to break this cycle is either disable AOT compilation of the ILCompiler; or do the multi-stage build thing.

[am11]

Just noticed that targetInfo wasn't correctly set for FreeBSD, opened dotnet/sdk#42144. (i.e. 70e1072 was overwritten as "Linux" by the SDK)

Whether or not it fix this particular issue, I think it's good to let runtime control platform resolution in one place.

[Thefrank]

@jkotas When doing a native build, turning off AOT for ILC (and other items?) via /p:PublishAot=false /p:UseNativeAotForComponents=false allows the building of ILCompiler without a prior version. While it runs, it does not appear to create usable binaries:

  oob -> Trimming freebsd-x64 out-of-band assemblies with ILLinker...
  ##vso[task.setvariable variable=_librariesBuildProducedPackages]true
  Microsoft.DotNet.ILCompiler -> /root/runtime/artifacts/packages/Release/Shipping/Microsoft.DotNet.ILCompiler.9.0.0-preview.7.24364.99.nupkg
  Microsoft.DotNet.ILCompiler -> /root/runtime/artifacts/packages/Release/Shipping/runtime.freebsd-x64.Microsoft.DotNet.ILCompiler.9.0.0-preview.7.24364.99.nupkg
  Microsoft.NETCore.App.Ref -> 
  The package Microsoft.NETCore.App.Ref.9.0.0-preview.7.24364.99 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important.
  Successfully created package '/root/runtime/artifacts/packages/Release/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.7.24364.99.nupkg'.
  Successfully created package '/root/runtime/artifacts/packages/Release/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.7.24364.99.symbols.nupkg'.
  Microsoft.NETCore.App.Host -> 
  /root/runtime/artifacts/obj/Microsoft.NETCore.App.Host/Release/net9.0/freebsd-x64/output/ -> /root/runtime/artifacts/packages/Release/Shipping//dotnet-apphost-pack-9.0.0-preview.7.24364.99-freebsd-x64.tar.gz
  The package Microsoft.NETCore.App.Host.freebsd-x64.9.0.0-preview.7.24364.99 is missing a readme. Go to https://aka.ms/nuget/authoring-best-practices/readme to learn why package readmes are important.
  Successfully created package '/root/runtime/artifacts/packages/Release/Shipping/Microsoft.NETCore.App.Host.freebsd-x64.9.0.0-preview.7.24364.99.nupkg'.
  Successfully created package '/root/runtime/artifacts/packages/Release/Shipping/Microsoft.NETCore.App.Host.freebsd-x64.9.0.0-preview.7.24364.99.symbols.nupkg'.
  crossgen2_publish -> /root/runtime/artifacts/bin/crossgen2_publish/x64/Release/crossgen2.dll
  Generating native code
  Segmentation fault (core dumped)
/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/build/Microsoft.NETCore.Native.targets(309,5): error MSB3073: The command ""/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/ilc-published/ilc" @"/root/runtime/artifacts/obj/coreclr/crossgen2_publish/freebsd.x64.Release/native/crossgen2.ilc.rsp"" exited with code 139. [/root/runtime/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj]
##vso[task.logissue type=error;sourcepath=/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/build/Microsoft.NETCore.Native.targets;linenumber=309;columnnumber=5;code=MSB3073;](NETCORE_ENGINEERING_TELEMETRY=Build) The command ""/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/ilc-published/ilc" @"/root/runtime/artifacts/obj/coreclr/crossgen2_publish/freebsd.x64.Release/native/crossgen2.ilc.rsp"" exited with code 139.

Build FAILED.

/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/build/Microsoft.NETCore.Native.targets(309,5): error MSB3073: The command ""/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/ilc-published/ilc" @"/root/runtime/artifacts/obj/coreclr/crossgen2_publish/freebsd.x64.Release/native/crossgen2.ilc.rsp"" exited with code 139. [/root/runtime/src/coreclr/tools/aot/crossgen2/crossgen2_publish.csproj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:24:23.00
Build failed with exit code 1. Check errors above.

The segmentation fault is SIGSEGV: address not mapped to object (fault address: 0x0). This occurs without parameters too.

I will create another Issue for when I figure out what is going on.

[jkotas]

/root/runtime/artifacts/bin/coreclr/freebsd.x64.Release/ilc-published/ilc is published as PublishSingleFile. My guess is that PublishSingleFile is broken on FreeBSD. If it is the case, you should be able to reproduce this crash by publishing a hello world console app with PublishSingleFile too.

You can try turning off PublishSingleFile / PublishReadyToRun / PublishTrimmed in https://github.com/dotnet/runtime/blob/main/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj and see whether it fixes the crash.

[Thefrank]

A simple app builds and runs OK. Steps if I missed something:

./dotnet --version
9.0.100-preview.5.24307.3
---
./dotnet new console -o /root/test
The template "Console App" was created successfully.

Processing post-creation actions...
Restoring /root/test/test.csproj:
Restore succeeded.
 ---
./dotnet publish /root/test/test.csproj /p:PublishSingleFile=true /p:PublishReadyToRun=true /p:PublishTrimmed=true
Restore complete (4.0s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  test succeeded (21.4s) → /root/test/bin/Release/net9.0/freebsd-x64/publish/

Build succeeded in 27.7s
---
ls /root/test/bin/Release/net9.0/freebsd-x64/publish
test            test.pdb
---
/root/test/bin/Release/net9.0/freebsd-x64/publish/test 
Hello, World!

same results with combinations of /p:PublishSingleFile=true /p:PublishReadyToRun and /p:PublishTrimmed=true. No issues.

However /p:PublishSingleFile=true /p:PublishAOT=true builds fine but will crash:

./dotnet publish -c Release /root/test/test.csproj /p:PublishSingleFile=true /p:PublishAOT=true
Restore complete (6.8s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  test succeeded (11.1s) → /root/test/bin/Release/net9.0/freebsd-x64/publish/

Build succeeded in 20.4s
/root/test/bin/Release/net9.0/freebsd-x64/publish/test
Segmentation fault (core dumped)

/p:PublishAOT=true alone will also build without issue but crash when run

[am11]

PublishSingleFile alone and PublishReadyToRun alone also run? Note that you will need to delete obj/ dir in between or pass -t:Rebuild (changing command-line args alone doesn't evict msbuild's cache).

[Thefrank]

Yes

./dotnet publish -c Release /root/test/test.csproj /p:PublishSingleFile=true -t:Rebuild && /root/test/bin/Release/net9.0/freebsd-x64/publish/test
Restore complete (2.1s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  test succeeded (4.3s) → /root/test/bin/Release/net9.0/freebsd-x64/publish/

Build succeeded in 9.5s
Hello, World!

./dotnet publish -c Release /root/test/test.csproj /p:PublishReadyToRun=true -t:Rebuild && /root/test/bin/Release/net9.0/freebsd-x64/publish/test
Restore complete (2.5s)
You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy
  test succeeded (6.6s) → /root/test/bin/Release/net9.0/freebsd-x64/publish/

Build succeeded in 11.5s
Hello, World!

PublishReadyToRun required me adding the SDK location to /etc/dotnet/install_location

[am11]

The issue is we are trying to restore linux-x64 apphost to PublishSingleFile. #105004 switches PublishSingleFile to use the live host. @Thefrank could you give it a try?

[Thefrank]

@am11 yep! That allows crossbuild on Linux to output a FreeBSD ELF for ILCompiler. I will open another issue to address the output from ILC

[sec]

Just my few cents on that one, running on arm64 hardware :) I've applied both #105587 and #105004, did the cross build of preview.7 and tried to native build using that as bootstrap.
It fails, with segfault when running ilc, so I've tried dotnet publish /p:PublishSingleFile=true /p:PublishAOT=true with console app and it also fails:

(lldb) target create "/usr/home/sec/9p7/sdk/a/bin/Debug/net9.0/freebsd-arm64/publish/a"
Current executable set to '/usr/home/sec/9p7/sdk/a/bin/Debug/net9.0/freebsd-arm64/publish/a' (aarch64).
(lldb) r
Process 35342 launched: '/usr/home/sec/9p7/sdk/a/bin/Debug/net9.0/freebsd-arm64/publish/a' (aarch64)
Process 35342 stopped
* thread #1, name = 'a', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x0)
    frame #0: 0x000025072fd0cd50 libc.so.7`__guard_setup at stack_protector.c:78:6
   75            * not allow optimizer to call memcpy() behind us.
   76            */
   77           error = _elf_aux_info(AT_CANARY,
-> 78               __DEQUALIFY(void *, tmp_stack_chk_guard),
   79               sizeof(tmp_stack_chk_guard));
   80           if (error == 0 && tmp_stack_chk_guard[0] != 0) {
   81                   for (idx = 0; idx < nitems(__stack_chk_guard); idx++) {
(lldb) bt
* thread #1, name = 'a', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x0)
  * frame #0: 0x000025072fd0cd50 libc.so.7`__guard_setup at stack_protector.c:78:6
    frame #1: 0x000079d01ff3cd2c ld-elf.so.1`objlist_call_init(list=0x000025072bd53740, lockstate=<unavailable>) at rtld.c:3134:7
    frame #2: 0x000079d01ff3ba70 ld-elf.so.1`_rtld(sp=<unavailable>, exit_proc=<unavailable>, objp=<unavailable>) at rtld.c:979:5
    frame #3: 0x000079d01ff39118 ld-elf.so.1`.rtld_start at rtld_start.S:39

@am11 Should I try to recompile runtime (and/or whole SDK) with -p:LinkerFlavor=bfd to get more info on that?

edit: turns out that /mnt/sec/dotnet-core-freebsd-source-build/runtime/artifacts/bin/coreclr/freebsd.arm64.Release/ilc-published/ilc is doing segfault right at the start, while the one from cross-compiled nuget package works, but produce not working code?

edit 2: with ilc from cross nuget, runtime repo builds natively, that's something at least :)

[Thefrank]

Which FreeBSD version and which LLVM/Clang version?

[sec]

13.3-RELEASE-p3 and llvm18 (had the same with llvm15). Tried to native build runtime with -p:LinkerFlavor=bfd switch, but this does not changed anything. Also there's one version of ilc that does work from native build:

sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % find . -type f -name ilc -exec ls -al {} +
-rwxr--r--  1 sec  sec  16606596 Aug 19 16:56 ./.packages/runtime.freebsd-arm64.microsoft.dotnet.ilcompiler/9.0.0-preview.7.24405.7/tools/ilc
-rwxr-xr-x  1 sec  sec  14560696 Aug 20 11:58 ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc-published/ilc
-rwxr-xr-x  1 sec  sec     72480 Aug 20 11:57 ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc/ilc
-rwxr-xr-x  1 sec  sec  14560696 Aug 20 11:58 ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc/native/ilc
sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % ./.packages/runtime.freebsd-arm64.microsoft.dotnet.ilcompiler/9.0.0-preview.7.24405.7/tools/ilc|head -n 2
Required argument missing for command: 'ilc'.

Description:
  .NET Native IL Compiler
sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc-published/ilc|head -n 2
Segmentation fault (core dumped)
sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc/ilc|head -n 2
Required argument missing for command: 'ilc'.

Description:
  .NET Native IL Compiler
sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc/native/ilc|head -n 2
Segmentation fault (core dumped)

edit: hm, looks like ilc is build without symbol exported from #105587 when doing native build, will look into that.

[Thefrank]

./artifacts/bin/coreclr/freebsd.arm64.Release/ilc/ilc might be the same from ./.packages/runtime.freebsd-arm64.microsoft.dotnet.ilcompiler/9.0.0-preview.7.24405.7/tools/ilc which might explain why it works but not why there is a difference between the crossbuilt and native build ILC

[sec]

The size of working ones are different - native one is 72480 bytes and the one from cross nuget is 16606596. the other two, that don't work don't have symbols exported that added them on crossbuild - have build running with symbol export added without Condition="'$(_targetOS)' == 'freebsd' and '$(IsNativeExecutable)' == 'true'" to verify if this is the issue or is it somewhere else, will know if few minutes.

edit: it sill segfault and there's no progname and environ symbol exported

sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % nm --dynamic ./.packages/runtime.freebsd-arm64.microsoft.dotnet.ilcompiler/9.0.0-preview.7.24405.7/tools/ilc|grep prog
00000000009efaf8 D __progname
sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime % nm --dynamic ./artifacts/bin/coreclr/freebsd.arm64.Release/ilc-published/ilc|grep prog
sec@amper:/mnt/sec/dotnet-core-freebsd-source-build/runtime %

@am11 maybe you could point in right direction, why those symbol exports could be missing when doing native build of ILC?

[sec]

Could anyone explain why with the same target WriteIlcRspFileForCompilation, arguments list is different?
this one is missing export of __progname and environ:

while this one have them:

[am11]

@sec, I haven't tested it with freebsd-arm64, but if all things being equal (to x64) it should work the same way. I think the first run of the target is getting targetOS == linux for some reason in your environment.

[sec]

@sec, I haven't tested it with freebsd-arm64, but if all things being equal (to x64) it should work the same way. I think the first run of the target is getting targetOS == linux for some reason in your environment.

I have changed those conditions to <IlcArg Include="--export-dynamic-symbol:__progname" /> so they should always be included, but I'm missing something.

[am11]

I have changed those conditions to <IlcArg Include="--export-dynamic-symbol:__progname" /> so they should always be included, but I'm missing something.

Have you changed it in all find ~/.nuget -name '*Unix.targets'?

[sec]

Hm, based on your PR, you only added those to src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets

I tried to look where --export-dynamic-symbol:DotNetRuntimeDebugHeader is added and it's only also in this file and it's added in both runs.

Microsoft.NETCore.Native.Unix.targets don't contains any IlcArgs

[am11]

Ah I meant find ~/.nuget -name '*Native.targets'. I'll create a VM on osx arm64 and try to debug it on the weekend.

[sec]

Ok, I think I've found the issue, indeed I had some old nuget's cached - will confirm that in few minutes.

edit: yes, I had old Microsoft.DotNet.ILCompiler.9.0.0-previ ew.7.24405.7 inside runtime/.packages that didn't had those args added (that was from the crossbuild before ilc patch...) - thanks @am11 :)

full build of runtime/aspnet/sdk was completed under freebsd-arm64 :)

[Thefrank]

One of those IlcArg items in both projects might(should?) contain a --targetos: value. If it does read linux or that entry is not there, then you might have to fish around the log and find where it got misassigned.