Uncaught exception - load.c(237) : Failed to load library : std.ndll

[e41q]

Hello. I'm trying to use neko nightly build for windows.
But when I using command lime test neko, i see this error.

But if I use neko what installed with haxe (2.0.0 I suppose) everything is ok.

[ppenzin]

I just got the same error while trying to build latest commit into a FreeBSD port (I am taking a look).

[ppenzin]

I apologize, not the same error, I will open an issue once I get more insight on it (so far I can see a potential problem with install.neko, but that is most likely unrelated).

[ncannasse]

Most likely you have a new std.ndll which loads an old neko.dll

[stevemayhew]

I see this problem on OS X too, with the pre-built 64 bit binaries of neko from the downloads page

First, neko is built with an RPATH

$ otool -L -v neko
neko:
	@rpath/libneko.2.dylib (compatibility version 2.0.0, current version 2.2.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)

When packaged VM with nekotool runs, it fails to find and load std.ndll

 haxelib 
...
dyld: loaded: /usr/lib/libobjc.A.dylib
dyld: loaded: /usr/lib/libc++abi.dylib
dyld: loaded: /usr/lib/libc++.1.dylib
Uncaught exception - load.c(237) : Failed to load library : std.ndll (dlopen(std.ndll, 1): image not found)

Isn't the neko VM added by nekotool complete standalone?

[stevemayhew]

That is, the neko binary attached to haxelib should be complete with no external dependencies.

[andyli]

nekotools boot is effectively a merge of the neko executable with a neko bytecode file. It only removes the dependency of the neko executable.

[kulick]

Hi guys. Steve, the std.ndll is really part of the Haxe installation not the Nemo installation. I agree with Andy; if your binary won’t run because std.ndll is not found, the problem is really with the install/environment of Haxe, not neko.

[andyli]

Ummm... not quite. The ndll files are parts of Neko. The official Haxe Windows/Mac installers also install Neko too, so people will get the ndll files when installing Haxe. However, the Haxe binary archives (.zip, .tar.gz) do not include Neko, so users will have to also get the Neko binary archives separately.

[stevemayhew]

Folks... Thanks for the tips. The issue is that the build of neko does not give a list for RPATH, just the executable path:

Load command 14
          cmd LC_RPATH
      cmdsize 32
         path @executable_path/ (offset 12)

standalone 'neko' command works fine (cause images, ndll and dyld are all in the same folder). For executables created by nekotool -boot, not so much, cause the library would need to be copied with the built binary (which defeats the purpose of packaging the VM and byte code together)

I see the loader code in neko looks for a path NEKOPATH and defaults to a reasonable path list for linux at least:

	if( path == NULL ) {
		allocated = strdup(NEKO_MODULE_PATH ":/usr/local/lib/neko:/usr/lib/neko:/usr/local/bin:/usr/bin");
		path = allocated;
	}

However none of that does any good for the OS-X loader, which cannot load the neko boot created object:

$ otool -l -v  /usr/local/lib/neko-2.2.0-osx64/neko | less 
stevemayhewmbpro15:srcroot smayhew$ 
stevemayhewmbpro15:srcroot smayhew$ 
stevemayhewmbpro15:srcroot smayhew$ echo $NEKOPATH 
/usr/local/lib/neko-2.2.0-osx64
stevemayhewmbpro15:srcroot smayhew$ neko
dyld: loaded: /usr/local/lib/neko-2.2.0-osx64/neko
dyld: loaded: /usr/local/lib/neko-2.2.0-osx64/libneko.2.dylib
dyld: loaded: /usr/lib/libSystem.B.dylib
dyld: loaded: /usr/lib/system/libcache.dylib
dyld: loaded: /usr/lib/system/libcommonCrypto.dylib
dyld: loaded: /usr/lib/system/libcompiler_rt.dylib
dyld: loaded: /usr/lib/system/libcopyfile.dylib
dyld: loaded: /usr/lib/system/libcorecrypto.dylib
dyld: loaded: /usr/lib/system/libdispatch.dylib
dyld: loaded: /usr/lib/system/libdyld.dylib
dyld: loaded: /usr/lib/system/libkeymgr.dylib
dyld: loaded: /usr/lib/system/liblaunch.dylib
dyld: loaded: /usr/lib/system/libmacho.dylib
dyld: loaded: /usr/lib/system/libquarantine.dylib
dyld: loaded: /usr/lib/system/libremovefile.dylib
dyld: loaded: /usr/lib/system/libsystem_asl.dylib
dyld: loaded: /usr/lib/system/libsystem_blocks.dylib
dyld: loaded: /usr/lib/system/libsystem_c.dylib
dyld: loaded: /usr/lib/system/libsystem_configuration.dylib
dyld: loaded: /usr/lib/system/libsystem_coreservices.dylib
dyld: loaded: /usr/lib/system/libsystem_coretls.dylib
dyld: loaded: /usr/lib/system/libsystem_dnssd.dylib
dyld: loaded: /usr/lib/system/libsystem_info.dylib
dyld: loaded: /usr/lib/system/libsystem_kernel.dylib
dyld: loaded: /usr/lib/system/libsystem_m.dylib
dyld: loaded: /usr/lib/system/libsystem_malloc.dylib
dyld: loaded: /usr/lib/system/libsystem_network.dylib
dyld: loaded: /usr/lib/system/libsystem_networkextension.dylib
dyld: loaded: /usr/lib/system/libsystem_notify.dylib
dyld: loaded: /usr/lib/system/libsystem_platform.dylib
dyld: loaded: /usr/lib/system/libsystem_pthread.dylib
dyld: loaded: /usr/lib/system/libsystem_sandbox.dylib
dyld: loaded: /usr/lib/system/libsystem_secinit.dylib
dyld: loaded: /usr/lib/system/libsystem_symptoms.dylib
dyld: loaded: /usr/lib/system/libsystem_trace.dylib
dyld: loaded: /usr/lib/system/libunwind.dylib
dyld: loaded: /usr/lib/system/libxpc.dylib
dyld: loaded: /usr/lib/libobjc.A.dylib
dyld: loaded: /usr/lib/libc++abi.dylib
dyld: loaded: /usr/lib/libc++.1.dylib
NekoVM 2.2.0 (c)2005-2017 Haxe Foundation
  Usage : neko <file>
stevemayhewmbpro15:srcroot smayhew$ /sandbox/b-haxetools-mainline/dev-host_osx/toolroot/bin/haxelib 
dyld: loaded: /sandbox/b-haxetools-mainline/dev-host_osx/toolroot/bin/haxelib
dyld: Library not loaded: @rpath/libneko.2.dylib
  Referenced from: /sandbox/b-haxetools-mainline/dev-host_osx/toolroot/bin/haxelib
  Reason: image not found
Abort trap: 6

So IMHO, this is a bug

[bjitivo]

Can we add an option to nekoboot to use a statically linked version of neko? That would mean that nekoboot would have to have access to a statically linked version of neko, but the neko build/install process could be changed to generate a statically linked version and keep it somewhere for nekoboot to use.

This issue is significant for us at TiVo because we have a very hard time keeping the haxe toolchain going for Mac based builds because of numerous issues like this. The haxe tooling is especially fragile on Mac and anything that can be done to improve it, such as fixing this bug, would be a big help. Thanks.

[stevemayhew]

Also, fixing it after the fact is impossible, because the nekotool boot constructed binary is not valid MachO format:

 install_name_tool -rpath @executable_path/ $NEKOPATH /sandbox/b-haxetools-mainline/dev-host_osx/toolroot/bin/haxelib 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: the __LINKEDIT segment does not cover the end of the file (can't be processed) in: /sandbox/b-haxetools-mainline/dev-host_osx/toolroot/bin/haxelib

This will also prevent code signing on Mac (which may become mandatory at some point). Putting the byte code in a valid segment might be a solution (https://stackoverflow.com/questions/1604673/how-do-i-embed-data-into-a-mac-os-x-mach-o-binary-files-text-section?rq=1)

I think the option Brian (@bjitivo ) suggested is best, include all the code in the nekotool boot created executable.

[andyli]

I think there are still some misconceptions here.

The use of rpath/executable_path is for finding libneko, but not the ndll files.
ndll loading can be configured by using NEKOPATH.

So, for a nekotools boot executable to work, you should put libneko next to the executable, and then set NEKOPATH as the path to the folder of the ndll files (or place the ndll next to the executable as well, which works on Windows, but I'm not sure for Mac).
I understand it is not really useful, but it is how it works. There is no simple method that I know to improve.
At best, we can static link libneko, but since ndll files are dynamic libraries in nature, they can't be statically linked.

It is also a known issue that the output binary is of invalid format. (See #130)
Nicolas mentioned he may had find a proper way in HaxeFoundation/hashlink#66.

[bjitivo]

I don't think the .ndll files are the real problem; as you said, all you need is a valid NEKOPATH to make those work. It's the shared libraries linked against the application that are the problem. It's really not a good solution to have to put share object libraries next to executables so that the operating system dynamic linker can find them.