Using Lucetc and Lucet-Runtime without hitting the filesystem

[DuncanUszkay1]

Hello, I'm trying to find a way to compile wasm using lucetc and run it with lucet_runtime without ever hitting the filesystem- ideally keeping the contents of the shared object file in a cache that I can directly load at runtime.

Currently the way I'm doing this involves reading and writing to temp files, but if there was a way to do this without hitting the filesystem that would be ideal.

Something like:

let bytes = lucetc::Lucetc.try_from_bytes(...).shared_object_bytes()
let module = lucet_runtime::DlModule::load_from_bytes(bytes)

If that makes sense, I wouldn't mind pitching in to add these methods

[iximeow]

So, I think this should be plausible, but I'm not sure adjusting DlModule to fit would be the way to do it - DlModule relies on libdl (via the rust libloading crate) to look up a few details and I don't think libdl itself has support for loading from a byte array.

That said, I'm fairly sure ModuleInternals::addr_details is the last place we depend on libdl, and it should be possible to rewrite that in terms of lucet_module::FunctionSpec. We also rely on the dynamic linker to resolve some relocations, but it's probably doable to reimplement that behavior using gimli to avoid needing dynamic linking (and the filesystem) at all.

These are essentially the same problems blocking a statically linked lucet module too, which we've had an open ticket for for a while now - if you want to get this sorted out, I'd be happy to work with you on it! As mentioned in #22 this all probably fits better under a not-DlModule name, like StaticModule.

[DuncanUszkay1]

Thanks for the quick response. I see it's not quite as trivial as I thought but I'd still like to take a stab at it when I get the time.