Can't resolve 'module' error when importing npm module for the brower + wasm compile error

[frading]

Hello,

As I mentioned in the Users of Manifold discussion, I'm starting to integrate Manifold in Polygonjs. But I'm quickly facing issues, as importing the library generates the following error:

ERROR in ./node_modules/manifold-3d/manifold.js 8:8370-8386
Module not found: Error: Can't resolve 'module' in '.../polygonjs/node_modules/manifold-3d'
resolve 'module' in '.../polygonjs/node_modules/manifold-3d'
Parsed request is a module
using description file: .../polygonjs/node_modules/manifold-3d/package.json (relative path: .)
Field 'browser' doesn't contain a valid alias configuration

It seems to come from await import('module') inside node_modules/manifold-3d/manifold.js. So I can temporarily work around it by removing the whole block inside if (ENVIRONMENT_IS_NODE) {...}. (as I'm building for the browser, not for node).

Then the app builds, but loading the wasm file generates a second error:

Aborted(CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 3c 21 44 4f @+0). Build with -sASSERTIONS for more info.

And this is unfortunately not an error I'm able to debug.

So this brings me 2 questions:

1 - Would it be possible to have a build in the npm module that is only for the browser and does not include nodejs specific instructions? Alternatively, could you suggest modifications to my webpack config that could solve this?

2 - Is there anything I should change to load the .wasm without the error above?

This issue may need to be split into 2 for clarity, feel free to let me know if so, I'll happily do it.

You can see my attempt to integrate manifold in the integration-manifold branch of polygonjs, and specifically this file. You can also start it with yarn && yarn start and view any errors in the dev console when opening it on http://localhost:8080/example.html?id=nodes/sop/SDFBox.

Don't hesitate to let me know if you need more info from me

[pca006132]

This is weird, we do have tests running with nodejs in our CI:

manifold/.github/workflows/manifold.yml

Line 132 in dca9cce

npm test

What is the version for your nodejs?

[frading]

Thank you for your quick reply.

I'm on v16.15.1, but to confirm, I'm trying to build for the browser, not for node.

And to help debug this, I've created a test repo https://github.com/frading/manifold-3d-test, where all I've done is:

• scaffold the project using vite with yarn create vite
• add a config that uses vite wasm plugin
• try to import manifold in https://github.com/frading/manifold-3d-test/blob/main/src/main.ts with:

import Module from 'manifold-3d'

async function loadManifold(){
  console.log(Module);
  const wasm = await Module()
  console.log(wasm);
  wasm.setup();
}
loadManifold()

And in this test, the error I get is different: GET http://localhost:5173/node_modules/.vite/deps/manifold.wasm net::ERR_ABORTED 404 (Not Found)

So webpack and vite both try and import it differently. I'll keep investigate to try and see how I can tweak those configs, but curious to hear what you think in the mean time.

Another suggestion that comes to mind, is that it would help import it if it was possible to give the url where to load the .wasm from to the Module() call. So it would be something like Module({wasmUrl: '/my-custom-folder/manifold.wasm'}). That would avoid having to configure the bundler to handle .wasm, which is always tricky.

[pca006132]

OK I think I misunderstood the problem. From emscripten documentation, it seems that you can modify the compiler args to build it for the web (https://emscripten.org/docs/getting_started/FAQ.html#can-i-build-javascript-that-only-runs-on-the-web) and bundle it as a single file which hopefully makes it easier for webpack/vite to work with (https://emscripten.org/docs/getting_started/FAQ.html#how-do-i-run-a-local-webserver-for-testing-why-does-my-program-stall-in-downloading-or-preparing)

[frading]

ah, I did not realise the current .wasm was only for node. Ok, thanks a lot for those links, I'll give that a go and report back.

[pca006132]

It should not be just for node, but I have not used frontend tools for a while and have no idea why it does not work with webpack. Please let us know your results, and if this works we can add it to the README.

[elalish]

Our npm library definitely works in the browser, simply look at manifoldcad.org, which uses it this way. From the file you link, the strangest part to me is that you await the module import and setup the WASM inside the cook function that does the work. Certainly you wouldn't want to do this if you call it multiple times - you should set up the WASM once and then use it from then on. See for instance how we do it here for manifoldcad.org: https://github.com/elalish/manifold/blob/master/bindings/wasm/examples/worker.js

[frading]

you should set up the WASM once and then use it from then on

Yes, you're totally right, what I linked to is just a preliminary test to try and load your library. Once I got that working, it'll be cached.

But for now, my attempts to load it, in both repos I've linked, using webpack or vite, have failed. And unless I'm missing something obvious, I'm doing it the same way as the file you just pointed to.

[elalish]

@frading That is odd - I'm not familiar with webpack or vite; have you tried with vanilla JS? I wonder if they need some configuration for ES6 modules?

[jirihon]

Hi @frading, I'm successfully using Manifold with Vite in a SvelteKit project. You just need to point the Module function to the location of the manifold.wasm. See:

import Module from 'manifold-3d';
import manifold_wasm from 'manifold-3d/manifold.wasm?url';
const manifold_module = await Module({
    locateFile: () => manifold_wasm,
});
manifold_module.setup();

The ?url syntax in the import is explicit URL import. Not sure if Webpack has something similar.

See Manifold running in an example Svelte project using Vite at https://stackblitz.com/edit/manifold-vite?file=src%2FApp.svelte&terminal=dev

If you use TypeScript, it will complain that the Module function does not have any argument. This is an inaccuracy in the TypeScript definitions of the Manifold's JS bindings.

So, currently, Manifold npm package is not usable without this extra knowledge with Vite (and Webpack). I don't know yet how this should be done properly. I'll try to find some other package that is also using wasm and is working out-of-the-box with Vite, for inspiration. But definitely, a least the TypeScript definitions could be fixed easily, so it's clear for the TypeScript users there is the locateFile parameter that can be set (and should be set when using bundlers).

[elalish]

Thanks for the fix to our TS types for this!

[frading]

I took the liberty to unmark this as solution, hope you don't mind, as it's still a vite-only solution (I expand on this in my other comment #372 (comment) )

[jirihon]

Thanks for the fix to our TS types for this!

Could you please trigger new release to npm with the fix? How does this process work for Manifold? Is it enough to bump the version number in the package.json?

[elalish]

Yeah, I'll put out a new npm version before too long. In the meanwhile, you should be able to work around any TS type issue with //@ts-ignore.

[jirihon]

It seems that npm packages using wasm modules encouters the exact same problem. And providing the URL to the wasm file to the module load function is the best option. For example, here is a relevant documentation for OpenCascade.js: Configuring Your Bundler.

[elalish]

We should probably link to this documentation in our README somewhere...

[frading]

ok, I could make some progress here. I now have a version that's working in polygon, but it has a few caveats that I describe below.

And @jirihon thanks a lot for your suggestion for vite. Using locateFile to load the wasm does help solve the .wasm loading issue.

The 2 remaining problems are:

1. webpack should ignore the line const {createRequire: createRequire} = await import('module'); since it is inside a if (ENVIRONMENT_IS_NODE) {} block, and it won't be run in the browser, but I'm unsure what config would make that work. Currently, it parses this, tries to import a file called module.js, module.ts or module.json but fails to find one and doesn't compile.

There are other similar lines that are only for node, such as crypto_module=require("crypto"), but those can be handled by adding crypto: false in the fallback entry of the webpack config.

2. Similarly, webpack tries to load ./manifold.wasm after parsing the line wasmBinaryFile = new URL('manifold.wasm', import.meta.url).href;. Even with the loader recommended by opencascade, that I have here and by using locateFile, webpack blocks on it.

So the solution I currently have is to copy the files from the node_modules/manifold-3d, then remove from manifold.js the parts that webpack doesn't handle. You can see them here: https://github.com/polygonjs/polygonjs/tree/integration-manifold/src/core/geometry/sdf/manifold

And that solution works for me at the moment, although I'm not sure if that does comply with manifold's license. It would be good if you could confirm if it does, in which case I'll happily proceed like this. If not, I'd love to hear suggestions on how to improve the webpack config, if possible, as @jirihon message marked as a solution is a vite-only solution.

[elalish]

Thanks for the details - have you filed an issue on webpack? Sounds like an issue on their side, though they may also know of a workaround. I think licensing-wise it's fine; Apache 2.0 is a lot like MIT; I think you just need the usual - include the copyright and the AUTHORS list. Still, it seems pretty suboptimal from a code maintenance standpoint. Hopefully we can find a better way.

[frading]

Thanks a lot for confirming regarding the license, I really appreciate. And I completely agree that it's sub-optimal, I consider that a temporary work around.

I haven't thought about filing an issue with webpack, and I might, but that led me to check the existing issues and found this one: "Webpack doesn’t work well with wasm modules created with Emscripten". Several possible solutions are mentioned, using either a webpack config change or emscripten compilation flags. And this repo is linked, showing that both may be needed.

I'll see if I can at least implement the suggested webpack config changes in the coming days.

[Loosetooth]

I used the 'workaround' changes that @frading proposed in his latest comment to create an npm package that should be compatible with webpack, for use in the browser.

This is just to save other people that are using webpack some headache, for the time being.

[elalish]

Thanks for sharing! Is there anything we can do here to make this more compatible? It's odd that our npm package works just fine in both browsers and node.js, yet somehow isn't working in webpack. What advantage does webpack have?

[Loosetooth]

Some advantages of webpack are listed here. Webpack is used by a lot of web development frameworks, forcing projects that already use one of these frameworks to remain compatible with it. In other words, if you have an existing react, vue, nextjs -or other- project, and you would like to integrate manifold, it would need to be compatible with webpack.

I'm not an expert here, and I'm not sure what the best fix is for now, but I'm willing to spend some time in the future to figure this out.

[pca006132]

It seems that there is an open issue since 2018 in webpack. People documented workaround here: https://gist.github.com/surma/b2705b6cca29357ebea1c9e6e15684cc

[dsafa]

Came across the same issues with webpack so here's what worked for me.
Current setup is webpack 5 (from an ejected create react app project). the only issue was the error about Error: Can't resolve 'module'. Setting module: false in the resolve fallback in the webpack config was good enough for me to get it working. Since it will never hit that import in the browser, it should be fine.

resolve: {
  fallback: {
      module: false
   },
}

ymmv, since create react app setup has a bunch of other configurations.