fs: RuntimeError: requested new array is too large

[tonyfettes]

For a file that is larger than some limit (400M in my case), @fs.read_to_string! will report that the allocation is too large for the runtime.

Environment

• Backend: WasmGC

• Version:

  moon 0.1.20240910 (0a5e0dc 2024-09-10) ~/Library/MoonBit/bin/moon
  moonc v0.1.20240910+3af041b9a ~/Library/MoonBit/bin/moonc
  moonrun 0.1.20240910 (0a5e0dc 2024-09-10) ~/Library/MoonBit/bin/moonrun
  

Steps to reproduce

1. Create a project folder with following content:

   main.mbt:

   test {
     @fs.read_to_string!(~path="mnist_handwritten_train.json") |> ignore
   }

   moon.mod.json:

   {
     "name": "username/hello",
     "version": "0.1.0",
     "deps": {
       "moonbitlang/x": "0.4.7"
     },
     "readme": "README.md",
     "repository": "",
     "license": "Apache-2.0",
     "keywords": [],
     "description": ""
   }

   moon.pkg.json:

   {
     "import": [
       "moonbitlang/x/fs"
     ]
   }

2. Download a large file.

   curl -LO https://github.com/lorenmh/mnist_handwritten_json/raw/master/mnist_handwritten_train.json.gz
   

3. Decompress the gzip-ed JSON file.

   gzip -d mnist_handwritten_train.json.gz
   

4. Install dependencies

   moon install

5. Run the test

   $ moon test
   test tonyfettes/torch/internal/mnist/test.mbt::0 failed: RuntimeError: requested new array is too large
       at moonbit.bytes_make (wasm://wasm/00034692:wasm-function[15]:0x92f)
       at Bytes::new.fn/13 (wasm://wasm/00034692:wasm-function[37]:0xc2a)
       at Buffer::grow_if_necessary.fn/20 (wasm://wasm/00034692:wasm-function[44]:0xe0b)
       at Buffer::write_char.fn/21 (wasm://wasm/00034692:wasm-function[45]:0xe42)
       at $moonbitlang/x/fs/internal/ffi.string_from_extern.fn/96 (wasm://wasm/00034692:wasm-function[127]:0x22a7)
       at $moonbitlang/x/fs._read_to_string.fn/99 (wasm://wasm/00034692:wasm-function[130]:0x22de)
       at $moonbitlang/x/fs.read_to_string.fn/101 (wasm://wasm/00034692:wasm-function[132]:0x2347)
       at $tonyfettes/torch/internal/mnist.__test_746573742e6d6274_0.fn/103 (wasm://wasm/00034692:wasm-function[136]:0x24e1)
       at $tonyfettes/torch/internal/mnist.__test_746573742e6d6274_0.clo/126 (wasm://wasm/00034692:wasm-function[23]:0xa31)
       at *main*/130 (wasm://wasm/00034692:wasm-function[149]:0x27bb)
   Total tests: 1, passed: 0, failed: 1.