Test cases

[elliott-wen]

Hi, I am attempting to improve the JIT engine. So it would be really great if you can provide test cases for the RVVM.
I saw in the makefile that riscv-tests.tar.gz was used. I tried to compile my own one from https://github.com/riscv-software-src/riscv-tests. But my RVVM was not taking any of the input and just stuck.

[LekKit]

Here is my old pack of testcases, drop them into release directory and run make test

riscv-tests.tar.gz

I will later try to figure out how to run them in CI, since you don't just compile them, you also need to define how they start, and how they report test success (In RVVM case I just patched to report to UART at 0x10000000 and poweroff via syscon at 0x00100000).
So to automate this I probably need to write an auto-patch script.

Also, what kind of JIT improvements? I'm interested, thanks for your help and keep me in touch.

I recently tried to implement FPU JIT on x86_64 locally, but the major showstopper is the need to workaround many IEEE754 shortcomings on x86_64 which make it non-compliant to RISC-V spec (Specifically canonical NaNs propagation, many lacking exceptions which need to be raised according to tests). Have a look at src/cpu/riscv_f.c for an overview of those fixups if you need.

[LekKit]

Also, major changes to the interpreter are expected after stable v0.6 release, so I will try to merge your work to next staging window (v0.7-git) first to prevent breaking your patches.

Basically my initial idea to decode instructions using a function table is not covering some new extensions (V, B) well and their entries collide in the decoding table with themselves or older instructions, so most likely interpreter will be rewritten to a usual switch, or something else if anyone comes with a better idea

[elliott-wen]

Thank you for your swift response and the test cases. I can cetainly help write an automatic script to patch riscv-test-cases.

For some background, I teach operating systems at my university and have been using v86 to run Linux on a web page. This approach allows students to write, compile, and test their kernel modules without the need for virtual machines, which can be inconvenient for certain machine such as M2.

Considering the future potential of RISC-V, I'm looking into RVVM, and I'm impressed with its code quality. My goal is to implement JIT compilation for WebAssembly in RVVM, similar to what v86 does.

The switch construct is a text-book method for intepretation. I agree with you on that. 👍

[LekKit]

Ah I see, wonderful if you can manage to do that. Beware that RVJIT is mainly designed for register machines yet WASM is a stack machine, but I guess you can treat entries on stack like registers and then WASM will (hopefully) optimize that. I guess v86 should do some similar tricks and you can reuse them in RVJIT WASM backend

My further work on the interpreter also hopefully can improve WASM performance (Interpreter is important even when JIT is present, because it is used before JIT warmup)

[elliott-wen]

Thanks :D. I will play around with code and keep you posted.

The other trick is that a wasm module requires compilation before its execution.
It would be way too slow if we compile each basic block we encounter. We may need to maintain an execution counter and only if it reaches a certain threshold, we proceed to compilation.

[LekKit]

There are other Emscripten target limitations, see #71.

I am not very familiar with web technologies related to lazy file loading, but if someone brings info on how to implement function that: reads bytes at offset from a file on a web server, then I can implement web-backed disk images;

Also note on networking: Emscripten implementation of select() is broken, so tap_user networking won't work. We can implement some kind of TAP proxy that sends Ethernet frames over WebSocket, but it needs to be hosted somewhere.

[elliott-wen]

Hi, V86 has implemented web-backed disk images, we may be able to reuse their code directly. As far as I know, It is based on the virtio filesystem. V86 also implemented the virtual ethernet card, again based on virtio if I am not mistaken. Cheers Elliott

…

On Mon, 26 Feb 2024 at 08:33, LekKit ***@***.***> wrote: There are other Emscripten target limitations, see #71 <#71>. I am not very familiar with web technologies related to lazy file loading, but if someone brings info on how to implement function that: reads bytes at offset from a file on a web server, then I can implement web-backed disk images; Also note on networking: Emscripten implementation of select() is broken, so tap_user networking won't work. We can implement some kind of TAP proxy that sends Ethernet frames over WebSocket, but it needs to be hosted somewhere. — Reply to this email directly, view it on GitHub <#125 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLPC3ER26N3XHSIQUYJ2WDYVOGXPAVCNFSM6AAAAABDYR4A66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRTGAZTQMZRGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[LekKit]

V86 also implemented the virtual ethernet card, again based on virtio if I am not mistaken.

We already have an ethernet card, just that accessing internet from Emscripten is problematic and won't work without an additional public proxy

[elliott-wen]

Thanks for all the hard work. RVVM looks very promising.

[LekKit]

Good news: My new interpreter (Which I wrote locally and is still WIP) is around 2x faster on WASM/Emscripten (And marginally faster on native targets, where most useful ones have a JIT anyways). So this new interpreter hopefully will go into the new v0.7-git branch

[elliott-wen]

Wonderful achievement. :D Hooray

…

On Sun, 17 Mar 2024 at 19:34, LekKit ***@***.***> wrote: Good news: My new interpreter (Which I wrote locally and is still WIP) is around 2x faster on WASM/Emscripten (And marginally faster on native targets, where most useful ones have a JIT anyways). So this new interpreter hopefully will go into the new v0.7-git branch — Reply to this email directly, view it on GitHub <#125 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLPC3CSJWVV4FOU54G6G43YYU2PVAVCNFSM6AAAAABDYR4A66VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBSGMZTANJRHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[LekKit]

The new interpreter is now in staging branch. Be aware that it's not tested widely enough to be considered production ready, but it passes riscv-tests, except for 1 privileged test which is unrelated (But I'm working on it), and it worked with every image I've thrown at it for now.

I now also provide my own fork of riscv-tests with patches for reporting results under RVVM: https://github.com/LekKit/riscv-tests. I will soon upload prebuilt test archive to releases in that repo, and implement automatic tests downloading inside the Makefile.

[LekKit]

Test rv64mi-p-illegal now passes. There are other new tests in my forked tree, which fail because there isn't an extension implemented which they are testing (Bitmanip, Zicbom, etc). So now that these tests exist I can begin implementing more extensions and be sure they are compliant.