-
Notifications
You must be signed in to change notification settings - Fork 15
inNative Tests
Usage: innative-test [-v] [-internal] [-benchmark] [-core] [test.wast]
inNative ships with a test harness that executes 3 stages of testing:
- Internal tests, intended to test inNative specific structures and non-standard webassembly extensions
- Benchmarks, which test the performance of a C++ version against several webassembly compiled versions of a function
- Standard webassembly tests, which are all the core
.wastfiles specified in the current specification implemented by inNative.
The internal tests and the benchmarks should always be compiled and pass, however the webassembly core tests can't all be tested on certain platforms. On windows, it is impossible to run the skip-stack-guard-page.wast test without encountering an unrecoverable critical error that immediately terminates the program. All platforms also cannot satisfy specific NaN bitpatterns because the underlying hardware simply doesn't preserve NaN bits after an operation is performed on them. All other webassembly tests currently pass.
The benchmarks return the number of microseconds that it took each function to complete. After all test cases have run, a second line displays a ratio consisting of the C++ runtime divided by that test case runtime. This means that the C++ function will always have a ratio of 1.00, and the webassembly versions will usually have something in the range of 0.6 to 0.98, depending on how close to native speed they get. Higher numbers are better.
To run all test stages, simply run innative-test with no arguments on any platform, (no command line arguments is equivilent to passing -internal -benchmark -core). It will be included in the SDK, but is absent from the redistributable, and is usually statically linked to whatever version of the runtime it's associated with. The test harness also accepts a few command line options.
This turns on verbose logging for all tests being run, which is intended for debugging. This will often result in a huge amount of logs, so don't turn this on unless it is necessary.
This will run the internal test stage, which is useful for verifying the internal components of inNative.
This runs the benchmarking stage, which compares the speed of various webassembly compilation settings to the native C++ version. These benchmarks are usually meaningless unless the test harness itself is compiled in release mode with all optimizations on and without a debugger attached.
This will run through all the WebAssembly core tests for the current platform.
If you provide the name (no path, just the filename) of a core webassembly test, the test harness will ignore all internal and benchmarks tests and instead run only that webassembly test file, unless you manually specify other test stages using -internal or -benchmark. This is useful if you are trying to debug a specific test failure but don't want to run the entire gauntlet.