Adding SymQEMU unit test suite to CI (eurecom-s3#44)

* Moved check-sym-runtime.c to unit tests and updated to work with QEMU8
* Build and run check-sym-runtime test with meson.build  
* Improve Dockerfile
* Added documentation about test suites

Dockerfile

@@ -1,4 +1,5 @@
-FROM ubuntu:22.04
+# prepare machine
+FROM ubuntu:22.04 as builder
 
 RUN apt update
 RUN apt install -y \
@@ -9,6 +10,9 @@ RUN apt install -y \
     python3 \
     python3-pip
 
+#
+FROM builder as symqemu
+
 COPY . /symqemu_source
 WORKDIR /symqemu_source
 
@@ -35,7 +39,6 @@ RUN ./configure                                                       \
 
 RUN make -j
 
-# Run QEMU base checks
 RUN make check
 
 WORKDIR /symqemu_source/tests/symqemu

README.md

@@ -102,14 +102,48 @@ cd ..
 
 Then build the SymQEMU image with (this will also run the tests):
 ```shell
-docker build -t symqemu  .
+docker build -t symqemu .
 ```
 
 You can use the docker with:
 ```shell
 docker run -it --rm symqemu
 ```
 
+## Contributing
+
+Use the GitHub project for reporting issues, and proposing changes.
+
+### Issues
+
+Please try to provide a minimal test case that demonstrates the problem, or ways
+to reproduce the behavior. If possible provide a precise line number if
+referring to some code. Ideally, make a PR with the test case demonstrating the
+failure (see next point).
+
+### Pull Requests
+
+Pull requests are very welcome. Pull requests will only be merged if all tests
+pass, and ideally with a new test case to validate the correctness of the
+proposed modifications. QEMU tests that are not specific to SymQEMU should pass
+(no regression).
+
+It is very valuable to also make a PR to add a test case for a known bug, this
+will facilitate correcting the issue.
+
+Current SymQEMU tests are run by the CI from the Docker container, the following
+test suites are currently in place:
+- [Unit tests](tests/unit/check-sym-runtime.c): Those tests are made to validate
+  specific instrumentation.
+- [Integration tests](tests/symqemu/): Those tests are running SymQEMU on a set
+  of binaries and compare the results to expected results. Note that those test
+  cases can legitimately fail if some changes are made to SymQEMU (because for
+  example, an improvement leads to generating new test cases). In that case,
+  update the relevant files in `expected_outputs` folders. It would be nice to
+  also validate those changes with a new test case.
+
+Also, refer to [QEMU's own tests suite documentation](https://www.qemu.org/docs/master/devel/testing.html).
+
 ## Documentation
 
 The [paper](http://www.s3.eurecom.fr/tools/symbolic_execution/symqemu.html) contains

tests/check-sym-runtime.c → tests/unit/check-sym-runtime.c

@@ -15,7 +15,9 @@
  */
 
 #include "qemu/osdep.h"
-#include "tcg.h"
+#include "tcg/tcg.h"
+#include "hw/i386/topology.h"
+#include "target/i386/cpu.h"
 #include "exec/helper-proto.h"
 
 #define SymExpr void*

tests/unit/meson.build

@@ -50,6 +50,7 @@ tests = {
   'test-qapi-util': [],
   'test-interval-tree': [],
   'test-xs-node': [qom],
+  'check-sym-runtime' : [qemuutil, qom, hwcore],
 }
 
 if have_system or have_tools
@@ -184,7 +185,32 @@ foreach test_name, extra: tests
     src += test_ss.all_sources()
     deps += test_ss.all_dependencies()
   endif
-  exe = executable(test_name, src, genh, dependencies: deps)
+  args =  []
+  lwith = []
+
+  # SymQEMU unit tests executable construction is a bit more complicated
+  if test_name == 'check-sym-runtime'
+    # lookup the libSymRuntime.so and add it as a dependence
+    libdir = meson.current_build_dir() / '../../' / config_host['SYMCC_BUILD']
+    symcc_runtime = cc.find_library('SymRuntime', dirs : libdir)
+    deps += [symcc_runtime]
+
+    # embeds most of qemu objects, including SymQEMU
+    lwith += [lib]
+    args +=  ['-I../target/i386/',
+	      '-I../../symcc/runtime/',
+	      '-DCONFIG_TARGET="x86_64-linux-user-config-target.h"',
+	      '-DNEED_CPU_H',
+	      '-Ix86_64-linux-user']
+
+    # Create the output file for symcc results
+    symcc_output=meson.current_build_dir() / 'symcc-tests-output'
+    test_env.append('SYMCC_OUTPUT_DIR', symcc_output)
+    r = run_command('mkdir', symcc_output, check : false)
+  endif
+
+  exe = executable(test_name, src, genh, dependencies: deps,
+                   c_args : args, link_with: lwith)
 
   test(test_name, exe,
        depends: test_deps.get(test_name, []),