Implement VirtIO GPU and input devices

[shengwen-tw]

Overview

This pull request adds basic virtio-gpu support for semu (conformed to spec. 1.2), which comprises the following changes:

• Implement virtio-gpu device with 2D rendering mode, in which the SDL2 library is used.
• Enable virtio-gpu kernel module
• Enable virtio-gpu in the device tree source
• Extend initrd region size to 35 MiB for containing virio-gpu moudles (~30 MiB)
• Enable libdrm from Buildroot setting
• Add shell script for packing kernel modules and launch script into the initrd image

Implemented Commands

• VIRTIO_GPU_CMD_GET_DISPLAY_INFO: Notifying the guest about the display size and state (i.e., enabled or disabled).
• VIRTIO_GPU_CMD_RESOURCE_CREATE_2D: Notifying the host to prepare data structure for storing image.
• VIRTIO_GPU_CMD_RESOURCE_UNREF: Notifying the host to destroy data structure of a 2D resource.
• VIRTIO_GPU_CMD_GET_EDID: Notifying the guest about the virtual connector's information (i.e., display).
• VIRTIO_GPU_CMD_GET_CAPSET_INFO: Notifying the guest the display capability sets.
• VIRTIO_GPU_CMD_SET_SCANOUT: Notifying the host to know which scanout (i.e., display) should link to which 2D resource.
• VIRTIO_GPU_CMD_RESOURCE_FLUSH: Notifying the host that the image data is ready to render.
• VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D: Notifying the host to copy the image data from the guest.
• VIRTIO_GPU_CMD_RESOURCE_ATTACH_BACKING: Notifying the host to record some memory pages (for storing image) for a 2D resource.

Test Procedures

1. Install SDL2:

$ sudo apt install libsdl2-dev libsdl2-2.0-0 libsdl2-image-dev libsdl2-image-2.0-0

2. Build kernel image:

$ ./scripts/build-image.sh

3. Build semu:

$ cd semu/
$ make

4. Pack virtio-gpu test image:

$ ./scripts/create-initrd-img.sh

5. Launch semu:

$ make check

6. After semu is launched, log in and execute run.sh to load the modules:

$ mkdir mnt
$ mount /dev/vda mnt
$ cd mnt
$ sh ./run.sh

7. After the modules are loaded, type:

$ modetest -M virtio_gpu -s 34:1024x768

A test image should then show up in the display window.

Reference output (text-only):

Welcome to Buildroot
buildroot login: root
# mkdir mnt
# mount /dev/vda mnt
[   21.086929] EXT4-fs (vda): recovery complete
[   21.087346] EXT4-fs (vda): mounted filesystem with ordered data mode. Quota mode: disabled.
# cd mnt
# sh ./run.sh
[   42.533320] [drm] features: -virgl +edid -resource_blob -host_visible
[   42.533399] [drm] features: -context_init
[   42.535654] [drm] number of scanouts: 1
[   42.535836] [drm] number of cap sets: 0
[   42.542754] [drm] Initialized virtio_gpu 0.1.0 0 for f4300000.virtio on minor 0
# modetest -M virtio_gpu -s 34:1024x768
setting mode 1024x768-60.00Hz on connectors 34, crtc 33
[   62.780039] virtio-mmio f4300000.virtio: [drm] drm_plane_enable_fb_damage_clips() not called
failed to set gamma: Function not implemented

[jserv]

How about using virglrenderer?
See https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg04235.html

[shengwen-tw]

Yes, I do plan to support VirGL for acceleration.

As far as I know, the offloading of rendering operations to the host GPU is called 3D mode by VirtIO's spec, whereas this pull request only implemented 2D mode due to the overall complexity.

Before implementing 3D rendering mode with VirGL, I think we need to configure the Buildroot to have a minimal desktop environment or set the Mesa benchmark tools without a desktop manager (I'm not sure if this is feasible).

In conclusion, I'll recommend first merging this pull request as a checkpoint for virtio-gpu and then adding support for VirGL in a later pull request.

[shengwen-tw]

Reference: How to run headless unit tests for GUIs on GitHub actions

[jserv]

Rather than relying on Xvfb as a heavy-duty solution for validating Linux DRM/framebuffer, it might be more efficient to use lighter tools like drm-framebuffer. This tool, designed for testing drm devices, takes data from stdin and displays it, offering a simpler and more straightforward approach.

[shengwen-tw]

Rather than relying on Xvfb as a heavy-duty solution for validating Linux DRM/framebuffer, it might be more efficient to use lighter tools like drm-framebuffer. This tool, designed for testing drm devices, takes data from stdin and displays it, offering a simpler and more straightforward approach.

Actually, the intention of using Xvfb here is to make sure the SDL window can be launched in the CI environment by GitHub Action.

[jserv]

Merely executing Xvfb :1 -screen 0 1600x1200x24 is inadequate because, although it confirms the initialization of SDL with libX11, it does not guarantee the accuracy of the window content rendered in Xvfb.

[shengwen-tw]

semu can now run DirectFB2 with DRM/KMS mode.

This is done by enforcing DirectFB2 to run with X8R8G8B8 format via code modification. (By default DirectFB2 uses A8R8G8B8 format but this failed to work so far, which requires further investigation)

DRM/KMS is crucial for integrating VirGL to the semu project as Linux framebuffer device does not trigger any 3D rendering operations.

[jserv]

Why did you check ENABLE_SDL here?

[shengwen-tw]

semu can now run DirectFB2 with DRM/KMS mode.

This is done by enforcing DirectFB2 to run with X8R8G8B8 format via code modification. (By default DirectFB2 uses A8R8G8B8 format but this failed to work so far, which requires further investigation)

DRM/KMS is crucial for integrating VirGL to the semu project as Linux framebuffer device does not trigger any 3D rendering operations.

A change to the DirectFB2 is proposed for resolving the issue:
directfb2/DirectFB2#154

Note: The issue is now resolved as the pull request is accepted.

[shengwen-tw]

Opened issue caramelli/yagears#4 to discuss using yagears as a test program for integrating VirGL into semu.

Simultaneously, a pull request caramelli/yagears#3 is submitted to yagears for resolving cross-compiling error with semu's Buildroot setup.

[shengwen-tw]

kmscube can be built directly with Buildroot.
The following image demonstrates kmscube running under virtio-gpu 2D mode:

The log message indicates that kmscube is using softpipe for rendering:

OpenGL ES 2.x information:
  version: "OpenGL ES 3.1 Mesa 24.0.9"
  shading language version: "OpenGL ES GLSL ES 3.10"
  vendor: "Mesa"
  renderer: "softpipe"
  extensions: "GL_EXT_blend_minmax GL_EXT_multi_draw_arrays GL_EXT_texture_filter_anisotropic GL_EXT_texture_compression_s3tc 
...

Since Mesa3D in Buildroot can be configured with VirGL support, further testing is needed to enable a hardware-accelerated pipeline. This might be achievable by setting:

export MESA_LOADER_DRIVER_OVERRIDE=virtio_gpu

# ls /usr/lib/dri/
kms_swrast_dri.so  swrast_dri.so      virtio_gpu_dri.so

However, kmscube currently tends to hang, requiring further investigation.

[jserv]

Rebase the latest master branch to pass CI pipeline for macOS.