Implement VirtIO GPU and input devices

[shengwen-tw]

Overview

This pull request adds basic virtio-gpu support for semu (conformed to spec. 1.2), which comprises the following changes:

• Implement virtio-gpu device with 2D rendering mode, in which the SDL2 library is used.
• Enable virtio-gpu kernel module
• Enable virtio-gpu in the device tree source
• Extend initrd region size to 35 MiB for containing virio-gpu moudles (~30 MiB)
• Enable libdrm from Buildroot setting
• Add shell script for packing kernel modules and launch script into the initrd image

Implemented Commands

• VIRTIO_GPU_CMD_GET_DISPLAY_INFO: Notifying the guest about the display size and state (i.e., enabled or disabled).
• VIRTIO_GPU_CMD_RESOURCE_CREATE_2D: Notifying the host to prepare data structure for storing image.
• VIRTIO_GPU_CMD_RESOURCE_UNREF: Notifying the host to destroy data structure of a 2D resource.
• VIRTIO_GPU_CMD_GET_EDID: Notifying the guest about the virtual connector's information (i.e., display).
• VIRTIO_GPU_CMD_GET_CAPSET_INFO: Notifying the guest the display capability sets.
• VIRTIO_GPU_CMD_SET_SCANOUT: Notifying the host to know which scanout (i.e., display) should link to which 2D resource.
• VIRTIO_GPU_CMD_RESOURCE_FLUSH: Notifying the host that the image data is ready to render.
• VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D: Notifying the host to copy the image data from the guest.
• VIRTIO_GPU_CMD_RESOURCE_ATTACH_BACKING: Notifying the host to record some memory pages (for storing image) for a 2D resource.

Test Procedures

1. Install SDL2:

$ sudo apt install libsdl2-dev libsdl2-2.0-0 libsdl2-image-dev libsdl2-image-2.0-0

2. Build kernel image:

$ ./scripts/build-image.sh

3. Build semu:

$ cd semu/
$ make

4. Pack virtio-gpu test image:

$ ./scripts/create-initrd-img.sh

5. Launch semu:

$ make check

6. After semu is launched, log in and execute run.sh to load the modules:

$ mkdir mnt
$ mount /dev/vda mnt
$ cd mnt
$ sh ./run.sh

7. After the modules are loaded, type:

$ modetest -M virtio_gpu -s 34:1024x768

A test image should then show up in the display window.

Reference output (text-only):

Welcome to Buildroot
buildroot login: root
# mkdir mnt
# mount /dev/vda mnt
[   21.086929] EXT4-fs (vda): recovery complete
[   21.087346] EXT4-fs (vda): mounted filesystem with ordered data mode. Quota mode: disabled.
# cd mnt
# sh ./run.sh
[   42.533320] [drm] features: -virgl +edid -resource_blob -host_visible
[   42.533399] [drm] features: -context_init
[   42.535654] [drm] number of scanouts: 1
[   42.535836] [drm] number of cap sets: 0
[   42.542754] [drm] Initialized virtio_gpu 0.1.0 0 for f4300000.virtio on minor 0
# modetest -M virtio_gpu -s 34:1024x768
setting mode 1024x768-60.00Hz on connectors 34, crtc 33
[   62.780039] virtio-mmio f4300000.virtio: [drm] drm_plane_enable_fb_damage_clips() not called
failed to set gamma: Function not implemented

[jserv]

How about using virglrenderer?
See https://mail.gnu.org/archive/html/qemu-devel/2021-02/msg04235.html

[shengwen-tw]

Yes, I do plan to support VirGL for acceleration.

As far as I know, the offloading of rendering operations to the host GPU is called 3D mode by VirtIO's spec, whereas this pull request only implemented 2D mode due to the overall complexity.

Before implementing 3D rendering mode with VirGL, I think we need to configure the Buildroot to have a minimal desktop environment or set the Mesa benchmark tools without a desktop manager (I'm not sure if this is feasible).

In conclusion, I'll recommend first merging this pull request as a checkpoint for virtio-gpu and then adding support for VirGL in a later pull request.

[shengwen-tw]

Reference: How to run headless unit tests for GUIs on GitHub actions

[jserv]

Rather than relying on Xvfb as a heavy-duty solution for validating Linux DRM/framebuffer, it might be more efficient to use lighter tools like drm-framebuffer. This tool, designed for testing drm devices, takes data from stdin and displays it, offering a simpler and more straightforward approach.

[shengwen-tw]

Rather than relying on Xvfb as a heavy-duty solution for validating Linux DRM/framebuffer, it might be more efficient to use lighter tools like drm-framebuffer. This tool, designed for testing drm devices, takes data from stdin and displays it, offering a simpler and more straightforward approach.

Actually, the intention of using Xvfb here is to make sure the SDL window can be launched in the CI environment by GitHub Action.

[jserv]

Merely executing Xvfb :1 -screen 0 1600x1200x24 is inadequate because, although it confirms the initialization of SDL with libX11, it does not guarantee the accuracy of the window content rendered in Xvfb.

[jserv]

Don't check in GPL-2.0-only code! Instead, extract the identifiers you need.

[shengwen-tw]

semu can now run DirectFB2 with DRM/KMS mode.

This is done by enforcing DirectFB2 to run with X8R8G8B8 format via code modification. (By default DirectFB2 uses A8R8G8B8 format but this failed to work so far, which requires further investigation)

DRM/KMS is crucial for integrating VirGL to the semu project as Linux framebuffer device does not trigger any 3D rendering operations.

[jserv]

Why did you check ENABLE_SDL here?

[jserv]

Don't change the above.

[jserv]

Change the configurations which are required. Minimize as possible as you can.

[jserv]

Merge the function into do_directfb.

[shengwen-tw]

semu can now run DirectFB2 with DRM/KMS mode.

This is done by enforcing DirectFB2 to run with X8R8G8B8 format via code modification. (By default DirectFB2 uses A8R8G8B8 format but this failed to work so far, which requires further investigation)

DRM/KMS is crucial for integrating VirGL to the semu project as Linux framebuffer device does not trigger any 3D rendering operations.

A change to the DirectFB2 is proposed for resolving the issue:
directfb2/DirectFB2#154

Note: The issue is now resolved as the pull request is accepted.

[shengwen-tw]

Opened issue caramelli/yagears#4 to discuss using yagears as a test program for integrating VirGL into semu.

Simultaneously, a pull request caramelli/yagears#3 is submitted to yagears for resolving cross-compiling error with semu's Buildroot setup.