Introspection for firecracker and crosvm

[halfadmin]

Amazon as well as Google have come up with some stripped down versions of KVM based virtualization. Will kvm-vmi work for those? If not, how would you estimate the porting effort?
Lots of stuff is moving in the cloud nowadays. Depending on what you run you can easily ensure that you start from a non compromised system . There is still the issue for runtime integrity protection in particular against advanced attacks and where you don't want or cannot run the analysis directly in the VM.

[Wenzel]

Hi @halfadmin,
That's a good question.

At the moment kvm-vmi only supports QEMU/KVM.

KVMi (the new KVM subsystem for introspection) opens new ioctls and expands the KVM interface.

The QEMU modifications are not that big.

cc @mdontu, @adlazar for more details

[mdontu]

There should not be any hard dependency on qemu. I have not taken a close look at firecracker or crosvm but I expect adding KVMI support to be a 3-4 month task (assuming some inevitable refactoring and patch ping-pong).

The basic KVMI flow is:

• create a connection to the introspection application (vsock)
• handle the handshake (version information, authorization token etc.)
• pass the connection to the kernel via an ioctl()
• pass the kernel an identifier for the VM (currently an UUID)
• monitor the connection for shutdown (by the kernel or introspection application)
• notify the introspection application about events such as: VM pause/suspend/snapshot and VM migrate. VM resume might be needed as well, but it can be aliased to VM power-on (ie. create a connection to the introspection application). A VM shutdown event might also be needed in certain circumstances (mostly for making it easier for the application to know when to do clean-up).

[Wenzel]

Thanks @mdontu.

Also cc @andreeaflorescu and @aghecenco from Firecracker's team, for your insights, if you are interested.

[andreeaflorescu]

Hey @mdontu @Wenzel. Let me open an issue in Firecracker to see if this is something we are interested in adding support for. We will discuss it in the team (on the Firecracker issue) and let you know.

[mtarral]

For crosvm, judging by the top 3 contributors, I can cc @danielverkamp, @zachreizner and @dgreid in this discussion.

[mtarral]

If you a are not familiar with the topic (Virtual Machine Introspection on KVM), I can suggest this presentation by @mdontu at the last KVM Forum:

Advanced VMI on KVM: A Progress Report
https://static.sched.com/hosted_files/kvmforum2019/f6/Advanced%20VMI%20on%20KVM%3A%20A%20progress%20Report.pdf