From f9a2173c356d103c2cf86507e10c6dfcf78af496 Mon Sep 17 00:00:00 2001 From: Christophe de Dinechin Date: Tue, 27 Oct 2020 15:57:27 +0100 Subject: [PATCH 1/4] docs: Document restricted annotations Document restricted annotations, as implemented in https://github.com/kata-containers/runtime/pull/3005 Fixes: #754 Signed-off-by: Christophe de Dinechin (cherry picked from commit 4809aee3905fb9ebab6b7334c2ddf6921061a648) --- how-to/how-to-set-sandbox-config-kata.md | 45 +++++++++++++++++++++--- 1 file changed, 41 insertions(+), 4 deletions(-) diff --git a/how-to/how-to-set-sandbox-config-kata.md b/how-to/how-to-set-sandbox-config-kata.md index 50067000..32838e07 100644 --- a/how-to/how-to-set-sandbox-config-kata.md +++ b/how-to/how-to-set-sandbox-config-kata.md @@ -3,6 +3,11 @@ Kata Containers gives users freedom to customize at per-pod level, by setting a wide range of Kata specific annotations in the pod specification. +Some annotations may be [restricted](#restricted-annotations) by the +configuration file for security reasons, notably annotations that could lead the +runtime to execute programs on the host. Such annotations are marked with _(R)_ in +the tables below. + # Kata Configuration Annotations There are several kinds of Kata configurations and they are listed below. @@ -38,6 +43,8 @@ There are several kinds of Kata configurations and they are listed below. | `io.katacontainers.config.hypervisor.block_device_cache_noflush` | `boolean` | Denotes whether flush requests for the device are ignored | | `io.katacontainers.config.hypervisor.block_device_cache_set` | `boolean` | cache-related options will be set to block devices or not | | `io.katacontainers.config.hypervisor.block_device_driver` | string | the driver to be used for block device, valid values are `virtio-blk`, `virtio-scsi`, `nvdimm`| +| `io.katacontainers.config.hypervisor.cpu_features` | `string` | Comma-separated list of CPU features to pass to the CPU (QEMU) | +| `io.katacontainers.config.hypervisor.ctlpath` (R) | `string` | Path to the `acrnctl` binary for the ACRN hypervisor | | `io.katacontainers.config.hypervisor.default_max_vcpus` | uint32| the maximum number of vCPUs allocated for the VM by the hypervisor | | `io.katacontainers.config.hypervisor.default_memory` | uint32| the memory assigned for a VM by the hypervisor in `MiB` | | `io.katacontainers.config.hypervisor.default_vcpus` | uint32| the default vCPUs assigned for a VM by the hypervisor | @@ -48,7 +55,7 @@ There are several kinds of Kata configurations and they are listed below. | `io.katacontainers.config.hypervisor.enable_mem_prealloc` | `boolean` | the memory space used for `nvdimm` device by the hypervisor | | `io.katacontainers.config.hypervisor.enable_swap` | `boolean` | enable swap of VM memory | | `io.katacontainers.config.hypervisor.entropy_source` | string| the path to a host source of entropy (`/dev/random`, `/dev/urandom` or real hardware RNG device) | -| `io.katacontainers.config.hypervisor.file_mem_backend` | string | file based memory backend root directory | +| `io.katacontainers.config.hypervisor.file_mem_backend` (R) | string | file based memory backend root directory | | `io.katacontainers.config.hypervisor.firmware_hash` | string | container firmware SHA-512 hash value | | `io.katacontainers.config.hypervisor.firmware` | string | the guest firmware that will run the container VM | | `io.katacontainers.config.hypervisor.guest_hook_path` | string | the path within the VM that will be used for drop in hooks | @@ -59,7 +66,7 @@ There are several kinds of Kata configurations and they are listed below. | `io.katacontainers.config.hypervisor.initrd_hash` | string | container guest initrd SHA-512 hash value | | `io.katacontainers.config.hypervisor.initrd` | string | the guest initrd image that will run in the container VM | | `io.katacontainers.config.hypervisor.jailer_hash` | string | container jailer SHA-512 hash value | -| `io.katacontainers.config.hypervisor.jailer_path` | string | the jailer that will constrain the container VM | +| `io.katacontainers.config.hypervisor.jailer_path` (R) | string | the jailer that will constrain the container VM | | `io.katacontainers.config.hypervisor.kernel_hash` | string | container kernel image SHA-512 hash value | | `io.katacontainers.config.hypervisor.kernel_params` | string | additional guest kernel parameters | | `io.katacontainers.config.hypervisor.kernel` | string | the kernel used to boot the container VM | @@ -68,12 +75,14 @@ There are several kinds of Kata configurations and they are listed below. | `io.katacontainers.config.hypervisor.memory_offset` | uint32| the memory space used for `nvdimm` device by the hypervisor | | `io.katacontainers.config.hypervisor.memory_slots` | uint32| the memory slots assigned to the VM by the hypervisor | | `io.katacontainers.config.hypervisor.msize_9p` | uint32 | the `msize` for 9p shares | -| `io.katacontainers.config.hypervisor.path` | string | the hypervisor that will run the container VM | +| `io.katacontainers.config.hypervisor.path` (R) | string | the hypervisor that will run the container VM | +| `io.katacontainers.config.hypervisor.pcie_root_port` | specify the number of PCIe Root Port devices. The PCIe Root Port device is used to hot-plug a PCIe device (QEMU) | | `io.katacontainers.config.hypervisor.shared_fs` | string | the shared file system type, either `virtio-9p` or `virtio-fs` | | `io.katacontainers.config.hypervisor.use_vsock` | `boolean` | specify use of `vsock` for agent communication | +| `io.katacontainers.config.hypervisor.vhost_user_store_path` (R) | `string` | specify the directory path where vhost-user devices related folders, sockets and device nodes should be (QEMU) | | `io.katacontainers.config.hypervisor.virtio_fs_cache_size` | uint32 | virtio-fs DAX cache size in `MiB` | | `io.katacontainers.config.hypervisor.virtio_fs_cache` | string | the cache mode for virtio-fs, valid values are `always`, `auto` and `none` | -| `io.katacontainers.config.hypervisor.virtio_fs_daemon` | string | virtio-fs `vhost-user` daemon path | +| `io.katacontainers.config.hypervisor.virtio_fs_daemon` (R) | string | virtio-fs `vhost-user` daemon path | | `io.katacontainers.config.hypervisor.virtio_fs_extra_args` | string | extra options passed to `virtiofs` daemon | # Example - Using annotations @@ -131,3 +140,31 @@ spec: stdin: true tty: true ``` + +# Restricted annotations + +Some annotations are _restricted_, meaning that the configuration file specifies +the acceptable values. Currently, only hypervisor annotations are restricted, +for security reason, with the intent to control which binaries the Kata +Containers runtime will launch on your behalf. + +The configuration file validates the annotation _name_ as well as the annotation +_value_. + +The acceptable annotation names are defined by the `enable_annotations` entry in +the configuration file. + +For restricted annotations, an additional configuration entry provides a list of +acceptable values. Since most restricted annotations are intended to control +which binaries the runtime can execute, the valid value is generally provided by +a shell pattern, as defined by `glob(3)`. The table below provides the name of +the configuration entry: + +| Key | Config file entry | Comments | +|-------| ----- | ----- | +| `ctlpath` | `valid_ctlpaths` | Valid paths for `acrnctl` binary | +| `file_mem_backend` | `valid_file_mem_backends` | Valid locations for the file-based memory backend root directory | +| `jailer_path` | `valid_jailer_paths`| Valid paths for the jailer constraining the container VM (Firecracker) | +| `path` | `valid_hypervisor_paths` | Valid hypervisors to run the container VM | +| `vhost_user_store_path` | `valid_vhost_user_store_paths` | Valid paths for vhost-user related files| +| `virtio_fs_daemon` | `valid_virtio_fs_daemon_paths` | Valid paths for the `virtiofsd` daemon | From c43cf9ecdc7179150e7340654345335e4f3d3fd5 Mon Sep 17 00:00:00 2001 From: Christophe de Dinechin Date: Tue, 27 Oct 2020 16:06:49 +0100 Subject: [PATCH 2/4] docs: Add a link to document describing how to use annotations Add a link to the document listing the available annotations Fixes: #756 Signed-off-by: Christophe de Dinechin (cherry picked from commit 0bd878b56e35452fe29ac171e214a5fbdf1d09ed) --- how-to/how-to-load-kernel-modules-with-kata.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/how-to/how-to-load-kernel-modules-with-kata.md b/how-to/how-to-load-kernel-modules-with-kata.md index 25122fdb..262bb3df 100644 --- a/how-to/how-to-load-kernel-modules-with-kata.md +++ b/how-to/how-to-load-kernel-modules-with-kata.md @@ -56,8 +56,9 @@ There are some limitations with this approach: As was mentioned above, not all containers need the same modules, therefore using the configuration file for specifying the list of kernel modules per [POD][3] can -be a pain. Unlike the configuration file, annotations provide a way to specify -custom configurations per POD. +be a pain. +Unlike the configuration file, [annotations](how-to-set-sandbox-config-kata.md) +provide a way to specify custom configurations per POD. The list of kernel modules and parameters can be set using the annotation `io.katacontainers.config.agent.kernel_modules` as a semicolon separated From e42025cf8b3c83fdc31562d17c6f6b60ceb946f7 Mon Sep 17 00:00:00 2001 From: Cria Hu Date: Tue, 13 Oct 2020 14:33:13 +0800 Subject: [PATCH 3/4] docs: modify the error url of logfmt There are many error urls, which will affect the correctness of the document. I changed the error url to the correct url. Fixes: #740 Signed-off-by: Cria Hu (cherry picked from commit 7f3bd7ab20ee98cc8c433d8a5812edd57a101837) --- how-to/how-to-use-kata-containers-with-acrn.md | 2 +- how-to/how-to-use-virtio-mem-with-kata.md | 2 +- how-to/service-mesh.md | 2 +- install/docker/opensuse-leap-docker-install.md | 2 +- install/docker/opensuse-tumbleweed-docker-install.md | 2 +- install/installing-with-kata-doc-to-script.md | 4 ++-- install/opensuse-leap-installation-guide.md | 2 +- install/opensuse-tumbleweed-installation-guide.md | 2 +- 8 files changed, 9 insertions(+), 9 deletions(-) diff --git a/how-to/how-to-use-kata-containers-with-acrn.md b/how-to/how-to-use-kata-containers-with-acrn.md index 27cad4ca..4fe97423 100644 --- a/how-to/how-to-use-kata-containers-with-acrn.md +++ b/how-to/how-to-use-kata-containers-with-acrn.md @@ -27,7 +27,7 @@ This document requires the presence of the ACRN hypervisor and Kata Containers o - ACRN supported [Hardware](https://projectacrn.github.io/latest/hardware.html#supported-hardware). > **Note:** Please make sure to have a minimum of 4 logical processors (HT) or cores. -- ACRN [software](https://projectacrn.github.io/latest/tutorials/kbl-nuc-sdc.html#use-the-script-to-set-up-acrn-automatically) setup. +- ACRN [software](https://projectacrn.github.io/1.6/tutorials/kbl-nuc-sdc.html#use-the-script-to-set-up-acrn-automatically) setup. - For networking, ACRN supports either MACVTAP or TAP. If MACVTAP is not enabled in the Service OS, please follow the below steps to update the kernel: ```sh diff --git a/how-to/how-to-use-virtio-mem-with-kata.md b/how-to/how-to-use-virtio-mem-with-kata.md index ce989a41..7b84184f 100644 --- a/how-to/how-to-use-virtio-mem-with-kata.md +++ b/how-to/how-to-use-virtio-mem-with-kata.md @@ -20,7 +20,7 @@ Please use following unofficial version of the Linux kernel and QEMU that suppor The Linux kernel is at https://github.com/davidhildenbrand/linux/tree/virtio-mem-rfc-v4. The Linux kernel config that can work with Kata Containers is at https://gist.github.com/teawater/016194ee84748c768745a163d08b0fb9. -The QEMU is at https://github.com/teawater/qemu/tree/kata-virtio-mem. (The original source is at https://github.com/davidhildenbrand/qemu/tree/virtio-mem. Its base version of QEMU cannot work with Kata Containers. So merge the commit of `virtio-mem` to upstream QEMU.) +The QEMU is at https://github.com/teawater/qemu/tree/kata-virtio-mem. (The original source is at https://github.com/davidhildenbrand/qemu/tree/virtio-mem-vfio. Its base version of QEMU cannot work with Kata Containers. So merge the commit of `virtio-mem` to upstream QEMU.) Set Linux and the QEMU that support `virtio-mem` with following line in the Kata Containers QEMU configuration `configuration-qemu.toml`: ```toml diff --git a/how-to/service-mesh.md b/how-to/service-mesh.md index 5a1727f9..ed8847da 100644 --- a/how-to/service-mesh.md +++ b/how-to/service-mesh.md @@ -76,7 +76,7 @@ is not able to perform a proper setup of the rules. ### Service Mesh Istio -As a reference, you can follow Istio [instructions](https://istio.io/docs/setup/kubernetes/quick-start/#download-and-prepare-for-the-installation). +As a reference, you can follow Istio [instructions](https://istio.io/latest/docs/setup/getting-started/#download-and-prepare-for-the-installation). The following is a summary of what you need to install Istio on your system: ``` diff --git a/install/docker/opensuse-leap-docker-install.md b/install/docker/opensuse-leap-docker-install.md index c743f4bd..e70730fa 100644 --- a/install/docker/opensuse-leap-docker-install.md +++ b/install/docker/opensuse-leap-docker-install.md @@ -7,7 +7,7 @@ You can ignore the content of this comment. ```bash $ echo "NOTE: this document is just a link to the generic openSUSE install guide located at: -https://raw.githubusercontent.com/kata-containers/documentation/master/install/docker/opensuse-docker-install.md +https://github.com/kata-containers/documentation/tree/master/install/docker/opensuse-docker-install.md Please download this file and run kata-doc-to-script.sh again." ``` diff --git a/install/docker/opensuse-tumbleweed-docker-install.md b/install/docker/opensuse-tumbleweed-docker-install.md index 8d1c5a36..56662c59 100644 --- a/install/docker/opensuse-tumbleweed-docker-install.md +++ b/install/docker/opensuse-tumbleweed-docker-install.md @@ -7,7 +7,7 @@ You can ignore the content of this comment. ```bash $ echo "NOTE: this document is just a link to the generic openSUSE install guide located at: -https://raw.githubusercontent.com/kata-containers/documentation/master/install/docker/opensuse-docker-install.md +https://github.com/kata-containers/documentation/tree/master/install/docker/opensuse-docker-install.md Please download this file and run kata-doc-to-script.sh again." ``` diff --git a/install/installing-with-kata-doc-to-script.md b/install/installing-with-kata-doc-to-script.md index 3c6e25b9..e11bcd33 100644 --- a/install/installing-with-kata-doc-to-script.md +++ b/install/installing-with-kata-doc-to-script.md @@ -17,7 +17,7 @@ to generate installation bash scripts. ```bash $ source /etc/os-release -$ curl -fsSL -O https://raw.githubusercontent.com/kata-containers/documentation/master/install/${ID}-installation-guide.md +$ curl -fsSL -O https://github.com/kata-containers/documentation/tree/master/install/${ID}-installation-guide.md $ bash -c "$(curl -fsSL https://raw.githubusercontent.com/kata-containers/tests/master/.ci/kata-doc-to-script.sh) ${ID}-installation-guide.md ${ID}-install.sh" ``` @@ -33,7 +33,7 @@ $ bash "./${ID}-install.sh" ```bash $ source /etc/os-release -$ curl -fsSL -O https://raw.githubusercontent.com/kata-containers/documentation/master/install/docker/${ID}-docker-install.md +$ curl -fsSL -O https://github.com/kata-containers/documentation/tree/master/install/docker${ID}-docker-install.md $ bash -c "$(curl -fsSL https://raw.githubusercontent.com/kata-containers/tests/master/.ci/kata-doc-to-script.sh) ${ID}-docker-install.md ${ID}-docker-install.sh" ``` diff --git a/install/opensuse-leap-installation-guide.md b/install/opensuse-leap-installation-guide.md index a4bf946f..733be1f3 100644 --- a/install/opensuse-leap-installation-guide.md +++ b/install/opensuse-leap-installation-guide.md @@ -8,7 +8,7 @@ You can ignore the content of this comment. ```bash $ echo "NOTE: this document is just a link to the generic openSUSE install guide located at: -https://raw.githubusercontent.com/kata-containers/documentation/master/install/opensuse-installation-guide.md +https://github.com/kata-containers/documentation/tree/master/install/opensuse-installation-guide.md Please download this file and run kata-doc-to-script.sh again." ``` diff --git a/install/opensuse-tumbleweed-installation-guide.md b/install/opensuse-tumbleweed-installation-guide.md index 35ec2f2e..f586df75 100644 --- a/install/opensuse-tumbleweed-installation-guide.md +++ b/install/opensuse-tumbleweed-installation-guide.md @@ -8,7 +8,7 @@ You can ignore the content of this comment. ```bash $ echo "NOTE: this document is just a link to the generic openSUSE install guide located at: -https://raw.githubusercontent.com/kata-containers/documentation/master/install/opensuse-installation-guide.md +https://github.com/kata-containers/documentation/tree/master/install/opensuse-installation-guide.md Please download this file and run kata-doc-to-script.sh again." ``` From 5ad45300ff1383a20b7c16629d64460639c9e7b5 Mon Sep 17 00:00:00 2001 From: Archana Shinde Date: Wed, 11 Nov 2020 11:03:59 -0800 Subject: [PATCH 4/4] docs: Update broken link Use correct url for fluentd logfmt Signed-off-by: Archana Shinde --- how-to/how-to-import-kata-logs-with-fluentd.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/how-to/how-to-import-kata-logs-with-fluentd.md b/how-to/how-to-import-kata-logs-with-fluentd.md index 056cb16e..1b62bec8 100644 --- a/how-to/how-to-import-kata-logs-with-fluentd.md +++ b/how-to/how-to-import-kata-logs-with-fluentd.md @@ -185,7 +185,7 @@ in Kibana: ![Kata tags in EFK](./images/efk_syslog_entry_detail.png). We can however further sub-parse the Kata entries using the -[Fluentd plugins](https://docs.fluentbit.io/manual/parser/logfmt) that will parse +[Fluentd plugins](https://docs.fluentbit.io/manual/pipeline/parsers/logfmt) that will parse `logfmt` formatted data. We can utilise these to parse the sub-fields using a Fluentd filter section. At the same time, we will prefix the new fields with `kata_` to make it clear where they have come from: