Skip to content

Commit

Permalink
KEP-3008: minor update
Browse files Browse the repository at this point in the history
Two more user stories. Clarify wording and fix some typos.
  • Loading branch information
marquiz committed Feb 1, 2022
1 parent e2cb478 commit 8e489e0
Showing 1 changed file with 19 additions and 10 deletions.
29 changes: 19 additions & 10 deletions keps/sig-node/3008-cri-class-based-resources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,16 +173,14 @@ We would like to add support for class-based resources in Kubernetes.
Class-based resources can be thought of as non-accountable resources, each of
which is presented by a set of classes. Being non-accountable means that
multiple containers can be assigned to the same class. They are also supposed
to be opaque to the CRI client in the sense that all details configuration and
control details of the resources and the classes within are taken care of by
the container runtime.
to be opaque to the CRI client in the sense that the container runtime takes
care of configuration and control of the resources and the classes within.

A prime example of a class-based resource is Intel RDT (Resource Director
Technology). RDT is a technology for controlling the cache lines and memory
bandwidth available to applications. RDT provides a class-based approach for
QoS control of these shared resources. That is, a limited set (by HW) of
classes (which processes are then assigned to) for limiting the available cache
lines and/or memory bandwidth.
QoS control of these shared resources: all processes in the same hardware class
share a portion of cache lines and memory bandwidth.

We also believe that the Linux Block IO controller (cgroup) should be handled
as a class-based resource on the level of container orchestration. This enables
Expand All @@ -207,7 +205,7 @@ bandwidth QoS of applications, providing a tool for mitigating noisy neighbors
and fulfilling SLAs. In Linux control happens via resctrl -- a
pseudo-filesystem provided by the kernel which makes it virtually agnostic of
the hardware architecture. The OCI runtime-spec has supported Intel RDT for a
while already. Other hardware vendoers have comparable technologies which use
while already. Other hardware vendors have comparable technologies which use
the same resctrl interface.

The Linux Block IO controller parameters depend very heavily on the underlying
Expand All @@ -223,7 +221,7 @@ in Kubernetes. CRI-O and containerd runtimes have support for RDT and blockio
classes and they provide an bridge-gap user interface through special pod
annotations. We would like to eventually get these types of resources first
class citizen and properly supported in Kubernetes, providing visibility, a
well-defined user interface, visibility and permission controls.
well-defined user interface, and permission controls.

### Goals

Expand Down Expand Up @@ -286,6 +284,17 @@ workload by assigning it to a class with exclusive cache allocation.
As a user I want to make sure my low-priority, I/O-intensive background task
will not disturb more important workloads running on the same node.

#### Story 3

As a cluster administrator I want to throttle I/O bandwidths of certain
DaemonSets, and I want that exact throttling values depend on the SSD model in
my heterogenous cluster.

#### Story 4

As a user I want to assign a low priority task into an (RDT) class that limits
the available memory bandwidth.

### Notes/Constraints/Caveats (Optional)

<!--
Expand Down Expand Up @@ -436,7 +445,7 @@ comparable to the PodClassResources message in PodSandboxConfig in the CRI API.
In practice, the class resource information will be directly used in the CRI
ContainerConfig (e.g. CreateContainerRequest message). At this point, without
resource discovery or access control kubelet does not do any validity checking
of the values. Invalide class assignments will cause an error in the container
of the values. Invalid class assignments will cause an error in the container
runtime.

### Container runtimes
Expand Down Expand Up @@ -471,7 +480,7 @@ Communicating Pod QoS class via class resources would advocate moving class
resources up to `ContainerConfig`.

Making this change, it would also be possible to separate `oom_score_adj` from
the pod qos class in the future. The runtime could provice a set of OOM
the pod qos class in the future. The runtime could provide a set of OOM
classes, making it possible for the user to specify a burstable pod with low
oom priority (low chance of being killed).

Expand Down

0 comments on commit 8e489e0

Please sign in to comment.