cgroup不匹配启动kubelet报错

[suichangyin]

缺陷描述

kubelet启动后无法创建容器，报错cgroup不匹配

环境 (请填写以下信息):

执行下面括号中的命令，提交返回结果

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):
  Linux 5.4.92-1.el7.elrepo.x86_64 x86_64
  NAME="CentOS Linux"
  VERSION="7 (Core)"
  ID="centos"
  ID_LIKE="rhel fedora"
  VERSION_ID="7"
  PRETTY_NAME="CentOS Linux 7 (Core)"
  ANSI_COLOR="0;31"
  CPE_NAME="cpe:/o:centos:centos:7"
  HOME_URL="https://www.centos.org/"
  BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

• Ansible版本 (ansible --version):
  /usr/local/lib/python3.6/site-packages/ansible/parsing/vault/init.py:44: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography. The next release of cryptography will remove support for Python 3.6.
  from cryptography.exceptions import InvalidSignature
  ansible 2.10.17
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.8 (default, Nov 14 2023, 16:29:52) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]

• Python版本 (python --version):
  Python 2.7.5

• Kubeadm-ha版本(commit) (git rev-parse --short HEAD):
  1.27
  如何复现

复现的步骤：

1. 第一步：编写 inventory.ini 文件，内容如下

   [all]
   192.168.56.11 ansible_port=22 ansible_user="vagrant" ansible_ssh_pass="vagrant"
   ......

2. 第二步：编写 variables.yaml 文件，内容如下

   skip_verify_node: false
   timezone: Asia/Shanghai
   ......

3. 第三步：执行部署命令，命令如下

   ansible-playbook -i inventory.ini -e @variables.yaml 90-init-cluster.yml

4. 出现错误

Nov 28 11:02:35 node84 containerd: time="2023-11-28T11:02:35.479206722+08:00" level=warning msg="cleanup warnings time="2023-11-28T11:02:35+08:00" level=info msg="starting signal loop" namespace=k8s.io pid=28805 runtime=io.containerd.runc.v2\ntime="2023-11-28T11:02:35+08:00" level=warning msg="failed to read init pid file" error="open /run/containerd/io.containerd.runtime.v2.task/k8s.io/f3577f95ce3419fd6739ffd6d41cdf2ddaa2c41d46d52a6fbe0007f852c7fbde/init.pid: no such file or directory" runtime=io.containerd.runc.v2\n"
Nov 28 11:02:35 node84 containerd: time="2023-11-28T11:02:35.479530882+08:00" level=error msg="copy shim log" error="read /proc/self/fd/21: file already closed"
Nov 28 11:02:35 node84 containerd: time="2023-11-28T11:02:35.493026274+08:00" level=error msg="RunPodSandbox for &PodSandboxMetadata{Name:lb-nginx-10.0.12.84,Uid:b1976a5836c3b2c5217a50b250fa545c,Namespace:kube-system,Attempt:0,} failed, error" error="failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format "slice:prefix:name" for systemd cgroups, got "/kubepods/besteffort/podb1976a5836c3b2c5217a50b250fa545c/f3577f95ce3419fd6739ffd6d41cdf2ddaa2c41d46d52a6fbe0007f852c7fbde" instead: unknown"
Nov 28 11:02:35 node84 kubelet: E1128 11:02:35.493556 28479 remote_runtime.go:176] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format "slice:prefix:name" for systemd cgroups, got "/kubepods/besteffort/podb1976a5836c3b2c5217a50b250fa545c/f3577f95ce3419fd6739ffd6d41cdf2ddaa2c41d46d52a6fbe0007f852c7fbde" instead: unknown"
Nov 28 11:02:35 node84 kubelet: E1128 11:02:35.493642 28479 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format "slice:prefix:name" for systemd cgroups, got "/kubepods/besteffort/podb1976a5836c3b2c5217a50b250fa545c/f3577f95ce3419fd6739ffd6d41cdf2ddaa2c41d46d52a6fbe0007f852c7fbde" instead: unknown" pod="kube-system/lb-nginx-10.0.12.84"
Nov 28 11:02:35 node84 kubelet: E1128 11:02:35.493677 28479 kuberuntime_manager.go:1122] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format "slice:prefix:name" for systemd cgroups, got "/kubepods/besteffort/podb1976a5836c3b2c5217a50b250fa545c/f3577f95ce3419fd6739ffd6d41cdf2ddaa2c41d46d52a6fbe0007f852c7fbde" instead: unknown" pod="kube-system/lb-nginx-10.0.12.84"
Nov 28 11:02:35 node84 kubelet: E1128 11:02:35.493800 28479 pod_workers.go:1294] "Error syncing pod, skipping" err="failed to "CreatePodSandbox" for "lb-nginx-10.0.12.84_kube-system(b1976a5836c3b2c5217a50b250fa545c)" with CreatePodSandboxError: "Failed to create sandbox for pod \"lb-nginx-10.0.12.84_kube-system(b1976a5836c3b2c5217a50b250fa545c)\": rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: expected cgroupsPath to be of format \"slice:prefix:name\" for systemd cgroups, got \"/kubepods/besteffort/podb1976a5836c3b2c5217a50b250fa545c/f3577f95ce3419fd6739ffd6d41cdf2ddaa2c41d46d52a6fbe0007f852c7fbde\" instead: unknown"" pod="kube-system/lb-nginx-10.0.12.84" podUID=b1976a5836c3b2c5217a50b250fa545c
N
```

预期结果

正常完成

屏幕截图

docker info显示使用的cgroup驱动为cgroupfs，导致kubelet使用cgroupfs启动后无法创建容器
修改docker为systemd后，正常完成。

其他事项
docker版本:
Client: Docker Engine - Community
Version: 24.0.7
API version: 1.43
Go version: go1.20.10
Git commit: afdd53b
Built: Thu Oct 26 09:11:35 2023
OS/Arch: linux/amd64
Context: default

Server: Docker Engine - Community
Engine:
Version: 24.0.7
API version: 1.43 (minimum version 1.12)
Go version: go1.20.10
Git commit: 311b9ff
Built: Thu Oct 26 09:10:36 2023
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.25
GitCommit: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
runc:
Version: 1.1.10
GitCommit: v1.1.10-0-g18a0cb0
docker-init:
Version: 0.19.0
GitCommit: de40ad0

在此处添加有关该问题的任何其他事项。

[TimeBye]

使用本项目前，服务器上的docker已近安装了吗？