Multi-node cluster failing to create - "Error applying plan -- check rancher-system-agent.service logs on node for more information"

[GrantMelvin]

What I did
Creating a multi-node K3s cluster with default settings and 3 machines fails with the error: "Error applying plan -- check rancher-system-agent.service logs on node for more information" on 2/3 nodes. One node is accessible and operating normally.

Inventory of machines:

Cloud configuration for the OS being used:

Web VNC of the two failed nodes:

The output of "journalctl -eu rancher-system-agent -f on the master node that is functioning:

If there are any more logs or commands that I can provide to help in solving the issue, please let me know.

Expectation:
A cluster is to be created with 3 functional nodes.

Environment:
Elemental release version (use cat /etc/os-release): 103.3.0+up1.5.3
Rancher version: 2.8.3
Kubernetes version (use kubectl version): v1.29.2
Cloud provider or hardware configuration: Harvester

[max06]

Maybe related, maybe not.... I've seen the same error messages just hours ago. These are misleading.

In my case the error resulted from k3s being unable to start, due to missing memory cgroups. Maybe the logs for k3s provide more insight on whats going on.