K2s agents on Nodes?

[jgriegershs]

What do you think of installing tiny (Go-based) agents on all cluster Nodes (similar to Kubelets) that:

• retrieve Node-specific data (e.g. status, etc.)
• execute commands (e.g. when setting up an addon, etc.)
• manage the K2s-specific configuration of "their" Node (e.g. firewall ports, etc.)

Right now SSH is used for remote commands which limits the control we have over logging and execution. Furthermore, the commands sent over SSH are OS-specific. Also, there is no push mechanism in the direction Node -> host that we can utilize for e.g. propagating events.

Lastly, the host needs to call different implementations for setting things up aka. executing remote commands depending on e.g. the Node OS. Having an agent with the same API surface would abstract those kind of details away.

[krotz-dieter]

This pattern we already have considered, the risk is always there that the solution gets to heavy and complicated by having on each node an extra agent. Also maintaining such a agent will cost some efforts on long term. To my opinion we shall utilize the K8s APIs for node status, cluster state, ... If there will be a need for addons to have node specific functionality that is not covered by the K8s APIs, than that addon should bring that in with normal K8s resources (daemonset or custom resources). We should start with low impact on the node and see how good this works on long term.