[Feature] Support foreign cluster nodes with their real names

[olljanat]

Is your feature request related to a problem? Please describe.
Ability to offloading namespaces to remote cluaters is very useful. However I would like to be able to control and see witch nodes actually run workloads on foreign clusters.

Describe the solution you'd like
Afaiu deploying virtual-kubelet as daemonset and passthrough node real node names for it would do the trick.

Describe alternatives you've considered
N/A

Additional context
We are discussing on loft-sh/vcluster#441 (comment) about ability to stretch vcluster to multiple host clusters and that liqo is one of the options to do so.

[giorio94]

Thanks @olljanat for this issue!

Ability to offloading namespaces to remote cluaters is very useful. However I would like to be able to control and see witch nodes actually run workloads on foreign clusters.

We feel that the feature you are proposing could be useful for certain use-cases, and it definitely has some advantages as it allows more fine-grained scheduling constraints compared to the single node approach. On the other hand, it would also increase the overall complexity and possibly lead to poorer scalability, as more information would need to be exchanged among clusters.

We are discussing internally about whether we could/should add the support for this approach in addition to the current one, while analyzing also the effort required. To bring more elements on the table, could you please elaborate a bit more why the current "entire cluster mapped as node" approach is not sufficient for your use-case? Is it that related to fine-grained scheduling or other aspects?

Afaiu deploying virtual-kubelet as daemonset and passthrough node real node names for it would do the trick.

I personally see two approaches to achieve the goal you are mentioning (I haven't investigated them thoroughly yet, though):

1. The virtual kubelet is modified to create locally a "shadow" node for each remote node, and manage all pods scheduled on these nodes.
2. A different virtual kubelet is started for each remote node, which is in charge of managing only the pods scheduled on that node (this could not be a daemonset, since we run the VK in the local clustrer, and not in the remote one). This would require modifications in the logic used to start the VK during the peering phase, as well as we should elect a single VK per cluster as leader to handle the reflection process (which is per-cluster, and not per-node).

[olljanat]

To bring more elements on the table, could you please elaborate a bit more why the current "entire cluster mapped as node" approach is not sufficient for your use-case? Is it that related to fine-grained scheduling or other aspects?

@giorio94 I mean that if vcluster syncer is running on three different host clusters where one of those have leading role and they always contact to local cluster it cannot works correctly if view is different depending on which process is leading.

However you are right, shadow nodes on all clusters should be enough still.

EDIT: However those shadow nodes should also have same IDs than original ones.

[Sharathmk99]

Shadow node of all remote cluster will really help in our case as we can get fine control over scheduling. Currently Management cluster see foreign cluster as one big cluster and due to fragmentation of resources in foreign cluster management cluster see node is still free to schedule jobs.
this is really important when you have really big foreign cluster.

[Sharathmk99]

I can contribute to this feature with some helps during design. Thank you

[cheina97]

Hi @Sharathmk99, at the moment we are discussing this feature and how to include you in our workflow. We will keep you informed.

Thanks for your interest in the project.