From 3e6b0937d598f6e6507d41b480e23c38422eeccb Mon Sep 17 00:00:00 2001 From: Denise Date: Mon, 21 Oct 2024 20:26:12 -0700 Subject: [PATCH 1/3] Update explaining syncing and bi-directional syncing --- vcluster/_fragments/how-does-syncing-work.mdx | 55 +++++++++++++++---- .../_fragments/sync-from-host-resources.mdx | 25 +++++++-- .../_fragments/sync-to-host-resources.mdx | 36 ++++++++++-- .../configure/vcluster-yaml/sync/README.mdx | 33 ++--------- .../vcluster-yaml/sync/from-host/README.mdx | 14 +---- .../vcluster-yaml/sync/to-host/README.mdx | 13 +---- 6 files changed, 109 insertions(+), 67 deletions(-) diff --git a/vcluster/_fragments/how-does-syncing-work.mdx b/vcluster/_fragments/how-does-syncing-work.mdx index 9b8bd7a6b..48078b88e 100644 --- a/vcluster/_fragments/how-does-syncing-work.mdx +++ b/vcluster/_fragments/how-does-syncing-work.mdx @@ -1,9 +1,37 @@ -Syncing is the process where vCluster copies a resource between the virtual cluster and the host cluster. It's the basic principle vCluster uses to emulate a fully functional Kubernetes cluster. +Since a virtual cluster does not have actual worker nodes or a network, syncing is the process where vCluster copies a resource between the virtual cluster and the host cluster. +It's the basic principle that vCluster uses to emulate a fully functional Kubernetes cluster. +By default, vCluster only syncs low-level resources, such as pods, secrets, configmaps, or services, but a user can +enable many other resources for all use cases. vCluster control plane runs with a minimal set of RBAC permissions to allow execution in restricted environments. Certain resources require +extra permissions to sync, which are automatically given to the vCluster ServiceAccount if you enable a resource that requires additional +permissions. -vCluster only syncs low-level resources by default, such as pods, secrets, configmaps or services. As soon as one of these resources are created in the virtual cluster, vCluster will sync it to the other side. As soon as a resource was copied by vCluster, it will start watching for changes to the resource on either side, so when a controller changes something on the host cluster side, the vCluster will sync that change to the virtual cluster or when a change on the virtual cluster side happens, it will sync it to the host cluster, to make sure the resource is in sync. +All fields of a resources are synced from one direction to the other based on what section the resource is part of in the `vcluster.yaml`. There +are a few exceptions for each resource where specific fields are kept in sync in both directions (i.e. bi-directional sync). +As soon as a resource is copied by vCluster, the syncer starts watching for changes to the resource on either side. When the resource +is updated in the origin cluster, the syncer updates and copies those changes in the destination cluster. + +When a resource is defined in the `toHost` section, as soon as that resource is created in the virtual cluster, vCluster copies it to the host cluster. +Any changes to the resource in the virtual cluster are automatically applied to the resource in the host cluster. + +When a resource is defined in the `fromHost` section, as soon as that resource is created in the host cluster, vCluster copies it to the virtual cluster. +Any changes to the resource in the host cluster are automatically applied to the resource in the virtual cluster. + + +When syncing resources, vCluster stores the mapping of the `VIRTUAL NAME <-> HOST NAME` for the actual resource, and the references of the other resources used by this resource. +Tracking references helps the vCluster to understand what resources are still needed in the host cluster. This important for secrets and configmaps because by default only the secrets and configmaps used are actually synced to the host cluster. + +Assume you've started your vCluster with ingress syncing to the host enabled. + +``` vcluster.yaml +sync: + toHost: + ingresses: + enabled: true +``` + +After the virtual cluster is up, the user creates an ingress in the virtual cluster. -Let's take `Ingress` as an example. A user that creates the following ingress in the virtual cluster: ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress @@ -26,7 +54,14 @@ spec: number: 80 ``` -When copying to the host cluster, the vCluster will change the ingress to: +When vCluster copies the ingress to the host cluster, the vCluster creates an ingress in the namespace of the virtual cluster with the following changes: + +* **Name Rewritten**: vCluster rewrites the name of the resource to avoid conflicts in the host cluster. The name is rewritten in the form of `NAME-x-NAMESPACE-x-VCLUSTER_NAME` +* **Namespace Rewritten**: vCluster rewrites the namespace of the resource to sync everything into the namespace of the deployed virtual cluster control plane. +* **Annotations / Labels Added**: vCluster adds annotations and labels to identify the resource in the host cluster, which are pre-fixed with `vcluster.loft.sh/`. +* **References Rewritten**: vCluster rewrites the `spec.rules[*].http.paths[*].backend.service.name` and `spec.tls[*].secretName` to match the rewritten names of the resources in the host cluster. + + ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress @@ -60,11 +95,11 @@ spec: secretName: testsecret-tls-x-default-x-vcluster ``` -There is a couple of important changes to be noticed: -* **Name Rewritten**: vCluster rewrites the name of the resource to avoid conflicts in the host cluster. The name is rewritten in the form of `NAME-x-NAMESPACE-x-VCLUSTER_NAME` -* **Namespace Rewritten**: vCluster rewrites the namespace of the resource to sync everything into the vcluster's namespace. -* **Annotations / Labels Added**: vCluster adds annotations and labels to identify the resource in the host cluster. -* **References Rewritten**: vCluster rewrites the `spec.rules[*].http.paths[*].backend.service.name` and `spec.tls[*].secretName` to match the rewritten names of the resources in the host cluster. +### Syncing labels to the host cluster -vCluster will also keep track from now of the resource within its internal name cache, that tracks the `VIRTUAL NAME <-> HOST NAME` as well as all references used by this resource. In the ingress case, this would be the service `nginx` and the secret `testsecret-tls`. Tracking references helps the vCluster to understand what resources are still needed in the host cluster. For secrets and configmaps this is important, because by default only the needed ones are actually synced to the host cluster. +When syncing labels to the host cluster, vCluster rewrites the following labels to avoid conflicts: +* `release` (Label is needed to avoid conflicts with the vCluster pods themselves). This is only excluded in single-namespace mode as multi-namespace mode does sync all pods into separate namespaces. +* `vcluster.loft.sh/namespace` +* `vcluster.loft.sh/managed-by` +* `vcluster.loft.sh/controlled-by` \ No newline at end of file diff --git a/vcluster/_fragments/sync-from-host-resources.mdx b/vcluster/_fragments/sync-from-host-resources.mdx index 47f1f6fd6..33215fde6 100644 --- a/vcluster/_fragments/sync-from-host-resources.mdx +++ b/vcluster/_fragments/sync-from-host-resources.mdx @@ -1,13 +1,24 @@ -vCluster can sync certain resources purely from host to make them available inside the vCluster. This is useful if you only want to view certain resources in read-only mode, but not want the users inside the vCluster to change them. +vCluster can sync certain resources from the host cluster to make them available inside the virtual cluster, but when these resources are +synced, they are only synced in read-only mode. No changes to the resource in the virtual cluster syncs back to the host cluster as the +resources are shared across the host cluster. -A good example would be nodes, which are nice to view inside the vCluster and can be also used to enabled certain features such as [scheduling inside the vCluster](/vcluster/configure/vcluster-yaml/control-plane/other/advanced/virtual-scheduler.mdx), but you wouldn't want your vCluster to change the node itself. Another benefit of purely syncing from host is that vCluster itself will only require read-only RBAC permissions. +A good example would be nodes, which are nice to view inside the virtual cluster and can be also used to enabled certain features such as [scheduling inside the vCluster](/vcluster/configure/vcluster-yaml/control-plane/other/advanced/virtual-scheduler.mdx), +but you wouldn't want your virtual cluster to change the node itself. Another benefit of only syncing from host is that vCluster itself only requires read-only RBAC permissions. vCluster also allows to sync custom resources via the [custom resource definitions syncer](../configure/vcluster-yaml/sync/from-host/custom-resources.mdx) -### Enabled by Default +There are a couple of labels that are created on the host cluster by vCluster that never get synced to the virtual cluster resource. These labels are: + +* `release` (Label is needed to avoid conflicts with the vCluster pods themselves). This is only excluded in single-namespace mode as multi-namespace mode does sync all pods into separate namespaces. +* `vcluster.loft.sh/namespace` +* `vcluster.loft.sh/managed-by` +* `vcluster.loft.sh/controlled-by` + +### Enabled by default + * [Events](../configure/vcluster-yaml/sync/from-host/events.mdx) -### Disabled by Default +### Disabled by default * [CustomResources](../configure/vcluster-yaml/sync/from-host/custom-resources.mdx) * [Nodes](../configure/vcluster-yaml/sync/from-host/nodes.mdx) * [IngressClasses](../configure/vcluster-yaml/sync/from-host/ingress-classes.mdx) @@ -15,3 +26,9 @@ vCluster also allows to sync custom resources via the [custom resource definitio * [CSINodes](../configure/vcluster-yaml/sync/from-host/csi-nodes.mdx) * [CSIDrivers](../configure/vcluster-yaml/sync/from-host/csi-drivers.mdx) * [CSIStorageCapacities](../configure/vcluster-yaml/sync/from-host/csi-storage-capacities.mdx) + +### No bi-directional syncing + +Since syncing resources from the host cluster is in read only mode and changes in the virtual cluster do not +get applied to the resource in the host cluster, bi-directional syncing does not exist across these resources. + diff --git a/vcluster/_fragments/sync-to-host-resources.mdx b/vcluster/_fragments/sync-to-host-resources.mdx index ff893cbf7..35bfb7d6e 100644 --- a/vcluster/_fragments/sync-to-host-resources.mdx +++ b/vcluster/_fragments/sync-to-host-resources.mdx @@ -1,8 +1,8 @@ -vCluster can sync certain resources from the virtual cluster to the host cluster and back. These resources are typically created within the virtual cluster and are expected to be changeable within the virtual cluster. These are typically namespace-scoped resources (although exceptions exist) such as Pods, Secrets, ConfigMaps, Services etc. +vCluster syncs specific Kubernetes resources from the virtual cluster to the host cluster. vCluster +expects these resources to be created in the virtual cluster and updated in the virtual cluster. Besides native +Kubernetes resources, custom resources can also be synced. -vCluster also allows to sync custom resources via the [custom resources syncer](../configure/vcluster-yaml/sync/to-host/advanced/custom-resources.mdx) - -### Enabled by Default +### Enabled by default * [Pods](../configure/vcluster-yaml/sync/to-host/core/pods.mdx) * [Secrets](../configure/vcluster-yaml/sync/to-host/core/secrets.mdx) * [ConfigMaps](../configure/vcluster-yaml/sync/to-host/core/config-maps.mdx) @@ -10,7 +10,7 @@ vCluster also allows to sync custom resources via the [custom resources syncer]( * [Endpoints](../configure/vcluster-yaml/sync/to-host/networking/endpoints.mdx) * [PersistentVolumeClaims](../configure/vcluster-yaml/sync/to-host/storage/persistent-volume-claims.mdx) -### Disabled by Default +### Disabled by default * [CustomResources](../configure/vcluster-yaml/sync/to-host/advanced/custom-resources.mdx) * [Ingresses](../configure/vcluster-yaml/sync/to-host/networking/ingresses.mdx) * [NetworkPolicies](../configure/vcluster-yaml/sync/to-host/networking/network-policies.mdx) @@ -20,3 +20,29 @@ vCluster also allows to sync custom resources via the [custom resources syncer]( * [ServiceAccounts](../configure/vcluster-yaml/sync/to-host/advanced/service-accounts.mdx) * [PodDisruptionBudgets](../configure/vcluster-yaml/sync/to-host/advanced/pod-disruption-budgets.mdx) * [PriorityClasses](../configure/vcluster-yaml/sync/to-host/advanced/priority-classes.mdx) +* [Custom Resources](../configure/vcluster-yaml/sync/to-host/advanced/custom-resources.mdx) + +### Bi-directional syncing + +For all resources synced from the virtual cluster to the host cluster, metadata labels and metadata annotations are synced in both directions. This means that not only changes in the +virtual cluster are reflected in the host cluster, but changes in host cluster are reflected in the virtual cluster. + +:::warning vCluster specific Labels +Any labels pre-fixed with `vcluster.loft.sh` on the host resources should not be updated by the user. These labels are used by the vCluster control plane to manage the resources and to avoid conflicts. +::: + +#### Additional Fields that support bi-directionally sync + +All `metadata.labels` and `metadata.annotations` support bi-directional sync for any resources being synced to the host cluster. +In addition to those fields, there are several other fields that vCluster supports bi-directional sync. + +| Resource | Fields | +| --- | --- | +| Pods | | +| Secrets | | +| ConfigMaps | | +| Services | +| Ingresses | | +| PersistentVolumes | | +| StorageClass | | + diff --git a/vcluster/configure/vcluster-yaml/sync/README.mdx b/vcluster/configure/vcluster-yaml/sync/README.mdx index 62bb765b2..a068901b4 100644 --- a/vcluster/configure/vcluster-yaml/sync/README.mdx +++ b/vcluster/configure/vcluster-yaml/sync/README.mdx @@ -6,40 +6,19 @@ description: Configure syncing. --- import Sync from '../../../_partials/config/sync.mdx' -import SyncToHostResources from '@site/vcluster/_fragments/sync-to-host-resources.mdx' -import SyncFromHostResources from '@site/vcluster/_fragments/sync-from-host-resources.mdx' +import SyncToHostResources from '../../../_fragments/sync-to-host-resources.mdx' +import SyncFromHostResources from '../../../_fragments/sync-from-host-resources.mdx' import HowDoesSyncingWork from '../../../_fragments/how-does-syncing-work.mdx' -{/* -Links to previous docs: -https://www.vcluster.com/docs/architecture/syncer/ - -TODO: Determine if we should remove this from the new docs since it's not multi-namespace mode has additional management overhead for users -https://www.vcluster.com/docs/architecture/syncer/single_vs_multins -*/} - -A virtual cluster does not have actual worker nodes or a network. Instead, the vCluster syncer component replicates the pods that are created within the virtual cluster to the host cluster. Subsequently, the host cluster schedules the pods, and the syncer keeps the virtual cluster pods and host cluster pods in sync. - -By default, vCluster runs with a minimal set of RBAC permissions to allow execution in restricted environments. Certain resources require extra permissions to sync, which will automatically be given to the vCluster ServiceAccount if you enable the resource sync with the associated config value. - -## Sync from Virtual Cluster to Host Cluster - - -## Sync from Host Cluster to Virtual Cluster - - - ## How does syncing work? -## Label Syncing +## Resources available to sync from virtual cluster to host cluster + -When syncing metadata, vCluster will rewrite the following labels to avoid conflicts: -* `release` (Label is needed to avoid conflicts with the vCluster pods themselves). This is only excluded in single-namespace mode as multi-namespace mode does sync all pods into separate namespaces. -* `vcluster.loft.sh/namespace` -* `vcluster.loft.sh/managed-by` -* `vcluster.loft.sh/controlled-by` +## Resources available to sync from host cluster to virtual cluster + ## Config reference diff --git a/vcluster/configure/vcluster-yaml/sync/from-host/README.mdx b/vcluster/configure/vcluster-yaml/sync/from-host/README.mdx index 333d63f19..b00e501f3 100644 --- a/vcluster/configure/vcluster-yaml/sync/from-host/README.mdx +++ b/vcluster/configure/vcluster-yaml/sync/from-host/README.mdx @@ -6,19 +6,11 @@ description: Configuration for ... --- import SyncFromHost from '../../../../_partials/config/sync/fromHost.mdx' -import SyncFromHostResources from '@site/vcluster/_fragments/sync-from-host-resources.mdx' +import SyncFromHostResources from '../../../../_fragments/sync-from-host-resources.mdx' -Declare which host resources you want to sync to the virtual cluster. - -{/* -This page covers all synced resources, but does not document which direction the syncing occurs: -https://www.vcluster.com/docs/syncer/core_resources - -General instructions for enabling / disabling synced resources: -https://www.vcluster.com/docs/syncer/config#enable-or-disable-synced-resources -*/} - +Read more about [how syncing works](../) before deciding which resources to +sync from the host cluster to the virtual cluster. diff --git a/vcluster/configure/vcluster-yaml/sync/to-host/README.mdx b/vcluster/configure/vcluster-yaml/sync/to-host/README.mdx index 7d510990b..4c59c67a2 100644 --- a/vcluster/configure/vcluster-yaml/sync/to-host/README.mdx +++ b/vcluster/configure/vcluster-yaml/sync/to-host/README.mdx @@ -6,17 +6,10 @@ description: Configuration for ... --- import SyncToHost from '../../../../_partials/config/sync/toHost.mdx' -import SyncToHostResources from '@site/vcluster/_fragments/sync-to-host-resources.mdx' +import SyncToHostResources from '../../../../_fragments/sync-to-host-resources.mdx' -Declare which virtual cluster resources you want to sync to the host cluster. - -{/* -This page covers all synced resources, but does not document which direction the syncing occurs: -https://www.vcluster.com/docs/syncer/core_resources - -General instructions for enabling / disabling synced resources: -https://www.vcluster.com/docs/syncer/config#enable-or-disable-synced-resources -*/} +Read more about [how syncing works](../) before deciding which resources to +sync from the virtual cluster to the host cluster. From f19bfd239577e70f4ab39a93653cba63637d3187 Mon Sep 17 00:00:00 2001 From: Denise Date: Tue, 22 Oct 2024 15:15:48 -0700 Subject: [PATCH 2/3] Small tweaks based on feedback --- vcluster/_fragments/how-does-syncing-work.mdx | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/vcluster/_fragments/how-does-syncing-work.mdx b/vcluster/_fragments/how-does-syncing-work.mdx index 48078b88e..2902c2379 100644 --- a/vcluster/_fragments/how-does-syncing-work.mdx +++ b/vcluster/_fragments/how-does-syncing-work.mdx @@ -6,10 +6,13 @@ enable many other resources for all use cases. vCluster control plane runs with extra permissions to sync, which are automatically given to the vCluster ServiceAccount if you enable a resource that requires additional permissions. -All fields of a resources are synced from one direction to the other based on what section the resource is part of in the `vcluster.yaml`. There +All fields of a resources are synced from either the virtual cluster to the host cluster or the inverse (i.e. from the host cluster to the virtual cluster). The direction on how +the sync of the resources is based on what section the resource is part of in the `vcluster.yaml`. There are a few exceptions for each resource where specific fields are kept in sync in both directions (i.e. bi-directional sync). -As soon as a resource is copied by vCluster, the syncer starts watching for changes to the resource on either side. When the resource -is updated in the origin cluster, the syncer updates and copies those changes in the destination cluster. + +If the resource is being asked to sync from the virtual cluster to the host cluster, vCluster copies the resources from the vCluster and sends it to the host cluster to be created. There are +a couple of additional labels added to the host cluster resource object for vCluster to keep it in sync. The syncer starts watching for changes to the resource on either side. When the resource +is updated in the virtual cluster, the syncer copies those changes in host cluster. When a resource is defined in the `toHost` section, as soon as that resource is created in the virtual cluster, vCluster copies it to the host cluster. Any changes to the resource in the virtual cluster are automatically applied to the resource in the host cluster. @@ -19,7 +22,7 @@ Any changes to the resource in the host cluster are automatically applied to the When syncing resources, vCluster stores the mapping of the `VIRTUAL NAME <-> HOST NAME` for the actual resource, and the references of the other resources used by this resource. -Tracking references helps the vCluster to understand what resources are still needed in the host cluster. This important for secrets and configmaps because by default only the secrets and configmaps used are actually synced to the host cluster. +Tracking references helps the vCluster to understand what resources are still needed in the host cluster. This important for secrets and configmaps because by default only the secrets and configmaps used by pods are actually synced to the host cluster. Assume you've started your vCluster with ingress syncing to the host enabled. From 9be9f5207d8b2cf22bab61b0cce50827f7d0f7bd Mon Sep 17 00:00:00 2001 From: Denise Date: Tue, 5 Nov 2024 12:44:53 -0800 Subject: [PATCH 3/3] changes based on review comments --- vcluster/_fragments/how-does-syncing-work.mdx | 9 +++++---- vcluster/introduction/architecture.mdx | 5 +---- 2 files changed, 6 insertions(+), 8 deletions(-) diff --git a/vcluster/_fragments/how-does-syncing-work.mdx b/vcluster/_fragments/how-does-syncing-work.mdx index 2902c2379..aab67b63e 100644 --- a/vcluster/_fragments/how-does-syncing-work.mdx +++ b/vcluster/_fragments/how-does-syncing-work.mdx @@ -14,13 +14,14 @@ If the resource is being asked to sync from the virtual cluster to the host clus a couple of additional labels added to the host cluster resource object for vCluster to keep it in sync. The syncer starts watching for changes to the resource on either side. When the resource is updated in the virtual cluster, the syncer copies those changes in host cluster. -When a resource is defined in the `toHost` section, as soon as that resource is created in the virtual cluster, vCluster copies it to the host cluster. -Any changes to the resource in the virtual cluster are automatically applied to the resource in the host cluster. +:::warning Syncing will overwrite resources +When you enable a sync, the syncer will usually copy changes in one direction. For example, if you are syncing pods from the virtual cluster to the host cluster, when changes +are made to one of the pods in the host cluster, those fields are overwritten when the sync occurs. There are a couple of fields that are exceptions that are supported due to bi-directional sync. +::: -When a resource is defined in the `fromHost` section, as soon as that resource is created in the host cluster, vCluster copies it to the virtual cluster. +If the resource is being asked to sync from the host cluster to the virtual cluster, as soon as that resource is created in the host cluster, vCluster copies it to the virtual cluster. Any changes to the resource in the host cluster are automatically applied to the resource in the virtual cluster. - When syncing resources, vCluster stores the mapping of the `VIRTUAL NAME <-> HOST NAME` for the actual resource, and the references of the other resources used by this resource. Tracking references helps the vCluster to understand what resources are still needed in the host cluster. This important for secrets and configmaps because by default only the secrets and configmaps used by pods are actually synced to the host cluster. diff --git a/vcluster/introduction/architecture.mdx b/vcluster/introduction/architecture.mdx index 780e150d5..0944563e1 100644 --- a/vcluster/introduction/architecture.mdx +++ b/vcluster/introduction/architecture.mdx @@ -41,11 +41,8 @@ By default, the syncer component synchronizes certain low-level virtual cluster While primarily focused on syncing from the virtual cluster to the host cluster, the syncer also propagates certain changes made in the host cluster back into the virtual cluster. This bi-directional syncing ensures that the virtual cluster remains up-to-date with the underlying infrastructure it depends on. -You can [configure what and how the syncer synchronizes](../configure/vcluster-yaml/sync/) to and from the host cluster depending on your use case. +You can [read more about how the syncer works in more detail](../configure/vcluster-yaml/sync/) and learn how to sync resources to and from the host cluster depending on your use case. -## How does syncing work? - - ## Host cluster and namespace