Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add classNamespace to topology #11352

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion api/v1beta1/cluster_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,11 @@ type Topology struct {
// The name of the ClusterClass object to create the topology.
Class string `json:"class"`

// The namespace of the ClusterClass object to create the topology.
//
// +optional
ClassNamespace string `json:"classNamespace,omitempty"`

// The Kubernetes version of the cluster.
Version string `json:"version"`

Expand Down Expand Up @@ -860,7 +865,17 @@ func (c *Cluster) GetClassKey() types.NamespacedName {
if c.Spec.Topology == nil {
return types.NamespacedName{}
}
return types.NamespacedName{Namespace: c.GetNamespace(), Name: c.Spec.Topology.Class}

return types.NamespacedName{Namespace: c.GetInfrastructureNamespace(), Name: c.Spec.Topology.Class}
}

// GetInfrastructureNamespace returns common namespace for the cluster infrastructure.
func (c *Cluster) GetInfrastructureNamespace() string {
if c.Spec.Topology == nil || c.Spec.Topology.ClassNamespace == "" {
return c.Namespace
}

return c.Spec.Topology.ClassNamespace
}

// GetConditions returns the set of conditions for this object.
Expand Down
7 changes: 7 additions & 0 deletions api/v1beta1/zz_generated.openapi.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions config/crd/bases/cluster.x-k8s.io_clusters.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -438,6 +438,89 @@ spec:
template: "{{ .cluster.name }}-{{ .machinePool.topologyName }}-{{ .random }}"
```

### Defining a custom namespace for ClusterClass object

As a user, I may need to create a `Cluster` from a `ClusterClass` object that exists only in a different namespace. To uniquely identify the `ClusterClass`, a `NamespaceName` ref is constructed from combination of:
* `cluster.spec.topology.classNamespace` - namespace of the `ClusterClass` object.
* `cluster.spec.topology.class` - name of the `ClusterClass` object.

Example of the `Cluster` object with the `name/namespace` reference:

```yaml
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: my-docker-cluster
namespace: default
spec:
topology:
class: docker-clusterclass-v0.1.0
classNamespace: default
version: v1.22.4
controlPlane:
replicas: 3
workers:
machineDeployments:
- class: default-worker
name: md-0
replicas: 4
failureDomain: region
```

<aside class="note warning">

<h1>Cluster rebase across namespaces</h1>

Class namespace referenced in the `Cluster` object is equivalent to a cluster being located in the referenced namespace from the validation perspective. Changing `classNamespace` is not allowed, while using a different `CluterClass` from the same namespace is permitted in the Cluster rebase procedure.

</aside>

#### Securing cross-namespace reference to the ClusterClass

It is often desirable to restrict free cross-namespace `ClusterClass` access for the `Cluster` object. This can be implemented by defining a [`ValidatingAdmissionPolicy`](https://kubernetes.io/docs/reference/access-authn-authz/validating-admission-policy/) on the `Cluster` object.
Copy link
Member

@enxebre enxebre Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

besides the on admission check which is nice, do we have any rbac recommendation for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VAP supports secondary authz for added RBAC control, and VAP binding can be used with label selectors, which addresses #5673 (comment). From the proposal itself, it seems using the policy for added restriction on top of RBAC is within the scope.

Webhook allows to use paramRef of any kind, which can be potentially explored with specific CRD to further restrict access beyond admission, with a controller acting on that resource.

Currently, this is just an example of how an on top policy can be defined (if needed) in k8s 1.30+, where a user may decide to use different policy mechanisms to further restrict access, including a more granular RBAC. I’m thinking to showcase it as an option, but to not enforce any specific solution within this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is also an RBAC recommendation in https://kccncna2024.sched.com/event/1hoyX, if a talk considered to be one.


An example of such policy may be:

```yaml
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
name: "cluster-class-ref.cluster.x-k8s.io"
spec:
failurePolicy: Fail
paramKind:
apiVersion: v1
kind: Secret
matchConstraints:
resourceRules:
- apiGroups: ["cluster.x-k8s.io"]
apiVersions: ["v1beta1"]
operations: ["CREATE", "UPDATE"]
resources: ["clusters"]
validations:
- expression: "!has(object.spec.topology.classNamespace) || object.spec.topology.classNamespace in params.data"
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
name: "cluster-class-ref-binding.cluster.x-k8s.io"
spec:
policyName: "cluster-class-ref.cluster.x-k8s.io"
validationActions: [Deny]
paramRef:
name: "ref-list"
namespace: "default"
parameterNotFoundAction: Deny
---
apiVersion: v1
kind: Secret
metadata:
name: "ref-list"
namespace: "default"
data:
default: ""
```

## Advanced features of ClusterClass with patches

This section will explain more advanced features of ClusterClass patches.
Expand Down
1 change: 1 addition & 0 deletions internal/apis/core/v1alpha4/conversion.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ func (src *Cluster) ConvertTo(dstRaw conversion.Hub) error {
if dst.Spec.Topology == nil {
dst.Spec.Topology = &clusterv1.Topology{}
}
dst.Spec.Topology.ClassNamespace = restored.Spec.Topology.ClassNamespace
dst.Spec.Topology.Variables = restored.Spec.Topology.Variables
dst.Spec.Topology.ControlPlane.Variables = restored.Spec.Topology.ControlPlane.Variables

Expand Down
1 change: 1 addition & 0 deletions internal/apis/core/v1alpha4/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions internal/controllers/topology/cluster/cluster_controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -362,7 +362,6 @@ func (r *Reconciler) clusterClassToCluster(ctx context.Context, o client.Object)
ctx,
clusterList,
client.MatchingFields{index.ClusterClassNameField: clusterClass.Name},
client.InNamespace(clusterClass.Namespace),
); err != nil {
return nil
}
Expand All @@ -371,7 +370,9 @@ func (r *Reconciler) clusterClassToCluster(ctx context.Context, o client.Object)
// create a request for each of the clusters.
requests := []ctrl.Request{}
for i := range clusterList.Items {
requests = append(requests, ctrl.Request{NamespacedName: util.ObjectKey(&clusterList.Items[i])})
if clusterList.Items[i].GetInfrastructureNamespace() == clusterClass.Namespace {
requests = append(requests, ctrl.Request{NamespacedName: util.ObjectKey(&clusterList.Items[i])})
}
}
return requests
}
Expand Down
11 changes: 9 additions & 2 deletions internal/webhooks/clusterclass.go
Original file line number Diff line number Diff line change
Expand Up @@ -380,12 +380,19 @@ func (webhook *ClusterClass) getClustersUsingClusterClass(ctx context.Context, c
clusters := &clusterv1.ClusterList{}
err := webhook.Client.List(ctx, clusters,
client.MatchingFields{index.ClusterClassNameField: clusterClass.Name},
client.InNamespace(clusterClass.Namespace),
)
if err != nil {
return nil, err
}
return clusters.Items, nil

referencedClusters := []clusterv1.Cluster{}
for _, cluster := range clusters.Items {
if cluster.GetInfrastructureNamespace() == clusterClass.Namespace {
referencedClusters = append(referencedClusters, cluster)
}
}

return referencedClusters, nil
}

func getClusterClassVariablesMapWithReverseIndex(clusterClassVariables []clusterv1.ClusterClassVariable) (map[string]*clusterv1.ClusterClassVariable, map[string]int) {
Expand Down