Use Flux to implement multi-cluster helm application distribution #2800

Poor12 · 2022-06-02T02:26:23Z

Poor12
Jun 2, 2022

Hello, everyone. I'm working on Karmada, a CNCF multi-cluster project. Now Karmada has implemented the distribution of Kubernetes native resources. At the same time, users hope to use Karmada to achieve multi-cluster distribution of helm applications and unified management. We found that with Flux, Karmada can easily implement multi-cluster management of helm applications.
For example, with Karmada's propagationpolicy, we can progagate helm releases to member clusters. Assume Flux is installed in member clusters.

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: podinfo
spec:
  interval: 1m
  url: https://stefanprodan.github.io/podinfo     
--- 
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: helm-repo
spec:
  resourceSelectors:
    - apiVersion: source.toolkit.fluxcd.io/v1beta2
      kind: HelmRepository
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: podinfo
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      version: 5.0.3
      sourceRef:
        kind: HelmRepository
        name: podinfo
---
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: helm-release
spec:
  resourceSelectors:
    - apiVersion: helm.toolkit.fluxcd.io/v2beta1
      kind: HelmRelease
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Then, we can find that helmrelease is created in member1 and member2 cluster.

helm --kubeconfig=/root/.kube/members.config --kube-context member1 list
NAME   	NAMESPACE	REVISION	UPDATED                               	STATUS  	CHART        	APP VERSION
podinfo	default  	1       	2022-05-27 01:44:35.24229175 +0000 UTC	deployed	podinfo-5.0.3	5.0.3

Moreover, With Karmada's OverridePolicy, it's very convenient to specify your helm release in specific clusters.
e.g

apiVersion: policy.karmada.io/v1alpha1
kind: OverridePolicy
metadata:
name: example-override
namespace: default
spec:
resourceSelectors:
  - apiVersion: helm.toolkit.fluxcd.io/v2beta1
    kind: HelmRelease
    name: podinfo
overrideRules:
  - targetCluster:
      clusterNames:
        - member1
    overriders:
      plaintext:
        - path: "/spec/values"
          operator: add
          value:
            replicaCount: 2

I wonder if Flux community has the same concern or has a plan to support multi-cluster environment. I'd love to share the pratice at community meetings，and contribute ‘Karmada + Flux’ document to the community.
Thanks. Looking forward to your reply.