Reconcile ServiceTemplates in ManagedCluster controller

[Kshatrix]

• Extend ManagedCluster spec with a list of serviceTemplates referencing resources in the same namespace:

beachHeadServices:
- serviceTemplate:
  config:

• Extend ManagedCluster reconciler to install requested services

[wahabmk]

Update

I am trying to get a service (defined by a ServiceTemplate) installed via Sveltos in the "managedcluster_controller". For now I am trying to get nginx installed this way on the target cluster:

apiVersion: hmc.mirantis.com/v1alpha1
kind: ServiceTemplate
metadata:
  name: ingress-nginx
spec:
  helm:
    chartName: ingress-nginx
    chartVersion: 2.0.0

Which is referenced via the ManagedCluster object as:

apiVersion: hmc.mirantis.com/v1alpha1
kind: ManagedCluster
metadata:
  name: wali-aws-dev
  namespace: ${NAMESPACE}
spec:
. . . . . . . .
  services:
    - template: ingress-nginx
      install: true
      config:
        releaseName: ingress-nginx
        releaseNamespace: ingress-nginx
        createNamespace: true
. . . . . . . .

Each template referes to a flux HelmChart, and while this works fine for ClusterTemplates because we can easily feed this into flux's HelmRelease. For ServiceTemplates, however, since we are using Sveltos, we need to retrieve the URL to pull the chart from, which I can retrieve as:

ServiceTemplate -> HelmChart -> HelmRepository -> HelmRepository.Spec.URL

This URL for our dev environment is oci://hmc-local-registry:5000/charts. So I created a Sevltos ClusterProfile during the reconcile process as below:

kind: ClusterProfile
. . . . . . . .
  helmCharts:
    # NOTE: The reason chartName == repositoryURL is because of
    # https://projectsveltos.github.io/sveltos/addons/helm_charts/#:~:text=For%20OCI%20charts%2C%20the%20chartName%20needs%20to%20have%20whole%20URL.
  - chartName: oci://hmc-local-registry:5000/charts/ingress-nginx
    chartVersion: 2.0.0
    releaseName: ingress-nginx
    releaseNamespace: ingress-nginx
    repositoryName: ingress-nginx
    repositoryURL: oci://hmc-local-registry:5000/charts/ingress-nginx
. . . . . . . .

However, Sveltos fails to install nginx on the target cluster due to following (seen in the ClusterSummary object):

  - failureMessage: 'failed to do request: Head "https://hmc-local-registry:5000/v2/charts/ingress-nginx/manifests/2.0.0":
      http: server gave HTTP response to HTTPS client'

There is a flag --plain-http which will resolve this because I can pull the chart successfully with:

# helm pull oci://hmc-local-registry:5000/charts/ingress-nginx --version 2.0.0 --plain-http                           
Pulled: hmc-local-registry:5000/charts/ingress-nginx:2.0.0
Digest: sha256:ae349c7ae29737912640fcf8c0d0514fbf6fc1a5de6209a37f34572c96e853a6

But ClusterProfile in Sveltos doesn't have that feature yet. It has skipTLSVerify option but that doesn't work:

# helm pull oci://hmc-local-registry:5000/charts/ingress-nginx --version 2.0.0 --insecure-skip-tls-verify
Error: failed to do request: Head "https://hmc-local-registry:5000/v2/charts/ingress-nginx/manifests/2.0.0": http: server gave HTTP response to HTTPS client

So we need Sveltos to support passing --plain-http option to it's helm client. I think it may be resolved by adding the following to this if condition:

options = append(options, registry.ClientOptPlainHTTP())