source-automation-controller

[saltyblu]

Hello,

currently we are using a git repository called -deployment for each service containing all corresponding kustomizations, these git repos get deployed via Flux, our main repo is adding a image-automation as well for each service.

I would like to have the same "image-automation-controller" for GitRepositories. That we could automate releases of the -deployment git like the image-automation-controller does.

Currently we are developing a bunch of microservices and they are close to the Kustomizations, due to the fact that the kustomization is rolling out infrastructure changes in our cloud environment as well.

[kingdonb]

Greetings,

Your timing is great, we've just released a feature which does satisfy the use case you're asking about, although it's slightly different than what you described.

So for reference, GitRepository has these fields in the spec:

KIND:     GitRepository
VERSION:  source.toolkit.fluxcd.io/v1beta2

RESOURCE: ref <Object>

DESCRIPTION:
     Reference specifies the Git reference to resolve and monitor for changes,
     defaults to the 'master' branch.

FIELDS:
   branch	<string>
     Branch to check out, defaults to 'master' if no other field is defined.
     When GitRepositorySpec.GitImplementation is set to 'go-git', a shallow
     clone of the specified branch is performed.

   commit	<string>
     Commit SHA to check out, takes precedence over all reference fields. When
     GitRepositorySpec.GitImplementation is set to 'go-git', this can be
     combined with Branch to shallow clone the branch, in which the commit is
     expected to exist.

   semver	<string>
     SemVer tag expression to check out, takes precedence over Tag.

   tag	<string>
     Tag to check out, takes precedence over Branch.

You can use either the semver or tag field to ship manifests in an automated way, you can use semver tags and select the semver with a wildcard such that the latest version is always used (except when manually reverted), or you can use a mutable tag such as production that represents your environment, really tag is the same thing as branch, they are both a ref, so you can also use branch here.

The ImageAutomationController makes git commits when a new image is available according to the policy. We don't have GitAutomationController or GitPolicy, to go with GitRepository here.

What we do have now, as of Flux 0.32, is the OCIRepository resource which you can use with ImageUpdateAutomation, if that's a requirement (to have a commit that can be reverted in the repository whenever there is an upgrade processed by some automation) or you can use without ImageUpdateAutomation, just as above GitRepository is capable of semver-driven automation or tag-based automation, the OCIRepository can be used in the same way.

$ k explain ocirepository.spec.ref
KIND:     OCIRepository
VERSION:  source.toolkit.fluxcd.io/v1beta2

RESOURCE: ref <Object>

DESCRIPTION:
     The OCI reference to pull and monitor for changes, defaults to the latest
     tag.

FIELDS:
   digest	<string>
     Digest is the image digest to pull, takes precedence over SemVer. The value
     should be in the format 'sha256:<HASH>'.

   semver	<string>
     SemVer is the range of tags to pull selecting the latest within the range,
     takes precedence over Tag.

   tag	<string>
     Tag is the image tag to pull, defaults to latest.

The major difference with respect to your question, I think, is that since every OCIRepository also can be represented as an ImageRepository, you can now put these tools together in a way that was not possible before. You can drive manifests from an OCIRepository that is driven by your GitRepository, and since you control when tags are pushed there in your CI process, you can have better control over what is released when, via Pull Request.

Here's the first major doc about working with OCIRepository resources, this is a brand new feature:

https://fluxcd.io/docs/cheatsheets/oci-artifacts/

Please let us know if it helps you satisfy your use case!

[saltyblu]

Hi,
that is a great thing.

The automation that is in place already using Semver in GitRepositories is not really suitable for us, because this approach does not reflect each change inside the git repository itself.

The OCIRepositoriy source would then do the same as the Image Repository and the ImageAutomationController is able to handle OCIRepo events?

That would mean we should use something like Kustomizer to push our Kustomize Artifacts into the OCIRegistry? Or is there a more production-ready solution :)

[stefanprodan]

Flux has full capabilities to build, push and reconcile OCI artifacts without any 3rd party tooling. Please see https://fluxcd.io/docs/cheatsheets/oci-artifacts/

[saltyblu]

Okay, but still I need something like

apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageUpdateAutomation

only for Source Repositories, adding a OCI Repo would mean to store my kustomization files inside an OCI Registry

here an example, what I thought about:

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: example-nginx-deployment
spec:
  interval: 1m0s
  ref:
    branch: v1.0.0
  secretRef:
    name: flux-system
  url: https://github.com/paymenttools/example-nginx.git
...
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: example-nginx
spec:
  interval: 5m0s
  path: k8s/dev
  prune: true
  force: true
  sourceRef:
    kind: GitRepository
    name: example-nginx-deployment
...
---
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: SourcePolicy
metadata:
  name: example-nginx
spec:
  SourceRepositoryRef:
    name: example-nginx
  filterTags:
    pattern: "^v(?P<version>[0-9]+)"
    extract: "$version"
  policy:
    numerical:
      order: asc
...
---
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: SourceUpdateAutomation
metadata:
  name: example-nginx
spec:
  interval: 1m0s
  sourceRef:
    kind: GitRepository
    name: flux-system
    namespace: flux-system
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: [email protected]
        name: Peter Payment
      messageTemplate: "{{range .Updated.Images}}{{println .}}{{end}}"
    push:
      branch: main
  update:
    path: ./clusters/dev/workload/example/example-nginx
    strategy: Setters

[5cat]

@kingdonb The OCI solution an interesting workaround and I liked it, and @stefanprodan gave a similar answer in #2978 (comment)

But sadly a lot of 3rd party apps such as metallb doesnt package the app in an oci artifact registry. Does this mean I would need to maintain an OCI mirror registry that contains all the apps I need? If that is a yes then it means:

• I would need to monitor the original apps' repos/charts then republish them in my OCI registry
• so I would need to have a tool that monitor git-repos/helm-charts to build and push OCI artifacts
• and now I'm back at square one currently not having that tool.

This means I would need to make every app in my cluster to use an OCI registry If I wanted to use the image-automation-controller
So the workflow is the following: |image-automation-controller| - |oci-registry| - |the-missing-tool| - [ |git-repos| , |helm-charts| ]
Lets assume we can design such a tool that integrates into flux in a DRY manner, so the git-repos and helm-charts will be of type source.toolkit.fluxcd.io/v1beta1 so we can offload the syncing task to the flux source-controller.
Then we just need to create oci-publisher-controller which monitors the source changes then build and push to the oci registry, the source-watcher example can help in here.

Now my question, is this a suitable solution assuming i have the time to create this tool? or is there a simpler solution that achieves what is in the original asked question for 3rd party libraries i have no control over their CI build process?