GitOps setup

[packituz-gttz]

Is your feature request related to a problem? Please describe.
I've been following this tutorial https://cyclops-ui.com/blog/2024/11/29/gitops-clickops , however when modifying the values in Cyclops UI these values are not saved back into the repo. I understand that is the way Cyclops works, but I would like to know if there are plans for Cyclops to save this values back to the repo, so a single source of truth is accomplished.

Describe the solution you'd like
I would like Cyclops to save this values back to the repo, so a single source of truth is accomplished.

Additional context
https://cyclops-ui.com/blog/2024/11/29/gitops-clickops

Thanks!

[petar-cvit]

Hey @packituz-gttz, thanks for reaching out!

We did think about Cyclops pushing Module changes to git and then leaving a GitOps tool (ArgoCD) to sync into the cluster where our controller could pick up the changes. You would get the same developer experience with GitOps in the loop.

We can add an option to commit all changes to a git repo. That option would be in the "advanced" section while creating a Module, and you could specify which repo and path to push your Module configuration to. By default, Modules would be applied directly to the cluster.

Would that make sense from your perspective? If yes, we can create a release candidate you can try out. Let us know what you think!

[packituz-gttz]

Hi, yes that would be really useful. Thanks!

[petar-cvit]

Hey @packituz-gttz, we made a new release candidate v0.16.0-rc.2, that can commit your module configuration to a git repository.

You can install it with the following command:

kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.16.0-rc.5/install/cyclops-install.yaml && kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.16.0-rc.5/install/demo-templates.yaml

Injecting credentials

Before deploying a module, you need to configure credentials for your git repos so changes can be pushed to a remote repo. For GitHub, you will need to create a token (can be fine-grained just for a single repo) that has read and write access to contents.

You can use that token to create a secret in your cluster with the following command:

kubectl create secret generic my-cyclops-secret \
    -n cyclops \
    --from-literal=username=<github-username> \
    --from-literal=password=<github-token>

Once that is created, you can apply the following resource which maps credentials to a specific repo. You can read more about it here

apiVersion: cyclops-ui.com/v1alpha1
kind: TemplateAuthRule
metadata:
  name: private-repo-rule
  namespace: cyclops            # has to be in cyclops namespace
spec:
  repo: <git-repo>           # git repo you want to push configuration to
  password:
    name: my-cyclops-secret     # name of the previously created secret
    key: password
  username:
    name: my-cyclops-secret     # name of the previously created secret
    key: username

The <git-repo> is the repo to which you want Cyclops to push the configuration to. In the example image below it would be https://github.com/petar-cvit/gitops-infra.

Deploying a module via GitOps

When creating a new repository, in the advanced section, you can toggle the Push changes to Git? and you will be able to see fields to specify to which repo you want to commit your Module configuration.

For the configuration in the image below, Cyclops will commit your Module as yaml to the repository https://github.com/petar-cvit/gitops-infra on branch main, and create a file applications/my-module.yaml. The name of the file of your module is constructed as {path}/{module name}.yaml. If you want to create a file in the root of your repo, you can set the path to “.”

From your git repository, you can sync Modules into the cluster with ArgoCD. With such a setup, you still get a custom UI for your apps, but you also get all the benefits of the GitOps approach. Let me know what you think or if you need help with the setup

[packituz-gttz]

Hi, I've been testing it and it works great, just one issue I found while testing is that the Module will keep unsynchronized due to iconURL and templateResolvedVersion being empty on the git repository but on Kubernetes they got values, as shown in the image:

[petar-cvit]

Thanks for reporting this @packituz-gttz! We made a new release candidate (v0.16.0-rc.5) that you can install with

kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.16.0-rc.5/install/cyclops-install.yaml && kubectl apply -f https://raw.githubusercontent.com/cyclops-ui/cyclops/v0.16.0-rc.5/install/demo-templates.yaml

It will push the iconURL and templateResolvedVersion to git to keep things in sync. Other status fields won't be pushed to git, and Argo will stop showing as out of sync. Let me know if you have any other issues with this feature!

[packituz-gttz]

Hi, thank you for the changes, I can see it working now. One thing I noticed is that when deleting the module from the UI it won't delete it from the git repo, I believe it should work that way, what do you think?

I've been experimenting with Cyclops and Argo Image Updater but failed to make them work together since Image Updater is not compatible with Directory Apps, this might be outside of the scope of this feature, but I believe it would be
valuable to have the possibility of Cyclops playing nice with it. Might be possible to use a Helm chart structure and save the spec.values into a separate file like shown in here What do you think?

[petar-cvit]

Hey @packituz-gttz, thanks for the feedback!

One thing I noticed is that when deleting the module from the UI it won't delete it from the git repo

We didn't delete the Module from the git repo, just in case you want to keep it there. I think we should make it configurable in the Module configuration so you can specify if you want to delete it from git when it is deleted in the UI. Would this be helpful for your use case?

I've been experimenting with Cyclops and Argo Image Updater

Oh, awesome! Thanks for the example repo. I didn't use the image updater that much, but yeah, since you can update only apps made from Helm and Kustomize, you can’t update the apps we mentioned in our example.

I think it would be hard to keep track of Modules if we start pushing them as separate files.

One idea that could work is that we could have Kustomize that references a Module in a module.yaml file and have a separate patch.yaml file, which would contain only the patch with the version. With that in place, Cyclops could still publish Module configuration to module.yaml, and the image updater would update only the patch.yaml, and when ArgoCD deploys it, it would patch it together and deploy the newer version.

This is just an idea, I still have to try it out, but might be worth checking out 😄

[packituz-gttz]

Hi,

We didn't delete the Module from the git repo, just in case you want to keep it there. I think we should make it configurable in the Module configuration so you can specify if you want to delete it from git when it is deleted in the UI. Would this be helpful for your use case?

Sure, sound good to me. Thank you.

[petar-cvit]

@packituz-gttz, after some digging through the Argo image updater, I found something that looks like the solution I mentioned above. I made a separate repo for this example so it is easier to follow - https://github.com/petar-cvit/image-updater-demo

So the idea is that Cyclops would push the module configuration to cyclops-module/module.yaml. There is also the transform.yaml that tells Kustomize how to patch images to the Module CRD.

Here is the Argo application that needs the argocd-image-updater.argoproj.io/<image name>.force-update so it updates the image.

With this in place, the image updater will only update the kustomization.yaml, and Cyclops will update the module.yaml.

Let me know what you think! We can keep discussing here :)

[petar-cvit]

Regarding the point above on configuring how a module should be deleted, we added a section in the delete pop-up where you need to specify if you want to delete the module directly in the cluster or if you want to remove the file from git.

This section is visible only if the module has configuration for writing back to git. We will build a new release candidate shortly, and I will post it here when we do

[packituz-gttz]

Thank you I was able to get Cyclops and Argo Image Updater working together on my setup based on the example you provided, it adjust perfectly to my current use case.

Regarding the point above on configuring how a module should be deleted, we added a section in the delete pop-up where you need to specify if you want to delete the module directly in the cluster or if you want to remove the file from git.
This section is visible only if the module has configuration for writing back to git. We will build a new release candidate shortly, and I will post it here when we do

Great thanks that will be pretty useful!