Preview with Helmfile

[vbehar]

See #26

TL;DR

• the preview environments are based on Helm 2 and "umbrella charts" - so we can add dependencies, but it's not very flexible
• we'll need to switch to Helm 3 - related to helm 3 and kustomize support  #34
• Helmfile is a higher-level tool build on top of Helm, which can be used to declaratively define an environment composed of multiple Helm releases. It supports both Helm 2 and Helm 3, and could be used to switch smoothly from one to the other for the preview env.

[vbehar]

FYI we've been deploying our preview envs at Dailymotion using Helmfile (and Helm 3) for a few months now, and... it's so much better ;-)

[nimajalali]

@vbehar Is the helmfile feature something that's available using off the shelf jx or are you running a fork internally?

[nimajalali]

I see now that jx v3 uses helmfile.

[jstrachan]

@vbehar i wonder how do you GC your preview environments with helmfile? Do you assume all resources are in the same namespace and remove that?

Just wondered. If we moved to helmfile for previews it would be awesome if folks could create a Canary in the staging namespace- which should be easy via a helmfile - am just not sure how removal of previews would work

[vbehar]

@jstrachan good question. we have our own GC, which is based on the std one. It does the following

• find all preview envs
• only keep the preview envs with closed/merged PRs
• delete all helm 3 releases in the preview env namespace
• delete the environment crd
• and delete the namespace
• in case of error, it writes a comment on the github PR

it works in our case because we're using helmfile with helm3, and 1 namespace per preview env. we need to delete each helm release because some projects rely on helm hooks (pre-install and post-delete) to create/delete cloud resources (gcs buckets, pubsub topics, ...)

ideally we should use the helmfile destroy command, but it requires to have access to the PR's source code.

[jstrachan]

here's the first spike of helmfile based previews... https://github.com/jenkins-x/jx-preview/blob/master/README.md

we're using a Preview custom resource to keep track of the source code (and user/token to clone it) so we can do a real helmfile destroy when we destroy or garbage collect a preview.

Its working great + is fully integrated into Jenkins X 3.x now and the BDD tests are green

would love to support Terraform based previews as well so we could spin up a custom cluster / cloud infrastructure used by the preview - then tear it all down afterwards

[vbehar]

that's awesome! Mainly the fact that you use helmfile destroy, so it's a clean deletion

[jenkins-x-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle stale

[jenkins-x-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://jenkins-x.io/community.
/lifecycle rotten

[jenkins-x-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

[jenkins-x-bot]

@jenkins-x-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://jenkins-x.io/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.