ci(test): add upgrade strategy matrix for helm tests

[tthvo]

Related to #192

Description of changes

Added a matrix to specify upgrade strategy. I think this would help us test all supported upgrade paths.

Notes

This does not fix the test fail, as we must resolve #200. See #201 (comment)

It would have been nicer to specifies helm-extra-args to include/overwrite arguments for helm upgrade. But:

• The upgrade strategy precedence: reset-then-reuse-values < reuse-values < reset-values. So, specifying --reset-then-reuse-values --reuse-values will just pick up reuse-values (incorrect).

Thus, it is a bit inconvenient to have a wrapper script :D

[tthvo]

Sorry for the noises from CI fails :D I took the chance to validate the behaviour of helm upgrade --force here, but unfortunately I misunderstood it for the helm v2 behaviour.

In helm v3, helm upgrade --force will work similarly to kubectl replace, which will still fail on immutable field changes. I think we do need #200 for the upgrade to work at all.

[tthvo]

In helm v3, helm upgrade --force will work similarly to kubectl replace, which will still fail on immutable field changes. I think we do need #200 for the upgrade to work at all.

I guess #200 would help allow a nice and seamless upgrade for users. Though, there are other ways to upgrade that requires a bit of work. Here are some ways I tested that worked.

1. Delete the release but keep the release history & install with replace

$ helm uninstall --keep-history <release-name>
$ helm install --replace <release-name> <chart-source>

All resources of previous release is removed but the release history is kept intact. However, rollback would fail (i.e. immutable field).

2. Delete the main deployment & upgrade

kubectl -n <release-namespace> delete deployment/<main-deployment>
helm upgrade <release-name> <chart-source>

Since the deployment is deleted, helm will recreate it when upgrading. Again, rollback would fail unless the current deployment is deleted.

I came across this in the Prometheus chart: https://github.com/prometheus-community/helm-charts/blob/main/charts/prometheus/README.md#to-220

3. Rename the deployment

This is the idea of #200 but I guess it is simpler since we can append -main to the main deployment. Rollback should work as the deployment is recreated.

All above ways are, like you said, one time temporary workaround. #200 is still nicest way around :D