Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(k8s): use Recreate strategy to deploy updates #3469

Merged
merged 1 commit into from
Sep 21, 2024

Conversation

themightychris
Copy link
Contributor

@themightychris themightychris commented Sep 20, 2024

Description

While watching the latest Metabase update deploy, I noticed that we don't have the Metabase deployment explicitly set to use the Recreate strategy, allowing replicas of different versions to run at the same time under the default rolling strategy

While deploying new versions, it is lower risk to shut down all the old version instances before spinning up the first new version instance and running migrations. This will result in a longer period of Metabase not being available (low minutes) but ensure no issues are created by Metabase's config database being simultaneously written to by different versions of Metabase.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

How has this been tested?

PR preview action will show applied change

Post-merge follow-ups

  • Watch the post-merge deployment of this PR shut down all metabase pods before spinning up new ones

Copy link

github-actions bot commented Sep 20, 2024

The following changes will be applied to the production Kubernetes cluster upon merge.

BE AWARE this may not reveal changes that have been manually applied to the cluster getting undone—applying manual changes to the cluster should be avoided.

metabase, metabase, Deployment (apps) has changed:
...
    namespace: metabase
    labels:
      name: metabase
  spec:
    replicas: 3
+   strategy:
+     type: Recreate
    selector:
      matchLabels:
        name: metabase
    template:
      metadata:
...
metabase-test, metabase, Deployment (apps) has changed:
...
    namespace: metabase-test
    labels:
      name: metabase
  spec:
    replicas: 1
+   strategy:
+     type: Recreate
    selector:
      matchLabels:
        name: metabase
    template:
      metadata:
        labels:
          name: metabase
      spec:
        containers:
          - name: metabase
-           image: metabase/metabase:v0.48.3
+           image: metabase/metabase:v0.50.26.1
            envFrom:
              - configMapRef:
                  name: metabase-config
            ports:
            - containerPort: 3000
...

The default is rolling, but while deploying updates it is lower risk to shut down all the old version instances before spinning up the first new version instance and running migrations
@ohrite ohrite added this to the Data Infrastructure Maintenance milestone Sep 20, 2024
@evansiroky evansiroky added the kubernetes Issues related to kubernetes config. Doc and Chris are product owners. label Sep 20, 2024
@themightychris themightychris merged commit dae006d into main Sep 21, 2024
2 checks passed
@themightychris themightychris deleted the fix/metabase-update-strategy branch September 21, 2024 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kubernetes Issues related to kubernetes config. Doc and Chris are product owners.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants