S3 snapshots not working on v1.22.0

[jonathon2nd]

On v1.22.0

16 snapshot_storage.cc:190] Creating AWS S3 client; region=ca-central-1; https=true; endpoint=s3.ca-central-1.wasabisys.com/"}
16 credentials_provider_chain.cc:28] aws: disabled EC2 metadata"}
16 credentials_provider_chain.cc:36] aws: loaded credentials; provider=environment"}
17 snapshot_storage.cc:247] Load snapshot: Searching for snapshot in S3 path: s3://preprod-dragonfly-operator/api-cache/dragonfly/snapshots"}
1 server_family.cc:988] Failed to load snapshot: Failed list objects in S3 bucket: Bucket not found: s3:"}

On v1.21.2 and v1.21.4

13 snapshot_storage.cc:190] Creating AWS S3 client; region=ca-central-1; https=true; endpoint=s3.ca-central-1.wasabisys.com/"}
13 credentials_provider_chain.cc:28] aws: disabled EC2 metadata"}
13 credentials_provider_chain.cc:36] aws: loaded credentials; provider=environment"}
14 snapshot_storage.cc:247] Load snapshot: Searching for snapshot in S3 path: s3://preprod-dragonfly-operator/api-cache/dragonfly/snapshots"}
1 server_family.cc:954] Loading s3://preprod-dragonfly-operator/api-cache/dragonfly/snapshots/dump-2024-09-04T17:10:16-summary.dfs"}

dragonfly yaml

apiVersion: dragonflydb.io/v1alpha1
kind: Dragonfly
metadata:
  name: api-cache
  namespace: api-cache
  resourceVersion: '845476149'
  uid: f4a7c374-6c0c-44d7-9cc4-a29425d25006
spec:
  args:
    - '--s3_endpoint=s3.ca-central-1.wasabisys.com/'
  authentication:
    passwordFromSecret:
      key: password
      name: db-password
      optional: false
  env:
    - name: AWS_ACCESS_KEY_ID
      valueFrom:
        secretKeyRef:
          key: awsAccessKeyId
          name: wasabi-credentials
    - name: AWS_SECRET_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          key: awsSecretAccessKey
          name: wasabi-credentials
    - name: AWS_REGION
      value: ca-central-1
    - name: AWS_BUCKET_NAME
      value: preprod-dragonfly-operator
  image: docker.dragonflydb.io/dragonflydb/dragonfly:v1.21.4
  replicas: 3
  resources:
    limits: {}
    requests:
      cpu: 250m
      memory: 250Mi
  snapshot:
    cron: 0 1 * * *
    dir: s3://preprod-dragonfly-operator/api-cache/dragonfly/snapshots
    persistentVolumeClaimSpec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi
      storageClassName: linstor-replica-one-local
status:
  isRollingUpdate: true
  phase: ready

[mircea-pavel-anton]

This is indeed troublesome especially since V1.22 is, as far as I can tell, the first version to fix the vector search implementation

[Pothulapati]

Sorry for the bad experience folks! Looks like it was a Dragonfly issue, and we fixed it. Once a new version is released, we will update the operator to use as the default.