New vernemq start script always purges

[rubengees]

I just tried out the updated vernemq.sh script which now has a better mechanism to decide if the node should leave the cluster or just be stopped.

I noticed that whenever the node is shut down, a cluster leave is performed. It makes no difference if I gracefully shut it down (with kubectl delete) or scale it down (with kubectl scale). I also tried to simulate a normal rolling release (with kubectl rollout restart statefulset vernemq).

I tried to debug the script and found that the jq expression in https://github.com/vernemq/docker-vernemq/blob/master/bin/vernemq.sh#L188 is always true. status.replicas and status.currentReplicas always have the same value, no matter in which way the pod is terminated. I also printed the status object and actually all values regarding the replicas have the same value.

Is this because of a behaviour change in kubernetes? I am testing with 1.12.3.

Also, the helm chart is currently missing the apps/statefulsets get permission here which is needed to run and test this.

cc @isavcic

[ioolkos]

@rubengees thanks, this seems relevant, going through it given the next release I'm preparing.

How would you add the stateful set permission here?:

rules:
  - apiGroups: [""] # "" indicates the core API group
    resources: ["pods"]
    verbs: ["get", "watch", "list"]

[rubengees]

I can't test right now, so please test these rules before if you want to take them into the chart.

An additional entry like this is required (see this comment on the original PR):

- apiGroups: ["apps"]
  resources: ["statefulsets"]
  verbs: ["get"]

Also, the watch permission can be removed from the pods rule since it is not actually used.

The complete block could look like this:

rules:
  - apiGroups: [ "" ]
    resources: [ "pods" ]
    verbs: [ "get", "list" ]
  - apiGroups: [ "apps" ]
    resources: [ "statefulsets" ]
    verbs: [ "get" ]