Default Canary/Stable ReplicaSet Labels to Improve AnalysisRun Accuracy and User Experience

[johnmwood]

Summary

By default, during an analysis run on a canary Rollout, Argo Rollouts considers the metrics between both canary and stable replicaSets. For our users, this causes unnecessary burn down of SLO on bad revisions and makes for slower rollbacks. Further, multiple users report that they are surprised that this is the default behavior.

We have identified that ephemeral metadata is a solution to our problem in that users can join prometheus queries on metrics which can see pod labels. E.g. if we added a label argo_rollouts_status=canary|stable, they could add that metadata into their analysis template's PromQL and exclusively target metrics from canary pods during a rollout.

While users have the option to add the ephemeral metadata themselves, this is not ideal from a platform perspective because we can't guarantee a standard naming convention or format for the labels. This complicates writing ClusterAnalysisTemplates or other PromQL queries to make these labels useful.

Solution

Since the ephemeral metadata is able to add labels to replicaSets, we would like an option to default this behavior on all canary/stable replicaSets directly through a config flag on the Argo Rollouts controller. For example:

Config flag: --enable-canary-status-labels=true
Outcome: The Argo Rollouts controller will automatically append the labels argo-rollouts-status=canary|stable to all canary and stable replicaSets respectively.

This saves users the overhead of having to remember to add ephemeral metadata fields to their yaml, and provides value in standardization for our Kubernetes platform team.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[johnmwood]

Any updates on if this proposal would be worth exploring or fits into the current roadmap for any Argo Rollouts improvements?