feat(controller): Allow specifying full annotations for nginx canary …

…ingresses. (#3671) * refactor according to suggestions Signed-off-by: cyrilico <[email protected]> * codegen Signed-off-by: cyrilico <[email protected]> * missing codegen Signed-off-by: cyrilico <[email protected]> * Update specification.md Signed-off-by: cyrilico <[email protected]> * further clarify when to use new field Signed-off-by: cyrilico <[email protected]> --------- Signed-off-by: cyrilico <[email protected]>
argoproj · Jul 25, 2024 · 872f2ac · 872f2ac
1 parent d3d630d
commit 872f2ac
Show file tree

Hide file tree

Showing 13 changed files with 807 additions and 526 deletions.
diff --git a/docs/features/specification.md b/docs/features/specification.md
@@ -425,6 +425,8 @@ spec:
           additionalIngressAnnotations:   # optional
             canary-by-header: X-Canary
             canary-by-header-value: iwantsit
+          canaryIngressAnnotations:   # optional
+            my-custom-annotation.mygroup.com/key: value
 
         # ALB Ingress Controller routing configuration
         alb:

diff --git a/docs/features/traffic-management/nginx.md b/docs/features/traffic-management/nginx.md
@@ -7,7 +7,8 @@ The Rollout controller will always set the following two annotations on the cana
 - `canary: true` to indicate that this is the canary Ingress
 - `canary-weight: <num>` to indicate what percentage of traffic to send to the canary. If all traffic is routed to the stable Service, this is set to `0`
 
-You can provide additional annotations to add to the canary Ingress via the `additionalIngressAnnotations` field to enable features like routing by header or cookie.
+You can provide additional annotations to add to the canary Ingress via the `additionalIngressAnnotations` or `canaryIngressAnnotations` field to enable features like routing by header or cookie.
+While the former offers the possibility of reusing a common, injectable nginx annotation prefix, the latter allows defining custom full annotations of any group, which may be relevant for other, independent mechanisms, such as a third-party metrics scraper or some other infrastructure integration.
 
 
 ## Integration with Argo Rollouts
@@ -34,6 +35,8 @@ spec:
           additionalIngressAnnotations:   # optional
             canary-by-header: X-Canary
             canary-by-header-value: iwantsit
+          canaryIngressAnnotations:     # optional
+            my-custom-annotation.mygroup.com/key: value
 ```
 
 The stable Ingress field is a reference to an Ingress in the same namespace of the Rollout. The Rollout requires the primary Ingress routes traffic to the stable Service. The Rollout checks that condition by confirming the Ingress has a backend that matches the Rollout's stableService.
@@ -42,6 +45,8 @@ The controller routes traffic to the canary Service by creating a second Ingress
 
 Since the Nginx Ingress controller allows users to configure the annotation prefix used by the Ingress controller, Rollouts can specify the optional `annotationPrefix` field. The canary Ingress uses that prefix instead of the default `nginx.ingress.kubernetes.io` if the field set.
 
+If full annotations, [as defined in the Kubernetes docs](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set), perhaps from different groups, need to be declared instead, the `canaryIngressAnnotations` field can be used, which accepts a similar key-value structure, but performs no prefix injection.
+Note that, in case of collision with `additionalIngressAnnotations`, the value under `canaryIngressAnnotations` prevails.
 
 ## Using Argo Rollouts with multiple NGINX ingress controllers per service
 Starting with v1.5, argo rollouts supports multiple Nginx ingress controllers pointing at one service with canary deployments. If only one ingress controller is needed, utilize the existing key `stableIngress`. If multiple ingress controllers are needed (e.g., separating internal vs external traffic), use the key `stableIngresses` instead. It takes an array of string values that are the names of the ingress controllers. Canary steps are applied identically across all ingress controllers.

diff --git a/manifests/crds/rollout-crd.yaml b/manifests/crds/rollout-crd.yaml
@@ -934,6 +934,10 @@ spec:
                                 type: object
                               annotationPrefix:
                                 type: string
+                              canaryIngressAnnotations:
+                                additionalProperties:
+                                  type: string
+                                type: object
                               stableIngress:
                                 type: string
                               stableIngresses:

diff --git a/manifests/install.yaml b/manifests/install.yaml
@@ -13448,6 +13448,10 @@ spec:
                                 type: object
                               annotationPrefix:
                                 type: string
+                              canaryIngressAnnotations:
+                                additionalProperties:
+                                  type: string
+                                type: object
                               stableIngress:
                                 type: string
                               stableIngresses:

diff --git a/pkg/apiclient/rollout/rollout.swagger.json b/pkg/apiclient/rollout/rollout.swagger.json
@@ -1678,6 +1678,13 @@
             "type": "string"
           },
           "title": "StableIngresses refers to the names of `Ingress` resources in the same namespace as the `Rollout` in a multi ingress scenario\n+optional"
+        },
+        "canaryIngressAnnotations": {
+          "type": "object",
+          "additionalProperties": {
+            "type": "string"
+          },
+          "title": "+optional"
         }
       },
       "title": "NginxTrafficRouting configuration for Nginx ingress controller to control traffic routing"