From efe3914dc90958568a86f2e655835a0199b619e9 Mon Sep 17 00:00:00 2001 From: Katrina Verey Date: Fri, 27 Aug 2021 15:56:12 -0700 Subject: [PATCH] Update Composition KEP for move to implementable --- keps/prod-readiness/sig-cli/2299.yaml | 6 + .../README.md | 576 +++++++++++------- .../kep.yaml | 4 +- 3 files changed, 363 insertions(+), 223 deletions(-) create mode 100644 keps/prod-readiness/sig-cli/2299.yaml diff --git a/keps/prod-readiness/sig-cli/2299.yaml b/keps/prod-readiness/sig-cli/2299.yaml new file mode 100644 index 00000000000..928f5a0a346 --- /dev/null +++ b/keps/prod-readiness/sig-cli/2299.yaml @@ -0,0 +1,6 @@ +# The KEP must have an approver from the +# "prod-readiness-approvers" group +# of http://git.k8s.io/enhancements/OWNERS_ALIASES +kep-number: 2299 +alpha: + approver: "@ehashman" diff --git a/keps/sig-cli/2299-kustomize-plugin-composition/README.md b/keps/sig-cli/2299-kustomize-plugin-composition/README.md index 84d035fceff..802d27efe7a 100644 --- a/keps/sig-cli/2299-kustomize-plugin-composition/README.md +++ b/keps/sig-cli/2299-kustomize-plugin-composition/README.md @@ -12,17 +12,20 @@ - [Story 2](#story-2) - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) - [Integration with Kustomization](#integration-with-kustomization) + - [Integration with Kubectl Kustomize](#integration-with-kubectl-kustomize) + - [Plugin execution flags](#plugin-execution-flags) - [Risks and Mitigations](#risks-and-mitigations) - - [Security of running containers locally](#security-of-running-containers-locally) - [Design Details](#design-details) - [Key terminology](#key-terminology) - [API schema](#api-schema) - - [Built-in modules](#built-in-modules) - - [Function invocation](#function-invocation) + - [Built-in transformers](#built-in-transformers) - [Composition evaluation](#composition-evaluation) - - [Function packaging and distribution](#function-packaging-and-distribution) - [Test Plan](#test-plan) - [Graduation Criteria](#graduation-criteria) + - [Alpha](#alpha) + - [Beta](#beta) + - [GA](#ga) + - [Deprecation](#deprecation) - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) - [Version Skew Strategy](#version-skew-strategy) - [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) @@ -41,39 +44,58 @@ Items marked with (R) are required *prior to targeting to a milestone / release*. -- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR) -- [ ] (R) KEP approvers have approved the KEP status as `implementable` -- [ ] (R) Design details are appropriately documented -- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input -- [ ] (R) Graduation criteria is in place -- [ ] (R) Production readiness review completed +- [x] (R) Enhancement issue in release milestone, which links to KEP dir in + [kubernetes/enhancements] (not the initial KEP PR) +- [x] (R) KEP approvers have approved the KEP status as `implementable` +- [x] (R) Design details are appropriately documented +- [x] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input +- [x] (R) Graduation criteria is in place +- [x] (R) Production readiness review completed - [ ] Production readiness review approved - [ ] "Implementation History" section is up-to-date for milestone -- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] -- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes +- [ ] User-facing documentation has been created in [kubernetes/website], for publication to + [kubernetes.io] +- [ ] Supporting documentation—e.g., additional design documents, links to mailing list + discussions/SIG meetings, relevant PRs/issues, release notes -[kubernetes.io]: https://kubernetes.io/ -[kubernetes/enhancements]: https://git.k8s.io/enhancements +[kubernetes.io]: https://kubernetes.io/[kubernetes/enhancements]: https://git.k8s.io/enhancements [kubernetes/kubernetes]: https://git.k8s.io/kubernetes -[kubernetes/website]: https://git.k8s.io/website +[kubernetes/website]: https://git.k8s.io/website[Configuration Functions Specification]: +https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md ## Summary -Introduces a new Kustomize API (`kind`) that is oriented around Kustomize plugins, making them a first-class expression of a resource configuration bundle. +Introduces a new Kustomize API (`kind`) that is oriented around Kustomize plugins, making them a +first-class expression of a resource configuration bundle. -This new API will provide new sophisticated capabilities for using plugins as composable -units and include support for automatic discovery and installation of plugins. It will showcase -plugins as a way to implement composable, declarative client-side abstractions. +This new API will provide new sophisticated capabilities for using plugins as composable units. It +will showcase plugins as a way to implement composable, declarative client-side abstractions. ## Motivation -The `Kustomization` API was designed around built-in Kustomize transformer and generator -operations applied to Kustomize bases. This API is suboptimal for workflows that -are primarily composed of Kustomize plugins. Challenges with the current approach include: - -1. Packaging, distribution and installation of plugins is immature and non-declarative. -2. `Kustomization` separates plugins into three categories: generators, transformers and validators. With abstraction-based plugins, which mechanism the plugin uses is opaque to the end user and may even change based on input. This leads all abstraction-based plugins to be placed under `transformers`. Because these are executed after built-in transformers, an additional `Kustomization` layer is typically required to use built-ins fruitfully in abstraction-based workflows. -3. Although it is possible to transform plugin configuration as resources before execution, it is not possible to have a sensible distribution of concerns in configuration sets composed this way. Notably, plugin configuration must be finalized (e.g. tailored to an environment) before it can be combined with built-in configuration at all, and built-in transformers cannot be specified in bases, or else they will not apply to the results of the plugins. +The `Kustomization` API was designed around built-in Kustomize transformer, generator and validator +operations applied to Kustomize bases. While plugin transformers can be invoked via the appropriate +field, `Kustomization` is suboptimal for workflows that are primarily composed of Kustomize +plugins. Challenges with the current approach include: + +1. **Sequencing of built-in and plugin transformers is static**. The end user can choose to +categorize a plugin as a generator, transformer or validator, but can only control relative +execution order within those fields. Especially in the case of abstraction-based plugins, the user +may not know which category they fall in, and indeed the plugin may change what it does based on +input. This can be addressed by placing all plugins under `transformers`. However, doing so means +they will all be executed after built-in transformers. The result is that an additional +`Kustomization` layer is typically required to use built-ins fruitfully in abstraction-based +workflows. + +1. **Plugin config transformation must be isolated**. Although it is possible to +transform plugin configuration as resources before execution, it is not possible to have a sensible +distribution of concerns in configuration sets composed this way. Notably, plugin configuration +must be finalized (e.g. tailored to an environment) before it can be combined with built-in +configuration at all, and built-in transformers cannot be specified in bases, or else they will not +apply to the results of the plugins. [example/kustomization](example/kustomization) demonstrates this difficulty. + +The sequencing issue could be solved within `Kustomization`; it is the difficulty with plugin +config transformation that motivates an entirely new Kind. ### Goals @@ -81,59 +103,62 @@ Develop an API (`kind`) for Kustomize that is focused on plugin-based workflows. A successful implementation of this API should have the following characteristics: -1. The orchestration model used to evaluate the API must be simplified in a way that is optimized for plugins. Notably, it must be easy to recursively compose and modify lists of plugins in a way that yields an overall configuration structure that reflects the user's intent. -1. Plugin orchestration must be central to the new API format. Support for existing Kustomize operations should be compiled in, but expressed in the same way as extensions plugin operations. -1. The machinery for the new API must enable seamless invocation of sets of approved -plugins, and must not require out-of-band imperative installation steps. -1. The new API must cleanly integrate with the existing Kustomize tool (i.e. `kustomize build`). +1. It must be easy to recursively compose and modify lists of plugins in a way that yields an +overall configuration structure that reflects the user's intent. + +1. Plugin orchestration must be +central to the new API format. Built-in and plugin transformer config should be treated the same +way in the new API. + +1. The new API must cleanly integrate with the existing Kustomize tool +(i.e. `kustomize build`) as well as existing Kustomize kinds. ### Non-Goals -- Replace Kustomize as "the way" to do anything. +- Replace Kustomization for use cases that involve few or no plugins. - Expand the scope of the existing `Kustomization` API. -- Directly integrate with `kubectl apply`, `kubectl diff`, etc. The new API will - be compatible with those and other tools via evaluation into a resource list - suitable for use in gitops workflows. -- Pull resources or files from a remote git source. +- Introduce plugin distribution or discovery mechanisms. +- Change the flags and parameters used for invoking plugins in any way. ## Proposal -Introduce `Composition` as a new API `kind` recognized by `kustomize build`. A `Composition` must be defined in a `composition.yaml` file, and inclusion of both `composition.yaml` and `kustomization.yaml` in the target directory of a build will be considered an error. +Introduce `Composition` as a new API `kind` recognized by `kustomize build`. A `Composition` must be +defined in a `composition.yaml` file. Inclusion of both `composition.yaml` and `kustomization.yaml` +in the target directory of a build will be considered an error. -The Composition API enables users to: +The `Composition` API enables users to: -- Define a list of Kubernetes-style configuration objects called **modules**. -A module is a client-side resource that expresses the desired state implemented by -a Kustomize plugin. -- Import modules from another Composition and add them to the list. -- Override an imported module's fields with new values. -- Reorder the list of modules prior to execution. +- Define a list of transformer configuration that intermingles built-in and plugin transformers. +- Import transformers from another `Composition` and add them to the list. +- Override an imported transformer's fields with new values. +- Reorder the list of transformers prior to execution. -Once a user has written a Composition, all they need to do is run `kustomize build` -to turn it into a list of Kubernetes resources that they can commit to git. +Once a user has written a `Composition`, they can run `kustomize build` to turn it into a list of +Kubernetes resources that they can commit to git, the same way they would with `Kustomization`. To provide this experience, Kustomize will need to do the following during builds: -- Consolidate the Composition and its imports into a finalized list of modules. -- Automatically fetch and execute the plugin that implements each module. -- Pass the output of each plugin invocation as the input to the next plugin. +- Consolidate the `Composition` and its imports into a finalized list of transformers (which may be + plugins). +- Execute each transformer in sequence (no changes to plugin execution are proposed by this KEP). +- Pass the output of each transformer as the input to the next. ### User Stories (Optional) #### Story 1 -As a user, I can define my configuration as a sequence of declarative abstractions -implemented by plugins that are automatically discovered and installed. +As a user, I can define my configuration as a sequence of declarative abstractions implemented by +plugins. ```yaml # app/composition.yaml kind: Composition -modules: +transformers: # generate resources for a Java application - apiVersion: example.com/v1 kind: JavaApplication - provider: {container: {image: example/module_providers/java:v1.0.0}} + provider: {container: {image: docker.example.co/kustomize_transformers/java:v1.0.0}} metadata: name: my-app spec: @@ -142,26 +167,26 @@ modules: # transform resources to inject a logger - apiVersion: example.com/v1 kind: Logger - provider: {container: {image: example/module_providers/logger:v1.0.3}} + provider: {container: {image: docker.example.co/kustomize_transformers/logger:v1.0.3}} metadata: name: logger ``` #### Story 2 -As a user, I can reuse and extend an existing Composition by importing its modules and -optionally overriding imported modules' fields. +As a user, I can reuse and extend an existing `Composition` by importing its transformers and +optionally overriding imported transformers' fields. ```yaml # staging/composition.yaml kind: Composition -modulesFrom: -# import the JavaApplication and Logger modules +transformersFrom: +# import the JavaApplication and Logger transformers - path: ../app/composition.yaml importMode: prepend -moduleOverrides: +transformerOverrides: # override the JavaApplication version before the function is run - apiVersion: example.com/v1 kind: JavaApplication @@ -170,11 +195,11 @@ moduleOverrides: spec: version: v1.1-beta -modules: -# transform resources from imported modules to inject metrics +transformers: +# transform resources from imported transformers to inject metrics - apiVersion: example.com/v1 kind: Prometheus - provider: {container: {image: example/module_providers/prometheus:v1.0.2}} + provider: {container: {image: docker.example.co/kustomize_transformers/prometheus:v1.0.2}} metadata: name: metrics spec: @@ -185,62 +210,55 @@ modules: #### Integration with Kustomization -For the alpha phase, the integration between Composition and Kustomization should be unidirectional: Composition can process a Kustomization via a built-in transformer, but Composition cannot be loaded from a Kustomization. - -For the beta phase, we should consider enabling Kustomizations to reference Compositions in the `resources` and various plugin fields. When used in `resources` or `generators`, the Composition would be executed with empty input and the results would be added to the resource list. When used in `validators` or `transformers`, the resource list would be provided as input to the first module in the consolidated Composition, and in the case of `transformers`, the output would replace the resource list. - -Regardless of Kustomization integration, all extensions plugin execution from either Kind should remain gated behind `--enable-alpha-plugins`. This means Composition will essentially be useless in `kubectl -k` for the time being, since plugins cannot be enabled in that context. If the evolution of plugins leads to enablement in kubectl becoming acceptable in the future, plugins defined in both Kinds should be enabled the same way. +By beta at the latest, `Kustomization`s should be able to reference `Composition`s in various fields +as follows: +- the `resources` and `generators` fields: the `Composition` will be executed with empty input to the + first transformer in the consolidated `Composition`, and the results will be appended to the + resource list. +- the `transformers` field: the resource list will be provided as input to the first transformer in + the consolidated `Composition`, and the output will replace the resource list. +- the `validators` field: the resource list will be provided as input to the first transformer in + the consolidated `Composition`, and the output (if any) will be discarded. -### Risks and Mitigations - -#### Security of running containers locally - -Composition has similar capabilities to Kustomization. +#### Integration with Kubectl Kustomize -The key difference for security is that it does not require plugins to be installed in advance / out of band. This difference is -key to the user experience, and it does decrease user awareness of exactly what is -being executed. As such, it must be made clear to users that Compositions are not inherently -secure and that they should only build Compositions they trust. +`Composition` support will neither be required nor suppressed in `kubectl kustomize` during alpha. In +other words, if a `kubectl kustomize` release happens during alpha, it can include the feature, but +graduation to beta does not require this. `Composition` support MUST be included in at least two +`kubectl kustomize` releases during beta. -This risk will be mitigated in the following ways: +#### Plugin execution flags -1. Container-based functions will be run without network or volume access. This does -limit the functionality they can implement, but it also encourages plugins -to be developed such that the module resources that configure them express -the entire desired state. Should this prove unacceptably limiting, network or volume access -may be introduced in a future version, but likely with a more explicit and restrictive -permissions model than plugins have today. The same is true for exec plugins: they -will not be initially supported, and additional restrictions will likely be placed on them -if support is introduced in the future. +This KEP does not propose any changes to the flags used to gate plugins, or to the parameters +available to configure container/exec/starlark plugin execution. All plugin execution from either +Kind will remain gated behind `--enable-alpha-plugins`, as well as existing additional flags for +particular plugin provider types. Any changes to plugin gating should apply identically to both +Kinds. This does limit the usefulness of `Composition` within `kubectl -k` for the time being, since +only a subset of plugins can be enabled in that context. -1. Even if network or volume access support is added, containers will still run -without it unless the user's Composition explicitly grants it. If exec support -is added, a checksum may be required in addition to the binary location. +### Risks and Mitigations -1. Plugins will not be trusted by default. The execution framework will feature -a pluggable trust model that enables users and organizations to allowlist plugin -sources. It should be possible for large organizations to distribute binaries with -specific allowlists compiled in, and for additional sources to be allowlisted at -invocation time. +This proposal effectively takes a feature set that exists in Kustomize today and makes it much more +usable in certain workflows. As such, it does not come with any novel technical risks. ## Design Details ### Key terminology -This terminology is not set in stone at this early stage, but should be -used consistently within this proposal. +This terminology is not set in stone at this early stage, but should be used consistently within +this proposal. * **Composition**: a new plugin-oriented API (`kind`) to be understood by `kustomize build`. - * **module**: a client-side resource that expresses the desired state implemented by -a Kustomize plugin. Modules are analogous to server-side custom resource instances. - * **Built-in module**: a module whose implementation is provided by Kustomize itself. - * **Extensions module**: a module whose implementation is provided by a plugin. - * **plugin/function**: a Starlark or container function that complies with -the existing [Configuration Functions Specification](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md). Functions are analogous to server-side controllers. -Although they power the new API, functions are not the center of the user experience, the same way -controller code is not the focus of (or even referred to in) server-side resource expressions. - * **module definition**: an OpenAPIv3 schema describing a module. Analogous - to a server-side CRD. + * **Transformer**: a program that generates, manipulates and/or validates a stream of Kubernetes + resources, along with the corresponding YAML config object used to configure that program. + * **Plugin transformer**: A user-authored transformer. + * **Built-in transformer**: A transformer whose configuration spec and implementation are part + of Kustomize itself. + * **Plugin provider**: The program that implements the plugin (e.g. container, script, Go + program). Analogous to the controller for a custom resource. + * **Plugin/transformer config**: The KRM-style YAML document that declares the desired state the + transformer implements. In the case of a plugin, it includes the provider to execute and the + specification to follow in doing so. Analogous to a custom resource object. ### API schema @@ -250,15 +268,16 @@ Example showing all fields: apiVersion: kustomize.config.k8s.io/v1alpha1 kind: Composition -# `modulesFrom` imports module lists from other Compositions. +# `transformersFrom` imports transformer lists from other Compositions. # The target Compositions' own imports, overrides and reorderings are applied before it is imported. -modulesFrom: +transformersFrom: - path: ../app/composition.yaml importMode: prepend # this is the default; append is also possible -# `moduleOverrides` allows fields of modules imported via `modulesFrom` to be changed before execution. +# `transformerOverrides` allows fields of transformers imported via `transformersFrom` to be changed +# before execution. # Entries are treated as strategic merge patches. -moduleOverrides: +transformerOverrides: - apiVersion: example.com/v1 kind: JavaApplication metadata: @@ -267,8 +286,8 @@ moduleOverrides: application: team/my-app version: v3.1.3 -# `modules` adds new modules to the list. -modules: +# `transformers` adds new transformers to the list. +transformers: - apiVersion: example.com/v1 kind: Prometheus provider: @@ -277,20 +296,29 @@ modules: metadata: name: metrics-injector -# `moduleOrder` allows advanced users to reorder the merged modules list explicitly. -# modules can be referred to by name alone as long as names are unique. -moduleOrder: +# `transformerOrder` allows advanced users to reorder the merged transformers list explicitly. +# transformers can be referred to by name alone as long as names are unique. +transformerOrder: - name: my-app - name: logger - name: metrics ``` +Notes: +- One notable difference from `Kustomization` is that in addition to being able to provide a path to + a file containing the transformer config, you can specify the config itself (not strigified) + inline. +- A second difference is the introduction of a top-level reserved `provider` field, replacing the + JSON annotation currently used for this information. +- The `metadata.name` field is defaulted to `Kind` in kebab case. However, `metadata.name` MUST be +specified when using multiple independent instances of the same GVK within a `Composition` tree. +
Full schema ```yaml definitions: - module: + transformer: type: object additionalProperties: true required: @@ -349,7 +377,7 @@ properties: kind: type: string enum: ["Composition"] - modulesFrom: + transformersFrom: type: array items: type: object @@ -363,15 +391,15 @@ properties: importMode: type: string enum: ["append", "prepend"] - modules: + transformers: type: array items: - "$ref": "#/definitions/module" - moduleOverrides: + "$ref": "#/definitions/transformer" + transformerOverrides: type: array items: - "$ref": "#/definitions/module" - moduleOrder: + "$ref": "#/definitions/transformer" + transformerOrder: type: array items: type: object @@ -393,139 +421,246 @@ properties: ```
-### Built-in modules +### Built-in transformers -The Composition implementation itself should provide a select few modules that -implement essential functionality, notably functionality that is available through -Kustomization built-ins today. +[everything is already a transformer]: +https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/#everything-is-a-transformer +[accumulateResources]: +https://github.com/kubernetes-sigs/kustomize/blob/bf6b207cc997b689887c5b1cfc9a4f81f3012b12/api/internal/target/kusttarget.go#L346 -Essential starter set: +In Kustomize, [everything is already a transformer]. In a Composition, users can include any of the +existing built-in transformers (including generators and validators) in their transformers list. -1. `kind: StaticResources`: a way to load custom-written Kubernetes resources from the -filesystem for processing. Equivalent to the `resources` field in Kustomization, but -not capable of fetching from remote sources. -1. `kind: Kustomize`: support basic transformations via fields such as `spec.commonLabels`, -as well as processing a Kustomization.yaml referred to by path. +```yaml +transformers: +- apiVersion: builtin + kind: PrefixSuffixTransformer + metadata: + name: myFancyNamePrefixer + prefix: bob- + fieldSpecs: + - path: metadata/name +``` -The following example shows how this core Kustomize functionality would be expressed -as modules that fit in elegantly alongside plugin-driven modules: +One important exception to this is the `resources:` field, which is not already a generator. Work +will need to be done to expose [accumulateResources] as one. ```yaml -... -modules: -- metadata: - name: local-resources - kind: StaticResources - apiVersion: kustomize.config.k8s.io/v1alpha1 - spec: - paths: - - ../resources -- metadata: - name: prod-customizations - kind: Kustomize - apiVersion: kustomize.config.k8s.io/v1alpha1 - spec: - commonLabels: - env: production - namespace: my-app-prod +transformers: +- apiVersion: builtin + kind: ResourceAccumulator + paths: + - path/to/file.yaml ``` -Other possibilities include: -- A validation module. -- Exposing individual Kustomization built-in transformers in Composition as well. The built-in HelmChartInflationGenerator would be particularly well-suited for this, since it provides module-like encapsulated functionality. +In addition to that, `kind: Kustomization` itself may be inlined as a transformer. Because it is a +transformer in this context, resources from previous transformers will be written to a temp file +that will be automatically prepended to the `Kustomization`'s `resources` field. -### Function invocation +```yaml +transformers: +- apiVersion: kustomize.config.k8s.io/v1beta1 + kind: Kustomization + commonLabels: + foo: bar + resources: + - config.yaml +``` -To work with Composition, functions need to comply with the existing [Configuration Functions Specification](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md). +### Composition evaluation -Function invocation machinery will be built using the existing `sigs.k8s.io/kustomize/kyaml/fn/runtime` package. -It will work largely the same way function invocation does in Kustomization, with -the following notable exceptions: -* The function to be invoked will be specified in a new reserved field called `provider` -rather than being embedded in module metadata. -* The `provider` field will only support the `container.image` and `starlark.path` options, -at least at first. -* Composition will _always_ use the `ResourceList` input format when -invoking functions, and will _always_ include the `functionConfig` field. -That field will contain the module. +Composition topologically sorts imported transformers with its own transformers and then runs the +combined list sequentially. This means that arbitrary depths of imports always result in a single +consolidated Composition prior to execution. Once that consolidated Composition has been compiled, +its transformers will be invoked the same way they are today from the `transformers` field in +`Kustomization`. -### Composition evaluation +For use cases that require resources to be evaluated eagerly before further processing, for example +diamond-shaped configuration, the built-in `ResourceAccumulator` can be given a Composition to render +into resources. -Composition topologically sorts imported modules with its own modules -and then runs the combined list sequentially. This means that arbitrary depths of -imports always result in a single consolidated Composition prior to execution. +If it is deemed useful for debugging, a command for consolidating Composition without executing it +may be introduced to the `kustomize cfg` command group (or equivalent, if `cfg` is replaced). -For use cases that require resources to be evaluated eagerly before further processing, -for example diamond-shaped configuration, the built-in `StaticResources` module can be given -a Composition to render into resources. +### Test Plan -### Function packaging and distribution +Testing can be done purely on the client-side without the need for a cluster. -The canonical way to build functionality for Composition will be to publish -functions as container images that are semantically versioned using tags, -and to include the image registry in Composition's allowlist. The format of -that list is TBD, but ideally it can be provided at compile time for organizational -use as well as at runtime. +### Graduation Criteria -Function authors will be encouraged to label their containers with module definitions. -Doing so will unlock additional functionality, such as making the `provider` field -optional in cases where a module's APIVersion+Kind resolve uniquely within the registry. +#### Alpha -Ad hoc functions can be written in Starlark and stored locally with the -Composition. If remote Starlark functions are supported in the future, -a similar "registry" model will be needed for them. +- `Composition` implemented with an alpha GV and supported by `kustomize build` + - Compatibility with existing built-in transformers + - `ResourceAccumulator` extracted as a transformer + - `transformer`, `transformersFrom` and `transformerOverrides` fields implemented. + - reserved `provider` field +- Basic documentation added to the Kustomize website. -### Test Plan +#### Beta -Testing can be done purely on the client-side without the need for a cluster. +- `Composition` beta GV supported by `kustomize build` + - `Kustomization` transformer implemented + - `transformerOrder` field implemented +- `Composition` supported by the `resources`, `transformers`, `generators` and `validators` fields + in `Kustomization` and `Component` +- Thorough documentation and examples published on the Kustomize website. -### Graduation Criteria +#### GA + +- At least two releases of `kubectl kustomize` have included `Composition` support, to allow for + feedback from the wider community. +- Inline transformer config support in the `transformers`, `generators` and `validators` fields of + `Kustomization` +- TBD -Not yet required. +#### Deprecation + +As with any Kustomize feature, deprecation post-alpha would need to take the `kubectl kustomize` +release cycle into consideration as well as Kustomize's own, to ensure all users are adequately +warned before removal. ### Upgrade / Downgrade Strategy -NA -- not part of the cluster +N/A -- feature is client-side-only ### Version Skew Strategy -NA -- not part of the cluster +N/A -- feature is client-side-only ## Production Readiness Review Questionnaire -NA -- not part of the cluster - ### Feature Enablement and Rollback -NA -- distributed as a client-side binary +No special flags will be needed to use `Composition`, which will have a GV indicative of its +maturity level. The same flags required to use plugins in existing contexts will be required to use +them via `Composition`. + +###### How can this feature be enabled / disabled in a live cluster? + +N/A -- feature is client-side-only + +###### Does enabling the feature change any default behavior? + +N/A -- feature is client-side-only + +###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? + +N/A -- feature is client-side-only + +###### What happens if we reenable the feature if it was previously rolled back? + +N/A -- feature is client-side-only + +###### Are there any tests for feature enablement/disablement? + +N/A -- no feature flag required ### Rollout, Upgrade and Rollback Planning -NA -- distributed as a client-side binary +###### How can a rollout or rollback fail? Can it impact already running workloads? + +N/A -- feature is client-side-only + +###### What specific metrics should inform a rollback? + +N/A -- feature is client-side-only + +###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? + +N/A -- feature is client-side-only + +###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? + +No ### Monitoring Requirements -NA -- distributed as a client-side binary +N/A -- feature is client-side-only + +###### How can an operator determine if the feature is in use by workloads? + +N/A -- feature is client-side-only + +###### How can someone using this feature know that it is working for their instance? + +N/A -- feature is client-side-only + +###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? + +N/A -- feature is client-side-only + +###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? + +N/A -- feature is client-side-only + +###### Are there any missing metrics that would be useful to have to improve observability of this feature? + +N/A -- feature is client-side-only ### Dependencies -NA -- distributed as a client-side binary +This feature does not require any new Golang dependencies. + +###### Does this feature depend on any specific services running in the cluster? + +No -- feature is client-side-only ### Scalability -NA -- distributed as a client-side binary +This feature is not expected to have significantly different performance characteristics than the +use of transformers in Kustomization. Performance in practice will depend largely on the type and +implementation of plugins used. + +###### Will enabling / using this feature result in any new API calls? + +No + +###### Will enabling / using this feature result in introducing new API types? + +Yes, but only client-side within Kustomize + +###### Will enabling / using this feature result in any new calls to the cloud provider? + +No + +###### Will enabling / using this feature result in increasing size or count of the existing API objects? + +No + +###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs? + +No + +###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components? + +N/A -- feature is client-side-only ### Troubleshooting -NA -- distributed as a client-side binary +###### How does this feature react if the API server and/or etcd is unavailable? + +N/A -- feature is client-side-only +###### What are other known failure modes? + +Kustomize plugins are immature (various alpha mechanisms) and when they fail to execute, it is not +always graceful. Golang plugins in particular come with a long list of caveats. KRM Exec and +Starlark plugins are not currently enabled in `kubectl kustomize` at all. + +###### What steps should be taken if SLOs are not being met to determine the problem? + +N/A -- feature is client-side-only ## Implementation History +- April 27, 2021: Provisional KEP merged +- August 2021: Proposal updated and marked implementable. +