Skip to content

Commit

Permalink
changed: tidy up the main example
Browse files Browse the repository at this point in the history
  • Loading branch information
aabouzaid committed Sep 10, 2023
1 parent 77df539 commit a199e4b
Show file tree
Hide file tree
Showing 16 changed files with 154 additions and 44 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*.dll
*.so
*.dylib
dist/*

# Test binary, built with `go test -c`
*.test
Expand Down
12 changes: 12 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
CONTROLLER_GEN_HOME := /tmp/controller-gen
CONTROLLER_GEN_VERSION := v0.13.0

#
# Schema targets.
#

.PHONY: schema.install-tools
schema.install-tools:
mkdir -p $(CONTROLLER_GEN_HOME);\
Expand All @@ -16,6 +20,14 @@ schema.generate:
-e 's|apiextensions.k8s.io/v1|config.kubernetes.io/v1alpha1|g' \
-i pkg/merger/schema/generators.kustomize.aabouzaid.com_mergers.yaml

#
# Golang targets.
#

.PHONY: go.build
go.build:
go build -o 'dist' .

.PHONY: go.format
go.format:
gofumpt -l -w .
36 changes: 28 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ A Kustomize generator plugin to merge YAML files seamlessly for real-world use c
- [1. Generate multiple manifests from a single base](#1-generate-multiple-manifests-from-a-single-base)
- [2. Merge lists in manifests without schema or a unique identifier](#2-merge-lists-in-manifests-without-schema-or-a-unique-identifier)
- [3. Structure long manifests into smaller ones](#3-structure-long-manifests-into-smaller-ones)
- [TODO](#todo)
- [TO-DO](#to-do)
- [Project status](#project-status)
- [Contributing](#contributing)
- [License](#license)
Expand All @@ -36,15 +36,18 @@ Kustomize's default patch strategy for the lists (arrays) is `replace`, and to c
for the Kubernetes `Custom Resource`, you must provide the OpenAPI schema of that custom resource,
which is only helpful if the lists of that CR have a unique id.

For more details, please read:
Many people are asking for such functionalities as [easy merging CRs](https://stackoverflow.com/q/73655002/4547221),
[creating a ConfigMap from arbitrary YAML files](https://stackoverflow.com/q/74547569/4547221),
and for more details on the challenge of providing OpenAPI schema to merge files, please read the following post:
[Set OpenAPI patch strategy for Kubernetes Custom Resources](https://tech.aabouzaid.com/2022/11/set-openapi-patch-strategy-for-kubernetes-custom-resources-kustomize.html).


## Features

- Generate multiple resources/manifests from a single base without copying the resources multiple times.
- Merge any manifests (even CustomResources) without needing their OpenAPI schema.
- Merge manifests with a list of maps without a unique identifier (when using `x-kubernetes-patch-merge-key` is not possible).
- Merge manifests with a list of maps without a unique identifier
(when using `x-kubernetes-patch-merge-key` is not possible).
- Merge YAML files with different merge strategies (StrategicMerge).
- Merge applications configuration YAML files into a ConfigMap or Secret (WIP).

Expand All @@ -57,6 +60,19 @@ apiVersion: generators.kustomize.aabouzaid.com/v1alpha1
kind: Merger
metadata:
name: merge
annotations:
# Containerized KRM function.
config.kubernetes.io/function: |
container:
image: ghcr.io/aabouzaid/kustomize-generator-merger
mounts:
- type: bind
src: ./
dst: /mnt
# Exec KRM functions.
# config.kubernetes.io/function: |
# exec:
# path: ./kustomize-plugin-merger
spec:
resources:
- name: example
Expand All @@ -66,6 +82,8 @@ spec:
# - Patch: Produce a single output by merging all sources together then with the destination.
method: overlay
files:
# The same as in the KRM container above.
root: /mnt
sources:
- src01.yaml
- src02.yaml
Expand All @@ -88,15 +106,16 @@ This section shows a couple of use cases where Merger can help.
### 1. Generate multiple manifests from a single base
In this case, you have multiple `CronJobs`, all of them share the same body, but each one has a different command or other config.
In this case, you have multiple `CronJobs`, all of them share the same body,
but each one has a different command or other config.

[Use case full example](./examples/generate-multiple-manifests/README.md).

### 2. Merge lists in manifests without schema or a unique identifier

Currently, in Kustomize, it's not possible to merge resources without a unique identifier, even with Open API schema.

It's possible to do that using the merge strategy `append` in Merger (later on, `combine` will also be supported).
It's possible to do that using the merge strategy `append` in Merger (later on, `combineWithKey` will also be supported).

[Use case full example](./examples/merge-lists-without-schema/README.md).

Expand All @@ -109,15 +128,16 @@ and use the Merger `patch` input method to make it a single manifest again.
[Use case full example](./examples/structure-long-manifests/README.md).


## TODO
## TO-DO

- Support `ConfigMap` or `Secret` as an output.
- Support `combine` merge strategy with an identifier key (similar to `x-kubernetes-patch-merge-key`).
- Provide better docs for Merger options.


## Project status

Please note that this project is still under development and could be breaking changes,
Please note that this project is still under development and could have breaking changes,
but it will follow the SemVer convention.


Expand All @@ -130,4 +150,4 @@ or [create a PR](https://github.com/aabouzaid/kustomize-plugin-merger/pulls).

## License

This is open-source software licensed using the Apache License 2.0. Please see [LICENSE](LICENSE) for details.
Merger is an open-source software licensed using the Apache License 2.0. Please see [LICENSE](LICENSE) for details.
8 changes: 8 additions & 0 deletions examples/krm-and-kustomize/input/base.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
apiVersion: v1
kind: Pod
metadata:
name: base
spec:
containers:
- name: sidecar
image: sidecar
8 changes: 8 additions & 0 deletions examples/krm-and-kustomize/input/dev.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
metadata:
name: dev
spec:
containers:
- name: dev
image: dev:1.0.0
ports:
- containerPort: 80
8 changes: 8 additions & 0 deletions examples/krm-and-kustomize/input/stage.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
metadata:
name: stage
spec:
containers:
- name: stage
image: stage:1.0.0
ports:
- containerPort: 80
5 changes: 5 additions & 0 deletions examples/krm-and-kustomize/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

generators:
- merger.yaml
34 changes: 34 additions & 0 deletions examples/krm-and-kustomize/merger.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
apiVersion: generators.kustomize.aabouzaid.com/v1alpha1
kind: Merger
metadata:
name: merge
annotations:
# Containerized KRM function.
config.kubernetes.io/function: |
container:
image: ghcr.io/aabouzaid/kustomize-generator-merger
mounts:
- type: bind
src: ./
dst: /mnt
# Exec KRM functions.
# config.kubernetes.io/function: |
# exec:
# path: ./kustomize-plugin-merger
spec:
resources:
- name: my-envs
input:
method: overlay
files:
# The same as in the KRM container above.
root: /mnt
sources:
- input/dev.yaml
- input/stage.yaml
destination: input/base.yaml
merge:
strategy: append
output:
format: raw
39 changes: 39 additions & 0 deletions examples/krm-and-kustomize/resourcelist.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
apiVersion: config.kubernetes.io/v1
kind: ResourceList
metadata:
name: krm-function-input
functionConfig:
apiVersion: generators.kustomize.aabouzaid.com/v1alpha1
kind: Merger
metadata:
name: merge
annotations:
# Containerized KRM function.
config.kubernetes.io/function: |
container:
image: ghcr.io/aabouzaid/kustomize-generator-merger
mounts:
- type: bind
src: ./
dst: /mnt
# Exec KRM functions.
# config.kubernetes.io/function: |
# exec:
# path: ./kustomize-plugin-merger
spec:
resources:
- name: my-envs
input:
method: overlay
files:
# The same as in the KRM container above.
root: /mnt
sources:
- input/dev.yaml
- input/stage.yaml
destination: input/base.yaml
merge:
strategy: append
output:
format: raw
3 changes: 0 additions & 3 deletions examples/krm-without-kustomize/input/common.yaml

This file was deleted.

3 changes: 0 additions & 3 deletions examples/krm-without-kustomize/input/dev.yaml

This file was deleted.

3 changes: 0 additions & 3 deletions examples/krm-without-kustomize/input/stage.yaml

This file was deleted.

24 changes: 0 additions & 24 deletions examples/krm-without-kustomize/resourcelist.yaml

This file was deleted.

7 changes: 5 additions & 2 deletions pkg/merger/merge.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,16 @@ func (r *mergerResource) setInputFilesOverlay() {
for _, inputFileSource := range r.Input.Files.Sources {
r.Input.items = append(r.Input.items,
resourceInputFiles{
Sources: []string{inputFileSource},
Destination: r.Input.Files.Destination,
Sources: []string{r.Input.Files.Root + inputFileSource},
Destination: r.Input.Files.Root + r.Input.Files.Destination,
})
}
}

func (r *mergerResource) setInputFilesPatch() {
for index, inputFileSource := range r.Input.Files.Sources {
r.Input.Files.Sources[index] = r.Input.Files.Root + inputFileSource
}
r.Input.items = append(r.Input.items, r.Input.Files)
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ spec:
- name: v1alpha1
schema:
openAPIV3Schema:
description: Merger manifest.
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
Expand All @@ -44,6 +45,8 @@ spec:
properties:
destination:
type: string
root:
type: string
sources:
items:
type: string
Expand Down
4 changes: 3 additions & 1 deletion pkg/merger/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ const (
ResourceKind string = "Merger"
)

// Merger manifest body.
// Merger manifest.
type Merger struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata"`
Expand Down Expand Up @@ -49,6 +49,8 @@ type mergerResource struct {
//

type resourceInputFiles struct {
// +optional
Root string `yaml:"root" json:"root"`
Sources []string `yaml:"sources" json:"sources"`
Destination string `yaml:"destination" json:"destination"`
}
Expand Down

0 comments on commit a199e4b

Please sign in to comment.