-
Notifications
You must be signed in to change notification settings - Fork 34
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(common): rework and update cnpg code (#509)
**Description** This PR aims to rework and update cnpg code to expose more features of CNPG to our users. Primarily a big change would be the support of more backup and restore options. - add some missing options for both cluster and pooler - seperates cluster and pooler sections - add labels/annotions for pooler and cluster seperatly as well as cnpg-wide labels-annotations - Simplify storage logic - Disable metrics by default **⚙️ Type of change** - [x] ⚙️ Feature/App addition - [x] 🪛 Bugfix - [x]⚠️ Breaking change (fix or feature that would cause existing functionality to not work as expected) - [x] 🔃 Refactor of current code **🧪 How Has This Been Tested?** <!-- Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration --> **📃 Notes:** It will be very loosely inspired by the work done here, mostly because it gives some nice values.yaml structure on how to deal with the many backup options: cloudnative-pg/charts#106 It's going to be flagged as "breaking", because of the shear scope and the fact it would likely also include a change in standard operating procedures when it comes to backup-and-restore for our users. **✔️ Checklist:** - [ ] ⚖️ My code follows the style guidelines of this project - [ ] 👀 I have performed a self-review of my own code - [ ] #️⃣ I have commented my code, particularly in hard-to-understand areas - [ ] 📄 I have made corresponding changes to the documentation - [ ]⚠️ My changes generate no new warnings - [ ] 🧪 I have added tests to this description that prove my fix is effective or that my feature works - [ ] ⬆️ I increased versions for any altered app according to semantic versioning **➕ App addition** If this PR is an app addition please make sure you have done the following. - [ ] 🪞 I have opened a PR on [truecharts/containers](https://github.com/truecharts/containers) adding the container to TrueCharts mirror repo. - [ ] 🖼️ I have added an icon in the Chart's root directory called `icon.png` --- _Please don't blindly check all the boxes. Read them and only check those that apply. Those checkboxes are there for the reviewer to see what is this all about and the status of this PR with a quick glance._ --------- Co-authored-by: Stavros kois <[email protected]> Co-authored-by: Stavros Kois <[email protected]>
- Loading branch information
1 parent
71995a4
commit 47daa5c
Showing
67 changed files
with
5,262 additions
and
326 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
words: | ||
- cnpg | ||
- netshoot | ||
- tailscale | ||
- Velero | ||
- velero |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
242 changes: 242 additions & 0 deletions
242
library/common-test/tests/cnpg/backup_metadata_test.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,242 @@ | ||
suite: cnpg backup metadata test | ||
templates: | ||
- common.yaml | ||
chart: | ||
appVersion: &appVer v9.9.9 | ||
release: | ||
name: test-release-name | ||
namespace: test-release-namespace | ||
tests: | ||
- it: should pass with cnpg manualBackups backup created with labels and annotations | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
label1: label1 | ||
label2: global_label2 | ||
label3: label3 | ||
annotation1: annotation1 | ||
annotation2: global_annotation2 | ||
annotation3: annotation3 | ||
global: | ||
labels: | ||
g_label1: global_label1 | ||
g_label2: "{{ .Values.label2 }}" | ||
annotations: | ||
g_annotation1: global_annotation1 | ||
g_annotation2: "{{ .Values.annotation2 }}" | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
labels: | ||
label1: "{{ .Values.label1 }}" | ||
label2: label2 | ||
annotations: | ||
annotation1: "{{ .Values.annotation1 }}" | ||
annotation2: annotation2 | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
labels: | ||
label5: label5 | ||
annotations: | ||
annotation5: annotation5 | ||
manualBackups: | ||
- name: today | ||
labels: | ||
label3: "{{ .Values.label3 }}" | ||
label4: label4 | ||
annotations: | ||
annotation3: "{{ .Values.annotation3 }}" | ||
annotation4: annotation4 | ||
asserts: | ||
- documentIndex: &backupDoc 0 | ||
isKind: | ||
of: Backup | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.annotations | ||
value: | ||
annotation1: annotation1 | ||
annotation2: annotation2 | ||
annotation3: annotation3 | ||
annotation4: annotation4 | ||
annotation5: annotation5 | ||
g_annotation1: global_annotation1 | ||
g_annotation2: global_annotation2 | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.labels | ||
value: | ||
app: common-test-1.0.0 | ||
release: test-release-name | ||
helm-revision: "0" | ||
helm.sh/chart: common-test-1.0.0 | ||
app.kubernetes.io/name: common-test | ||
app.kubernetes.io/instance: test-release-name | ||
app.kubernetes.io/managed-by: Helm | ||
app.kubernetes.io/version: *appVer | ||
cnpg.io/cluster: test-release-name-common-test-cnpg-my-pg | ||
g_label1: global_label1 | ||
g_label2: global_label2 | ||
label1: label1 | ||
label2: label2 | ||
label3: label3 | ||
label4: label4 | ||
label5: label5 | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.namespace | ||
value: test-release-namespace | ||
|
||
- it: should pass with cnpg manualBackups backup created with namespace | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
namespace: some-namespace | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: today | ||
asserts: | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.namespace | ||
value: some-namespace | ||
|
||
- it: should pass with cnpg manualBackups backup created with object namespace from tpl | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
key: some-namespace | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
namespace: "{{ .Values.key }}" | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: today | ||
asserts: | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.namespace | ||
value: some-namespace | ||
|
||
- it: should pass with cnpg manualBackups backup created with namespace from global with tpl | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
key: global-namespace | ||
global: | ||
namespace: "{{ .Values.key }}" | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: today | ||
asserts: | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.namespace | ||
value: global-namespace | ||
|
||
- it: should pass with cnpg manualBackups backup created with namespace from root with tpl | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
key: local-namespace | ||
namespace: "{{ .Values.key }}" | ||
global: | ||
namespace: global-namespace | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: today | ||
asserts: | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.namespace | ||
value: local-namespace | ||
|
||
- it: should pass with cnpg manualBackups backup created with namespace in TrueNAS SCALE | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
global: | ||
ixChartContext: | ||
iAmNotEmpty: true | ||
storageClassName: some-storage-class | ||
namespace: ix-namespace | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: today | ||
asserts: | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.namespace | ||
value: ix-namespace |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,62 @@ | ||
suite: cnpg backup name test | ||
templates: | ||
- common.yaml | ||
release: | ||
name: test-release-name | ||
namespace: test-release-namespace | ||
tests: | ||
- it: should generate correct name | ||
set: | ||
operator: | ||
verify: | ||
enabled: false | ||
cnpg: | ||
my-pg: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: today | ||
my-pg2: | ||
enabled: true | ||
user: test-user | ||
database: test-db | ||
backups: | ||
enabled: true | ||
target: primary | ||
retentionPolicy: 10d | ||
provider: azure | ||
destinationPath: some-path | ||
azure: | ||
connectionString: some-connection-string | ||
manualBackups: | ||
- name: other | ||
asserts: | ||
- documentIndex: &backupDoc 0 | ||
isKind: | ||
of: Backup | ||
- documentIndex: *backupDoc | ||
isAPIVersion: | ||
of: postgresql.cnpg.io/v1 | ||
- documentIndex: *backupDoc | ||
equal: | ||
path: metadata.name | ||
value: test-release-name-common-test-cnpg-my-pg-backup-today | ||
- documentIndex: &otherbackupDoc 3 | ||
isKind: | ||
of: Backup | ||
- documentIndex: *otherbackupDoc | ||
isAPIVersion: | ||
of: postgresql.cnpg.io/v1 | ||
- documentIndex: *otherbackupDoc | ||
equal: | ||
path: metadata.name | ||
value: test-release-name-common-test-cnpg-my-pg2-backup-other |
Oops, something went wrong.