Skip to content

Commit

Permalink
feat(common): rework and update cnpg code (#509)
Browse files Browse the repository at this point in the history
**Description**
This PR aims to rework and update cnpg code to expose more features of
CNPG to our users.
Primarily a big change would be the support of more backup and restore
options.

- add some missing options for both cluster and pooler
- seperates cluster and pooler sections
- add labels/annotions for pooler and cluster seperatly as well as
cnpg-wide labels-annotations
- Simplify storage logic
- Disable metrics by default

**⚙️ Type of change**

- [x] ⚙️ Feature/App addition
- [x] 🪛 Bugfix
- [x] ⚠️ Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [x] 🔃 Refactor of current code

**🧪 How Has This Been Tested?**
<!--
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration
-->

**📃 Notes:**

It will be very loosely inspired by the work done here, mostly because
it gives some nice values.yaml structure on how to deal with the many
backup options:
cloudnative-pg/charts#106

It's going to be flagged as "breaking", because of the shear scope and
the fact it would likely also include a change in standard operating
procedures when it comes to backup-and-restore for our users.

**✔️ Checklist:**

- [ ] ⚖️ My code follows the style guidelines of this project
- [ ] 👀 I have performed a self-review of my own code
- [ ] #️⃣ I have commented my code, particularly in hard-to-understand
areas
- [ ] 📄 I have made corresponding changes to the documentation
- [ ] ⚠️ My changes generate no new warnings
- [ ] 🧪 I have added tests to this description that prove my fix is
effective or that my feature works
- [ ] ⬆️ I increased versions for any altered app according to semantic
versioning

**➕ App addition**

If this PR is an app addition please make sure you have done the
following.

- [ ] 🪞 I have opened a PR on
[truecharts/containers](https://github.com/truecharts/containers) adding
the container to TrueCharts mirror repo.
- [ ] 🖼️ I have added an icon in the Chart's root directory called
`icon.png`

---

_Please don't blindly check all the boxes. Read them and only check
those that apply.
Those checkboxes are there for the reviewer to see what is this all
about and
the status of this PR with a quick glance._

---------

Co-authored-by: Stavros kois <[email protected]>
Co-authored-by: Stavros Kois <[email protected]>
  • Loading branch information
3 people authored Dec 4, 2023
1 parent 71995a4 commit 47daa5c
Show file tree
Hide file tree
Showing 67 changed files with 5,262 additions and 326 deletions.
4 changes: 3 additions & 1 deletion cspell.config.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
words:
- cnpg
- netshoot
- tailscale
- Velero
- velero
2 changes: 1 addition & 1 deletion library/common-test/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ appVersion: ""
dependencies:
- name: common
repository: file://../common
version: ~15.3.0
version: ~16.0.0
deprecated: false
description: Helper chart to test different use cases of the common library
home: https://github.com/truecharts/apps/tree/master/charts/library/common-test
Expand Down
242 changes: 242 additions & 0 deletions library/common-test/tests/cnpg/backup_metadata_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
suite: cnpg backup metadata test
templates:
- common.yaml
chart:
appVersion: &appVer v9.9.9
release:
name: test-release-name
namespace: test-release-namespace
tests:
- it: should pass with cnpg manualBackups backup created with labels and annotations
set:
operator:
verify:
enabled: false
label1: label1
label2: global_label2
label3: label3
annotation1: annotation1
annotation2: global_annotation2
annotation3: annotation3
global:
labels:
g_label1: global_label1
g_label2: "{{ .Values.label2 }}"
annotations:
g_annotation1: global_annotation1
g_annotation2: "{{ .Values.annotation2 }}"
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
labels:
label1: "{{ .Values.label1 }}"
label2: label2
annotations:
annotation1: "{{ .Values.annotation1 }}"
annotation2: annotation2
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
labels:
label5: label5
annotations:
annotation5: annotation5
manualBackups:
- name: today
labels:
label3: "{{ .Values.label3 }}"
label4: label4
annotations:
annotation3: "{{ .Values.annotation3 }}"
annotation4: annotation4
asserts:
- documentIndex: &backupDoc 0
isKind:
of: Backup
- documentIndex: *backupDoc
equal:
path: metadata.annotations
value:
annotation1: annotation1
annotation2: annotation2
annotation3: annotation3
annotation4: annotation4
annotation5: annotation5
g_annotation1: global_annotation1
g_annotation2: global_annotation2
- documentIndex: *backupDoc
equal:
path: metadata.labels
value:
app: common-test-1.0.0
release: test-release-name
helm-revision: "0"
helm.sh/chart: common-test-1.0.0
app.kubernetes.io/name: common-test
app.kubernetes.io/instance: test-release-name
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/version: *appVer
cnpg.io/cluster: test-release-name-common-test-cnpg-my-pg
g_label1: global_label1
g_label2: global_label2
label1: label1
label2: label2
label3: label3
label4: label4
label5: label5
- documentIndex: *backupDoc
equal:
path: metadata.namespace
value: test-release-namespace

- it: should pass with cnpg manualBackups backup created with namespace
set:
operator:
verify:
enabled: false
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
namespace: some-namespace
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: today
asserts:
- documentIndex: *backupDoc
equal:
path: metadata.namespace
value: some-namespace

- it: should pass with cnpg manualBackups backup created with object namespace from tpl
set:
operator:
verify:
enabled: false
key: some-namespace
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
namespace: "{{ .Values.key }}"
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: today
asserts:
- documentIndex: *backupDoc
equal:
path: metadata.namespace
value: some-namespace

- it: should pass with cnpg manualBackups backup created with namespace from global with tpl
set:
operator:
verify:
enabled: false
key: global-namespace
global:
namespace: "{{ .Values.key }}"
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: today
asserts:
- documentIndex: *backupDoc
equal:
path: metadata.namespace
value: global-namespace

- it: should pass with cnpg manualBackups backup created with namespace from root with tpl
set:
operator:
verify:
enabled: false
key: local-namespace
namespace: "{{ .Values.key }}"
global:
namespace: global-namespace
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: today
asserts:
- documentIndex: *backupDoc
equal:
path: metadata.namespace
value: local-namespace

- it: should pass with cnpg manualBackups backup created with namespace in TrueNAS SCALE
set:
operator:
verify:
enabled: false
global:
ixChartContext:
iAmNotEmpty: true
storageClassName: some-storage-class
namespace: ix-namespace
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: today
asserts:
- documentIndex: *backupDoc
equal:
path: metadata.namespace
value: ix-namespace
62 changes: 62 additions & 0 deletions library/common-test/tests/cnpg/backup_name_test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
suite: cnpg backup name test
templates:
- common.yaml
release:
name: test-release-name
namespace: test-release-namespace
tests:
- it: should generate correct name
set:
operator:
verify:
enabled: false
cnpg:
my-pg:
enabled: true
user: test-user
database: test-db
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: today
my-pg2:
enabled: true
user: test-user
database: test-db
backups:
enabled: true
target: primary
retentionPolicy: 10d
provider: azure
destinationPath: some-path
azure:
connectionString: some-connection-string
manualBackups:
- name: other
asserts:
- documentIndex: &backupDoc 0
isKind:
of: Backup
- documentIndex: *backupDoc
isAPIVersion:
of: postgresql.cnpg.io/v1
- documentIndex: *backupDoc
equal:
path: metadata.name
value: test-release-name-common-test-cnpg-my-pg-backup-today
- documentIndex: &otherbackupDoc 3
isKind:
of: Backup
- documentIndex: *otherbackupDoc
isAPIVersion:
of: postgresql.cnpg.io/v1
- documentIndex: *otherbackupDoc
equal:
path: metadata.name
value: test-release-name-common-test-cnpg-my-pg2-backup-other
Loading

0 comments on commit 47daa5c

Please sign in to comment.