K8SPSMDB-1140: Allow users to override hostnames in replset config

[egegunes]

CHANGE DESCRIPTION

Problem:
There's no way to specify hostnames in replset config. Some of our users want to override hostnames to support their multi-cluster topologies.

Solution:

This PR introduces a new field called replsetOverrides to directly influence replset configuration:

replsets:
- name: rs0
   replsetOverrides:
     cluster1-rs0-0:
       host: rs0-0.egedemo.xyz:27017
     cluster1-rs0-1:
       host: rs0-1.egedemo.xyz:27017
     cluster1-rs0-2:
       host: rs0-2.egedemo.xyz:27017

Once you override hostnames, operator will start changing hosts in replset configuration one by one (secondaries first, primary last).

While doing this, I also wanted to allow users to add custom tags to replset members. replsetOverrides field also allow specifying tags:

replsets:
- name: rs0
   replsetOverrides:
     cluster1-rs0-0:
       host: rs0-0.egedemo.xyz:27017
       tags:
         key: value
     cluster1-rs0-1:
       host: rs0-1.egedemo.xyz:27017
       tags:
         key: value
     cluster1-rs0-2:
       host: rs0-2.egedemo.xyz:27017
       tags:
         key: value

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?
• Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• Is an E2E test/test case added for the new feature/change?
• Are unit tests added where appropriate?
• Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

• Are all needed new/changed options added to default YAML files?
• Did we add proper logging messages for operator actions?
• Did we ensure compatibility with the previous version or cluster upgrade process?
• Does the change support oldest and newest supported MongoDB version?
• Does the change support oldest and newest supported Kubernetes version?

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

	sed "s/NAMESPACE/${namespace}/g" ${test_dir}/conf/${cluster}-overridden.yml > ${tmp_dir}/${cluster}-overridden.yml
	sed "s/NAMESPACE/${namespace}/g" ${test_dir}/conf/${cluster}-overridden.yml >${tmp_dir}/${cluster}-overridden.yml

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

	sed "s/NAMESPACE/${namespace}/g" ${test_dir}/conf/${cluster}-overridden.yml > ${tmp_dir}/${cluster}-overridden.yml
	sed "s/NAMESPACE/${namespace}/g" ${test_dir}/conf/${cluster}-overridden.yml >${tmp_dir}/${cluster}-overridden.yml

[hors]

Please add it as an example into cr.yaml

[egegunes]

19b3be3

[JNKPercona]

Test name	Status
arbiter	passed
balancer	passed
custom-replset-name	passed
custom-tls	passed
cross-site-sharded	passed
data-at-rest-encryption	passed
data-sharded	passed
demand-backup	passed
demand-backup-eks-credentials	passed
demand-backup-physical	passed
demand-backup-physical-sharded	passed
demand-backup-sharded	passed
expose-sharded	passed
ignore-labels-annotations	passed
init-deploy	passed
finalizer	passed
ldap	passed
ldap-tls	passed
limits	passed
liveness	passed
mongod-major-upgrade	passed
mongod-major-upgrade-sharded	passed
monitoring-2-0	failure
multi-cluster-service	passed
non-voting	passed
one-pod	passed
operator-self-healing-chaos	passed
pitr	passed
pitr-sharded	passed
pitr-physical	passed
pvc-resize	passed
recover-no-primary	passed
replset-overrides	passed
rs-shard-migration	passed
scaling	passed
scheduled-backup	passed
security-context	passed
self-healing-chaos	passed
service-per-pod	passed
serviceless-external-nodes	passed
smart-update	passed
split-horizon	passed
storage	passed
tls-issue-cert-manager	passed
upgrade	passed
upgrade-consistency	passed
upgrade-consistency-sharded-tls	passed
upgrade-sharded	passed
users	passed
version-service	passed
We run 50 out of 50

commit: d4f82a7
image: perconalab/percona-server-mongodb-operator:PR-1640-d4f82a71