Skip to content

Commit

Permalink
Update pcr guides to use encode-uri cli for managing certificates
Browse files Browse the repository at this point in the history
  • Loading branch information
kathancox committed May 10, 2024
1 parent 193b3e0 commit 85d48a4
Show file tree
Hide file tree
Showing 4 changed files with 69 additions and 16 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

{% include_cached copy-clipboard.html %}
~~~ sql
ALTER VIRTUAL CLUSTER {original_primary_vc} START REPLICATION FROM {promoted_standby_vc} ON connection_string_standby;
ALTER VIRTUAL CLUSTER {original_primary_vc} START REPLICATION FROM {promoted_standby_vc} ON {connection_string_standby};
~~~

The original primary virtual cluster may be almost up to date with the promoted standby's virtual cluster. The difference in data between the two virtual clusters will include only the writes that have been applied to the promoted standby after cutover from the primary cluster.
30 changes: 22 additions & 8 deletions src/current/v24.1/cutover-replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,12 +187,30 @@ This section illustrates the steps to cut back to the original primary cluster f
ALTER VIRTUAL CLUSTER {cluster_a} STOP SERVICE;
~~~

1. Open another terminal window and connect to the system virtual cluster for **Cluster B**:
1. Open another terminal window and generate a connection string for **Cluster B** using `cockroach encode-uri`:

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach encode-uri {replication user}:{password}@{cluster B node IP or hostname}:26257 --ca-cert certs/ca.crt --inline
~~~

Copy the output ready for starting the replication stream, which requires the connection string to **Cluster B**:

{% include_cached copy-clipboard.html %}
~~~
postgresql://{replication user}:{password}@{cluster B node IP or hostname}:26257/defaultdb?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded_cert}-----END+CERTIFICATE-----%0A
~~~

{{site.data.alerts.callout_success}}
For details on connection strings, refer to the [Connection reference]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}#connection-reference).
{{site.data.alerts.end}}

1. Connect to the system virtual cluster for **Cluster B**:

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql --url \
"postgresql://{user}@{node IP or hostname cluster B}:26257?options=-ccluster=system&sslmode=verify-full" \
"postgresql://{user}@{cluster B node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full" \
--certs-dir "certs"
~~~

Expand All @@ -203,19 +221,15 @@ This section illustrates the steps to cut back to the original primary cluster f
SET CLUSTER SETTING kv.rangefeed.enabled = 'true';
~~~

1. From the system virtual cluster on **Cluster A**, start the replication from cluster B to cluster A:
1. From the system virtual cluster on **Cluster A**, start the replication from **Cluster B** to **Cluster A**. Include the connection string for **Cluster B**:

{% include_cached copy-clipboard.html %}
~~~ sql
ALTER VIRTUAL CLUSTER {cluster_a} START REPLICATION OF {cluster_b} ON 'postgresql://{user}@{ node IP or hostname cluster B}:26257?options=-ccluster=system&sslmode=verify-full&sslrootcert=certs/{standby cert}.crt';
ALTER VIRTUAL CLUSTER {cluster_a} START REPLICATION OF {cluster_b} ON 'postgresql://{replication user}:{password}@{cluster B node IP or hostname}:26257/defaultdb?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded_cert}-----END+CERTIFICATE-----%0A';
~~~

This will reset the virtual cluster on **Cluster A** back to the time at which the same virtual cluster on **Cluster B** diverged from it. **Cluster A** will check with **Cluster B** to confirm that its virtual cluster was replicated from **Cluster A** as part of the original [physical cluster replication stream]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}).

{{site.data.alerts.callout_success}}
For details on connection strings, refer to the [Connection reference]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}#connection-reference).
{{site.data.alerts.end}}

1. Check the status of the virtual cluster on **A**:

{% include_cached copy-clipboard.html %}
Expand Down
2 changes: 1 addition & 1 deletion src/current/v24.1/physical-cluster-replication-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ The node topology of the two clusters does not need to be the same. For example,
- During a failover scenario, the standby will need to handle the full production load. However, the clusters cannot have different region topologies (refer to [Limitations](#known-limitations)).

{{site.data.alerts.callout_info}}
Every node in the standby cluster must be able to make a network connection to every node in the primary cluster to start a replication stream successfully. Refer to [Copy certificates]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}#step-3-copy-certificates) for details.
Every node in the standby cluster must be able to make a network connection to every node in the primary cluster to start a replication stream successfully. Refer to [Manage the cluster certificates]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}#step-3-manage-the-cluster-certificates) for details.
{{site.data.alerts.end}}

### Connect to the system virtual cluster and virtual cluster
Expand Down
51 changes: 45 additions & 6 deletions src/current/v24.1/set-up-physical-cluster-replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ The high-level steps in this tutorial are:
- Two separate CockroachDB clusters (primary and standby) with a minimum of three nodes each, and each using the same CockroachDB {{page.version.version}} version.
- To set up each cluster, you can follow [Deploy CockroachDB on Premises]({% link {{ page.version.version }}/deploy-cockroachdb-on-premises.md %}). When you initialize the cluster with the [`cockroach init`]({% link {{ page.version.version }}/cockroach-init.md %}) command, you **must** pass the `--virtualized` or `--virtualized-empty` flag. Refer to the cluster creation steps for the [primary cluster](#initialize-the-primary-cluster) and for the [standby cluster](#initialize-the-standby-cluster) for details.
- The [Deploy CockroachDB on Premises]({% link {{ page.version.version }}/deploy-cockroachdb-on-premises.md %}) tutorial creates a self-signed certificate for each {{ site.data.products.core }} cluster. To create certificates signed by an external certificate authority, refer to [Create Security Certificates using OpenSSL]({% link {{ page.version.version }}/create-security-certificates-openssl.md %}).
- All nodes in each cluster will need access to the Certificate Authority for the other cluster. Refer to [Copy certificates](#step-3-copy-certificates).
- All nodes in each cluster will need access to the Certificate Authority for the other cluster. Refer to [Manage the cluster certificates](#step-3-manage-the-cluster-certificates).
- An [{{ site.data.products.enterprise }} license]({% link {{ page.version.version }}/enterprise-licensing.md %}) on the primary **and** standby clusters. You must use the system virtual cluster on the primary and standby clusters to enable your {{ site.data.products.enterprise }} license.
- The primary and standby clusters **must have the same [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported.

Expand Down Expand Up @@ -268,16 +268,45 @@ If you would like to access the [DB Console]({% link {{ page.version.version }}/

Open the DB Console in your web browser: `https://{node IP or hostname}:8080/`, where you will be prompted for these credentials. Refer to [Physical Cluster Replication Monitoring]({% link {{ page.version.version }}/physical-cluster-replication-monitoring.md %}) for more detail on tracking relevant metrics for your replication stream.

## Step 3. Copy certificates

At this point, the primary and standby clusters are both running. The next step allows the standby cluster to connect to the primary cluster and begin ingesting its data. Depending on how you manage certificates, you must ensure that all nodes on the primary and the standby cluster have access to the certificate of the other cluster.
## Step 3. Manage the cluster certificates

{{site.data.alerts.callout_danger}}
It is important to carefully manage the exchange of CA certificates between clusters if you have generated self-signed certificates with `cockroach cert` as part of the [prerequisite deployment tutorial]({% link {{ page.version.version }}/deploy-cockroachdb-on-premises.md %}).

To create certificates signed by an external certificate authority, refer to [Create Security Certificates using OpenSSL]({% link {{ page.version.version }}/create-security-certificates-openssl.md %}).
{{site.data.alerts.end}}

At this point, the primary and standby clusters are both running. The next step allows the standby cluster to connect to the primary cluster and begin ingesting its data. Depending on how you manage certificates, you must ensure that all nodes on the primary and the standby cluster have access to the certificate of the other cluster.

You can do this by either:

- [Using the `cockroach encode-uri` command to generate a connection string containing the certificate](#generate-a-connection-string-containing-the-certificate).
- [Adding the primary cluster's certificate to the standby cluster](#copy-certificates).
### Generate a connection string containing the certificate
{% include_cached new-in.html version="v24.1" %} Use the `cockroach encode-uri` command to generate a connection string that contains a cluster's certifcate. You can pass this connection string whenever necessary to connect to the other cluster.

For example, in this tutorial you will need a connection string for the primary cluster when you start the replication stream from the standby.

To generate a connection string, pass the replication user, IP and port, along with the directory to the certificate for the primary cluster:

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach encode-uri {replication user}:{password}@{node IP or hostname}:26257 --ca-cert certs/ca.crt --inline
~~~

The connection string output contains the primary cluster's certificate:
{% include_cached copy-clipboard.html %}
~~~
postgresql://{replication user}:{password}@{node IP or hostname}:26257/defaultdb?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded_cert}-----END+CERTIFICATE-----%0A
~~~
Copy the output ready for [Step 4](#step-4-start-replication), which requires the connection string to the primary cluster.
### Copy certificates
For example, if you followed the [Deploy CockroachDB]({% link {{ page.version.version }}/deploy-cockroachdb-on-premises.md %}) prerequisite, you need to add the `ca.crt` from the primary cluster to the `certs` directory on all the nodes in the standby cluster.
1. Name the `ca.crt` from the primary cluster to a new name on the standby cluster. For example, `ca_primary.crt`.
Expand All @@ -294,7 +323,16 @@ The system virtual cluster in the standby cluster initiates and controls the rep
1. From the **standby** cluster, use your connection string to the primary.
The connection string contains:
If you generated the connection string using [`cockroach encode-uri`](#generate-a-connection-string-containing-the-certificate):
{% include_cached copy-clipboard.html %}
~~~ sql
CREATE VIRTUAL CLUSTER main
FROM REPLICATION OF main
ON 'postgresql://{replication user}:{password}@{node IP or hostname}:26257/defaultdb?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded_cert}-----END+CERTIFICATE-----%0A';
~~~
Otherwise, pass the connection string that contains:
- The replication user and password that you [created for the primary cluster](#create-a-replication-user-and-password).
- The node IP address or hostname of one node from the primary cluster.
- The path to the primary node's certificate on the standby cluster.
Expand Down Expand Up @@ -361,10 +399,11 @@ This table outlines the connection strings you will need for this setup tutorial
For additional detail on the standard CockroachDB connection parameters, refer to [Client Connection Parameters]({% link {{ page.version.version }}/connection-parameters.md %}#connect-using-a-url).
Cluster | Virtual Cluster | Usage | URL and Parameters
--------+-----------+-------+------------+----
--------+-----------------+-------+-------------------
Primary | System | Set up a replication user and view running virtual clusters. Connect with [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}). | `"postgresql://root@{node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full"`<br><br><ul><li>`options=-ccluster=system`</li><li>`sslmode=verify-full`</li></ul>Use the `--certs-dir` flag to specify the path to your certificate.
Primary | Main | Add and run a workload with [`cockroach workload`]({% link {{ page.version.version }}/cockroach-workload.md %}). | `"postgresql://root@{node IP or hostname}:{26257}?options=-ccluster=main&sslmode=verify-full&sslrootcert=certs/ca.crt&sslcert=certs/client.root.crt&sslkey=certs/client.root.key"`<br><br>{% include {{ page.version.version }}/connect/cockroach-workload-parameters.md %} As a result, for the example in this tutorial, you will need:<br><br><ul><li>`options=-ccluster={virtual_cluster_name}`</li><li>`sslmode=verify-full`</li><li>`sslrootcert={path}/certs/ca.crt`</li><li>`sslcert={path}/certs/client.root.crt`</li><li>`sslkey={path}/certs/client.root.key`</li></ul>
Standby | System | Manage the replication stream. Connect with [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}). | `"postgresql://root@{node IP or hostname}:26257?options=-ccluster=system&sslmode=verify-full"`<br><br><ul><li>`options=-ccluster=system`</li><li>`sslmode=verify-full`</li></ul>Use the `--certs-dir` flag to specify the path to your certificate.
Standby/Primary | System | Connect to the other cluster. | `"postgresql://{replication user}:{password}@{node IP or hostname}:26257/defaultdb?options=-ccluster%3Dsystem&sslinline=true&sslmode=verify-full&sslrootcert=-----BEGIN+CERTIFICATE-----{encoded_cert}-----END+CERTIFICATE-----%0A"`<br><br>Generate the connection string with [`cockroach encode-uri`](#generate-a-connection-string-containing-the-certificate). Use the generated connection string in:<br><br><ul><li>`CREATE VIRTUAL CLUSTER` statements to [start the replication stream](#step-4-start-replication).</li><li>`ALTER VIRTUAL CLUSTER` statements to [cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster).</li></ul>
## What's next

Expand Down

0 comments on commit 85d48a4

Please sign in to comment.