Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade rewrites for v24.3 #19098

Merged
merged 41 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
7238268
Upgrade rewrites for v24.3
mdlinville Nov 5, 2024
49a5ec9
Update Kubernetes self-hosted upgrade docs
mdlinville Nov 5, 2024
5196b90
Fixes
mdlinville Nov 6, 2024
1ce087e
Dan's feedback
mdlinville Nov 7, 2024
ff8b51f
Fix errors in rollback content
mdlinville Nov 7, 2024
5b0b533
Merge branch 'main' into DOC-11170
mdlinville Nov 7, 2024
c053ef7
Merge branch 'main' into DOC-11170
mdlinville Nov 7, 2024
08fe33b
Merge branch 'main' into DOC-11170
mdlinville Nov 8, 2024
8fe4434
Apply suggestions from code review
mdlinville Nov 8, 2024
be7024c
Update src/current/v24.3/upgrade-cockroachdb-kubernetes.md
mdlinville Nov 8, 2024
f31c3b0
Merge branch 'main' into DOC-11170
mdlinville Nov 8, 2024
5cda3b6
Merge branch 'main' into DOC-11170
mdlinville Nov 8, 2024
04b1af3
Merge remote-tracking branch 'origin/main' into DOC-11170
mdlinville Nov 8, 2024
ef222f4
Remove irrelevant pre-upgrade tasks from Cloud prep steps
mdlinville Nov 8, 2024
741aa37
Merge remote-tracking branch 'origin/DOC-11170' into DOC-11170
mdlinville Nov 8, 2024
ef0b913
Merge branch 'main' into DOC-11170
mdlinville Nov 8, 2024
10b34ae
Merge branch 'main' into DOC-11170
mdlinville Nov 11, 2024
0b156d8
Merge branch 'main' into DOC-11170
mdlinville Nov 11, 2024
7d3dcb3
Merge remote-tracking branch 'origin/main' into DOC-11170
mdlinville Nov 12, 2024
dbe6565
Most of David's feedback
mdlinville Nov 12, 2024
5728639
Merge remote-tracking branch 'origin/DOC-11170' into DOC-11170
mdlinville Nov 12, 2024
f3cc19b
Feedback from Alicia and Yandu
mdlinville Nov 12, 2024
13c639a
Merge branch 'main' into DOC-11170
mdlinville Nov 12, 2024
d9b89bc
Merge branch 'main' into DOC-11170
mdlinville Nov 13, 2024
91ba3d6
Merge branch 'main' into DOC-11170
mdlinville Nov 13, 2024
2f3c848
Merge remote-tracking branch 'origin/main' into DOC-11170
mdlinville Nov 13, 2024
3d26155
Update Cluster Virtualization upgrade details for version skipping
mdlinville Nov 14, 2024
e9b5c3c
Merge remote-tracking branch 'origin/DOC-11170' into DOC-11170
mdlinville Nov 14, 2024
c3cc42c
Merge branch 'main' into DOC-11170
mdlinville Nov 14, 2024
fe938a4
Merge remote-tracking branch 'origin/main' into DOC-11170
mdlinville Nov 15, 2024
b1a2bb7
Apply suggestions from code review
mdlinville Nov 15, 2024
271769a
Conditionalize the high-level steps intro
mdlinville Nov 15, 2024
7d848ff
Mike's feedback
mdlinville Nov 15, 2024
3b98c78
Merge remote-tracking branch 'origin/main' into DOC-11170
mdlinville Nov 15, 2024
00f8fb1
Merge remote-tracking branch 'origin/DOC-11170' into DOC-11170
mdlinville Nov 15, 2024
18161dc
Remove liquid fragment
mdlinville Nov 15, 2024
39d2791
Fix typo
mdlinville Nov 15, 2024
d6bcbed
Fix links
mdlinville Nov 15, 2024
379587d
Add missing spaces
mikeCRL Nov 18, 2024
bbfb376
Merge branch 'main' into DOC-11170
mikeCRL Nov 18, 2024
2fc3ce1
Remove old New In statement from Virtual Clusters page
mikeCRL Nov 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions src/current/_includes/common/upgrade/disable-auto-finalization.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
{% capture new_flag %}
{{site.data.alerts.callout_info}}
Previously, to disable automatic finalization and preserve the ability to roll back a major-version upgrade, it was required to set the cluster setting `cluster.preserve_downgrade_option` to the cluster's current major version before beginning the major-version upgrade, and then to unset the setting to finalize the upgrade.

We now recommend managing a cluster's finalization policy using the cluster setting `cluster.auto_upgrade.enabled`, which was introduced in v23.2. The setting does not need to be modified after it is initially set.

Either of these settings prevents automatic finalization.
{{site.data.alerts.end}}
{% endcapture %}

{% if page.path contains 'kubernetes' %}

<section class="filter-content" markdown="1" data-scope="operator">
For clusters managed by the Operator, auto-finalization is disabled and cannot be enabled. A major version upgrade is not complete until it is manually [finalized](#finalize-a-major-version-upgrade-manually).
</section>

<section class="filter-content" markdown="1" data-scope="manual">

By default, auto-finalization is enabled, and a major-version upgrade is finalized when all nodes have rejoined the cluster using the new `cockroach` binary. This means that by default, a major-version upgrade cannot be rolled back. Instead, you must [restore the cluster to the previous version]({% link {{ page.version.version }}/restoring-backups-across-versions.md %}#support-for-restoring-backups-into-a-newer-version).

To disable auto-finalization:

1. Connect to the cluster using the SQL shell:

~~~ shell
$ kubectl exec -it cockroachdb-client-secure \
-- ./cockroach sql \
--certs-dir=/cockroach-certs \
--host=cockroachdb-public
~~~

1. Set the [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) `cluster.auto_upgrade.enabled` to `false`.

Now, to complete a major-version upgrade, you must manually [finalize it](#finalize-a-major-version-upgrade-manually) or [roll it back](#roll-back-a-major-version-upgrade).

{{ new_flag }}

</section>

<section class="filter-content" markdown="1" data-scope="helm">

By default, auto-finalization is enabled, and a major-version upgrade is finalized when all nodes have rejoined the cluster using the new `cockroach` binary. This means that by default, a major-version upgrade cannot be rolled back. Instead, you must [restore the cluster to the previous version]({% link {{ page.version.version }}/restoring-backups-across-versions.md %}#support-for-restoring-backups-into-a-newer-version).

To disable auto-finalization:

1. Connect to the cluster using the SQL shell:

~~~ shell
$ kubectl exec -it cockroachdb-client-secure \
-- ./cockroach sql \
--certs-dir=/cockroach-certs \
--host=cockroachdb-public
~~~

1. Set the [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) `cluster.preserve_downgrade_option` to the cluster's current major version.

Now, to complete a major-version upgrade, you must manually [finalize it](#finalize-a-major-version-upgrade-manually) or [roll it back](#roll-back-a-major-version-upgrade).

{{ new_flag }}

</section>

{% else %}

By default, auto-finalization is enabled, and a major-version upgrade is finalized when all nodes have rejoined the cluster using the new `cockroach` binary. This means that by default, a major-version upgrade cannot be rolled back. Instead, you must [restore the cluster to the previous version]({% link {{ page.version.version }}/restoring-backups-across-versions.md %}#support-for-restoring-backups-into-a-newer-version).

To disable auto-finalization:

1. Connect to the cluster using the SQL shell:

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql
~~~

1. Set the [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) `cluster.auto_upgrade.enabled` to `false`.

Now, to complete a major-version upgrade, you must manually [finalize it](#finalize-a-major-version-upgrade-manually) or [roll it back](#roll-back-a-major-version-upgrade).

{{ new_flag }}

{% endif %}
16 changes: 16 additions & 0 deletions src/current/_includes/common/upgrade/finalize-cloud.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
To finalize a major-version upgrade of CockroachDB {{ site.data.products.advanced }} or CockroachDB {{ site.data.products.standard }} manually,
mdlinville marked this conversation as resolved.
Show resolved Hide resolved

1. Click the cluster's name to open the **Cluster Details** page.
1. At the top of the page, click **Finalize**.

When finalization begins, a series of migration jobs run to enable certain types of features and changes in the new major version that cannot be rolled back. These include changes to system schemas, indexes, and descriptors, and enabling certain types of improvements and new features.

Until the upgrade is finalized, these features and functions will not be available, and the command `SHOW CLUSTER SETTING version` will continue to report the previous major version. Features that are available only after finalization are listed in the release notes for the new major version.

You can monitor the process of finalization in the CockroachDB {{ site.data.products.cloud }} [Jobs page]({% link cockroachcloud/jobs-page.md %}). Migration jobs have names in the format `{major-version}-{migration-id}`. If a migration job fails or stalls, Cockroach Labs can use the migration ID to help diagnose and troubleshoot the problem. Each major version has a unique set of migration jobs and IDs.

When finalization is complete, the **Cluster List** and **Cluster Details** page report the new version, and you can no longer roll back to the previous major version.

{{site.data.alerts.callout_info}}
CockroachDB {{ site.data.products.basic }} clusters are upgraded automatically and finalization is not required.
{{site.data.alerts.end}}
36 changes: 36 additions & 0 deletions src/current/_includes/common/upgrade/finalize-kubernetes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{% assign major_version_numeric = page.version.version | remove_first: 'v' %}

To finalize a major-version upgrade:

1. Connect to the cluster using the SQL shell:

~~~ shell
$ kubectl exec -it cockroachdb-client-secure \
-- ./cockroach sql \
--certs-dir=/cockroach-certs \
--host=cockroachdb-public
~~~

1. Run the following command:

{% include_cached copy-clipboard.html %}
~~~ sql
> RESET CLUSTER SETTING cluster.preserve_downgrade_option;
~~~

A series of migration jobs runs to enable certain types of features and changes in the new major version that cannot be rolled back. These include changes to system schemas, indexes, and descriptors, and enabling certain types of improvements and new features. Until the upgrade is finalized, these features and functions will not be available and the command `SHOW CLUSTER SETTING version` will return the previous version`.

You can monitor the process of the migration in the DB Console [Jobs page]({% link {{ page.version.version }}/ui-jobs-page.md %}). Migration jobs have names in the format `{{ major_version_numeric }}-{migration-id}`. If a migration job fails or stalls, Cockroach Labs can use the migration ID to help diagnose and troubleshoot the problem. Each major version has different migration jobs with different IDs.

The amount of time required for finalization depends on the amount of data in the cluster, because finalization runs various internal maintenance and migration tasks. During this time, the cluster will experience a small amount of additional load.

When all migration jobs have completed, the upgrade is complete.

1. To confirm that finalization has completed, check the cluster version:

{% include_cached copy-clipboard.html %}
~~~ sql
> SHOW CLUSTER SETTING version;
~~~

If the cluster continues to report that it is on the previous version, finalization has not completed. If auto-finalization is enabled but finalization has not completed, check for the existence of [decommissioning nodes]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#status-change) where decommission has stalled. In most cases, issuing the `decommission` command again resolves the issue. If you have trouble upgrading, [contact Support](https://cockroachlabs.com/support/hc/).
38 changes: 38 additions & 0 deletions src/current/_includes/common/upgrade/finalize-self-hosted.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{% assign major_version_numeric = page.version.version | remove_first: 'v' %}

To finalize a major-version upgrade:

1. Connect to the cluster using the SQL shell:

{% include_cached copy-clipboard.html %}
~~~ shell
cockroach sql
~~~

1. Run the following command. Replace `{VERSION}` with the new major version, such as `{{ major_version_numeric }}`.

{% include_cached copy-clipboard.html %}
~~~ sql
SET CLUSTER SETTING version '{VERSION}';
~~~

A series of migration jobs runs to enable certain types of features and changes in the new major version that cannot be rolled back. These include changes to system schemas, indexes, and descriptors, and enabling certain types of improvements and new features. Until the upgrade is finalized, these features and functions will not be available and the command `SHOW CLUSTER SETTING version` will return the previous version.

You can monitor the process of the migration in the DB Console [Jobs page]({% link {{ page.version.version }}/ui-jobs-page.md %}). Migration jobs have names in the format `{{ major_version_numeric }}-{migration-id}`. If a migration job fails or stalls, Cockroach Labs can use the migration ID to help diagnose and troubleshoot the problem. Each major version has different migration jobs with different IDs.

The amount of time required for finalization depends on the amount of data in the cluster, because finalization runs various internal maintenance and migration tasks. During this time, the cluster will experience a small amount of additional load.

{{site.data.alerts.callout_info}}
Finalization is not complete until all [schema change]({% link {{ page.version.version }}/online-schema-changes.md %}) jobs reach a terminal state. Finalization can take as long as the longest-running schema change.
{{site.data.alerts.end}}

When all migration jobs have completed, the upgrade is complete.

1. To confirm that finalization has completed, check the cluster version:

{% include_cached copy-clipboard.html %}
~~~ sql
> SHOW CLUSTER SETTING version;
~~~

If the cluster continues to report that it is on the previous version, finalization has not completed. If auto-finalization is enabled but finalization has not completed, check for the existence of [decommissioning nodes]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#status-change) where decommission has stalled. In most cases, issuing the `decommission` command again resolves the issue. If you have trouble upgrading, [contact Support](https://cockroachlabs.com/support/hc/).
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
This section shows how to perform a major-version upgrade for a cluster in CockroachDB {{ site.data.products.cloud }}. Patch upgrades within a cluster's major version are applied automatically, and no action is required.

1. Verify the cluster's current major version, and which versions it can be upgraded to:
1. Sign into [CockroachDB {{ site.data.products.cloud }}](https://cockroachlabs.cloud).
1. From the **Clusters** page, find the cluster by name. If the cluster is in a folder, click the name of the folder to view its descendants. The cluster's major version and patch are shown in the **Version** column.
1. Check which upgrades are available to the cluster, if any.

Beginning with v24.1, major versions alternate in type between Regular releases, which are required, and Innovation releases, which can be skipped.

From a Regular release, you can upgrade to the next major release (an Innovation release) or the subsequent major release (another Regular release). From an Innovation release, you must upgrade to the next Regular release.

These releases also provide different support periods.

For details, refer the [CockroachDB Cloud Support Policy]({% link cockroachcloud/upgrade-policy.md %}#cockroachdb-cloud-support-policy).

To check whether major-version upgrades are available, click the three-dot **Action** menu. If upgrades are available, **Upgrade major version** will be enabled. Click it. In the dialog, if only one newer major version upgrade is available, it is selected automatically. If multiple upgrades are available, the latest Regular release is selected by default. If you intend to upgrade the cluster, keep this dialog open while completing the next steps. Otherwise, close the dialog.
1. Before beginning a major-version upgrade, review its [Release notes]({% link releases/index.md %}), as well as the release notes for any skipped Innovation Releases. If any backward-incompatible changes or new features impact the cluster's workloads, you may need to make adjustments before beginning the upgrade.
1. A CockroachDB {{ site.data.products.standard }} or {{ site.data.products.basic }} cluster remains fully available while it is upgraded. For a multi-node CockroachDB {{ site.data.products.advanced }} cluster, nodes are upgraded one at a time in a rolling fashion so the cluster remains available, with one node unavailable at a time. A single-node {{ site.data.products.advanced }} cluster will be unavailable while the cluster is upgraded and restarted. If necessary, prepare for the upgrade by communicating ahead of time with your users, and plan to begin the upgrade during a time when the impact on the cluster's workload will be minimized.
1. To begin a major-version upgrade, go back to the dialog in the CockroachDB {{ site.data.products.cloud }} Console. If multiple upgrades are available, select the desired version. Review the details, then click **Start upgrade**. The dialog closes, and the **Version** columnin the **Cluster List** page changes to **Upgrading**. When the upgrade has finished but has not yet been finalized, the **Version** column reports that the new version is **pending**.
mdlinville marked this conversation as resolved.
Show resolved Hide resolved

The upgrade is not complete until it is [finalized](#finalize-a-major-version-upgrade-manually). After the upgrade is finalized, the cluster can no longer be [rolled back](#roll-back-a-major-version-upgrade) to its previous major version. If you take no action, finalization occurs automatically after approximately 72 hours.
Loading
Loading