-
@@ -96,8 +94,11 @@
-
+
+ Enterprise: This paid license allows usage of all CockroachDB features in accordance with the terms specified in the CockroachDB Software License. +
+
+ -
+
+ Enterprise Free: Same functionality as Enterprise, but free of charge for businesses with less than $10M in annual revenue. Clusters will be throttled after 7 days without sending telemetry. License must be renewed annually. +
+
+ -
+
+ Enterprise Trial: A 30 day self-service trial license. Telemetry is required during the trial. Clusters will be throttled after 7 days without sending telemetry. Telemetry can be disabled once the cluster is upgraded to a paid Enterprise license. +
+
+ -
+
+ Added on-hover cursor support that will display the closest time-series value and highlight the node in the legend to allow users to quickly pinpoint outliers. +
+
+ -
+
+ Improved legend visibillity and made legends scrollable to improve usability and reduce vertical scrolling. +
+
+ -
+
+ Foreground (regular) CPU work +
+
+ -
+
+ Store (IO) work +
+
+ -
+
+ Background (elastic) CPU work +
+
+ -
+
+ Replication admission control (store overload on replicas) +
+
+ - [`lagging_ranges_threshold`]({% link {{ page.version.version }}/create-changefeed.md %}#lagging-ranges-threshold), which is the amount of time that a range checkpoint needs to be in the past to be considered lagging.
- [`lagging_ranges_polling_interval`]({% link {{ page.version.version }}/create-changefeed.md %}#lagging-ranges-polling-interval), which is the frequency at which lagging ranges are polled and the metric is updated.
- `retry`: Retry the backup right away.
- `reschedule`: Retry the backup by rescheduling it based on the `RECURRING` expression.
- `pause`: Pause the schedule. This requires manual intervention to [resume the schedule]({% link {{ page.version.version }}/resume-schedules.md %}).
- `start`: Start the new backup anyway, even if the previous one is still running.
- `skip`: Skip the new backup and run the next backup based on the `RECURRING` expression.
- `wait`: Wait for the previous backup to complete.
- If you're just getting started with CockroachDB:
+
-
+
- Start a cluster locally and talk to it via the built-in SQL client +
- Learn more about CockroachDB SQL +
- Build a simple application with CockroachDB using PostgreSQL-compatible client drivers and ORMs +
- Explore CockroachDB features like automatic replication, rebalancing, and fault tolerance +
+ - If you're ready to run CockroachDB in production: + + +
- If your cluster's certificates are managed using
cert-manager(recommended but not default), get the names of the cluster'sissuersand delete them: {{ get_issuers_command }} - If your cluster uses self-signed certificates (the default), get the names of any CSRs for the cluster, then delete them: {{ get_csrs_command }}
- If your cluster's certificates are managed using
cert-manager(recommended but not default), get the names of the cluster'sissuersand delete them: {{ get_issuers_command }} - If your cluster uses self-signed certificates (the default), get the names of any CSRs for the cluster, then delete them: {{ get_csrs_command }}
- To use these defaults, apply the Operator manifest without modifying it: {{ apply_default_operator_manifest_command }}
- To change these defaults:
- Download the Operator manifest: {{ download_operator_manifest_command }}
- To use a custom namespace, edit all instances of
namespace: cockroach-operator-systemwith your desired namespace. - To limit the namespaces that will be monitored, set the
WATCH_NAMESPACEenvironment variable in theDeploymentpod spec. This can be set to a single namespace or a comma-delimited set of namespaces. When set, only thoseCrdbClusterresources in the supplied namespace(s) will be reconciled. - Apply your local version of the Operator manifest to the cluster: {{ apply_local_operator_manifest_command }}
"alteredSettings": {
- "cluster.organization": "\u003credacted\u003e", "kv.lease.expiration_leases_only.enabled": "false",
- "kv.rangefeed.enabled": "true"
+ "cluster.organization": "\u003credacted\u003e",
+ "cluster.secret": "\u003credacted\u003e",
+ "kv.rangefeed.enabled": "true",
+ "server.oidc_authentication.client_id": "\u003credacted\u003e",
+ "server.oidc_authentication.client_secret": "\u003credacted\u003e",
}
Security updates
+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of CockroachDB's diagnostics telemetry. The transmitted payload includes a record of modified cluster settings and their values when they are not strings. If you previously applied the mitigations in [Technical Advisory 133479]({% link advisories/a133479.md %}), you can safely turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster settings values. [#134063][#134063] + +General changes
+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134083][#134083] + +Bug fixes
+ +- Fixed a bug where CockroachDB could encounter an internal error `interface conversion: coldata.Column is` in an edge case. The bug was present in v22.2.13+, v23.1.9+, and v23.2+. [#133758][#133758] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT ... ON CONFLICT ... DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug had been present since at least v20.1.0. [#133824][#133824] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE FROM ... user` on an object that is not a sequence. [#133706][#133706] +- Addressed a panic that could occur inside `CREATE TABLE AS` if sequence builtin expressions had invalid function overloads. [#133866][#133866] +- Previously, when executing queries with index / lookup joins where ordering needed to be maintained, CockroachDB's behavior could lead to increased query latency, potentially by several orders of magnitude. This bug was introduced in v22.2, and is now fixed. [#134363][#134363] +- Fixed a bug where `DROP CASCADE` would occasionally panic with an `un-dropped backref` message on partitioned tables. [#134524][#134524] +- Reduced the duration of partitions in the gossip network when a node crashes. This eliminates false positives in the `ranges.unavailable` metric. [#134603][#134603] + +[#133706]: https://github.com/cockroachdb/cockroach/pull/133706 +[#133758]: https://github.com/cockroachdb/cockroach/pull/133758 +[#133824]: https://github.com/cockroachdb/cockroach/pull/133824 +[#133866]: https://github.com/cockroachdb/cockroach/pull/133866 +[#134063]: https://github.com/cockroachdb/cockroach/pull/134063 +[#134083]: https://github.com/cockroachdb/cockroach/pull/134083 +[#134363]: https://github.com/cockroachdb/cockroach/pull/134363 +[#134524]: https://github.com/cockroachdb/cockroach/pull/134524 +[#134603]: https://github.com/cockroachdb/cockroach/pull/134603 +[#134649]: https://github.com/cockroachdb/cockroach/pull/134649 +[154e9f0e0]: https://github.com/cockroachdb/cockroach/commit/154e9f0e0 diff --git a/src/current/_includes/releases/v23.2/v23.2.13.md b/src/current/_includes/releases/v23.2/v23.2.13.md index 28ce188820c..26f4a561790 100644 --- a/src/current/_includes/releases/v23.2/v23.2.13.md +++ b/src/current/_includes/releases/v23.2/v23.2.13.md @@ -16,7 +16,7 @@ Release Date: October 17, 2024 [#130664][#130664] -- The new [metric]({% link v23.2/metrics.md %}) `changefeed.total_ranges` allows observation of the number of changes that are watched by a changefeed aggregator. It uses the same polling interval as `changefeed.lagging_ranges`, which is controlled by the changefeed option `lagging_ranges_polling_interval`. [#130984][#130984] +- The new [metric]({% link v23.2/metrics.md %}) `changefeed.total_ranges` allows observation of the number of ranges that are watched by a changefeed aggregator. It uses the same polling interval as `changefeed.lagging_ranges`, which is controlled by the changefeed option `lagging_ranges_polling_interval`. [#130984][#130984] - The following groups of [metrics]({% link v23.2/metrics.md %}) and [logs]({% link v23.2/logging.md %}) have been renamed to include the buffer they are associated with. The previous metrics are still maintained for backward compatibility. - `changefeed.buffer_entries.*` - `changefeed.buffer_entries_mem.*` @@ -81,6 +81,7 @@ Release Date: October 17, 2024 [#130664]: https://github.com/cockroachdb/cockroach/pull/130664 [#130790]: https://github.com/cockroachdb/cockroach/pull/130790 [#130919]: https://github.com/cockroachdb/cockroach/pull/130919 +[#130984]: https://github.com/cockroachdb/cockroach/pull/130984 [#130988]: https://github.com/cockroachdb/cockroach/pull/130988 [#131065]: https://github.com/cockroachdb/cockroach/pull/131065 [#131128]: https://github.com/cockroachdb/cockroach/pull/131128 diff --git a/src/current/_includes/releases/v23.2/v23.2.16.md b/src/current/_includes/releases/v23.2/v23.2.16.md index 5793f589357..6e205c889b3 100644 --- a/src/current/_includes/releases/v23.2/v23.2.16.md +++ b/src/current/_includes/releases/v23.2/v23.2.16.md @@ -1,6 +1,6 @@ ## v23.2.16 -Release Date: November 15, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} diff --git a/src/current/_includes/releases/v23.2/v23.2.17.md b/src/current/_includes/releases/v23.2/v23.2.17.md new file mode 100644 index 00000000000..29855bc2638 --- /dev/null +++ b/src/current/_includes/releases/v23.2/v23.2.17.md @@ -0,0 +1,63 @@ +## v23.2.17 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +Security updates
+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of Cockroach Labs' diagnostics telemetry. The payload includes a record of modified cluster settings and their values when they are not strings. If you previously applied the mitigations in Technical Advisory 133479, you can safely set the value of cluster setting `server.redact_sensitive_settings.enabled` to `false` and turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster setting values. [#134015][#134015] + +General changes
+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134085][#134085] + +{{ site.data.products.enterprise }} edition changes
+ +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps (PTS). This table is required for CDC queries. [#134233][#134233] + +Operational changes
+ +- Added a new cluster setting `ui.database_locality_metadata.enabled`, which allows operators to disable loading extended database and table region information in the DB Console's Databases and Table Details pages. This information can cause significant CPU load on large clusters with many ranges. Versions of this page from v24.3 onwards do not have this problem. If you require this data, you can use the `SHOW RANGES FROM {DATABASE| TABLE}` query via SQL to compute on-demand. [#134093][#134093] + +Bug fixes
+ +- Previously, CockroachDB could encounter an internal error of the form `interface conversion: coldata.Column is` in an edge case. This is now fixed. The bug was present in versions v22.2.13 and later, v23.1.9 and later, and v23.2 and later. [#133759][#133759] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT ... ON CONFLICT ... DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug had been present since at least v20.1.0. [#133823][#133823] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE ... FROM user` on an object that was not a sequence. [#133707][#133707] +- Addressed a panic that could occur inside `CREATE TABLE AS` that occurred if sequence builtin expressions had invalid function overloads. [#133867][#133867] +- String constants can now be compared against collated strings. [#134114][#134114] +- Previously, when executing queries with index or lookup joins when the ordering needed to be maintained, CockroachDB in some cases could get into a pathological state which would lead to increased query latency, possibly by several orders of magnitude. This bug was introduced in v22.2 and is now fixed. [#134364][#134364] +- Addressed a bug with `DROP CASCADE` that would occasionally panic with an `un-dropped backref` message on partitioned tables. [#134523][#134523] +- Reduced the duration of partitions in the gossip network when a node crashes. This eliminates false positives in the `ranges.unavailable` metric. [#134602][#134602] +- An error message is no longer returned when a non-admin user runs `DROP ROLE IF EXISTS` on a user that does not exist. [#134967][#134967] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g., with an equality filter). [#135113][#135113] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135196][#135196] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#135691][#135691] + +Performance improvements
+ +- CockroachDB now avoids loading unnecessary file blocks shortly after a rebalance in a rare case. [#134526][#134526] [#135303][#135303] [#135577][#135577] +- Reduced the write-amplification impact of rebalances by splitting snapshot sstable files into smaller ones before ingesting them into Pebble. [#134526][#134526] [#135303][#135303] [#135577][#135577] + +[#133707]: https://github.com/cockroachdb/cockroach/pull/133707 +[#133759]: https://github.com/cockroachdb/cockroach/pull/133759 +[#133823]: https://github.com/cockroachdb/cockroach/pull/133823 +[#133867]: https://github.com/cockroachdb/cockroach/pull/133867 +[#134015]: https://github.com/cockroachdb/cockroach/pull/134015 +[#134085]: https://github.com/cockroachdb/cockroach/pull/134085 +[#134093]: https://github.com/cockroachdb/cockroach/pull/134093 +[#134114]: https://github.com/cockroachdb/cockroach/pull/134114 +[#134233]: https://github.com/cockroachdb/cockroach/pull/134233 +[#134364]: https://github.com/cockroachdb/cockroach/pull/134364 +[#134523]: https://github.com/cockroachdb/cockroach/pull/134523 +[#134526]: https://github.com/cockroachdb/cockroach/pull/134526 +[#134602]: https://github.com/cockroachdb/cockroach/pull/134602 +[#134647]: https://github.com/cockroachdb/cockroach/pull/134647 +[#134967]: https://github.com/cockroachdb/cockroach/pull/134967 +[#135113]: https://github.com/cockroachdb/cockroach/pull/135113 +[#135196]: https://github.com/cockroachdb/cockroach/pull/135196 +[#135303]: https://github.com/cockroachdb/cockroach/pull/135303 +[#135577]: https://github.com/cockroachdb/cockroach/pull/135577 +[#135691]: https://github.com/cockroachdb/cockroach/pull/135691 +[#136006]: https://github.com/cockroachdb/cockroach/pull/136006 diff --git a/src/current/_includes/releases/v24.1/v24.1.7.md b/src/current/_includes/releases/v24.1/v24.1.7.md index 9517b2ec9bb..f0d339f9c81 100644 --- a/src/current/_includes/releases/v24.1/v24.1.7.md +++ b/src/current/_includes/releases/v24.1/v24.1.7.md @@ -1,6 +1,6 @@ ## v24.1.7 -Release Date: November 14, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} diff --git a/src/current/_includes/releases/v24.1/v24.1.8.md b/src/current/_includes/releases/v24.1/v24.1.8.md new file mode 100644 index 00000000000..a6b427cf04b --- /dev/null +++ b/src/current/_includes/releases/v24.1/v24.1.8.md @@ -0,0 +1,73 @@ +## v24.1.8 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +Security updates
+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of our diagnostics telemetry. The transmitted payload includes a record of modified cluster settings and their values when they are not strings. If you previously applied the mitigations in [Technical Advisory 133479]({% link advisories/a133479.md %}), you can safely set the value of cluster setting `server.redact_sensitive_settings.enabled` to false and turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster settings values. [#134016][#134016] + +General changes
+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134087][#134087] +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps (PTS). This table is required for change data capture queries. [#134836][#134836] +- Added changefeed support for the `mvcc_timestamp` option with the `avro` format. If both options are specified, the `avro` schema includes an `mvcc_timestamp` metadata field and emits the row's MVCC timestamp with the row data. [#136482][#136482] + +Operational changes
+ +- To prevent unnecessary queuing in admission control CPU queues, the `goschedstats.always_use_short_sample_period.enabled` setting should be set to `true` for any production cluster. [#133583][#133583] +- A new cluster setting `ui.database_locality_metadata.enabled`, when set to `true`, disables loading database and table region information in the DB Console Database and Table pages. This information can cause significant CPU load on large clusters with many ranges. The Database and Table pages in v24.3 onwards do not have this problem. If you require this data, use the `SHOW RANGES FROM {DATABASE|TABLE}` SQL statement to compute this information on-demand. [#134094][#134094] +- The row-level TTL job now will periodically log progress by showing the number of table spans that have been processed so far. [#135179][#135179] + +Bug fixes
+ +- Fixed a bug where CockroachDB could encounter an internal error `interface conversion: coldata.Column is` in an edge case. The bug was present in v22.2.13+, v23.1.9+, and v23.2+. [#133760][#133760] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE ... FROM user` on an object that is not a sequence. [#133708][#133708] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT ... ON CONFLICT ... DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug had been present since at least v20.1.0. [#133822][#133822] +- Addressed a panic that could occur inside `CREATE TABLE AS` if sequence builtin expressions had invalid function overloads. [#133868][#133868] +- String constants can now be compared against collated strings. [#134105][#134105] +- Previously, when executing queries with index / lookup joins where ordering needed to be maintained, CockroachDB's behavior could lead to increased query latency, potentially by several orders of magnitude. This bug was introduced in v22.2, and is now fixed. [#134365][#134365] +- Fixed a bug where `DISCARD ALL` statements were counted under the `sql.ddl.count` metric. Now these statements will be counted under the `sql.misc.count` metric. [#134508][#134508] +- Fixed a bug where `DROP CASCADE` would occasionally panic with an `un-dropped backref` message on partitioned tables. [#134522][#134522] +- Reduced the duration of partitions in the gossip network when a node crashes. This eliminates false positives in the `ranges.unavailable` metric. [#134601][#134601] +- As a non-admin user who runs `DROP ROLE IF EXISTS` on a user that does not exist, you no longer get an error message. [#134968][#134968] +- Fixed a bug that caused quotes around the name of a routine to be dropped when the routine was called within another routine. This could prevent the correct routine name from being resolved if the nested routine name was case-sensitive. The bug has existed since v24.1, when nested routines were introduced. [#134001][#134001] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (for example, with an equality filter). [#135111][#135111] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema, such as `pg_catalog` or `information_schema`. [#135195][#135195] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#135690][#135690] +- `CREATE SCHEMA` now returns the correct error if a the schema name is missing. [#135926][#135926] + +Performance improvements
+ +- Unnecessary block loads of SSTable files are now avoided in some rare cases after a replica rebalance. [#134541][#134541] + +[#133583]: https://github.com/cockroachdb/cockroach/pull/133583 +[#133708]: https://github.com/cockroachdb/cockroach/pull/133708 +[#133760]: https://github.com/cockroachdb/cockroach/pull/133760 +[#133822]: https://github.com/cockroachdb/cockroach/pull/133822 +[#133868]: https://github.com/cockroachdb/cockroach/pull/133868 +[#134001]: https://github.com/cockroachdb/cockroach/pull/134001 +[#134016]: https://github.com/cockroachdb/cockroach/pull/134016 +[#134087]: https://github.com/cockroachdb/cockroach/pull/134087 +[#134094]: https://github.com/cockroachdb/cockroach/pull/134094 +[#134100]: https://github.com/cockroachdb/cockroach/pull/134100 +[#134105]: https://github.com/cockroachdb/cockroach/pull/134105 +[#134365]: https://github.com/cockroachdb/cockroach/pull/134365 +[#134446]: https://github.com/cockroachdb/cockroach/pull/134446 +[#134508]: https://github.com/cockroachdb/cockroach/pull/134508 +[#134522]: https://github.com/cockroachdb/cockroach/pull/134522 +[#134541]: https://github.com/cockroachdb/cockroach/pull/134541 +[#134601]: https://github.com/cockroachdb/cockroach/pull/134601 +[#134648]: https://github.com/cockroachdb/cockroach/pull/134648 +[#134731]: https://github.com/cockroachdb/cockroach/pull/134731 +[#134836]: https://github.com/cockroachdb/cockroach/pull/134836 +[#134968]: https://github.com/cockroachdb/cockroach/pull/134968 +[#135111]: https://github.com/cockroachdb/cockroach/pull/135111 +[#135179]: https://github.com/cockroachdb/cockroach/pull/135179 +[#135195]: https://github.com/cockroachdb/cockroach/pull/135195 +[#135614]: https://github.com/cockroachdb/cockroach/pull/135614 +[#135690]: https://github.com/cockroachdb/cockroach/pull/135690 +[#135926]: https://github.com/cockroachdb/cockroach/pull/135926 +[#136008]: https://github.com/cockroachdb/cockroach/pull/136008 +[#136482]: https://github.com/cockroachdb/cockroach/pull/136482 diff --git a/src/current/_includes/releases/v24.2/v24.2.5.md b/src/current/_includes/releases/v24.2/v24.2.5.md index 643d7a89006..844401e23df 100644 --- a/src/current/_includes/releases/v24.2/v24.2.5.md +++ b/src/current/_includes/releases/v24.2/v24.2.5.md @@ -1,6 +1,6 @@ ## v24.2.5 -Release Date: November 14, 2024 +Release Date: November 18, 2024 {% include releases/new-release-downloads-docker-image.md release=include.release %} diff --git a/src/current/_includes/releases/v24.2/v24.2.6.md b/src/current/_includes/releases/v24.2/v24.2.6.md new file mode 100644 index 00000000000..3cedad5d3a0 --- /dev/null +++ b/src/current/_includes/releases/v24.2/v24.2.6.md @@ -0,0 +1,78 @@ +## v24.2.6 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +Security updates
+ +- All cluster settings that accept strings are now fully redacted when transmitted as part of diagnostics telemetry. This payload includes a record of modified cluster settings and their values when they are not strings. Customers who previously applied the mitigations in [Technical Advisory 133479]({% link advisories/a133479.md %}) can safely set the value of cluster setting `server.redact_sensitive_settings.enabled` to false and turn on diagnostic reporting via the `diagnostics.reporting.enabled` cluster setting without leaking sensitive cluster settings values. [#134017][#134017] + +General changes
+ +- `COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING` is no longer mentioned in the `cockroach demo` command. [#134088][#134088] +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps. This table is required for change data capture queries. [#134837][#134837] + +Operational changes
+ +- The `goschedstats.always_use_short_sample_period.enabled` setting should be set to true for any production cluster, to prevent unnecessary queuing in admission control CPU queues. [#133584][#133584] +- Added a new cluster setting `ui.database_locality_metadata.enabled` that allows operators to disable loading extended database and table region information in the DB Console Database and Table pages. This information can cause significant CPU load on large clusters with many ranges. Versions of this page from v24.3 and later do not have this problem. If customers require this data, they can use the `SHOW RANGES FROM {DATABASE| TABLE}` query via SQL to compute on-demand. [#134095][#134095] +- Row-level TTL jobs now periodically log progress by showing the number of table spans that have been processed so far. [#135170][#135170] + +Bug fixes
+ +- Fixed a bug that caused non-reusable query plans, e.g., plans for DDL and `SHOW ...` statements, to be cached and reused in future executions, possibly causing stale results to be returned. This bug only occurred when `plan_cache_mode` was set to `auto` or `force_generic_plan`, both of which are not currently the default settings. [#133074][#133074] +- Previously, CockroachDB could encounter an internal error of the form `interface conversion: coldata.Column is` in an edge case and this is now fixed. The bug is present in v22.2.13+, v23.1.9+, v23.2+. [#133761][#133761] +- Fixed a bug that caused incorrect `NOT NULL` constraint violation errors on `UPSERT` and `INSERT .. ON CONFLICT .. DO UPDATE` statements when those statements updated an existing row and a subset of columns that did not include a `NOT NULL` column of the table. This bug has been present since at least v20.1.0. [#133821][#133821] +- Fixed an unhandled error that could occur when using `REVOKE ... ON SEQUENCE FROM ... user` on an object that is not a sequence. [#133709][#133709] +- Addressed a panic inside `CREATE TABLE AS` if sequence builtin expressions had invalid function overloads. [#133869][#133869] +- `STRING`constants can now be compared against collated strings. [#134084][#134084] +- When executing queries with index / lookup joins when the ordering needs to be maintained, previously CockroachDB could experience increased query latency, possibly by several orders of magnitude. This bug was introduced in v22.2 and is now fixed. [#134366][#134366] +- Fixed a minor bug where `DISCARD ALL` statements were counted under the `sql.ddl.count` metric. Now these will be counted under the `sql.misc.count` metric. [#134509][#134509] +- Addressed a bug with `DROP CASCADE` that would occasionally panic with an undropped `backref` message on partitioned tables. [#134472][#134472] +- Reduced the duration of partitions in the gossip network when a node crashes in order to eliminate false positives in the `ranges.unavailable` metric. [#134600][#134600] +- Non-`admin` users that run `DROP ROLE IF EXISTS` on a user that does not exist will no longer receive an error message. [#134969][#134969] +- Fixed a bug that caused quotes around the name of a routine to be dropped when it was called within another routine. This could prevent the correct routine from being resolved if the nested routine name was case sensitive. The bug has existed since v24.1, when nested routines were introduced. [#134000][#134000] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g., with an equality filter). [#135076][#135076] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135180][#135180] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#135689][#135689] +- Fixed a bug where `ALTER DATABASE` operations that modify the zone config would hang if an invalid zone config already exists. [#135215][#135215] +- `CREATE SCHEMA` now returns the correct error if a the schema name is missing. [#135927][#135927] +- Using more than one `DECLARE` statment in the definition of a user-defined function now correctly declares additional variables. [#135738][#135738] +- Fixed a bug where CockroachDB would encounter an internal error when evaluating `FETCH ABSOLUTE 0` statements. The bug has been present since v22.1. [#134992][#134992] +- Fixed an issue where corrupted table statistics could cause the `cockroach` process to crash. [#136041][#136041] +- Fixed a bug that causes the optimizer to use stale table statistics after altering an `ENUM` type used in the table. [#136812][#136812] + +[#133074]: https://github.com/cockroachdb/cockroach/pull/133074 +[#133584]: https://github.com/cockroachdb/cockroach/pull/133584 +[#133709]: https://github.com/cockroachdb/cockroach/pull/133709 +[#133761]: https://github.com/cockroachdb/cockroach/pull/133761 +[#133821]: https://github.com/cockroachdb/cockroach/pull/133821 +[#133869]: https://github.com/cockroachdb/cockroach/pull/133869 +[#134000]: https://github.com/cockroachdb/cockroach/pull/134000 +[#134017]: https://github.com/cockroachdb/cockroach/pull/134017 +[#134084]: https://github.com/cockroachdb/cockroach/pull/134084 +[#134088]: https://github.com/cockroachdb/cockroach/pull/134088 +[#134095]: https://github.com/cockroachdb/cockroach/pull/134095 +[#134099]: https://github.com/cockroachdb/cockroach/pull/134099 +[#134366]: https://github.com/cockroachdb/cockroach/pull/134366 +[#134447]: https://github.com/cockroachdb/cockroach/pull/134447 +[#134472]: https://github.com/cockroachdb/cockroach/pull/134472 +[#134509]: https://github.com/cockroachdb/cockroach/pull/134509 +[#134600]: https://github.com/cockroachdb/cockroach/pull/134600 +[#134646]: https://github.com/cockroachdb/cockroach/pull/134646 +[#134730]: https://github.com/cockroachdb/cockroach/pull/134730 +[#134837]: https://github.com/cockroachdb/cockroach/pull/134837 +[#134969]: https://github.com/cockroachdb/cockroach/pull/134969 +[#134992]: https://github.com/cockroachdb/cockroach/pull/134992 +[#135076]: https://github.com/cockroachdb/cockroach/pull/135076 +[#135170]: https://github.com/cockroachdb/cockroach/pull/135170 +[#135180]: https://github.com/cockroachdb/cockroach/pull/135180 +[#135215]: https://github.com/cockroachdb/cockroach/pull/135215 +[#135611]: https://github.com/cockroachdb/cockroach/pull/135611 +[#135689]: https://github.com/cockroachdb/cockroach/pull/135689 +[#135738]: https://github.com/cockroachdb/cockroach/pull/135738 +[#135927]: https://github.com/cockroachdb/cockroach/pull/135927 +[#136010]: https://github.com/cockroachdb/cockroach/pull/136010 +[#136041]: https://github.com/cockroachdb/cockroach/pull/136041 +[#136812]: https://github.com/cockroachdb/cockroach/pull/136812 diff --git a/src/current/_includes/releases/v24.3/v24.3.1.md b/src/current/_includes/releases/v24.3/v24.3.1.md new file mode 100644 index 00000000000..4db85eb32a8 --- /dev/null +++ b/src/current/_includes/releases/v24.3/v24.3.1.md @@ -0,0 +1,78 @@ +## v24.3.1 + +Release Date: December 12, 2024 + +{% include releases/new-release-downloads-docker-image.md release=include.release %} + +SQL language changes
+ +- When triggers fire one another cyclically, the new `recursion_depth_limit` setting now limits the depth of the recursion. By default, the limit is `1000` nested trigger executions. [#135046][#135046] + +Operational changes
+ +- The metrics scrape HTTP endpoint at `/ _status/vars` will now truncate HELP text at the first sentence, reducing the metadata for metrics with large descriptions. Customers can still access these descriptions via our docs. [#135021][#135021] +- The row-level TTL job now periodically updates the progress meter in the jobs introspection interfaces, including `SHOW JOBS` and the Jobs page in the DB console. [#135171][#135171] +- Telemetry delivery is now considered successful even in cases where we experience a network timeout. This will prevent throttling in cases outside an operator's control. [#136481][#136481] + +DB Console changes
+ +- When activating statement diagnostics in the DB Console, users now have the option to produce a redacted bundle as output. This bundle will omit sensitive data. [#134993][#134993] + +Other changes
+ +- Protected timestamp records for changefeeds now include the `system.users` table. This ensures that user information remains available when running CDC queries against historical data. [#134238][#134238] + +Bug fixes
+ +- Fixed a bug that could cause `DELETE` triggers not to fire on cascading delete, and which could cause `INSERT` triggers to match incorrectly in the same scenario. [#134896][#134896] +- Reduced the duration of partitions in the gossip network when a node crashes in order to eliminate false positives in the `ranges.unavailable` metric. [#134480][#134480] +- When a non-admin user runs `DROP ROLE IF EXISTS` on a user that does not exist, an error is no longer returned. [#134970][#134970] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g. with an equality filter). [#135037][#135037] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135181][#135181] +- A bug has been fixed that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#134710][#134710] +- Prevent `ALTER DATABASE` operations that modify the zone config from hanging if an invalid zone config already exists. [#135216][#135216] +- `CREATE SCHEMA` now returns the correct error if a schema name is missing. [#135928][#135928] +- `percentile_cont` and `percentile_disc` aggregate functions now support `float4` data type inputs. Previously, these functions would return an error when used with `float4` values. [#135764][#135764] +- `security.certificate.*` metrics now update correctly when certificates are reloaded during node runtime. Previously, these metrics would not reflect changes to certificates after node startup. [#136227][#136227] +- SQL roles created from LDAP groups that contain periods (.) or hyphens (-) in their Common Names (CN) no longer result in authorization failures. [#134942][#134942] +- LDAP authorization now supports partial group mapping, allowing users to authenticate even when some LDAP groups do not have corresponding CockroachDB roles. Previously, authentication would fail if any LDAP group lacked a matching database role. [#135587][#135587] +- Regional by row tables with uniqueness constraints where the region is not part of those uniqueness constraints and which also contain non-unique indices will now have those constraints properly enforced when modified at `READ COMMITTED` isolation. This bug was introduced in v24.3.0. [#137367][#137367] + +Performance improvements
+ +- The `_status/nodes_ui` API no longer returns unnecessary metrics in its response. This decreases the payload size of the API and improves the load time of various DB Console pages and components. [#135209][#135209] +- PL/pgSQL loops now execute up to 3-4x faster through improved optimization, particularly when they contain subqueries. This enhancement improves performance for routines with many iterations or nested operations. [#135648][#135648] + +[#133230]: https://github.com/cockroachdb/cockroach/pull/133230 +[#134238]: https://github.com/cockroachdb/cockroach/pull/134238 +[#134480]: https://github.com/cockroachdb/cockroach/pull/134480 +[#134710]: https://github.com/cockroachdb/cockroach/pull/134710 +[#134729]: https://github.com/cockroachdb/cockroach/pull/134729 +[#134896]: https://github.com/cockroachdb/cockroach/pull/134896 +[#134942]: https://github.com/cockroachdb/cockroach/pull/134942 +[#134970]: https://github.com/cockroachdb/cockroach/pull/134970 +[#134993]: https://github.com/cockroachdb/cockroach/pull/134993 +[#135021]: https://github.com/cockroachdb/cockroach/pull/135021 +[#135037]: https://github.com/cockroachdb/cockroach/pull/135037 +[#135046]: https://github.com/cockroachdb/cockroach/pull/135046 +[#135094]: https://github.com/cockroachdb/cockroach/pull/135094 +[#135120]: https://github.com/cockroachdb/cockroach/pull/135120 +[#135171]: https://github.com/cockroachdb/cockroach/pull/135171 +[#135181]: https://github.com/cockroachdb/cockroach/pull/135181 +[#135209]: https://github.com/cockroachdb/cockroach/pull/135209 +[#135216]: https://github.com/cockroachdb/cockroach/pull/135216 +[#135587]: https://github.com/cockroachdb/cockroach/pull/135587 +[#135648]: https://github.com/cockroachdb/cockroach/pull/135648 +[#135764]: https://github.com/cockroachdb/cockroach/pull/135764 +[#135928]: https://github.com/cockroachdb/cockroach/pull/135928 +[#136011]: https://github.com/cockroachdb/cockroach/pull/136011 +[#136227]: https://github.com/cockroachdb/cockroach/pull/136227 +[#136481]: https://github.com/cockroachdb/cockroach/pull/136481 +[#137367]: https://github.com/cockroachdb/cockroach/pull/137367 +[0d7f6eed3]: https://github.com/cockroachdb/cockroach/commit/0d7f6eed3 +[1f2b1b084]: https://github.com/cockroachdb/cockroach/commit/1f2b1b084 +[3cbd07fbd]: https://github.com/cockroachdb/cockroach/commit/3cbd07fbd +[3f5305a4c]: https://github.com/cockroachdb/cockroach/commit/3f5305a4c +[965dded2a]: https://github.com/cockroachdb/cockroach/commit/965dded2a +[989a49c3f]: https://github.com/cockroachdb/cockroach/commit/989a49c3f +[9951e3e61]: https://github.com/cockroachdb/cockroach/commit/9951e3e61 diff --git a/src/current/_includes/releases/v25.1/backward-incompatible.md b/src/current/_includes/releases/v25.1/backward-incompatible.md new file mode 100644 index 00000000000..d81c2f57213 --- /dev/null +++ b/src/current/_includes/releases/v25.1/backward-incompatible.md @@ -0,0 +1,10 @@ +Before [upgrading to CockroachDB v24.3]({% link v24.3/upgrade-cockroach-version.md %}), be sure to review the following backward-incompatible changes, as well as [key cluster setting changes](#v24-3-0-cluster-settings), and adjust your deployment as necessary. + +If you plan to upgrade to v24.3 directly from v24.1 and skip v24.2, be sure to also review the [v24.2 release notes]({% link releases/v24.2.md %}) for backward-incompatible changes from v24.1. + +- Upgrading to v24.3 is blocked if no [license]({% link v24.3/licensing-faqs.md %}) is installed, or if a trial/free license is installed with telemetry disabled. [#130576][#130576] + +[#130576]: https://github.com/cockroachdb/cockroach/pull/130576 + +{% comment %}Remove this anchor when it is added to the v24.3.0 GA release notes{% endcomment %} + diff --git a/src/current/_includes/releases/v25.1/deprecations.md b/src/current/_includes/releases/v25.1/deprecations.md new file mode 100644 index 00000000000..abf7fd16114 --- /dev/null +++ b/src/current/_includes/releases/v25.1/deprecations.md @@ -0,0 +1,5 @@ +The following deprecations are announced in v24.3. If you plan to upgrade to v24.3 directly from v24.1 and skip v24.2, be sure to also review the [v24.2 release notes]({% link releases/v24.2.md %}) for deprecations. + +- The session variable [`enforce_home_region_follower_reads_enabled`]({% link v24.3/session-variables.md %}#enforce-home-region-follower-reads-enabled) is now deprecated, and will be removed in a future release. The related session variable [`enforce_home_region`]({% link v24.3/session-variables.md %}#enforce-home-region) is **not** deprecated. [#129024][#129024] + +[#129024]: https://github.com/cockroachdb/cockroach/pull/129024 diff --git a/src/current/_includes/releases/v25.1/feature-detail-key.html b/src/current/_includes/releases/v25.1/feature-detail-key.html new file mode 100644 index 00000000000..e11ead7e232 --- /dev/null +++ b/src/current/_includes/releases/v25.1/feature-detail-key.html @@ -0,0 +1,25 @@ +| Feature detail key | +|
|---|---|
| ★ | +Features marked "All★" were recently made available in the CockroachDB Cloud platform. They are available for all supported versions of CockroachDB, under the deployment methods specified in their row under Availability. | +
| ★★ | +Features marked "All★★" were recently made available via tools maintained outside of the CockroachDB binary. They are available to use with all supported versions of CockroachDB, under the deployment methods specified in their row under Availability. | +
| {% include icon-yes.html %} | +Feature is available for this deployment method of CockroachDB as specified in the icon’s column: CockroachDB Self-hosted, CockroachDB Advanced, CockroachDB Standard, or CockroachDB Basic. | +
| {% include icon-no.html %} | +Feature is not available for this deployment method of CockroachDB as specified in the icon’s column: CockroachDB Self-hosted, CockroachDB Advanced, CockroachDB Standard, or CockroachDB Basic. | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ IAM authentication support for Amazon MSK Serverless ++ Changefeeds support IAM Authentication with Amazon MSK Serverless clusters (Amazon Managed Streaming for Apache Kafka). This feature is generally available. + + |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ Free trial on CockroachDB Cloud ++ New CockroachDB Cloud organizations can benefit from a 30-day free trial that enables you to consume up to $400 worth of free credits. Get started by signing up for CockroachDB Cloud + + |
+ All + ★ + | +{% include icon-no.html %} | +{% include icon-no.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ SELECT now supported on PCR standby clusters +
+ Physical cluster replication (PCR) has been enhanced to support |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +
|
+ Logical Data Replication in Preview ++ Logical data replication (LDR) continuously replicates tables between an active source CockroachDB cluster to an active destination CockroachDB cluster. Both source and destination can receive application reads and writes, and participate in bidirectional LDR replication for eventual consistency in the replicating tables. + ++ The active-active setup between clusters can provide protection against cluster, datacenter, or region failure while still achieving single-region low latency reads and writes in the individual CockroachDB clusters. Each cluster in an LDR job still benefits individually from multi-active availability with CockroachDB's built-in Raft replication providing data consistency across nodes, zones, and regions. + ++ This feature is in Preview. + + |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ Licensing changes ++ All versions of CockroachDB starting from the release date of 24.3.0 onward, including patch fixes for versions 23.1-24.2, are made available under the CockroachDB Software License. + ++ See below for a summary of license options for self-hosted deployments. All Cloud deployments automatically have a valid Enterprise license. + +
+ See the Licensing FAQs page for more details on the CockroachDB Software License and license options. + + You may acquire CockroachDB licenses through the CockroachDB Cloud console. + + |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ Improved usability for the DB Console Metrics page ++ Introduced several enhancements to the DB Console Metrics page to support large scale clusters, including the following: +
|
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +
|
+ Improved peformance and scalability for the DB Console Databases pages ++ CockroachDB now caches the data that is surfaced in the Databases page. This enhances the performance and scalability of the Databases page for large-scale clusters. + + |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +
|
+ Improved admission control observability ++ The DB Console Overload page now provides additional metrics to help identify overload in the system. Graphs and metrics on this page provide quick signals on which resource is exhausted and whether it is due to background activity or foreground. + ++ There are now 4 graphs for admission queue delay: + +
|
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ LDAP support in Preview ++ CockroachDB supports [authentication and authorization using LDAP-compatible directory services]({% link v24.3/ldap-authentication.md %}), such as Active Directory and Microsoft Entra ID. This allows you to integrate CockroachDB clusters with your organization's existing identity infrastructure for centralized user management and access control. This feature is available in Preview. + + |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +{% include icon-no.html %} | +
| Feature | +Availability | +||||
|---|---|---|---|---|---|
| Ver. | +Self-hosted | +Advanced | +Standard | +Basic | +|
|
+ User-defined functions and stored procedures support SECURITY DEFINER +
+ You can create or alter a user-defined function (UDF) or stored procedure with |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +
|
+ CockroachDB now supports triggers in Preview ++ CockroachDB now supports triggers. Triggers allow automatic execution of specified functions in response to specified events on a particular table or view. They can be used for automating tasks, enforcing business rules, and maintaining data integrity. This feature is in Preview. + + |
+ 24.3 | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +{% include icon-yes.html %} | +
Backward-incompatible changes
+ +- The old `BACKUP TO`, `RESTORE FROMSecurity updates
+ +- Added support for partial roles from LDAP synced group to be mapped to CockroachDB roles and ensure appropriate erroring for undesired behavior. [#135552][#135552] + +General changes
+ +- To improve the granularity of changefeed pipeline metrics, the changefeed metrics `changefeed.admit_latency` and `changefeed.commit_latency` now have histogram buckets from `5ms` to `60m` (previously `500ms` to `5m`). The following changefeed metrics now have histogram buckets from `5ms` to `10m` (previously `500ms` to `5m`): + - `changefeed.parallel_io_queue_nanos` + - `changefeed.parallel_io_result_queue_nanos` + - `changefeed.sink_batch_hist_nanos` + - `changefeed.flush_hist_nanos` + - `changefeed.kafka_throttling_hist_nanos` +[#136265][#136265] +- Added support for multiple seed brokers in the new Kafka sink. [#136632][#136632] +- Added the new metric `distsender.rangefeed.catchup_ranges_waiting_client_side` that counts how many rangefeeds are waiting on the client-side limiter to start performing catchup scans. [#136741][#136741] +- Added `system.users` to the list of system tables that changefeeds protect with protected timestamps. This table is required for change data capture queries. [#133568][#133568] +- Added the `AWS_USE_PATH_STYLE` parameter to S3 URI parsing. [#136715][#136715] + +SQL language changes
+ +- Two new builtin functions, `crdb_internal.{lease_holder, range_stats}_with_errors`, include errors encountered while fetching leaseholder and range stats. These new builtins are used by the `crdb_internal.ranges` table, which includes a new column, `errors`, that combines the errors from the builtins. [#131232][#131232] +- The cluster setting `sql.stats.automatic_partial_collection.enabled` is now enabled by default, which enables automatic collection of partial table stats. Partial table stats (i.e., those created with `CREATE STATISTICS ... USING EXTREMES`) scan the lower and upper ends of indexes to collect statistics outside the range covered by the previous full statistics collection. [#133988][#133988] +- When triggers fire one another cyclically, the new `recursion_depth_limit` session variable now limits the depth of the recursion. By default, the limit is `1000` nested trigger executions. [#134498][#134498] +- The names of `BEFORE` triggers fired by a mutation now show up in the `EXPLAIN` output. The trigger-function invocations are visible in the output of verbose `EXPLAIN`. [#135556][#135556] +- `AFTER` triggers will now show up in the output of `EXPLAIN`, as well as `EXPLAIN ANALYZE`. [#135556][#135556] +- Added support for `SHOW TRIGGERS`, which displays the names of all triggers on a table, as well as whether each trigger is enabled. The user must have any privilege on the table, or be its owner. [#135778][#135778] +- Added support for `SHOW CREATE TRIGGER`, which displays the `CREATE` statement for a trigger. The user must have any privilege on the table, or be its owner. [#135778][#135778] +- Added an informational notice to the result of `CREATE TABLE ... AS` statements that describes that indexes and constraints are not copied to the new table. [#135845][#135845] +- Altering a column’s type no longer requires enabling the `enable_experimental_alter_column_type_general` session variable. This change makes the feature generally available. [#135936][#135936] +- Added support for `COLLATE` expressions on arrays of strings to match PostgreSQL more closely. [#133751][#133751] +- Added the column `readable_high_water_timestamp` to the output of `SHOW CHANGEFEED JOBS`. This human-readable form will be easier to consume. `high_water_timestamp` still exists and is in epoch nanoseconds. [#135623][#135623] +- The `sql_safe_updates` session variable must be disabled to perform `ALTER COLUMN TYPE` operations that require a column rewrite. [#136110][#136110] +- Added the `CREATE LOGICALLY REPLICATED` syntax that will direct logical data replication jobs to create the destination table(s) using a copy of the source table(s). [#136841][#136841] +- It is now possible to execute queries with correlated joins with sub-queries or common table expressions in both the `INNER` and `OUTER` context. Errors with the following message: `unimplemented: apply joins with subqueries in the "inner" and "outer" contexts are not supported` will no longer occur. [#136506][#136506] +- It is now possible to include a common table expression within the body of a user-defined function or stored procedure. [#136506][#136506] + +Operational changes
+ +- Retired the cluster setting `kv.rangefeed.scheduler.enabled`. The rangefeed scheduler is now unconditionally enabled. [#132825][#132825] +- Added the cluster setting `ui.database_locality_metadata.enabled` that allows operators to disable loading extended database and table region information in the DB Console Database and Table pages. This information can cause significant CPU load on large clusters with many ranges. Versions of this page from v24.3 and later do not have this problem. If customers require this data, they can use the `SHOW RANGES FROM {DATABASE| TABLE}` query via SQL to compute on-demand. [#133075][#133075] +- The metrics scrape HTTP endpoint at `/ _status/vars` will now truncate `HELP` text at the first sentence, reducing the metadata for metrics with large descriptions. Descriptions are still accessible in the documentation. [#134724][#134724] +- The row-level TTL job will now periodically update the progress meter in the jobs introspection interfaces, including `SHOW JOBS` and the Jobs page in the DB console. [#135142][#135142] +- The `kv.bulk_io_write.min_capacity_remaining_fraction` cluster setting can be be set between `0.04` and `0.3`. [#135779][#135779] +- Added two new metrics, `sql.distsql.select.distributed_exec.count` and `sql.distsql.select.distributed_exec.count.internal`. These metrics count the number of `SELECT` statements that actually execute with full or partial distribution. These metrics differ from `sql.distsql.select.count` and `sql.distsql.select.count.internal` in that the latter count the number of `SELECT` statements that are **planned** with full or partial distribution, but might not necessarily execute with full or partial distribution, depending on the location of data. [#135236][#135236] +- Added the new metric `sql.distsql.distributed_exec.count` that counts the number of invocations of the execution engine with full or partial distribution. (This is in contrast to `sql.distsql.queries.total`, which counts the total number of invocations of the execution engine.) [#135236][#135236] +- Added some clarification that the following metrics count invocations of the execution engine and not SQL queries (which could each result in multiple invocations of the execution engine): + - `sql.distsql.queries.active` + - `sql.distsql.queries.total` + - `sql.distsql.distributed_exec.count` [#135236][#135236] +- The default value for the cluster setting `trace.span_registry.enabled` has been changed from `true` to `false`. [#135682][#135682] +- Removed the `sql.auth.resolve_membership_single_scan.enabled` cluster setting. This was added in case it was necessary to revert back to the previous behavior for looking up role memberships, but this cluster setting has not been needed in practice since this was added in v23.1. [#135852][#135852] +- Telemetry delivery is now considered successful even in cases where CockroachDB experiences a network timeout. This will prevent throttling in cases outside an operator's control. [#136219][#136219] +- When a schema change job is completed, rolls back, or encounters a failure, the time taken since the job began is now logged in a structured log in the `SQL_SCHEMA` log channel. [#136899][#136899] + +DB Console changes
+ +- As of v25.1 the legacy Database page, which was previously available through the Advanced Debug page, is no longer available. [#134005][#134005] +- When activating statement diagnostics in the DB Console, users now have the option to produce a redacted bundle as output. This bundle will omit sensitive data. [#134723][#134723] +- Fixed a list of UI bugs on the DB Console Overview and Node Overview pages. [#135293][#135293] +- Removed the link for the legacy table page on the Plan Details page. [#136311][#136311] +- Changed the table and index contents of the Hot Ranges page in DB console. [#134106][#134106] + +Bug fixes
+ +- Fixed a bug that prevented the `CREATE` statement for a routine from being shown in a statement bundle. This happened when the routine was created on a schema other than `public`. The bug has existed since v23.1. [#132147][#132147] +- Reduced the duration of partitions in the gossip network when a node crashes in order to eliminate false positives in the `ranges.unavailable` metric. [#133300][#133300] +- Fixed a bug that caused incorrect evaluation of some binary expressions involving `CHAR(N)` values and untyped string literals with trailing whitespace characters. For example, the expression `'f'::CHAR = 'f '` now correctly evaluates to `true`. [#133037][#133037] +- Fixed a race condition in Sarama when Kafka throttling is enabled. [#133563][#133563] +- Fixed a metrics bug in rangefeed restarts introduced in v23.2. [#133947][#133947] +- Fixed a bug that could result in incorrect metrics related to retryable rangefeed errors. [#133991][#133991] +- Fixed a bug that could cause `DELETE` triggers not to fire on cascading `DELETE`, and which could cause `INSERT` triggers to match incorrectly in the same scenario. [#134759][#134759] +- Non-`admin` users that run `DROP ROLE IF EXISTS` on a user that does not exist will no longer receive an error message. [#134850][#134850] +- Fixed a bug where CockroachDB would encounter an internal error when evaluating `FETCH ABSOLUTE 0` statements. The bug had been present since v22.1. [#134919][#134919] +- Fixed a bug that could cause incorrect query results when the optimizer planned a lookup join on an index containing a column of type `CHAR(N)`, `VARCHAR(N)`, `BIT(N)`, `VARBIT(N)`, or `DECIMAL(M, N)`, and the query held that column constant to a single value (e.g., with an equality filter). [#134891][#134891] +- Fixed a bug that prevented restoring cluster backups taken in a multi-region cluster that had configured the `system` database with a region configuration into a non-multi-region cluster. [#134604][#134604] +- Fixed an unhandled error that would occur if `DROP SCHEMA` was executed on the `public` schema of the `system` database, or on an internal schema like `pg_catalog` or `information_schema`. [#135149][#135149] +- `ALTER DATABASE` operations no longer hang when the operation modifies the zone config if an invalid zone config already exists. [#135168][#135168] +- CockroachDB now correctly evaluates `percentile_cont` and `percentile_disc` aggregates over `FLOAT4` values. [#135130][#135130] +- The schema changer's backfill process now includes a retry mechanism that reduces the batch size when memory issues occur. This improves the likelihood of operation success without requiring manual adjustment of the `bulko.index_backfill.batch_size` cluster setting. [#135563][#135563] +- `CREATE SCHEMA` now returns the correct error if a the schema name is missing. [#135910][#135910] +- Fixed an issue where corrupted table statistics could cause the `cockroach` process to crash. [#135944][#135944] +- Table statistics collection in CockroachDB could previously run into `no bytes in account to release` errors in some edge cases (when the SQL memory budget, configured via `--max-sql-memory` flag, was close to being exhausted). The bug has been present since v21.2 and is now fixed. [#135970][#135970] +- `security.certificate.*` metrics will now be updated if a node loads new certificates while running. [#135596][#135596] +- A table that is participating in logical data replication can no longer be dropped. Previously, this was allowed, which would cause all the replicated rows to end up in the dead-letter queue. [#136172][#136172] +- `ALTER COLUMN SET NOT NULL` was not enforced consistently when the table was created in the same transaction. [#136298][#136298] +- `CREATE` relation / type could leave dangling namespace entries if the schema was concurrently being dropped. [#136325][#136325] +- The `idle_in_session_timeout` session variable now excludes the time spent waiting for schema changer jobs to complete, preventing unintended session termination during schema change operations. [#136463][#136463] +- Fixed a bug that causes the optimizer to use stale table statistics after altering an `ENUM` type used in the table. [#136538][#136538] +- CockroachDB now better respects the `statement_timeout` limit on queries involving the top K sort and merge join operations. [#136631][#136631] +- Fixed a bug that would cause the `make_timestamp` and `make_timestamptz` builtin functions to incorrectly extract the `seconds` argument if the value was less than `1`. [#136804][#136804] +- Fixed possible index corruption caused by triggers that could occur when the following conditions were satisfied: + 1. A query calls a user-defined function or stored procedure, and also performs a mutation on a table. + 2. The user-defined function or storage procedure contains a statement that either fires an `AFTER` trigger, or fires a `CASCADE` that itself fires a trigger. + 3. The trigger modifies the same row as the outer statement. + 4. Either the outer or inner mutation is something other than an `INSERT` without an `ON CONFLICT` clause. [#136076][#136076] +- Fixed an issue where license enforcement was not consistently disabled for single-node clusters started with `cockroach start-single-node`, ensuring proper behavior on cluster restarts. [#136926][#136926] +- Fixed a bug that caused an incorrect filesystem to be logged as part of the store information. [#137024][#137024] + +Performance improvements
+ +- The `/_status/nodes_ui` API no longer returns unnecessary metrics in its response. This decreases the payload size of the API and improves the load time of various DB Console pages and components. [#135186][#135186] +- Performance for some PL/pgSQL loops is now significantly improved, by as much as 3–4 times. This is due to applying tail-call optimization in more cases to the recursive sub-routines that implement loops. [#135145][#135145] +- Improved the internal caching logic for role membership information. This reduces the latency impact of commands such as `DROP ROLE`, `CREATE ROLE`, and `GRANT role TO user`, which cause the role membership cache to be invalidated. [#135852][#135852] +- The session variable `plan_cache_mode` now defaults to `auto`, enabling generic query plans for some queries. [#135668][#135668] +- GRPC streams are now pooled across unary intra-cluster RPCs, allowing for reuse of gRPC resources to reduce the cost of remote key-value layer access. This pooling can be disabled using the `rpc.batch_stream_pool.enabled` cluster setting. [#136648][#136648] + +Multi-tenancy
+ +- The `nodes` endpoint should work for `shared` secondary tenants. Since nodes are common to all the tenants, this API endpoint behaves similarly to the system tenant's endpoint. [#131644][#131644] + +[#131232]: https://github.com/cockroachdb/cockroach/pull/131232 +[#131644]: https://github.com/cockroachdb/cockroach/pull/131644 +[#132147]: https://github.com/cockroachdb/cockroach/pull/132147 +[#132825]: https://github.com/cockroachdb/cockroach/pull/132825 +[#133037]: https://github.com/cockroachdb/cockroach/pull/133037 +[#133075]: https://github.com/cockroachdb/cockroach/pull/133075 +[#133080]: https://github.com/cockroachdb/cockroach/pull/133080 +[#133215]: https://github.com/cockroachdb/cockroach/pull/133215 +[#133300]: https://github.com/cockroachdb/cockroach/pull/133300 +[#133563]: https://github.com/cockroachdb/cockroach/pull/133563 +[#133568]: https://github.com/cockroachdb/cockroach/pull/133568 +[#133610]: https://github.com/cockroachdb/cockroach/pull/133610 +[#133751]: https://github.com/cockroachdb/cockroach/pull/133751 +[#133947]: https://github.com/cockroachdb/cockroach/pull/133947 +[#133988]: https://github.com/cockroachdb/cockroach/pull/133988 +[#133991]: https://github.com/cockroachdb/cockroach/pull/133991 +[#134005]: https://github.com/cockroachdb/cockroach/pull/134005 +[#134106]: https://github.com/cockroachdb/cockroach/pull/134106 +[#134498]: https://github.com/cockroachdb/cockroach/pull/134498 +[#134604]: https://github.com/cockroachdb/cockroach/pull/134604 +[#134723]: https://github.com/cockroachdb/cockroach/pull/134723 +[#134724]: https://github.com/cockroachdb/cockroach/pull/134724 +[#134759]: https://github.com/cockroachdb/cockroach/pull/134759 +[#134850]: https://github.com/cockroachdb/cockroach/pull/134850 +[#134891]: https://github.com/cockroachdb/cockroach/pull/134891 +[#134919]: https://github.com/cockroachdb/cockroach/pull/134919 +[#135130]: https://github.com/cockroachdb/cockroach/pull/135130 +[#135142]: https://github.com/cockroachdb/cockroach/pull/135142 +[#135145]: https://github.com/cockroachdb/cockroach/pull/135145 +[#135149]: https://github.com/cockroachdb/cockroach/pull/135149 +[#135168]: https://github.com/cockroachdb/cockroach/pull/135168 +[#135186]: https://github.com/cockroachdb/cockroach/pull/135186 +[#135236]: https://github.com/cockroachdb/cockroach/pull/135236 +[#135293]: https://github.com/cockroachdb/cockroach/pull/135293 +[#135552]: https://github.com/cockroachdb/cockroach/pull/135552 +[#135556]: https://github.com/cockroachdb/cockroach/pull/135556 +[#135563]: https://github.com/cockroachdb/cockroach/pull/135563 +[#135596]: https://github.com/cockroachdb/cockroach/pull/135596 +[#135623]: https://github.com/cockroachdb/cockroach/pull/135623 +[#135668]: https://github.com/cockroachdb/cockroach/pull/135668 +[#135682]: https://github.com/cockroachdb/cockroach/pull/135682 +[#135778]: https://github.com/cockroachdb/cockroach/pull/135778 +[#135779]: https://github.com/cockroachdb/cockroach/pull/135779 +[#135845]: https://github.com/cockroachdb/cockroach/pull/135845 +[#135852]: https://github.com/cockroachdb/cockroach/pull/135852 +[#135910]: https://github.com/cockroachdb/cockroach/pull/135910 +[#135936]: https://github.com/cockroachdb/cockroach/pull/135936 +[#135944]: https://github.com/cockroachdb/cockroach/pull/135944 +[#135970]: https://github.com/cockroachdb/cockroach/pull/135970 +[#136076]: https://github.com/cockroachdb/cockroach/pull/136076 +[#136110]: https://github.com/cockroachdb/cockroach/pull/136110 +[#136172]: https://github.com/cockroachdb/cockroach/pull/136172 +[#136219]: https://github.com/cockroachdb/cockroach/pull/136219 +[#136265]: https://github.com/cockroachdb/cockroach/pull/136265 +[#136298]: https://github.com/cockroachdb/cockroach/pull/136298 +[#136311]: https://github.com/cockroachdb/cockroach/pull/136311 +[#136325]: https://github.com/cockroachdb/cockroach/pull/136325 +[#136463]: https://github.com/cockroachdb/cockroach/pull/136463 +[#136506]: https://github.com/cockroachdb/cockroach/pull/136506 +[#136538]: https://github.com/cockroachdb/cockroach/pull/136538 +[#136631]: https://github.com/cockroachdb/cockroach/pull/136631 +[#136632]: https://github.com/cockroachdb/cockroach/pull/136632 +[#136648]: https://github.com/cockroachdb/cockroach/pull/136648 +[#136715]: https://github.com/cockroachdb/cockroach/pull/136715 +[#136741]: https://github.com/cockroachdb/cockroach/pull/136741 +[#136804]: https://github.com/cockroachdb/cockroach/pull/136804 +[#136841]: https://github.com/cockroachdb/cockroach/pull/136841 +[#136899]: https://github.com/cockroachdb/cockroach/pull/136899 +[#136926]: https://github.com/cockroachdb/cockroach/pull/136926 +[#137024]: https://github.com/cockroachdb/cockroach/pull/137024 \ No newline at end of file diff --git a/src/current/_includes/sidebar-data-v25.1.json b/src/current/_includes/sidebar-data-v25.1.json new file mode 100644 index 00000000000..4fe4209a469 --- /dev/null +++ b/src/current/_includes/sidebar-data-v25.1.json @@ -0,0 +1,28 @@ +[ + { + "title": "Docs Home", + "is_top_level": true, + "urls": [ + "/" + ] + }, + {% include_cached v25.1/sidebar-data/get-started.json %}, + {% include_cached v25.1/sidebar-data/releases.json %}, + {% include_cached v25.1/sidebar-data/feature-overview.json %}, + {% include_cached v25.1/sidebar-data/resilience.json %}, + {% include_cached v25.1/sidebar-data/connect-to-cockroachdb.json %}, + {% include_cached v25.1/sidebar-data/migrate.json %}, + {% include_cached v25.1/sidebar-data/cloud-deployments.json %}, + {% include_cached v25.1/sidebar-data/self-hosted-deployments.json %}, + {% include_cached v25.1/sidebar-data/schema-design.json %}, + {% include_cached v25.1/sidebar-data/reads-and-writes.json %}, + {% include_cached v25.1/sidebar-data/stream-data.json %}, + {% include_cached v25.1/sidebar-data/cross-cluster-replication.json %}, + {% include_cached v25.1/sidebar-data/multi-region-capabilities.json %}, + {% include_cached v25.1/sidebar-data/optimize-performance.json %}, + {% include_cached v25.1/sidebar-data/troubleshooting.json %}, + {% include_cached v25.1/sidebar-data/sql.json %}, + {% include_cached v25.1/sidebar-data/reference.json %}, + {% include_cached v25.1/sidebar-data/faqs.json %}, + {% include_cached sidebar-data-cockroach-university.json %} +] diff --git a/src/current/_includes/v23.1/cdc/lagging-ranges.md b/src/current/_includes/v23.1/cdc/lagging-ranges.md index 339a5571443..e0061df7d4b 100644 --- a/src/current/_includes/v23.1/cdc/lagging-ranges.md +++ b/src/current/_includes/v23.1/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -{% include_cached new-in.html version="v23.1.12" %} Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}): +{% include_cached new-in.html version="v23.1.12" %} Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) that are behind in a changefeed. This is calculated based on the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}): - `changefeed.lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `changefeed.lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v23.1.29" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `changefeed.lagging_ranges_polling_interval` cluster setting. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v23.1/storage/free-up-disk-space.md b/src/current/_includes/v23.1/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v23.1/storage/free-up-disk-space.md +++ b/src/current/_includes/v23.1/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v23.2/cdc/lagging-ranges.md b/src/current/_includes/v23.2/cdc/lagging-ranges.md index b784a93cbfb..35d269bc706 100644 --- a/src/current/_includes/v23.2/cdc/lagging-ranges.md +++ b/src/current/_includes/v23.2/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -{% include_cached new-in.html version="v23.2" %} Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +{% include_cached new-in.html version="v23.2" %} Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v23.2.13" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md b/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md index 0a9fc624cf2..7d0d79a79b5 100644 --- a/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v23.2/known-limitations/physical-cluster-replication.md @@ -1,6 +1,6 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new v23.2 clusters. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. - Read queries are not supported on the standby cluster before [cutover]({% link {{ page.version.version }}/cutover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Cutting back to the primary cluster after a cutover is a manual process. Refer to [Cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster). In addition, after cutover, to continue using physical cluster replication, you must configure it again. - Before cutover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - After a cutover, there is no mechanism to stop applications from connecting to the original primary cluster. It is necessary to redirect application traffic manually, such as by using a network load balancer or adjusting DNS records. diff --git a/src/current/_includes/v23.2/storage/free-up-disk-space.md b/src/current/_includes/v23.2/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v23.2/storage/free-up-disk-space.md +++ b/src/current/_includes/v23.2/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v23.2/ui/databases.md b/src/current/_includes/v23.2/ui/databases.md index 94ae7fc0585..1b8ec510206 100644 --- a/src/current/_includes/v23.2/ui/databases.md +++ b/src/current/_includes/v23.2/ui/databases.md @@ -15,7 +15,7 @@ The following information is displayed for each database: {% endif -%} | Tables | The number of tables in the database. | {% if page.cloud != true -%} -| Regions/Nodes | The regions and nodes on which the tables in the database are located. This is not displayed on a single-node cluster. | +| Regions/Nodes | The regions and nodes on which the tables in the database are located. This is not displayed on a single-node cluster.On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). | | Index Recommendations | The number of index recommendations for the database. | {%- else -%} | Regions | The regions where the tables in the database are located. | @@ -26,6 +26,11 @@ Click a **database name** to open the **Tables** page. - Select **View: Tables** in the pulldown menu to display the [Tables view](#tables-view). - Select **View: Grants** in the pulldown menu to display the [Grants view](#grants-view). +{% if page.cloud != true -%} +### `ui.database_locality_metadata.enabled` cluster setting +{% include_cached new-in.html version="v23.2.17" %} Retrieving extended database and table region information can cause significant CPU load on large multi-node clusters with many ranges. You can prevent the retrieval of this data and the associated CPU load by disabling the [`ui.database_locality_metadata.enabled` cluster setting]({{ link_prefix }}cluster-settings.html#setting-ui-database-locality-metadata-enabled). When set to `false`, “No data” will be displayed for region data and replica counts. If you require this data, use the SQL statement [`SHOW RANGES FROM {DATABASE|TABLE}`]({{ link_prefix }}show-ranges.html) to compute this information. +{% endif -%} + ## Search and filter By default, the Databases page shows all databases running on the cluster. By default, the [**Tables** view](#tables-view) and the [**Grants** view](#grants-view) show all tables in a selected database. @@ -63,7 +68,7 @@ The following information is displayed for each table: | Columns | The number of columns in the table. | | Indexes | The number of indexes in the table. | {% if page.cloud != true -%} -| Regions | The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. | +| Regions | The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster.
On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). | {% else -%} | Regions | The regions where the table data is stored. {% endif -%} @@ -84,14 +89,14 @@ The table details include: {% if page.cloud != true %} - **Size**: The approximate disk size of all replicas of this table on the cluster. -- **Replicas**: The number of [replicas]({{ link_prefix }}architecture/replication-layer.html) of this table on the cluster. +- **Replicas**: The number of [replicas]({{ link_prefix }}architecture/replication-layer.html) of this table on the cluster. On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). - **Ranges**: The number of [ranges]({{ link_prefix }}architecture/glossary.html#architecture-range) in this table. - **% of Live Data**: Percentage of total uncompressed logical data that has not been modified (updated or deleted). - **Table Stats Last Updated**: The last time table statistics were created or updated. {% endif %} - **Auto Stats Collection**: Whether [automatic statistics collection]({{ link_prefix }}cost-based-optimizer.html#table-statistics) is enabled. {% if page.cloud != true %} -- **Regions/Nodes**: The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. +- **Regions/Nodes**: The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). {% else %} - **Regions**: The regions where the table data is stored. {% endif %} diff --git a/src/current/_includes/v24.1/cdc/lagging-ranges.md b/src/current/_includes/v24.1/cdc/lagging-ranges.md index 8d0b5eb6c23..17a7e035916 100644 --- a/src/current/_includes/v24.1/cdc/lagging-ranges.md +++ b/src/current/_includes/v24.1/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v24.1.6" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md b/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md index f914c7eced2..a6c7edf2f32 100644 --- a/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v24.1/known-limitations/physical-cluster-replication.md @@ -1,6 +1,6 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new clusters on v23.2 or above. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. - Read queries are not supported on the standby cluster before [cutover]({% link {{ page.version.version }}/cutover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Cutting back to the primary cluster after a cutover is a manual process. Refer to [Cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster). In addition, after cutover, to continue using physical cluster replication, you must configure it again. - Before cutover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - Large data imports, such as those produced by [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) or [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}), may dramatically increase [replication lag]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}#cutover-and-promotion-process). diff --git a/src/current/_includes/v24.1/storage/free-up-disk-space.md b/src/current/_includes/v24.1/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v24.1/storage/free-up-disk-space.md +++ b/src/current/_includes/v24.1/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v24.1/ui/databases.md b/src/current/_includes/v24.1/ui/databases.md index 94ae7fc0585..529b2b11efb 100644 --- a/src/current/_includes/v24.1/ui/databases.md +++ b/src/current/_includes/v24.1/ui/databases.md @@ -15,7 +15,7 @@ The following information is displayed for each database: {% endif -%} | Tables | The number of tables in the database. | {% if page.cloud != true -%} -| Regions/Nodes | The regions and nodes on which the tables in the database are located. This is not displayed on a single-node cluster. | +| Regions/Nodes | The regions and nodes on which the tables in the database are located. This is not displayed on a single-node cluster.
On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). | | Index Recommendations | The number of index recommendations for the database. | {%- else -%} | Regions | The regions where the tables in the database are located. | @@ -26,6 +26,11 @@ Click a **database name** to open the **Tables** page. - Select **View: Tables** in the pulldown menu to display the [Tables view](#tables-view). - Select **View: Grants** in the pulldown menu to display the [Grants view](#grants-view). +{% if page.cloud != true -%} +### `ui.database_locality_metadata.enabled` cluster setting +{% include_cached new-in.html version="v24.1.8" %} Retrieving extended database and table region information can cause significant CPU load on large multi-node clusters with many ranges. You can prevent the retrieval of this data and the associated CPU load by disabling the [`ui.database_locality_metadata.enabled` cluster setting]({{ link_prefix }}cluster-settings.html#setting-ui-database-locality-metadata-enabled). When set to `false`, “No data” will be displayed for region data and replica counts. If you require this data, use the SQL statement [`SHOW RANGES FROM {DATABASE|TABLE}`]({{ link_prefix }}show-ranges.html) to compute this information. +{% endif -%} + ## Search and filter By default, the Databases page shows all databases running on the cluster. By default, the [**Tables** view](#tables-view) and the [**Grants** view](#grants-view) show all tables in a selected database. @@ -63,7 +68,7 @@ The following information is displayed for each table: | Columns | The number of columns in the table. | | Indexes | The number of indexes in the table. | {% if page.cloud != true -%} -| Regions | The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. | +| Regions | The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster.
On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). | {% else -%} | Regions | The regions where the table data is stored. {% endif -%} @@ -84,14 +89,14 @@ The table details include: {% if page.cloud != true %} - **Size**: The approximate disk size of all replicas of this table on the cluster. -- **Replicas**: The number of [replicas]({{ link_prefix }}architecture/replication-layer.html) of this table on the cluster. +- **Replicas**: The number of [replicas]({{ link_prefix }}architecture/replication-layer.html) of this table on the cluster. On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). - **Ranges**: The number of [ranges]({{ link_prefix }}architecture/glossary.html#architecture-range) in this table. - **% of Live Data**: Percentage of total uncompressed logical data that has not been modified (updated or deleted). - **Table Stats Last Updated**: The last time table statistics were created or updated. {% endif %} - **Auto Stats Collection**: Whether [automatic statistics collection]({{ link_prefix }}cost-based-optimizer.html#table-statistics) is enabled. {% if page.cloud != true %} -- **Regions/Nodes**: The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. +- **Regions/Nodes**: The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). {% else %} - **Regions**: The regions where the table data is stored. {% endif %} diff --git a/src/current/_includes/v24.2/cdc/lagging-ranges.md b/src/current/_includes/v24.2/cdc/lagging-ranges.md index 8d0b5eb6c23..eb22275849a 100644 --- a/src/current/_includes/v24.2/cdc/lagging-ranges.md +++ b/src/current/_includes/v24.2/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v24.2.4" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md b/src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md deleted file mode 100644 index 31fbf83187c..00000000000 --- a/src/current/_includes/v24.2/known-limitations/pcr-scheduled-changefeeds.md +++ /dev/null @@ -1 +0,0 @@ -After the [cutover process]({% link {{ page.version.version }}/cutover-replication.md %}) for [physical cluster replication]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}), [scheduled changefeeds]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) will continue on the promoted cluster. You will need to manage [pausing]({% link {{ page.version.version }}/pause-schedules.md %}) or [canceling]({% link {{ page.version.version }}/drop-schedules.md %}) the schedule on the promoted standby cluster to avoid two clusters running the same changefeed to one sink. [#123776](https://github.com/cockroachdb/cockroach/issues/123776) \ No newline at end of file diff --git a/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md b/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md index f914c7eced2..a6c7edf2f32 100644 --- a/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md +++ b/src/current/_includes/v24.2/known-limitations/physical-cluster-replication.md @@ -1,6 +1,6 @@ - Physical cluster replication is supported only on CockroachDB {{ site.data.products.core }} in new clusters on v23.2 or above. Physical Cluster Replication cannot be enabled on clusters that have been upgraded from a previous version of CockroachDB. - Read queries are not supported on the standby cluster before [cutover]({% link {{ page.version.version }}/cutover-replication.md %}). -- The primary and standby cluster **cannot have different [region topology]({% link {{ page.version.version }}/topology-patterns.md %})**. For example, replicating a multi-region primary cluster to a single-region standby cluster is not supported. Mismatching regions between a multi-region primary and standby cluster is also not supported. +- The primary and standby clusters must have the same [zone configurations]({% link {{ page.version.version }}/configure-replication-zones.md %}). - Cutting back to the primary cluster after a cutover is a manual process. Refer to [Cut back to the primary cluster]({% link {{ page.version.version }}/cutover-replication.md %}#cut-back-to-the-primary-cluster). In addition, after cutover, to continue using physical cluster replication, you must configure it again. - Before cutover to the standby, the standby cluster does not support running [backups]({% link {{ page.version.version }}/backup-and-restore-overview.md %}) or [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}). - Large data imports, such as those produced by [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) or [`IMPORT INTO`]({% link {{ page.version.version }}/import-into.md %}), may dramatically increase [replication lag]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}#cutover-and-promotion-process). diff --git a/src/current/_includes/v24.2/storage/free-up-disk-space.md b/src/current/_includes/v24.2/storage/free-up-disk-space.md index c63b70b766e..e4a6b08a57a 100644 --- a/src/current/_includes/v24.2/storage/free-up-disk-space.md +++ b/src/current/_includes/v24.2/storage/free-up-disk-space.md @@ -1 +1 @@ -For instructions on how to free up disk space as quickly as possible after deleting data, see [How can I free up disk space quickly?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-quickly) +For instructions on how to free up disk space as quickly as possible after dropping a table, see [How can I free up disk space that was used by a dropped table?]({% link {{ page.version.version }}/operational-faqs.md %}#how-can-i-free-up-disk-space-when-dropping-a-table) diff --git a/src/current/_includes/v24.2/ui/databases.md b/src/current/_includes/v24.2/ui/databases.md index 94ae7fc0585..0465cb16860 100644 --- a/src/current/_includes/v24.2/ui/databases.md +++ b/src/current/_includes/v24.2/ui/databases.md @@ -15,7 +15,7 @@ The following information is displayed for each database: {% endif -%} | Tables | The number of tables in the database. | {% if page.cloud != true -%} -| Regions/Nodes | The regions and nodes on which the tables in the database are located. This is not displayed on a single-node cluster. | +| Regions/Nodes | The regions and nodes on which the tables in the database are located. This is not displayed on a single-node cluster.
On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). | | Index Recommendations | The number of index recommendations for the database. | {%- else -%} | Regions | The regions where the tables in the database are located. | @@ -26,6 +26,11 @@ Click a **database name** to open the **Tables** page. - Select **View: Tables** in the pulldown menu to display the [Tables view](#tables-view). - Select **View: Grants** in the pulldown menu to display the [Grants view](#grants-view). +{% if page.cloud != true -%} +### `ui.database_locality_metadata.enabled` cluster setting +{% include_cached new-in.html version="v24.2.6" %} Retrieving extended database and table region information can cause significant CPU load on large multi-node clusters with many ranges. You can prevent the retrieval of this data and the associated CPU load by disabling the [`ui.database_locality_metadata.enabled` cluster setting]({{ link_prefix }}cluster-settings.html#setting-ui-database-locality-metadata-enabled). When set to `false`, “No data” will be displayed for region data and replica counts. If you require this data, use the SQL statement [`SHOW RANGES FROM {DATABASE|TABLE}`]({{ link_prefix }}show-ranges.html) to compute this information. +{% endif -%} + ## Search and filter By default, the Databases page shows all databases running on the cluster. By default, the [**Tables** view](#tables-view) and the [**Grants** view](#grants-view) show all tables in a selected database. @@ -63,7 +68,7 @@ The following information is displayed for each table: | Columns | The number of columns in the table. | | Indexes | The number of indexes in the table. | {% if page.cloud != true -%} -| Regions | The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. | +| Regions | The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster.
On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). | {% else -%} | Regions | The regions where the table data is stored. {% endif -%} @@ -84,14 +89,14 @@ The table details include: {% if page.cloud != true %} - **Size**: The approximate disk size of all replicas of this table on the cluster. -- **Replicas**: The number of [replicas]({{ link_prefix }}architecture/replication-layer.html) of this table on the cluster. +- **Replicas**: The number of [replicas]({{ link_prefix }}architecture/replication-layer.html) of this table on the cluster. On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). - **Ranges**: The number of [ranges]({{ link_prefix }}architecture/glossary.html#architecture-range) in this table. - **% of Live Data**: Percentage of total uncompressed logical data that has not been modified (updated or deleted). - **Table Stats Last Updated**: The last time table statistics were created or updated. {% endif %} - **Auto Stats Collection**: Whether [automatic statistics collection]({{ link_prefix }}cost-based-optimizer.html#table-statistics) is enabled. {% if page.cloud != true %} -- **Regions/Nodes**: The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. +- **Regions/Nodes**: The regions and nodes on which the table data is stored. This is not displayed on a single-node cluster. On a multi-node cluster, the display of this information is controlled by the cluster setting [`ui.database_locality_metadata.enabled`](#ui-database_locality_metadata-enabled-cluster-setting) (default `true`). {% else %} - **Regions**: The regions where the table data is stored. {% endif %} diff --git a/src/current/_includes/v24.3/backups/backup-to-deprec.md b/src/current/_includes/v24.3/backups/backup-to-deprec.md deleted file mode 100644 index 77a2b6cae18..00000000000 --- a/src/current/_includes/v24.3/backups/backup-to-deprec.md +++ /dev/null @@ -1,7 +0,0 @@ -{{site.data.alerts.callout_danger}} -The `BACKUP ... TO` and `RESTORE ... FROM` syntax is **deprecated** as of v22.1 and will be removed in a future release. - -We recommend using the `BACKUP ... INTO {collectionURI}` syntax, which creates or adds to a [backup collection]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) in your storage location. For restoring backups, we recommend using `RESTORE FROM {backup} IN {collectionURI}` with `{backup}` being [`LATEST`]({% link {{ page.version.version }}/restore.md %}#restore-the-most-recent-full-or-incremental-backup) or a specific [subdirectory]({% link {{ page.version.version }}/restore.md %}#subdir-param). - -For guidance on the syntax for backups and restores, see the [`BACKUP`]({% link {{ page.version.version }}/backup.md %}#examples) and [`RESTORE`]({% link {{ page.version.version }}/restore.md %}#examples) examples. -{{site.data.alerts.end}} diff --git a/src/current/_includes/v24.3/backups/old-syntax-removed.md b/src/current/_includes/v24.3/backups/old-syntax-removed.md new file mode 100644 index 00000000000..7052fe0d3af --- /dev/null +++ b/src/current/_includes/v24.3/backups/old-syntax-removed.md @@ -0,0 +1,5 @@ +{{site.data.alerts.callout_danger}} +The `BACKUP ... TO` and `RESTORE ... FROM {storage_uri}` syntax has been removed from CockroachDB v24.3 and later. + +For details on the syntax to run `BACKUP` and `RESTORE`, refer to the {% if page.name == "backup.md" %} [backup](#examples) {% else %} [backup]({% link {{ page.version.version }}/backup.md %}#examples) {% endif %} and {% if page.name == "restore.md" %} [restore](#examples) {% else %} [restore]({% link {{ page.version.version }}/restore.md %}#examples) {% endif %} examples. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md b/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md index 375f4489914..2ef075abf9c 100644 --- a/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md +++ b/src/current/_includes/v24.3/backups/recommend-backups-for-upgrade.md @@ -3,7 +3,4 @@ When upgrading to a major release, you can optionally [take a self-managed backup]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) of your cluster to your own cloud storage, as an extra layer of protection in case the upgrade leads to issues. {% else %} -CockroachDB is designed with high fault tolerance. However, taking regular backups of your data is an operational best practice for [disaster recovery]({% link {{ page.version.version }}/disaster-recovery-planning.md %}) planning. - -We recommend that you enable [managed backups]({% link cockroachcloud/managed-backups.md %}#managed-backup-settings) and confirm that the cluster is backed up before beginning a major-version upgrade. This provides an extra layer of protection in case the upgrade leads to issues. -{% endif %} +CockroachDB is designed with high fault tolerance. However, taking regular backups of your data is an operational best practice for [disaster recovery]({% link {{ page.version.version }}/disaster-recovery-planning.md %}) planning.{% endif %} diff --git a/src/current/_includes/v24.3/cdc/lagging-ranges.md b/src/current/_includes/v24.3/cdc/lagging-ranges.md index 8d0b5eb6c23..45180baa57f 100644 --- a/src/current/_includes/v24.3/cdc/lagging-ranges.md +++ b/src/current/_includes/v24.3/cdc/lagging-ranges.md @@ -1,10 +1,12 @@ -Use the `changefeed.lagging_ranges` metric to track the number of ranges that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): +Use the `changefeed.lagging_ranges` metric to track the number of [ranges]({% link {{ page.version.version }}/architecture/overview.md %}#range) that are behind in a changefeed. This is calculated based on the [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options): - `lagging_ranges_threshold` sets a duration from the present that determines the length of time a range is considered to be lagging behind, which will then track in the [`lagging_ranges`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#lagging-ranges-metric) metric. Note that ranges undergoing an [initial scan]({% link {{ page.version.version }}/create-changefeed.md %}#initial-scan) for longer than the threshold duration are considered to be lagging. Starting a changefeed with an initial scan on a large table will likely increment the metric for each range in the table. As ranges complete the initial scan, the number of ranges lagging behind will decrease. - **Default:** `3m` - `lagging_ranges_polling_interval` sets the interval rate for when lagging ranges are checked and the `lagging_ranges` metric is updated. Polling adds latency to the `lagging_ranges` metric being updated. For example, if a range falls behind by 3 minutes, the metric may not update until an additional minute afterward. - **Default:** `1m` +{% include_cached new-in.html version="v24.3" %} Use the `changefeed.total_ranges` metric to monitor the number of ranges that are watched by [aggregator processors]({% link {{ page.version.version }}/how-does-an-enterprise-changefeed-work.md %}) participating in the changefeed job. If you're experiencing lagging ranges, `changefeed.total_ranges` may indicate that the number of ranges watched by aggregator processors in the job is unbalanced. You may want to try [pausing]({% link {{ page.version.version }}/pause-job.md %}) the changefeed and then [resuming]({% link {{ page.version.version }}/resume-job.md %}) it, so that the changefeed replans the work in the cluster. `changefeed.total_ranges` shares the same polling interval as the `changefeed.lagging_ranges` metric, which is controlled by the `lagging_ranges_polling_interval` option. + {{site.data.alerts.callout_success}} -You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` metric per changefeed. +You can use the [`metrics_label`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#using-changefeed-metrics-labels) option to track the `lagging_ranges` and `total_ranges` metric per changefeed. {{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.3/child-metrics-table.md b/src/current/_includes/v24.3/child-metrics-table.md index 05d57c55453..2b1a16f092a 100644 --- a/src/current/_includes/v24.3/child-metrics-table.md +++ b/src/current/_includes/v24.3/child-metrics-table.md @@ -7,7 +7,7 @@ Following is a list of the metrics that have child metrics:
| CockroachDB Metric Name | +Description | +Type | +Unit | +
{{ m.metric_id }} |
+ {% comment %} Use the value from the metrics-list, if any, followed by the value in the available-metrics-not-in-metrics-list, if any. {% endcomment %}
+ {{ metrics-list[0].description }}{{ m.description }} | +{{ metrics-list[0].type }}{{ m.type }} | +{{ metrics-list[0].unit }}{{ m.unit }} | +
+
+
+
+
+
+
+
+
+
+You can specify a backup with revision history without any value e.g., `WITH revision_history`. Or, you can explicitly define `WITH revision_history = 'true' / 'false'`. The `revision_history` option defaults to `true` when used with [`BACKUP`]({% link {{ page.version.version }}/backup.md %}) or `CREATE SCHEDULE FOR BACKUP`. A value is **required** when using `ALTER BACKUP SCHEDULE` to {% if page.name == "alter-backup-schedule.md" %} [apply different options to scheduled backups](#apply-different-options-to-scheduled-backups). {% else %} [alter a backup schedule](alter-backup-schedule.html). {% endif %} +`encryption_passphrase` | [`STRING`]({% link {{ page.version.version }}/string.md %}) | The passphrase used to [encrypt the files]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}) (`BACKUP` manifest and data files) that the `BACKUP` statement generates. This same passphrase is needed to decrypt the file when it is used to [restore]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}) and to list the contents of the backup when using [`SHOW BACKUP`]({% link {{ page.version.version }}/show-backup.md %}). There is no practical limit on the length of the passphrase. +`detached` | [`BOOL`]({% link {{ page.version.version }}/bool.md %}) / None | **Note:** Backups running on a schedule have the `detached` option applied implicitly. Therefore, you cannot modify this option for scheduled backups.
When a backup runs in `detached` mode, it will execute asynchronously. The job ID will be returned after the backup [job creation]({% link {{ page.version.version }}/backup-architecture.md %}#job-creation-phase) completes. Note that with `detached` specified, further job information and the job completion status will not be returned. For more on the differences between the returned job data, see the [example]({% link {{ page.version.version }}/backup.md %}#run-a-backup-asynchronously). To check on the job status, use the [`SHOW JOBS`](show-jobs.html) statement. +`EXECUTION LOCALITY` | Key-value pairs | Restricts the execution of the backup to nodes that match the defined locality filter requirements. For example, `WITH EXECUTION LOCALITY = 'region=us-west-1a,cloud=aws'`.
Refer to [Take Locality-restricted backups]({% link {{ page.version.version }}/take-locality-restricted-backups.md %}) for usage and reference detail. +`kms` | [`STRING`]({% link {{ page.version.version }}/string.md %}) | The URI of the cryptographic key stored in a key management service (KMS), or a comma-separated list of key URIs, used to [take and restore encrypted backups]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#examples). Refer to [URI Formats]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#uri-formats). The key or keys are used to encrypt the manifest and data files that the `BACKUP` statement generates and to decrypt them during a [restore]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#examples) operation, and to list the contents of the backup when using [`SHOW BACKUP`]({% link {{ page.version.version }}/show-backup.md %}).
AWS KMS, Google Cloud KMS, and Azure Key Vault are supported. +`incremental_location` | [`STRING`]({% link {{ page.version.version }}/string.md %}) | Create an incremental backup in a different location than the default incremental backup location.
`WITH incremental_location = 'explicit_incrementals_URI'`
See [Incremental backups with explicitly specified destinations]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#incremental-backups-with-explicitly-specified-destinations) for usage. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/backup-options.md b/src/current/_includes/v25.1/backups/backup-options.md new file mode 100644 index 00000000000..a4c3eeae595 --- /dev/null +++ b/src/current/_includes/v25.1/backups/backup-options.md @@ -0,0 +1,8 @@ + Option | Value | Description +-----------------------------------------------------------------+-------------------------+------------------------------ +`revision_history` | [`BOOL`]({% link {{ page.version.version }}/bool.md %}) / None | Create a backup with full [revision history]({% link {{ page.version.version }}/take-backups-with-revision-history-and-restore-from-a-point-in-time.md %}), which records every change made to the cluster within the garbage collection period leading up to and including the given timestamp.
You can specify a backup with revision history without any value e.g., `WITH revision_history`. Or, you can explicitly define `WITH revision_history = 'true' / 'false'`. `revision_history` defaults to `true` when used with `BACKUP` or `CREATE SCHEDULE FOR BACKUP`. A value is **required** when using [`ALTER BACKUP SCHEDULE`]({% link {{ page.version.version }}/alter-backup-schedule.md %}). +`encryption_passphrase` | [`STRING`]({% link {{ page.version.version }}/string.md %}) | The passphrase used to [encrypt the files]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}) (`BACKUP` manifest and data files) that the `BACKUP` statement generates. This same passphrase is needed to decrypt the file when it is used to [restore]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}) and to list the contents of the backup when using [`SHOW BACKUP`]({% link {{ page.version.version }}/show-backup.md %}). There is no practical limit on the length of the passphrase. +`detached` | [`BOOL`]({% link {{ page.version.version }}/bool.md %}) / None | When a backup runs in `detached` mode, it will execute asynchronously. The job ID will be returned after the backup [job creation]({% link {{ page.version.version }}/backup-architecture.md %}#job-creation-phase) completes. Note that with `detached` specified, further job information and the job completion status will not be returned. For more on the differences between the returned job data, see the [example]({% link {{ page.version.version }}/backup.md %}#run-a-backup-asynchronously). To check on the job status, use the [`SHOW JOBS`](show-jobs.html) statement. Backups running on a [schedule](create-schedule-for-backup.html) have the `detached` option applied implicitly.
To run a backup within a [transaction](transactions.html), use the `detached` option. +`EXECUTION LOCALITY` | Key-value pairs | Restricts the execution of the backup to nodes that match the defined locality filter requirements. For example, `WITH EXECUTION LOCALITY = 'region=us-west-1a,cloud=aws'`.
Refer to [Take Locality-restricted backups]({% link {{ page.version.version }}/take-locality-restricted-backups.md %}) for usage and reference detail. +`kms` | [`STRING`]({% link {{ page.version.version }}/string.md %}) | The URI of the cryptographic key stored in a key management service (KMS), or a comma-separated list of key URIs, used to [take and restore encrypted backups]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#examples). Refer to [URI Formats]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#uri-formats). The key or keys are used to encrypt the manifest and data files that the `BACKUP` statement generates and to decrypt them during a [restore]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#examples) operation, and to list the contents of the backup when using [`SHOW BACKUP`]({% link {{ page.version.version }}/show-backup.md %}).
AWS KMS, Google Cloud KMS, and Azure Key Vault are supported. +`incremental_location` | [`STRING`]({% link {{ page.version.version }}/string.md %}) | Create an incremental backup in a different location than the default incremental backup location.
`WITH incremental_location = 'explicit_incrementals_URI'`
See [Incremental backups with explicitly specified destinations]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#incremental-backups-with-explicitly-specified-destinations) for usage. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/backup-storage-collision.md b/src/current/_includes/v25.1/backups/backup-storage-collision.md new file mode 100644 index 00000000000..c52cc1524e5 --- /dev/null +++ b/src/current/_includes/v25.1/backups/backup-storage-collision.md @@ -0,0 +1 @@ +You will encounter an error if you run multiple [backup collections]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#backup-collections) to the same storage URI. Each collection's URI must be unique. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/bulk-auth-options.md b/src/current/_includes/v25.1/backups/bulk-auth-options.md new file mode 100644 index 00000000000..57bc67ea190 --- /dev/null +++ b/src/current/_includes/v25.1/backups/bulk-auth-options.md @@ -0,0 +1,6 @@ +The examples in this section use one of the following storage URIs: + +- External connections, which allow you to represent an external storage or sink URI. You can then specify the external connection's name in statements rather than the provider-specific URI. For detail on using external connections, see the [`CREATE EXTERNAL CONNECTION`]({% link {{ page.version.version }}/create-external-connection.md %}) page. +- Amazon S3 connection strings with the **default** `AUTH=specified` parameter. For guidance on using `AUTH=implicit` authentication with Amazon S3 buckets instead, read [Cloud Storage Authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}). + +For guidance on connecting to other storage options or using other authentication parameters instead, read [Use Cloud Storage]({% link {{ page.version.version }}/use-cloud-storage.md %}#example-file-urls). \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/cap-parameter-ext-connection.md b/src/current/_includes/v25.1/backups/cap-parameter-ext-connection.md new file mode 100644 index 00000000000..2628b8527a1 --- /dev/null +++ b/src/current/_includes/v25.1/backups/cap-parameter-ext-connection.md @@ -0,0 +1 @@ +If you are creating an {% if page.name == "create-external-connection.md" %}external connection{% else %}[external connection]({% link {{ page.version.version }}/create-external-connection.md %}){% endif %} with [`BACKUP` query parameters]({% link {{ page.version.version }}/backup.md %}#query-parameters) or [authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}) parameters, you must pass them in uppercase otherwise you will receive an `unknown query parameters` error. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/check-files-validate.md b/src/current/_includes/v25.1/backups/check-files-validate.md new file mode 100644 index 00000000000..b54cf5ce9a6 --- /dev/null +++ b/src/current/_includes/v25.1/backups/check-files-validate.md @@ -0,0 +1,32 @@ +1. Use `SHOW BACKUP ... check_files` with a backup for validation: + + {% include_cached copy-clipboard.html %} + ~~~sql + SHOW BACKUP "2022/09/19-134123.64" IN "s3://bucket?AWS_ACCESS_KEY_ID={Access Key ID}&AWS_SECRET_ACCESS_KEY={Secret Access Key}" WITH check_files; + ~~~ + + This will return the following output after validating that the backup files are correct and present: + + ~~~ + database_name | parent_schema_name | object_name | object_type | backup_type | start_time | end_time | size_bytes | rows | is_full_cluster | file_bytes + ----------------+--------------------+----------------------------+-------------+-------------+------------+----------------------------+------------+-------+-----------------+------------- + NULL | NULL | movr | database | full | NULL | 2022-09-19 13:41:23.645189 | NULL | NULL | f | NULL + movr | NULL | public | schema | full | NULL | 2022-09-19 13:41:23.645189 | NULL | NULL | f | NULL + movr | public | users | table | full | NULL | 2022-09-19 13:41:23.645189 | 31155 | 340 | f | 16598 + movr | public | vehicles | table | full | NULL | 2022-09-19 13:41:23.645189 | 22282 | 113 | f | 12459 + movr | public | rides | table | full | NULL | 2022-09-19 13:41:23.645189 | 261950 | 902 | f | 135831 + movr | public | vehicle_location_histories | table | full | NULL | 2022-09-19 13:41:23.645189 | 742557 | 10850 | f | 318583 + movr | public | promo_codes | table | full | NULL | 2022-09-19 13:41:23.645189 | 228320 | 1034 | f | 118376 + movr | public | user_promo_codes | table | full | NULL | 2022-09-19 13:41:23.645189 | 9320 | 111 | f | 4832 + ~~~ + + The output will return `file_bytes` along with the columns you receive from `SHOW BACKUP` without `check_files`. The `file_bytes` column indicates the estimated bytes in external storage for a particular table object. For more detail on the output columns, see the `SHOW BACKUP` [Response]({% link {{ page.version.version }}/show-backup.md %}#response) table. + +1. If `SHOW BACKUP ... check_files` cannot read from a file, it will return an error message similar to the following: + + ~~~ + ERROR: The following files are missing from the backup: + s3:/bucket-name/2022/09/19-134123.64/data/797981063156727810.sst + ~~~ + + `SHOW BACKUP ... check_files` will return up to ten file paths for incorrect or missing files. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/control-schedule-privileges.md b/src/current/_includes/v25.1/backups/control-schedule-privileges.md new file mode 100644 index 00000000000..13f0012ea7f --- /dev/null +++ b/src/current/_includes/v25.1/backups/control-schedule-privileges.md @@ -0,0 +1,3 @@ +- Members of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#default-roles). By default, the `root` user belongs to the `admin` role. +- Owners of a backup schedule, i.e., the user that [created the backup schedule]({% link {{ page.version.version }}/create-schedule-for-backup.md %}). +- Owners of a changefeed schedule, i.e., the user that [created the changefeed schedule]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}). \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/destination-privileges.md b/src/current/_includes/v25.1/backups/destination-privileges.md new file mode 100644 index 00000000000..fd5d019e97d --- /dev/null +++ b/src/current/_includes/v25.1/backups/destination-privileges.md @@ -0,0 +1,14 @@ +You can grant a user the `EXTERNALIOIMPLICITACCESS` [system-level privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges). + +Either the `EXTERNALIOIMPLICITACCESS` system-level privilege or the [`admin`]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role) role is required for the following scenarios: + +- Interacting with a cloud storage resource using [`IMPLICIT` authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}). +- Using a [custom endpoint](https://docs.aws.amazon.com/sdk-for-go/api/aws/endpoints/) on S3. +- Using the [`cockroach nodelocal upload`]({% link {{ page.version.version }}/cockroach-nodelocal-upload.md %}) command. + +No special privilege is required for: + +- Interacting with an Amazon S3 and Google Cloud Storage resource using `SPECIFIED` credentials. Azure Storage is always `SPECIFIED` by default. +- Using [Userfile]({% link {{ page.version.version }}/use-userfile-storage.md %}) storage. + +{% include {{ page.version.version }}/misc/bulk-permission-note.md %} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/encrypted-backup-description.md b/src/current/_includes/v25.1/backups/encrypted-backup-description.md new file mode 100644 index 00000000000..a81f545aaf6 --- /dev/null +++ b/src/current/_includes/v25.1/backups/encrypted-backup-description.md @@ -0,0 +1,11 @@ +You can encrypt full or incremental backups with a passphrase by using the [`encryption_passphrase` option]({% link {{ page.version.version }}/backup.md %}#with-encryption-passphrase). Files written by the backup (including `BACKUP` manifests and data files) are encrypted using the specified passphrase to derive a key. To restore the encrypted backup, the same `encryption_passphrase` option (with the same passphrase) must be included in the [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) statement. + +When used with [incremental backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#incremental-backups), the `encryption_passphrase` option is applied to all the [backup file URLs]({% link {{ page.version.version }}/backup.md %}#backup-file-urls), which means the same passphrase must be used when appending another incremental backup to an existing backup. Similarly, when used with [locality-aware backups]({% link {{ page.version.version }}/take-and-restore-locality-aware-backups.md %}), the passphrase provided is applied to files in all localities. + +Encryption is done using [AES-256-GCM](https://wikipedia.org/wiki/Galois/Counter_Mode), and GCM is used to both encrypt and authenticate the files. A random [salt](https://wikipedia.org/wiki/Salt_(cryptography)) is used to derive a once-per-backup [AES](https://wikipedia.org/wiki/Advanced_Encryption_Standard) key from the specified passphrase, and then a random [initialization vector](https://wikipedia.org/wiki/Initialization_vector) is used per-file. CockroachDB uses [PBKDF2](https://wikipedia.org/wiki/PBKDF2) with 64,000 iterations for the key derivation. + +{{site.data.alerts.callout_info}} +`BACKUP` and `RESTORE` will use more memory when using encryption, as both the plain-text and cipher-text of a given file are held in memory during encryption and decryption. +{{site.data.alerts.end}} + +For an example of an encrypted backup, see [Create an encrypted backup]({% link {{ page.version.version }}/take-and-restore-encrypted-backups.md %}#take-an-encrypted-backup-using-a-passphrase). diff --git a/src/current/_includes/v25.1/backups/existing-operation-service-account.md b/src/current/_includes/v25.1/backups/existing-operation-service-account.md new file mode 100644 index 00000000000..dc0f0996320 --- /dev/null +++ b/src/current/_includes/v25.1/backups/existing-operation-service-account.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +If you already have the service account that contains permissions for the operation, ensure that you give the identity service account access to this service account. Click on your service account and navigate to the **Permissions** tab. Then, use the process in [step 3](#step-3-give-the-identity-service-account-the-token-creator-role) to complete this. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/external-io-implicit-flag.md b/src/current/_includes/v25.1/backups/external-io-implicit-flag.md new file mode 100644 index 00000000000..4abb21f78f6 --- /dev/null +++ b/src/current/_includes/v25.1/backups/external-io-implicit-flag.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +If the use of implicit credentials is disabled with the [`--external-io-disable-implicit-credentials` flag]({% link {{ page.version.version }}/cockroach-start.md %}#security), you will receive an error when you access external cloud storage services with `AUTH=implicit`. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/file-size-setting.md b/src/current/_includes/v25.1/backups/file-size-setting.md new file mode 100644 index 00000000000..2ddef08efac --- /dev/null +++ b/src/current/_includes/v25.1/backups/file-size-setting.md @@ -0,0 +1,5 @@ +{{site.data.alerts.callout_info}} +To set a target for the amount of backup data written to each backup file, use the `bulkio.backup.file_size` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}). + +See the [`SET CLUSTER SETTING`]({% link {{ page.version.version }}/set-cluster-setting.md %}) page for more details on using cluster settings. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/backups/full-cluster-restore-validation.md b/src/current/_includes/v25.1/backups/full-cluster-restore-validation.md new file mode 100644 index 00000000000..c4196e6e198 --- /dev/null +++ b/src/current/_includes/v25.1/backups/full-cluster-restore-validation.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +Validation of full-cluster restores with `schema_only` must be run on an empty cluster in the same way as a complete [full-cluster restore]({% link {{ page.version.version }}/restore.md %}#full-cluster). Once you have successfully validated the restore, you can destroy the test cluster. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/gcs-auth-note.md b/src/current/_includes/v25.1/backups/gcs-auth-note.md new file mode 100644 index 00000000000..4c52b8625b7 --- /dev/null +++ b/src/current/_includes/v25.1/backups/gcs-auth-note.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +The examples in this section use the `AUTH=specified` parameter, which will be the default behavior in v21.2 and beyond for connecting to Google Cloud Storage. For more detail on how to pass your Google Cloud Storage credentials with this parameter, or, how to use `implicit` authentication, read [Use Cloud Storage for Bulk Operations — Authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}). +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/backups/gcs-default-deprec.md b/src/current/_includes/v25.1/backups/gcs-default-deprec.md new file mode 100644 index 00000000000..008ad61f4f9 --- /dev/null +++ b/src/current/_includes/v25.1/backups/gcs-default-deprec.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +**Deprecation notice:** Currently, GCS connections default to the `cloudstorage.gs.default.key` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}). This default behavior will no longer be supported in v21.2. If you are relying on this default behavior, we recommend adjusting your queries and scripts to now specify the `AUTH` parameter you want to use. Similarly, if you are using the `cloudstorage.gs.default.key` cluster setting to authorize your GCS connection, we recommend switching to use `AUTH=specified` or `AUTH=implicit`. `AUTH=specified` will be the default behavior in v21.2 and beyond. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/backups/locality-aware-access.md b/src/current/_includes/v25.1/backups/locality-aware-access.md new file mode 100644 index 00000000000..d0a57842341 --- /dev/null +++ b/src/current/_includes/v25.1/backups/locality-aware-access.md @@ -0,0 +1 @@ +A successful locality-aware backup job requires that each node in the cluster has access to each storage location. This is because any node in the cluster can claim the job and become the [_coordinator_]({% link {{ page.version.version }}/backup-architecture.md %}#job-creation-phase) node. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/locality-aware-backups.md b/src/current/_includes/v25.1/backups/locality-aware-backups.md new file mode 100644 index 00000000000..8ce87c53654 --- /dev/null +++ b/src/current/_includes/v25.1/backups/locality-aware-backups.md @@ -0,0 +1,39 @@ +{{site.data.alerts.callout_info}} +`SHOW BACKUP` is able to display metadata using `check_files` for locality-aware backups taken with the [`incremental_location`]({% link {{ page.version.version }}/show-backup.md %}#show-a-backup-taken-with-the-incremental-location-option) option. +{{site.data.alerts.end}} + +To view a list of [locality-aware backups]({% link {{ page.version.version }}/take-and-restore-locality-aware-backups.md %}), pass the endpoint [collection URI]({% link {{ page.version.version }}/backup.md %}#backup-file-urls) that is set as the `default` location with `COCKROACH_LOCALITY=default`: + +{% include_cached copy-clipboard.html %} +~~~ sql +> SHOW BACKUPS IN 's3://{default collection URI}/{path}?AWS_ACCESS_KEY_ID={placeholder}&AWS_SECRET_ACCESS_KEY={placeholder}'; +~~~ + +~~~ + path +------------------------- +/2023/02/23-150925.62 +/2023/03/08-192859.44 +(2 rows) +~~~ + +To view a [locality-aware backup]({% link {{ page.version.version }}/take-and-restore-locality-aware-backups.md %}), pass locality-aware backup URIs to `SHOW BACKUP`: + +{% include_cached copy-clipboard.html %} +~~~ sql +> SHOW BACKUP FROM LATEST IN ('s3://{bucket name}/locality?AWS_ACCESS_KEY_ID={placeholder}&AWS_SECRET_ACCESS_KEY={placeholder}&COCKROACH_LOCALITY=default', 's3://{bucket name}/locality?AWS_ACCESS_KEY_ID={placeholder}&AWS_SECRET_ACCESS_KEY={placeholder}&COCKROACH_LOCALITY=region%3Dus-west'); +~~~ + +~~~ + database_name | parent_schema_name | object_name | object_type | backup_type | start_time | end_time | size_bytes | rows | is_full_cluster +----------------+--------------------+----------------------------+-------------+-------------+------------+----------------------------+------------+------+------------------ + NULL | NULL | movr | database | full | NULL | 2023-02-23 15:09:25.625777 | NULL | NULL | f + movr | NULL | public | schema | full | NULL | 2023-02-23 15:09:25.625777 | NULL | NULL | f + movr | public | users | table | full | NULL | 2023-02-23 15:09:25.625777 | 5633 | 58 | f + movr | public | vehicles | table | full | NULL | 2023-02-23 15:09:25.625777 | 3617 | 17 | f + movr | public | rides | table | full | NULL | 2023-02-23 15:09:25.625777 | 159269 | 511 | f + movr | public | vehicle_location_histories | table | full | NULL | 2023-02-23 15:09:25.625777 | 79963 | 1092 | f + movr | public | promo_codes | table | full | NULL | 2023-02-23 15:09:25.625777 | 221763 | 1003 | f + movr | public | user_promo_codes | table | full | NULL | 2023-02-23 15:09:25.625777 | 927 | 11 | f +(8 rows) +~~~ diff --git a/src/current/_includes/v25.1/backups/locality-aware-multi-tenant.md b/src/current/_includes/v25.1/backups/locality-aware-multi-tenant.md new file mode 100644 index 00000000000..896d29db2d6 --- /dev/null +++ b/src/current/_includes/v25.1/backups/locality-aware-multi-tenant.md @@ -0,0 +1 @@ +Both CockroachDB {{ site.data.products.standard }} and CockroachDB {{ site.data.products.basic }} clusters operate with a different architecture compared to CockroachDB {{ site.data.products.core }}. These architectural differences have implications for how locality-aware backups can run. {{ site.data.products.standard }} and {{ site.data.products.basic }} clusters will scale resources depending on whether they are actively in use. This makes it less likely to have a SQL pod available in every locality. As a result, your Serverless cluster may not have a SQL pod in the locality where the data resides, which can lead to the cluster uploading that data to a storage bucket in a locality where you do have active SQL pods. You should consider this as you plan a backup strategy that must comply with [data domiciling]({% link v23.2/data-domiciling.md %}) requirements. diff --git a/src/current/_includes/v25.1/backups/metrics-per-node.md b/src/current/_includes/v25.1/backups/metrics-per-node.md new file mode 100644 index 00000000000..a1ec6e0b350 --- /dev/null +++ b/src/current/_includes/v25.1/backups/metrics-per-node.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +Metrics are reported per node. Therefore, it is necessary to retrieve metrics from every node in the cluster. For example, if you are monitoring whether a backup fails, it is necessary to track `scheduled_backup_failed` on each node. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/no-incremental-restore.md b/src/current/_includes/v25.1/backups/no-incremental-restore.md new file mode 100644 index 00000000000..6415ec7e034 --- /dev/null +++ b/src/current/_includes/v25.1/backups/no-incremental-restore.md @@ -0,0 +1 @@ +When you restore from an incremental backup, you're restoring the **entire** table, database, or cluster. CockroachDB uses both the latest (or a [specific]({% link {{ page.version.version }}/restore.md %}#restore-a-specific-full-or-incremental-backup)) incremental backup and the full backup during this process. You cannot restore an incremental backup without a full backup. Furthermore, it is not possible to restore over a [table]({% link {{ page.version.version }}/restore.md %}#tables), [database]({% link {{ page.version.version }}/restore.md %}#databases), or [cluster](restore.html#full-cluster) with existing data. Refer to [Restore types](restore.html#restore-types) for detail on the types of backups you can restore. diff --git a/src/current/_includes/v25.1/backups/object-dependency.md b/src/current/_includes/v25.1/backups/object-dependency.md new file mode 100644 index 00000000000..07bcbc698f3 --- /dev/null +++ b/src/current/_includes/v25.1/backups/object-dependency.md @@ -0,0 +1,13 @@ +Dependent objects must be {% if page.name == "restore.md" %} restored {% else %} backed up {% endif %} at the same time as the objects they depend on. When you back up a table, it will not include any dependent tables, [views]({% link {{ page.version.version }}/views.md %}), or [sequences]({% link {{ page.version.version }}/create-sequence.md %}). + +For example, if you back up [view]({% link {{ page.version.version }}/views.md %}) `v` that depends on table `t`, it will only back up `v`, not `t`. When you try to restore `v`, the restore will fail because the referenced table is not present in the backup. + +Alternatively, you can pass a `skip` option with {% if page.name == "restore.md" %} `RESTORE` {% else %} [`RESTORE`]({% link {{ page.version.version }}/restore.md %}) {% endif %} to skip the dependency instead: + +Dependent object | Depends on | Skip option +-------|------------+------------- +Table with [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) constraints | The table it `REFERENCES`. | [`skip_missing_foreign_keys`]({% link {{ page.version.version }}/restore.md %}#skip_missing_foreign_keys) +Table with a [sequence]({% link {{ page.version.version }}/create-sequence.md %}) | The sequence. | [`skip_missing_sequences`]({% link {{ page.version.version }}/restore.md %}#skip-missing-sequences) +[Views]({% link {{ page.version.version }}/views.md %}) | The tables used in the view's `SELECT` statement. | [`skip_missing_views`]({% link {{ page.version.version }}/restore.md %}#skip-missing-views) + +We recommend treating tables with [foreign keys]({% link {{ page.version.version }}/foreign-key.md %}), which contribute to [views]({% link {{ page.version.version }}/views.md %}), or that use sequences or user-defined types as a single unit with their dependencies. While you can restore individual tables, you may find that backing up and restoring at the database level is more convenient. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/old-syntax-removed.md b/src/current/_includes/v25.1/backups/old-syntax-removed.md new file mode 100644 index 00000000000..7052fe0d3af --- /dev/null +++ b/src/current/_includes/v25.1/backups/old-syntax-removed.md @@ -0,0 +1,5 @@ +{{site.data.alerts.callout_danger}} +The `BACKUP ... TO` and `RESTORE ... FROM {storage_uri}` syntax has been removed from CockroachDB v24.3 and later. + +For details on the syntax to run `BACKUP` and `RESTORE`, refer to the {% if page.name == "backup.md" %} [backup](#examples) {% else %} [backup]({% link {{ page.version.version }}/backup.md %}#examples) {% endif %} and {% if page.name == "restore.md" %} [restore](#examples) {% else %} [restore]({% link {{ page.version.version }}/restore.md %}#examples) {% endif %} examples. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/protected-timestamps.md b/src/current/_includes/v25.1/backups/protected-timestamps.md new file mode 100644 index 00000000000..31b931f4cc4 --- /dev/null +++ b/src/current/_includes/v25.1/backups/protected-timestamps.md @@ -0,0 +1,5 @@ +Scheduled backups ensure that the data to be backed up is protected from garbage collection until it has been successfully backed up. This active management of [protected timestamps]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) means that you can run scheduled backups at a cadence independent from the [GC TTL]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds) of the data. This is unlike non-scheduled backups that are tightly coupled to the GC TTL. See [Garbage collection and backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#garbage-collection-and-backups) for more detail. + +The data being backed up will not be eligible for garbage collection until a successful backup completes. At this point, the schedule will release the existing protected timestamp record and write a new one to protect data for the next backup that is scheduled to run. It is important to consider that when a scheduled backup fails there will be an accumulation of data until the next successful backup. Resolving the backup failure or [dropping the backup schedule]({% link {{ page.version.version }}/drop-schedules.md %}) will make the data eligible for garbage collection once again. + +You can also use the `exclude_data_from_backup` option with a scheduled backup as a way to prevent protected timestamps from prolonging garbage collection on a table. See the example [Exclude a table's data from backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#exclude-a-tables-data-from-backups) for usage information. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/pts-schedules-incremental.md b/src/current/_includes/v25.1/backups/pts-schedules-incremental.md new file mode 100644 index 00000000000..b8cb26aff0b --- /dev/null +++ b/src/current/_includes/v25.1/backups/pts-schedules-incremental.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +If you are creating incremental backups as part of a [backup schedule]({% link {{ page.version.version }}/create-schedule-for-backup.md %}), [protected timestamps]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) will ensure the backup revision data is not garbage collected, which allows you to lower the GC TTL. See [Protected timestamps and scheduled backups]({% link {{ page.version.version }}/create-schedule-for-backup.md %}#protected-timestamps-and-scheduled-backups) for more detail. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/recommend-backups-for-upgrade.md b/src/current/_includes/v25.1/backups/recommend-backups-for-upgrade.md new file mode 100644 index 00000000000..2ef075abf9c --- /dev/null +++ b/src/current/_includes/v25.1/backups/recommend-backups-for-upgrade.md @@ -0,0 +1,6 @@ +{% if page.path contains "cockroachcloud" %} +[Managed backups]({% link cockroachcloud/managed-backups.md %}) are automated backups of CockroachDB {{ site.data.products.cloud }} clusters that are stored by Cockroach Labs in cloud storage. By default, Cockroach Labs takes and retains managed backups in all Cloud clusters. + +When upgrading to a major release, you can optionally [take a self-managed backup]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}) of your cluster to your own cloud storage, as an extra layer of protection in case the upgrade leads to issues. +{% else %} +CockroachDB is designed with high fault tolerance. However, taking regular backups of your data is an operational best practice for [disaster recovery]({% link {{ page.version.version }}/disaster-recovery-planning.md %}) planning.{% endif %} diff --git a/src/current/_includes/v25.1/backups/retry-failure.md b/src/current/_includes/v25.1/backups/retry-failure.md new file mode 100644 index 00000000000..e29f6a8d3a6 --- /dev/null +++ b/src/current/_includes/v25.1/backups/retry-failure.md @@ -0,0 +1 @@ +If a backup job encounters too many retryable errors, it will enter a [`failed` state]({% link {{ page.version.version }}/show-jobs.md %}#job-status) with the most recent error, which allows subsequent backups the chance to succeed. Refer to the [Backup and Restore Monitoring]({% link {{ page.version.version }}/backup-and-restore-monitoring.md %}) page for metrics to track backup failures. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/schedule-options.md b/src/current/_includes/v25.1/backups/schedule-options.md new file mode 100644 index 00000000000..2743f788b85 --- /dev/null +++ b/src/current/_includes/v25.1/backups/schedule-options.md @@ -0,0 +1,7 @@ + Option | Value | Description +----------------------------+-----------------------------------------+------------------------------ +`first_run` | [`TIMESTAMPTZ`]({% link {{ page.version.version }}/timestamp.md %}) / `now` | Execute the schedule at the specified time in the future. If not specified, the default behavior is to execute the schedule based on its next `RECURRING` time. +`on_execution_failure` | `retry` / `reschedule` / `pause` | If an error occurs during the backup execution, do the following:
**Default**: `reschedule` +`on_previous_running` | `start` / `skip` / `wait` | If the previous backup started by the schedule is still running, do the following:
**Default**: `wait`. The option affects backups started by the full backup schedule only. Incremental backups are always set to `wait`. +`ignore_existing_backups` | N/A | If backups were already created in the [destination]({% link {{ page.version.version }}/use-cloud-storage.md %}) that the new schedule references, this option must be passed to acknowledge that the new schedule may be backing up different objects. +`updates_cluster_last_backup_time_metric` | N/A | ([`admin` privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role) required) When set during backup schedule creation, this option updates the [`schedules_backup_last_completed_time`]({% link {{ page.version.version }}/backup-and-restore-monitoring.md %}#available-metrics) metric for the scheduled backup. diff --git a/src/current/_includes/v25.1/backups/scheduled-backups-tip.md b/src/current/_includes/v25.1/backups/scheduled-backups-tip.md new file mode 100644 index 00000000000..62ced81e069 --- /dev/null +++ b/src/current/_includes/v25.1/backups/scheduled-backups-tip.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_success}} +We recommend using scheduled backups to automate daily backups of your cluster. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/show-backup-replace-diagram.html b/src/current/_includes/v25.1/backups/show-backup-replace-diagram.html new file mode 100644 index 00000000000..539b72b45da --- /dev/null +++ b/src/current/_includes/v25.1/backups/show-backup-replace-diagram.html @@ -0,0 +1,50 @@ + \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/storage-collision-examples.md b/src/current/_includes/v25.1/backups/storage-collision-examples.md new file mode 100644 index 00000000000..b6c3b85de05 --- /dev/null +++ b/src/current/_includes/v25.1/backups/storage-collision-examples.md @@ -0,0 +1 @@ +For example, if you have a backup schedule running backups for the database `users` the full backup and incremental backup should have the same storage URI for the full and incremental schedule. (`CREATE SCHEDULE FOR BACKUP` will automatically create two schedules for the full and incremental backup to the same storage URI.) If there is another backup schedule, for the database `accounts`, the full and incremental backups for `accounts` should have the same storage URI. However, the storage URI for the `accounts` backup collection should be different to the storage URI for the `users` backup collection. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/support-products.md b/src/current/_includes/v25.1/backups/support-products.md new file mode 100644 index 00000000000..d4687c816d2 --- /dev/null +++ b/src/current/_includes/v25.1/backups/support-products.md @@ -0,0 +1,3 @@ +## Supported products + +The feature described on this page is available in **CockroachDB {{ site.data.products.basic }}**, **CockroachDB {{ site.data.products.standard }}**, **CockroachDB {{ site.data.products.advanced }}**, and **CockroachDB {{ site.data.products.core }}** clusters when you are running [self-managed backups]({% link cockroachcloud/take-and-restore-self-managed-backups.md %}). For a full list of features, refer to [Backup and restore product support]({% link {{ page.version.version }}/backup-and-restore-overview.md %}#backup-and-restore-support). diff --git a/src/current/_includes/v25.1/backups/updated-backup-privileges.md b/src/current/_includes/v25.1/backups/updated-backup-privileges.md new file mode 100644 index 00000000000..013fa0f4d8d --- /dev/null +++ b/src/current/_includes/v25.1/backups/updated-backup-privileges.md @@ -0,0 +1,36 @@ +{{site.data.alerts.callout_info}} +Starting in v22.2, CockroachDB introduces a new [system-level privilege model]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) that provides finer control over a user's privilege to work with the database, including taking backups. + +There is continued support for the [legacy privilege model](#required-privileges-using-the-legacy-privilege-model) for backups in v22.2, however it **will be removed** in a future release of CockroachDB. We recommend implementing the new privilege model that follows in this section for all new and existing backups. +{{site.data.alerts.end}} + +You can [grant]({% link {{ page.version.version }}/grant.md %}#grant-privileges-on-specific-tables-in-a-database) the `BACKUP` privilege to a user or role depending on the type of backup: + +Backup | Privilege +-------+----------- +Cluster | Grant a user the `BACKUP` [system-level privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges). For example, `GRANT SYSTEM BACKUP TO user;`. +Database | Grant a user the `BACKUP` privilege on the target database. For example, `GRANT BACKUP ON DATABASE test_db TO user;`. +Table | Grant a user the `BACKUP` privilege at the table level. This gives the user the privilege to back up the schema and all user-defined types that are associated with the table. For example, `GRANT BACKUP ON TABLE test_db.table TO user;`. + +The listed privileges do not cascade to objects lower in the schema tree. For example, if you are granted database-level `BACKUP` privileges, this does not give you the privilege to back up a table. If you need the `BACKUP` privilege on a database to apply to all newly created tables in that database, use [`DEFAULT PRIVILEGES`]({% link {{ page.version.version }}/security-reference/authorization.md %}#default-privileges). You can add `BACKUP` to the user or role's default privileges with [`ALTER DEFAULT PRIVILEGES`]({% link {{ page.version.version }}/alter-default-privileges.md %}#grant-default-privileges-to-a-specific-role). + +{{site.data.alerts.callout_info}} +You can grant the `BACKUP` privilege to a user or role **without** the `SELECT` privilege on a table. As a result, these users will be able to take backups, but they will not be able to run a `SELECT` query on that data directly. However, these users could still read this data indirectly, by restoring it from any backups they produce. +{{site.data.alerts.end}} + +Members of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role) can run all three types of backups (cluster, database, and table) without the need to grant a specific `BACKUP` privilege. However, we recommend using the `BACKUP` privilege model to create users or roles and grant them `BACKUP` privileges as necessary for stronger access control. + +### Privileges for managing a backup job + +To manage a backup job with [`PAUSE JOB`]({% link {{ page.version.version }}/pause-job.md %}), [`RESUME JOB`]({% link {{ page.version.version }}/resume-job.md %}), or [`CANCEL JOB`]({% link {{ page.version.version }}/cancel-job.md %}), users must have at least one of the following: + +- Be a member of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role). +- The [`CONTROLJOB` role option]({% link {{ page.version.version }}/security-reference/authorization.md %}#role-options). + +To view a backup job with [`SHOW JOB`]({% link {{ page.version.version }}/show-jobs.md %}), users must have at least one of the following: + +- The [`VIEWJOB` privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges), which allows you to view all jobs (including `admin`-owned jobs). +- Be a member of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role). +- The [`CONTROLJOB` role option]({% link {{ page.version.version }}/security-reference/authorization.md %}#role-options). + +See [`GRANT`]({% link {{ page.version.version }}/grant.md %}) for detail on granting privileges to a role or user. \ No newline at end of file diff --git a/src/current/_includes/v25.1/backups/view-scheduled-backups.md b/src/current/_includes/v25.1/backups/view-scheduled-backups.md new file mode 100644 index 00000000000..5f3fee97e56 --- /dev/null +++ b/src/current/_includes/v25.1/backups/view-scheduled-backups.md @@ -0,0 +1,7 @@ + When a [backup is created by a schedule]({% link {{ page.version.version }}/create-schedule-for-backup.md %}), it is stored within a collection of backups in the given location. To view details for a backup created by a schedule, you can use the following: + +- `SHOW BACKUPS IN collectionURI` statement to [view a list of the full backup's subdirectories]({% link {{ page.version.version }}/show-backup.md %}#view-a-list-of-the-available-full-backup-subdirectories). +- `SHOW BACKUP FROM subdirectory IN collectionURI` statement to [view a list of the full and incremental backups that are stored in a specific full backup's subdirectory]({% link {{ page.version.version }}/show-backup.md %}#view-a-list-of-the-full-and-incremental-backups-in-a-specific-full-backup-subdirectory). +- Use the [Schedules page]({% link {{ page.version.version }}/ui-schedules-page.md %}) in the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}) to view a list of created backup schedules and their individual details. + +For more details, see [`SHOW BACKUP`]({% link {{ page.version.version }}/show-backup.md %}). \ No newline at end of file diff --git a/src/current/_includes/v25.1/backward-incompatible/alpha.1.md b/src/current/_includes/v25.1/backward-incompatible/alpha.1.md new file mode 100644 index 00000000000..d5aaef86c6b --- /dev/null +++ b/src/current/_includes/v25.1/backward-incompatible/alpha.1.md @@ -0,0 +1,15 @@ +- CockroachDB no longer performs environment variable expansion in the parameter `--certs-dir`. Uses like `--certs-dir='$HOME/path'` (expansion by CockroachDB) can be replaced by `--certs-dir="$HOME/path"` (expansion by the Unix shell). [#81298][#81298] +- In the Cockroach CLI, [`BOOL` values](../v23.1/bool.html) are now formatted as `t` or `f` instead of `True` or `False`. [#81943][#81943] +- Removed the `cockroach quit` command. It has been deprecated since v20.1. To [shut down a node](../v23.1/node-shutdown.html) gracefully, send a `SIGTERM` signal to it. [#82988][#82988] +- Added a cluster version to allow the [Pebble storage engine](../v23.1/architecture/storage-layer.html#pebble) to recombine certain SSTables (specifically, user keys that are split across multiple files in a level of the [log-structured merge-tree](../v23.1/architecture/storage-layer.html#log-structured-merge-trees)). Recombining the split user keys is required for supporting the range keys feature. The migration to recombine the SSTables is expected to be short (split user keys are rare in practice), but will block subsequent migrations until all tables have been recombined. The `storage.marked-for-compaction-files` time series metric can show the progress of the migration. [#84887][#84887] +- Using a single TCP port listener for both RPC (node-node) and SQL client connections is now deprecated. This capability **will be removed** in the next version of CockroachDB. Instead, make one of the following configuration changes to your CockroachDB deployment: + - Preferred: keep port `26257` for SQL, and allocate a new port, e.g., `26357`, for node-node RPC connections. For example, you might configure a node with the flags `--listen-addr=:26357 --sql-addr=:26257`, where subsequent nodes seeking to join would then use the flag `--join=othernode:26357,othernode:26257`. This will become the default configuration in the next version of CockroachDB. When using this mode of operation, care should be taken to use a `--join` flag that includes both the previous and new port numbers for other nodes, so that no network partition occurs during the upgrade. + - Optional: keep port `26257` for RPC, and allocate a new port, e.g., `26357`, for SQL connections. For example, you might configure a node with the flags `--listen-addr=:26257 --sql-addr=:26357`. When using this mode of operation, the `--join` flags do not need to be modified. However, SQL client apps or the SQL load balancer configuration (when in use) must be updated to use the new SQL port number. [#85671][#85671] +- If no `nullif` option is specified while using [`IMPORT CSV`](../v23.1/import.html), then a zero-length string in the input is now treated as `NULL`. The quoted empty string in the input is treated as an empty string. Similarly, if `nullif` is specified, then an unquoted value is treated as `NULL`, and a quoted value is treated as that string. These changes were made to make `IMPORT CSV` behave more similarly to `COPY CSV`. If the previous behavior (i.e., treating either quoted or unquoted values that match the `nullif` setting as `NULL`) is desired, you can use the new `allow_quoted_null` option in the `IMPORT` statement. [#84487][#84487] +- [`COPY FROM`](../v23.1/copy-from.html) operations are now atomic by default instead of being segmented into 100 row transactions. Set the `copy_from_atomic_enabled` session setting to `false` for the previous behavior. [#85986][#85986] +- The `GRANT` privilege has been removed and replaced by the more granular [`WITH GRANT OPTION`]({% link v22.2/grant.md %}#grant-privileges-with-the-option-to-grant-to-others), which provides control over which privileges are allowed to be granted. [#81310][#81310] +- Removed the ability to cast `int`, `int2`, and `int8` to a `0` length `BIT` or `VARBIT`. [#81266][#81266] +- Removed the deprecated `GRANT` privilege. [#81310][#81310] +- Removed the `ttl_automatic_column` storage parameter. The `crdb_internal_expiration` column is created when `ttl_expire_after` is set and removed when `ttl_expire_after` is reset. [#83134][#83134] +- Removed the byte string parameter in the `crdb_internal.schedule_sql_stats_compaction` function. [#82560][#82560] +- Changed the default value of the `enable_implicit_transaction_for_batch_statements` to `true`. This means that a [batch of statements]({% link v22.2/transactions.md %}#batched-statements) sent in one string separated by semicolons is treated as an implicit transaction. [#76834][#76834] diff --git a/src/current/_includes/v25.1/cdc/apache-pulsar-unsupported.md b/src/current/_includes/v25.1/cdc/apache-pulsar-unsupported.md new file mode 100644 index 00000000000..fecb7931784 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/apache-pulsar-unsupported.md @@ -0,0 +1,8 @@ +Changefeeds emitting to an Apache Pulsar sink do **not** support: + +- [`format=avro`]({% link {{ page.version.version }}/create-changefeed.md %}#format) +- [`confluent_schema_registry`]({% link {{ page.version.version }}/create-changefeed.md %}#confluent-schema-registry) +- [`topic_prefix`]({% link {{ page.version.version }}/create-changefeed.md %}#topic-prefix) +- Any batching configuration +- [Authentication query parameters]({% link {{ page.version.version }}/create-changefeed.md %}#query-parameters) +- [External connections]({% link {{ page.version.version }}/create-external-connection.md %}) \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/apache-pulsar-uri.md b/src/current/_includes/v25.1/cdc/apache-pulsar-uri.md new file mode 100644 index 00000000000..28e8f6438d8 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/apache-pulsar-uri.md @@ -0,0 +1,6 @@ +{% include_cached copy-clipboard.html %} +~~~ +pulsar://{host IP}:6650 +~~~ + +By default, Apache Pulsar listens for client connections on port `:6650`. For more detail on configuration, refer to the [Apache Pulsar documentation](https://pulsar.apache.org/docs/2.10.x/reference-configuration). \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/avro-limitations.md b/src/current/_includes/v25.1/cdc/avro-limitations.md new file mode 100644 index 00000000000..8580ac64cf7 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/avro-limitations.md @@ -0,0 +1,30 @@ +- [Decimals]({% link {{ page.version.version }}/decimal.md %}) must have precision specified. +- [`BYTES`]({% link {{ page.version.version }}/bytes.md %}) (or its aliases `BYTEA` and `BLOB`) are often used to store machine-readable data. When you stream these types through a changefeed with [`format=avro`]({% link {{ page.version.version }}/create-changefeed.md %}#format), CockroachDB does not encode or change the data. However, Avro clients can often include escape sequences to present the data in a printable format, which can interfere with deserialization. A potential solution is to hex-encode `BYTES` values when initially inserting them into CockroachDB. This will ensure that Avro clients can consistently decode the hexadecimal. Note that hex-encoding values at insertion will increase record size. +- [`BIT`]({% link {{ page.version.version }}/bit.md %}) and [`VARBIT`]({% link {{ page.version.version }}/bit.md %}) types are encoded as arrays of 64-bit integers. + + For efficiency, CockroachDB encodes `BIT` and `VARBIT` bitfield types as arrays of 64-bit integers. That is, [base-2 (binary format)](https://wikipedia.org/wiki/Binary_number#Conversion_to_and_from_other_numeral_systems) `BIT` and `VARBIT` data types are converted to base 10 and stored in arrays. Encoding in CockroachDB is [big-endian](https://wikipedia.org/wiki/Endianness), therefore the last value may have many trailing zeroes. For this reason, the first value of each array is the number of bits that are used in the last value of the array. + + For instance, if the bitfield is 129 bits long, there will be 4 integers in the array. The first integer will be `1`; representing the number of bits in the last value, the second integer will be the first 64 bits, the third integer will be bits 65–128, and the last integer will either be `0` or `9223372036854775808` (i.e., the integer with only the first bit set, or `1000000000000000000000000000000000000000000000000000000000000000` when base 2). + + This example is base-10 encoded into an array as follows: + + ~~~ + {"array": [1,
**Note:** Using `http(s)` without the `file-` prefix is deprecated as a [changefeed sink]({% link {{ page.version.version }}/changefeed-sinks.md %}) scheme. There is continued support for `http(s)`, but it will be removed in a future release. We recommend implementing the `file-http(s)` scheme for changefeed messages. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/message-format-list.md b/src/current/_includes/v25.1/cdc/message-format-list.md new file mode 100644 index 00000000000..7f77dbba575 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/message-format-list.md @@ -0,0 +1,6 @@ +By default, changefeeds emit messages in JSON format. You can use a different format by [creating a changefeed](create-changefeed.html) with the [`format`](create-changefeed.html#format) option and specifying one of the following: + +- `json` +- `csv` +- `avro` +- `parquet` \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/metrics-labels.md b/src/current/_includes/v25.1/cdc/metrics-labels.md new file mode 100644 index 00000000000..6f97eaffcdd --- /dev/null +++ b/src/current/_includes/v25.1/cdc/metrics-labels.md @@ -0,0 +1,8 @@ +To measure metrics per changefeed, you can define a "metrics label" for one or multiple changefeed(s). The changefeed(s) will increment each [changefeed metric]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#metrics). Metrics label information is sent with time-series metrics to `http://{host}:{http-port}/_status/vars`, viewable via the [Prometheus endpoint]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#prometheus-endpoint). An aggregated metric of all changefeeds is also measured. + +It is necessary to consider the following when applying metrics labels to changefeeds: + +- The `server.child_metrics.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) must be set to `true` before using the `metrics_label` option. `server.child_metrics.enabled` is enabled by default in {{ site.data.products.standard }} and {{ site.data.products.basic }}. +- Metrics label information is sent to the `_status/vars` endpoint, but will **not** show up in [`debug.zip`]({% link {{ page.version.version }}/cockroach-debug-zip.md %}) or the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}). +- Introducing labels to isolate a changefeed's metrics can increase cardinality significantly. There is a limit of 1024 unique labels in place to prevent cardinality explosion. That is, when labels are applied to high-cardinality data (data with a higher number of unique values), each changefeed with a label then results in more metrics data to multiply together, which will grow over time. This will have an impact on performance as the metric-series data per changefeed quickly populates against its label. +- The maximum length of a metrics label is 128 bytes. diff --git a/src/current/_includes/v25.1/cdc/modify-changefeed.md b/src/current/_includes/v25.1/cdc/modify-changefeed.md new file mode 100644 index 00000000000..fde29d8687e --- /dev/null +++ b/src/current/_includes/v25.1/cdc/modify-changefeed.md @@ -0,0 +1,9 @@ +To modify an {{ site.data.products.enterprise }} changefeed, [pause]({% link {{ page.version.version }}/create-and-configure-changefeeds.md %}#pause) the job and then use: + +~~~ sql +ALTER CHANGEFEED job_id {ADD table DROP table SET option UNSET option}; +~~~ + +You can add new table targets, remove them, set new [changefeed options]({% link {{ page.version.version }}/create-changefeed.md %}#options), and unset them. + +For more information, see [`ALTER CHANGEFEED`]({% link {{ page.version.version }}/alter-changefeed.md %}). diff --git a/src/current/_includes/v25.1/cdc/msk-dedicated-support.md b/src/current/_includes/v25.1/cdc/msk-dedicated-support.md new file mode 100644 index 00000000000..9c3c119b838 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/msk-dedicated-support.md @@ -0,0 +1 @@ +If you would like to connect a changefeed running on a CockroachDB Dedicated cluster to an Amazon MSK Serverless cluster, contact your Cockroach Labs account team. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/msk-iam-policy-role-step.md b/src/current/_includes/v25.1/cdc/msk-iam-policy-role-step.md new file mode 100644 index 00000000000..0758d426419 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/msk-iam-policy-role-step.md @@ -0,0 +1,51 @@ +1. In the AWS Management Console, go to the [IAM console](https://console.aws.amazon.com/iam/), select **Policies** from the navigation, and then **Create Policy**. +1. Using the **JSON** tab option, update the policy with the following JSON. These permissions will allow you to connect to the cluster, manage topics, and consume messages. You may want to adjust the permissions to suit your permission model. For more details on the available permissions, refer to the AWS documentation on [IAM Access Control](https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html#kafka-actions) for MSK. + + Replace the instances of `arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}` with the MSK ARN from your cluster's summary page and add `/*` to the end, like the following: + + {% include_cached copy-clipboard.html %} + ~~~json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "kafka-cluster:Connect", + "kafka-cluster:AlterCluster", + "kafka-cluster:DescribeCluster" + ], + "Resource": [ + "arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" + ] + }, + { + "Effect": "Allow", + "Action": [ + "kafka-cluster:*Topic", + "kafka-cluster:WriteData", + "kafka-cluster:ReadData" + ], + "Resource": [ + "arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" + ] + }, + { + "Effect": "Allow", + "Action": [ + "kafka-cluster:AlterGroup", + "kafka-cluster:DescribeGroup" + ], + "Resource": [ + "arn:aws:kafka:{region}:{account ID}:cluster/{msk-cluster-name}/*" + ] + } + ] + } + ~~~ + +1. Once you have added your policy, add a policy name (for example, `msk-policy`), click **Next**, and **Create policy**. +1. Return to the [IAM console](https://console.aws.amazon.com/iam/), select **Roles** from the navigation, and then **Create role**. +1. Select **AWS service** for the **Trusted entity type**. For **Use case**, select **EC2** from the dropdown. Click **Next**. +1. On the **Add permissions** page, search for the IAM policy (`msk-policy`) you just created. Click **Next**. +1. Name the role (for example, `msk-role`) and click **Create role**. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/msk-tutorial-crdb-setup.md b/src/current/_includes/v25.1/cdc/msk-tutorial-crdb-setup.md new file mode 100644 index 00000000000..40de46f2af7 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/msk-tutorial-crdb-setup.md @@ -0,0 +1,33 @@ +1. (Optional) On the EC2 instance running CockroachDB, run the [Movr]({% link {{ page.version.version }}/movr.md %}) application workload to set up some data for your changefeed. + + Create the schema for the workload: + + {% include_cached copy-clipboard.html %} + ~~~shell + cockroach workload init movr + ~~~ + + Then run the workload: + + {% include_cached copy-clipboard.html %} + ~~~shell + cockroach workload run movr --duration=1m + ~~~ + +1. Start a SQL session. For details on the available flags, refer to the [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}) page. + + {% include_cached copy-clipboard.html %} + ~~~ shell + cockroach sql --insecure + ~~~ + + {{site.data.alerts.callout_info}} + To set your {{ site.data.products.enterprise }} license, refer to the [Licensing FAQs]({% link {{ page.version.version }}/licensing-faqs.md %}#set-a-license) page. + {{site.data.alerts.end}} + +1. Enable the `kv.rangefeed.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}): + + {% include_cached copy-clipboard.html %} + ~~~ sql + SET CLUSTER SETTING kv.rangefeed.enabled = true; + ~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/mux-rangefeed.md b/src/current/_includes/v25.1/cdc/mux-rangefeed.md new file mode 100644 index 00000000000..964918545df --- /dev/null +++ b/src/current/_includes/v25.1/cdc/mux-rangefeed.md @@ -0,0 +1,3 @@ +`MuxRangefeed` is enabled by default. + +`MuxRangefeed` is a subsystem that improves the performance of rangefeeds with scale. It significantly reduces the overhead of running rangefeeds. Without `MuxRangefeed`, the number of RPC streams is proportional with the number of ranges in a table. For example, a large table could have tens of thousands of ranges. With `MuxRangefeed`, this proportion improves so that the number of RPC streams is relative to the number of nodes in a cluster. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/note-changefeed-message-page.md b/src/current/_includes/v25.1/cdc/note-changefeed-message-page.md new file mode 100644 index 00000000000..4570458c8be --- /dev/null +++ b/src/current/_includes/v25.1/cdc/note-changefeed-message-page.md @@ -0,0 +1 @@ +For an overview of the messages emitted from changefeeds, see the [Changefeed Messages]({% link {{ page.version.version }}/changefeed-messages.md %}) page. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/oauth-description.md b/src/current/_includes/v25.1/cdc/oauth-description.md new file mode 100644 index 00000000000..a12ea818927 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/oauth-description.md @@ -0,0 +1 @@ +[OAuth 2.0](https://oauth.net/2/) authentication uses credentials managed by a third-party provider (IdP) to authenticate with Kafka instead of requiring you to provide your Kafka cluster credentials directly in a [`CREATE CHANGEFEED`]({% link {{ page.version.version }}/create-changefeed.md %}) statement. Your provider's authentication server will issue a temporary token, giving you flexibility to apply access rules on the credentials that your IdP provides. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/options-table-note.md b/src/current/_includes/v25.1/cdc/options-table-note.md new file mode 100644 index 00000000000..7ae80055e08 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/options-table-note.md @@ -0,0 +1 @@ +This table shows the parameters for changefeeds to a specific sink. The `CREATE CHANGEFEED` page provides a list of all the available [options]({% link {{ page.version.version }}/create-changefeed.md %}#options). diff --git a/src/current/_includes/v25.1/cdc/print-key.md b/src/current/_includes/v25.1/cdc/print-key.md new file mode 100644 index 00000000000..ab0b0924d30 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/print-key.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +This example only prints the value. To print both the key and value of each message in the changefeed (e.g., to observe what happens with `DELETE`s), use the `--property print.key=true` flag. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/cdc/privilege-model.md b/src/current/_includes/v25.1/cdc/privilege-model.md new file mode 100644 index 00000000000..e7b8153004c --- /dev/null +++ b/src/current/_includes/v25.1/cdc/privilege-model.md @@ -0,0 +1,43 @@ +{{site.data.alerts.callout_info}} +Starting in v22.2, CockroachDB introduces a new [system-level privilege model]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) that provides finer control over a user's privilege to work with the database, including creating and managing changefeeds. + +There is continued support for the [legacy privilege model](#legacy-privilege-model) for changefeeds in v23.1, however it **will be removed** in a future release of CockroachDB. We recommend implementing the new privilege model that follows in this section for all changefeeds. +{{site.data.alerts.end}} + +You can [grant]({% link {{ page.version.version }}/grant.md %}#grant-privileges-on-specific-tables-in-a-database) a user the `CHANGEFEED` privilege to allow them to create changefeeds on a specific table: + +{% include_cached copy-clipboard.html %} +~~~sql +GRANT CHANGEFEED ON TABLE example_table TO user; +~~~ + +When you grant a user the `CHANGEFEED` privilege on a set of tables, they can: + +- Create changefeeds on the target tables even if the user does **not** have the [`CONTROLCHANGEFEED` role option]({% link {{ page.version.version }}/alter-role.md %}#role-options) or the `SELECT` privilege on the tables. +- Manage the changefeed jobs running on the tables using the [`SHOW CHANGEFEED JOB`]({% link {{ page.version.version }}/show-jobs.md %}#show-changefeed-jobs), [`PAUSE JOB`]({% link {{ page.version.version }}/pause-job.md %}), [`RESUME JOB`]({% link {{ page.version.version }}/resume-job.md %}), and [`CANCEL JOB`](cancel-job.html) commands. + +These users will be able to create changefeeds, but they will not be able to run a `SELECT` query on that data directly. However, they could still read this data indirectly if they have read access to the [sink]({% link {{ page.version.version }}/changefeed-sinks.md %}). + +{% include {{ page.version.version }}/cdc/ext-conn-cluster-setting.md %} + +### Privilege model + +The following summarizes the operations users can run when they have changefeed privileges on a table: + +Granted privileges | Usage +-------------------+------- +`CHANGEFEED` | Create changefeeds on tables.
Manage changefeed jobs on tables. +`CHANGEFEED` + [`USAGE`]({% link {{ page.version.version }}/create-external-connection.md %}#required-privileges) on external connection | Create changefeeds on tables to an external connection URI.
Manage changefeed jobs on tables.
**Note:** If you need to manage access to changefeed sink URIs, set the `changefeed.permissions.require_external_connection_sink.enabled=true` cluster setting. This will mean that users with these privileges can **only** create changefeeds on external connections. +`SELECT` | Create a sinkless changefeed that emits messages to a SQL client. +**Deprecated** `CONTROLCHANGEFEED` role option + `SELECT` | Create changefeeds on tables. + +You can add `CHANGEFEED` to the user or role's [default privileges]({% link {{ page.version.version }}/security-reference/authorization.md %}#default-privileges) with [`ALTER DEFAULT PRIVILEGES`]({% link {{ page.version.version }}/alter-default-privileges.md %}#grant-default-privileges-to-a-specific-role): + +{% include_cached copy-clipboard.html %} +~~~sql +ALTER DEFAULT PRIVILEGES GRANT CHANGEFEED ON TABLES TO user; +~~~ + +{{site.data.alerts.callout_info}} +Users with the `CONTROLCHANGEFEED` role option must have `SELECT` on each table, even if they are also granted the `CHANGEFEED` privilege. The `CONTROLCHANGEFEED` role option will be deprecated in a future release. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/pts-gc-monitoring.md b/src/current/_includes/v25.1/cdc/pts-gc-monitoring.md new file mode 100644 index 00000000000..11a2c4d1fd0 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/pts-gc-monitoring.md @@ -0,0 +1,6 @@ +You can monitor changefeed jobs for [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) usage. We recommend setting up {% if page.name == "monitor-and-debug-changefeeds.md" %} monitoring {% else %} [monitoring]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}) {% endif %}for the following metrics: + +- `jobs.changefeed.protected_age_sec`: Tracks the age of the oldest [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) record protected by changefeed jobs. We recommend monitoring if `protected_age_sec` is greater than [`gc.ttlseconds`]({% link {{ page.version.version }}/configure-replication-zones.md %}#gc-ttlseconds). As `protected_age_sec` increases, garbage accumulation increases. [Garbage collection]({% link {{ page.version.version }}/architecture/storage-layer.md %}#garbage-collection) will not progress on a table, database, or cluster if the protected timestamp record is present. +- `jobs.changefeed.currently_paused`: Tracks the number of changefeed jobs currently considered [paused]({% link {{ page.version.version }}/pause-job.md %}). Since paused changefeed jobs can accumulate garbage, it is important to [monitor the number of paused changefeeds]({% link {{ page.version.version }}/pause-job.md %}#monitoring-paused-jobs). +- `jobs.changefeed.expired_pts_records`: Tracks the number of expired [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) records owned by changefeed jobs. You can monitor this metric in conjunction with the [`gc_protect_expires_after` option]({% link {{ page.version.version }}/create-changefeed.md %}#gc-protect-expires-after). +- `jobs.changefeed.protected_record_count`: Tracks the number of [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) records held by changefeed jobs. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/recommendation-monitoring-pts.md b/src/current/_includes/v25.1/cdc/recommendation-monitoring-pts.md new file mode 100644 index 00000000000..c802caf5cd3 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/recommendation-monitoring-pts.md @@ -0,0 +1 @@ +Cockroach Labs recommends monitoring your changefeeds to track [retryable errors]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#changefeed-retry-errors) and [protected timestamp]({% link {{ page.version.version }}/architecture/storage-layer.md %}#protected-timestamps) usage. Refer to the [Monitor and Debug Changefeeds]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}) page for more information. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/schedule-query-example.md b/src/current/_includes/v25.1/cdc/schedule-query-example.md new file mode 100644 index 00000000000..7a692f21eb4 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/schedule-query-example.md @@ -0,0 +1,14 @@ +This example creates a nightly export of some filtered table data with a [scheduled changefeed]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}) that will run just after midnight every night. The changefeed uses [CDC queries]({% link {{ page.version.version }}/cdc-queries.md %}) to query the table and filter the data it will send to the sink: + +{% include_cached copy-clipboard.html %} +~~~ sql +CREATE SCHEDULE sf_skateboard FOR CHANGEFEED INTO 'external://cloud-sink' WITH format=csv + AS SELECT current_location, id, type, status FROM vehicles + WHERE city = 'san francisco' AND type = 'skateboard' + RECURRING '1 0 * * *' WITH SCHEDULE OPTIONS on_execution_failure=retry, on_previous_running=start; +~~~ + +The [schedule options]({% link {{ page.version.version }}/create-schedule-for-changefeed.md %}#schedule-options) control the schedule's behavior: + +- If it runs into a failure, `on_execution_failure=retry` will ensure that the schedule retries the changefeed immediately. +- If the previous scheduled changefeed is still running, `on_previous_running=start` will start a new changefeed at the defined cadence. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/schema-registry-metric.md b/src/current/_includes/v25.1/cdc/schema-registry-metric.md new file mode 100644 index 00000000000..b9482feafdc --- /dev/null +++ b/src/current/_includes/v25.1/cdc/schema-registry-metric.md @@ -0,0 +1 @@ +Use the `changefeed.schema_registry.retry_count` metric to measure the number of request retries performed when sending requests to the schema registry. For more detail on monitoring changefeeds, refer to [Monitor and Debug Changefeeds]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}). \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/schema-registry-timeout.md b/src/current/_includes/v25.1/cdc/schema-registry-timeout.md new file mode 100644 index 00000000000..eec8371f282 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/schema-registry-timeout.md @@ -0,0 +1 @@ +Use the {% if page.name == "create-changefeed.md" %} `timeout={duration}` query parameter {% else %} [`timeout={duration}` query parameter]({% link {{ page.version.version }}/create-changefeed.md %}#confluent-schema-registry) {% endif %}([duration string](https://pkg.go.dev/time#ParseDuration)) in your Confluent Schema Registry URI to change the default timeout for contacting the schema registry. By default, the timeout is 30 seconds. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/show-changefeed-job-retention.md b/src/current/_includes/v25.1/cdc/show-changefeed-job-retention.md new file mode 100644 index 00000000000..74706999656 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/show-changefeed-job-retention.md @@ -0,0 +1 @@ +All changefeed jobs will display regardless of if the job completed and when it completed. You can define a retention time and delete completed jobs by using the `jobs.retention_time` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}). \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/show-changefeed-job.md b/src/current/_includes/v25.1/cdc/show-changefeed-job.md new file mode 100644 index 00000000000..1515b2eeb95 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/show-changefeed-job.md @@ -0,0 +1,24 @@ +{% include_cached copy-clipboard.html %} +~~~ sql +SHOW CHANGEFEED JOBS; +~~~ +~~~ + job_id | description | ... ++----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ ... + 685724608744325121 | CREATE CHANGEFEED FOR TABLE mytable INTO 'kafka://localhost:9092' WITH confluent_schema_registry = 'http://localhost:8081', format = 'avro', resolved, updated | ... + 685723987509116929 | CREATE CHANGEFEED FOR TABLE mytable INTO 'kafka://localhost:9092' WITH confluent_schema_registry = 'http://localhost:8081', format = 'avro', resolved, updated | ... +(2 rows) +~~~ + +To show an individual {{ site.data.products.enterprise }} changefeed: + +{% include_cached copy-clipboard.html %} +~~~ sql +SHOW CHANGEFEED JOB {job_id}; +~~~ +~~~ + job_id | description | user_name | status | running_status | created | started | finished | modified | high_water_timestamp | error | sink_uri | full_table_names | topics | format +---------------------+--------------------------------------------------------------------------------------+-----------+---------+------------------------------------------+----------------------------+----------------------------+----------+----------------------------+--------------------------------+-------+----------------+---------------------+--------+---------- + 866218332400680961 | CREATE CHANGEFEED FOR TABLE movr.users INTO 'external://aws' WITH format = 'parquet' | root | running | running: resolved=1684438482.937939878,0 | 2023-05-18 14:14:16.323465 | 2023-05-18 14:14:16.360245 | NULL | 2023-05-18 19:35:16.120407 | 1684438482937939878.0000000000 | | external://aws | {movr.public.users} | NULL | parquet +(1 row) +~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/sink-URI-external-connection.md b/src/current/_includes/v25.1/cdc/sink-URI-external-connection.md new file mode 100644 index 00000000000..90ab96f315f --- /dev/null +++ b/src/current/_includes/v25.1/cdc/sink-URI-external-connection.md @@ -0,0 +1 @@ +You can create an external connection to represent a changefeed sink URI. This allows you to specify the external connection's name in statements rather than the provider-specific URI. For detail on using external connections, see the [`CREATE EXTERNAL CONNECTION`]({% link {{ page.version.version }}/create-external-connection.md %}) page. \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/sink-configuration-detail.md b/src/current/_includes/v25.1/cdc/sink-configuration-detail.md new file mode 100644 index 00000000000..ed96a0ead52 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/sink-configuration-detail.md @@ -0,0 +1,28 @@ +{{site.data.alerts.callout_danger}} +Setting either `Messages` or `Bytes` with a non-zero value without setting `Frequency` will cause the sink to assume `Frequency` has an infinity value. If either `Messages` or `Bytes` have a non-zero value, then a non-zero value for `Frequency` **must** be provided. This configuration is invalid and will cause an error, since the messages could sit in a batch indefinitely if the other conditions do not trigger. +{{site.data.alerts.end}} + +Some complexities to consider when setting `Flush` fields for batching: + +- When all batching parameters are zero (`"Messages"`, `"Bytes"`, and `"Frequency"`) the sink will interpret this configuration as "send batch every time a message is available." This would be the same as not providing any configuration at all: + + ~~~ + { + "Flush": { + "Messages": 0, + "Bytes": 0, + "Frequency": "0s" + } + } + ~~~ + +- If one or more fields are set as non-zero values, any fields with a zero value the sink will interpret as infinity. For example, in the following configuration, the sink will send a batch whenever the size reaches 100 messages, **or**, when 5 seconds has passed since the batch was populated with its first message. `Bytes` is unset, so the batch size is unlimited. No flush will be triggered due to batch size: + + ~~~ + { + "Flush": { + "Messages": 100, + "Frequency": "5s" + } + } + ~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/sink-list.md b/src/current/_includes/v25.1/cdc/sink-list.md new file mode 100644 index 00000000000..6468b3d317c --- /dev/null +++ b/src/current/_includes/v25.1/cdc/sink-list.md @@ -0,0 +1,8 @@ +- {% if page.name == "changefeed-sinks.md" %} [Amazon MSK](#amazon-msk) {% else %} [Amazon MSK]({% link {{ page.version.version }}/changefeed-sinks.md %}#amazon-msk) {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Apache Pulsar](#apache-pulsar) (in Preview) {% else %} [Apache Pulsar]({% link {{ page.version.version }}/changefeed-sinks.md %}#apache-pulsar) (in Preview) {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Azure Event Hubs](#azure-event-hubs) {% else %} [Azure Event Hubs]({% link {{ page.version.version }}/changefeed-sinks.md %}#azure-event-hubs) {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Cloud Storage](#cloud-storage-sink) / HTTP {% else %} [Cloud Storage]({% link {{ page.version.version }}/changefeed-sinks.md %}#cloud-storage-sink) / HTTP {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Confluent Cloud](#confluent-cloud) {% else %} [Confluent Cloud]({% link {{ page.version.version }}/changefeed-sinks.md %}#confluent-cloud) {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Google Cloud Pub/Sub](#google-cloud-pub-sub) {% else %} [Google Cloud Pub/Sub]({% link {{ page.version.version }}/changefeed-sinks.md %}#google-cloud-pub-sub) {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Kafka](#kafka) {% else %} [Kafka]({% link {{ page.version.version }}/changefeed-sinks.md %}#kafka) {% endif %} +- {% if page.name == "changefeed-sinks.md" %} [Webhook](#webhook-sink) {% else %} [Webhook]({% link {{ page.version.version }}/changefeed-sinks.md %}#webhook-sink) {% endif %} \ No newline at end of file diff --git a/src/current/_includes/v25.1/cdc/sql-cluster-settings-example.md b/src/current/_includes/v25.1/cdc/sql-cluster-settings-example.md new file mode 100644 index 00000000000..fa2887967a1 --- /dev/null +++ b/src/current/_includes/v25.1/cdc/sql-cluster-settings-example.md @@ -0,0 +1,27 @@ +1. As the `root` user, open the [built-in SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}): + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach sql --insecure + ~~~ + +1. Set your organization name and [{{ site.data.products.enterprise }} license]({% link {{ page.version.version }}/licensing-faqs.md %}#types-of-licenses) key: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SET CLUSTER SETTING cluster.organization = '
| CockroachDB Metric Name | +{% if feature == "ldr" %}Description{% else %}Description When Aggregated{% endif %} | +Type | +Unit | +
{{ m.child_metric_id }} |
+ {% if metrics-list[0].description == null %}{{ m.description }}{% else %}{{ metrics-list[0].description }}{% endif %} | +{% if metrics-list[0].type == null %}{{ m.type }}{% else %}{{ metrics-list[0].type }}{% endif %} | +{% if metrics-list[0].unit == null %}{{ m.unit }}{% else %}{{ metrics-list[0].unit }}{% endif %} | +
+Set a `DATABASE_URL` environment variable to your connection string.
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+export DATABASE_URL="{connection string}"
+~~~
+
+
+
+
+Set a `DATABASE_URL` environment variable to your connection string.
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+$env:DATABASE_URL = "{connection string}"
+~~~
+
+
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/connect/core-note.md b/src/current/_includes/v25.1/connect/core-note.md
new file mode 100644
index 00000000000..7b701cafb80
--- /dev/null
+++ b/src/current/_includes/v25.1/connect/core-note.md
@@ -0,0 +1,7 @@
+{{site.data.alerts.callout_info}}
+The connection information shown on this page uses [client certificate and key authentication]({% link {{ page.version.version }}/authentication.md %}#client-authentication) to connect to a secure, CockroachDB {{ site.data.products.core }} cluster.
+
+To connect to a CockroachDB {{ site.data.products.core }} cluster with client certificate and key authentication, you must first [generate server and client certificates]({% link {{ page.version.version }}/authentication.md %}#using-digital-certificates-with-cockroachdb).
+
+For instructions on starting a secure cluster, see [Start a Local Cluster (Secure)]({% link {{ page.version.version }}/secure-a-cluster.md %}).
+{{site.data.alerts.end}}
diff --git a/src/current/_includes/v25.1/connect/jdbc-connection-url.md b/src/current/_includes/v25.1/connect/jdbc-connection-url.md
new file mode 100644
index 00000000000..c055a390b4e
--- /dev/null
+++ b/src/current/_includes/v25.1/connect/jdbc-connection-url.md
@@ -0,0 +1,19 @@
+Set a `JDBC_DATABASE_URL` environment variable to your JDBC connection string.
+
+
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+export JDBC_DATABASE_URL="{connection string}"
+~~~
+
+
+
+
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+$env:JDBC_DATABASE_URL = "{connection string}"
+~~~
+
+
diff --git a/src/current/_includes/v25.1/dedicated-pci-compliance.md b/src/current/_includes/v25.1/dedicated-pci-compliance.md
new file mode 100644
index 00000000000..59dd7790daf
--- /dev/null
+++ b/src/current/_includes/v25.1/dedicated-pci-compliance.md
@@ -0,0 +1,7 @@
+{{site.data.alerts.callout_info}}
+CockroachDB {{ site.data.products.dedicated }} clusters comply with the Payment Card Industry Data Security Standard (PCI DSS). Compliance is certified by a PCI Qualified Security Assessor (QSA).
+
+To achieve compliance with PCI DSS on a CockroachDB {{ site.data.products.dedicated }} cluster, you must enable all required features in your CockroachDB {{ site.data.products.cloud }} organization and your cluster, and you must take additional steps to ensure that your organization's applications and procedures comply with PCI DSS. For details, refer to [PCI DSS Compliance in CockroachDB {{ site.data.products.dedicated }} advanced](https://cockroachlabs.com/docs/cockroachcloud/pci-dss.html).
+
+To learn more about achieving PCI DSS compliance with CockroachDB {{ site.data.products.dedicated }}, contact your Cockroach Labs account team.
+{{site.data.alerts.end}}
diff --git a/src/current/_includes/v25.1/demo_movr.md b/src/current/_includes/v25.1/demo_movr.md
new file mode 100644
index 00000000000..5a8a431193a
--- /dev/null
+++ b/src/current/_includes/v25.1/demo_movr.md
@@ -0,0 +1,6 @@
+Start the [MovR database]({% link {{ page.version.version }}/movr.md %}) on a 3-node CockroachDB demo cluster with a larger data set.
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+cockroach demo movr --num-histories 250000 --num-promo-codes 250000 --num-rides 125000 --num-users 12500 --num-vehicles 3750 --nodes 3
+~~~
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/essential-alerts.md b/src/current/_includes/v25.1/essential-alerts.md
new file mode 100644
index 00000000000..f71d81dd2f5
--- /dev/null
+++ b/src/current/_includes/v25.1/essential-alerts.md
@@ -0,0 +1,533 @@
+{% if include.deployment == 'self-hosted' %}
+## Platform
+
+### High CPU
+
+A node with a high CPU utilization, an *overloaded* node, has a limited ability to process the user workload and increases the risks of cluster instability.
+
+**Metric**
+[`sys.cpu.combined.percent-normalized`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-cpu-combined-percent-normalized) +
[`sys.cpu.host.combined.percent-normalized`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-cpu-host-combined-percent-normalized) + +**Rule** +
Set alerts for each node for each of the listed metrics: +
WARNING: Metric greater than `0.80` for `4 hours` +
CRITICAL: Metric greater than `0.90` for `1 hour` + +**Action** + +- Refer to [CPU Usage]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#cpu-usage) and [Workload Concurrency]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#workload-concurrency). + +- In the DB Console, navigate to **Metrics**, [**Hardware** dashboard]({% link {{ page.version.version }}/ui-hardware-dashboard.md %}) for the cluster and check for high values on the [**CPU Percent** graph]({% link {{ page.version.version }}/ui-hardware-dashboard.md %}#cpu-percent) and the [**Host CPU Percent** graph]({% link {{ page.version.version }}/ui-hardware-dashboard.md %}#host-cpu-percent). + +- In the DB Console, navigate to **Metrics**, [**SQL** dashboard]({% link {{ page.version.version }}/ui-sql-dashboard.md %}) for the cluster and check for high values on the [**Active SQL Statements** graph]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#active-sql-statements). This graph shows the true concurrency of the workload, which may exceed the cluster capacity planning guidance of no more than 4 active statements per vCPU or core. + +- A persistently high CPU utilization of all nodes in a CockroachDB cluster suggests the current compute resources may be insufficient to support the user workload's concurrency requirements. If confirmed, the number of processors (vCPUs or cores) in the CockroachDB cluster needs to be adjusted to sustain the required level of workload concurrency. For a prompt resolution, either add cluster nodes or throttle the workload concurrency, for example, by reducing the number of concurrent connections to not exceed 4 active statements per vCPU or core. + +### Hot node (hot spot) + +Unbalanced utilization of CockroachDB nodes in a cluster may negatively affect the cluster's performance and stability, with some nodes getting overloaded while others remain relatively underutilized. + +**Metric** +
[`sys.cpu.combined.percent-normalized`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-cpu-host-combined-percent-normalized) +
[`sys.cpu.host.combined.percent-normalized`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-cpu-host-combined-percent-normalized) + +**Rule** +
Set alerts for each of the listed metrics: +
WARNING: The max CPU utilization across all nodes exceeds the cluster's median CPU utilization by `30` for `2 hours` + +**Action** + +- Refer to [Hot spots]({% link {{ page.version.version }}/performance-recipes.md %}#hot-spots). + +### Node memory utilization + +One node with high memory utilization is a cluster stability risk. High memory utilization is a prelude to a node's [out-of-memory (OOM) crash]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#out-of-memory-oom-crash) — the process is terminated by the OS when the system is critically low on memory. An OOM condition is not expected to occur if a CockroachDB cluster is provisioned and sized per [Cockroach Labs guidance]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#memory-planning). + +**Metric** +
[`sys.rss`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-rss) + +**Rule** +
Set alerts for each node: +
WARNING: `sys.rss` greater than `0.80` for `4 hours` +
CRITICAL: `sys.rss` greater than `0.90` for `1 hour` + +**Action** + +- Provision all CockroachDB VMs or machines with [sufficient RAM]({% link {{ page.version.version }}/recommended-production-settings.md %}#memory). + +### Node storage performance + +Under-configured or under-provisioned disk storage is a common root cause of inconsistent CockroachDB cluster performance and could also lead to cluster instability. Refer to [Disk IOPS]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#disk-iops). + +**Metric** +
[`sys.host.disk.iopsinprogress`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-host-disk-iopsinprogress) + +**Rule** +
WARNING: `sys.host.disk.iopsinprogress` greater than `10` for `10 seconds` +
CRITICAL: `sys.host.disk.iopsinprogress` greater than `20` for `10 seconds` + +**Action** + +- Provision enough storage capacity for CockroachDB data, and configure your volumes to maximize disk I/O. Refer to [Storage and disk I/O]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#storage-and-disk-i-o). + +### Version mismatch + +All CockroachDB cluster nodes should be running the same exact executable (with identical build label). This warning guards against an operational error where some nodes were not upgraded. + +**Metric** +
`build.timestamp` + +**Rule** +
Set alerts for each node: +
WARNING: `build.timestamp` not the same across cluster nodes for more than `4 hours` + +**Action** + +- Ensure all cluster nodes are running exactly the same CockroachDB version, including the patch release version number. + +### High open file descriptor count + +Send an alert when a cluster is getting close to the open file descriptor limit. + +**Metric** +
`sys.fd.open` +
`sys.fd.softlimit` + +**Rule** +
Set alerts for each node: +
WARNING: `sys_fd_open` / `sys_fd_softlimit` greater than `0.8` for `10 minutes` + +**Action** + +- Refer to [File descriptors limit]({% link {{ page.version.version }}/recommended-production-settings.md %}#file-descriptors-limit). +{% endif %} + +## Storage + +### Node storage capacity + +A CockroachDB node will not able to operate if there is no free disk space on a CockroachDB [store]({% link {{ page.version.version }}/cockroach-start.md %}#store) volume. + +**Metric** +
[`capacity`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#capacity) +
[`capacity.available`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#capacity-available) + +**Rule** +
Set alerts for each node: +
WARNING: `capacity.available`/`capacity` is less than `0.30` for `24 hours` +
CRITICAL: `capacity.available`/`capacity` is less than `0.10` for `1 hour` + +**Action** + +- Refer to [Storage Capacity]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#storage-capacity). +- Increase the size of CockroachDB node storage capacity. CockroachDB storage volumes should not be utilized more than 60% (40% free space). +- In a "disk full" situation, you may be able to get a node "unstuck" by removing the [automatically created emergency ballast file]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#automatic-ballast-files). + +{% if include.deployment == 'self-hosted' %} +### Write stalls + +A high `write-stalls` value means CockroachDB is unable to write to a disk in an acceptable time, resulting in CockroachDB facing a disk latency issue and not responding to writes. + +**Metric** +
[`storage.write-stalls`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#storage-write-stalls) + +**Rule** +
Set alerts for each node: +
WARNING: `storage.write-stalls` per minute is greater than or equal to `1` per minute +
CRITICAL: `storage.write-stalls` per second is greater than or equal to `1` per second + +**Action** + +- Refer to [Disk stalls]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#disk-stalls). +{% endif %} + +{% if include.deployment == 'self-hosted' %} +## Health + +### Node restarting too frequently + +Send an alert if a node has restarted more than once in the last 10 minutes. Calculate this using the number of times the `sys.uptime` metric was reset back to zero. + +**Metric** +
[`sys.uptime`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sys-uptime) + +**Rule** +
Set alerts for each node: +
WARNING: `sys.uptime` resets greater than `1` in the last `10 minutes` + +**Action** + +- Refer to [Node process restarts]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#node-process-restarts). + +### Node LSM storage health + +CockroachDB uses the [Pebble]({% link {{ page.version.version }}/architecture/storage-layer.md %}#pebble) storage engine that uses a [Log-structured Merge-tree (LSM tree)]({% link {{ page.version.version }}/architecture/storage-layer.md %}#log-structured-merge-trees) to manage data storage. The health of an LSM tree can be measured by the [*read amplification*]({% link {{ page.version.version }}/architecture/storage-layer.md %}#inverted-lsms), which is the average number of [SST files]({% link {{ page.version.version }}/architecture/storage-layer.md %}#log-structured-merge-trees) being checked per read operation. A value in the single digits is characteristic of a healthy LSM tree. A value in the double, triple, or quadruple digits suggests an [inverted LSM]({% link {{ page.version.version }}/architecture/storage-layer.md %}#inverted-lsms). A node reporting a high read amplification is an indication of a problem on that node that is likely to affect the workload. + +**Metric** +
`rocksdb.read-amplification` + +**Rule** +
Set alerts for each node: +
WARNING: `rocksdb.read-amplification` greater than `50` for `1 hour` +
CRITICAL: `rocksdb.read-amplification` greater than `150` for `15 minutes` + +**Action** + +- Refer to [LSM Health]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#lsm-health). + +## Expiration of license and certificates + +### Enterprise license expiration + +Avoid [license]({% link {{ page.version.version }}/licensing-faqs.md %}#types-of-licenses) expiration to avoid any disruption to feature access. + +**Metric** +
[`seconds.until.enterprise.license.expiry`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#seconds-until-enterprise-license-expiry) + +**Rule** +
WARNING: `seconds.until.enterprise.license.expiry` is greater than `0` and less than `1814400` seconds (3 weeks) +
CRITICAL: `seconds.until.enterprise.license.expiry` is greater than `0` and less than `259200` seconds (3 days) + +**Action** + +[Renew the enterprise license]({% link {{ page.version.version }}/licensing-faqs.md %}#renew-an-expired-license). + +### Security certificate expiration + +Avoid [security certificate]({% link {{ page.version.version }}/cockroach-cert.md %}) expiration. + +**Metric** +
[`security.certificate.expiration.ca`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#security-certificate-expiration-ca) +
[`security.certificate.expiration.client-ca`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#security-certificate-expiration-client-ca) +
[`security.certificate.expiration.ui`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#security-certificate-expiration-ui) +
[`security.certificate.expiration.ui-ca`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#security-certificate-expiration-ui-ca) +
[`security.certificate.expiration.node`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#security-certificate-expiration-node) +
[`security.certificate.expiration.node-client`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#security-certificate-expiration-node-client) + +**Rule** +
Set alerts for each of the listed metrics: +
WARNING: Metric is greater than `0` and less than `1814400` seconds (3 weeks) until enterprise license expiration +
CRITICAL: Metric is greater than `0` and less than `259200` seconds (3 days) until enterprise license expiration + +**Action** + +[Rotate the expiring certificates]({% link {{ page.version.version }}/rotate-certificates.md %}). +{% endif %} + +{% if include.deployment == 'self-hosted' %} +## KV distributed + +{{site.data.alerts.callout_info}} +During [rolling maintenance]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}) or planned cluster resizing, the nodes' state and count will be changing. **Mute KV distributed alerts described in the following sections during routine maintenance procedures** to avoid unnecessary distractions. +{{site.data.alerts.end}} + +### Heartbeat latency + +Monitor the cluster health for early signs of instability. If this metric exceeds 1 second, it is a sign of instability. + +**Metric** +
[`liveness.heartbeatlatency`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#liveness-heartbeatlatency) + +**Rule** +
WARNING: `liveness.heartbeatlatency` greater than `0.5s` +
CRITICAL: `liveness.heartbeatlatency` greater than `3s` + +**Action** + +- Refer to [Node liveness issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#node-liveness-issues). + +### Live node count change + +The liveness checks reported by a node is inconsistent with the rest of the cluster. Number of live nodes in the cluster (will be 0 if this node is not itself live). This is a critical metric that tracks the live nodes in the cluster. + +**Metric** +
[`liveness.livenodes`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#liveness-livenodes) + +**Rule** +
Set alerts for each node: +
WARNING: max(`liveness.livenodes`) for the cluster - min(`liveness.livenodes`) for node > `0` for `2 minutes` +
CRITICAL: max(`liveness.livenodes`) for the cluster - min(`liveness.livenodes`) for node > `0` for `5 minutes` + +**Action** + +- Refer to [Node liveness issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#node-liveness-issues). + +### Intent buildup + +Send an alert when very large transactions are [locking]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#write-intents) millions of keys (rows). A common example is a transaction with a [`DELETE`]({% link {{ page.version.version }}/delete.md %}) that affects a large number of rows. Transactions with an excessively large scope are often inadvertent, perhaps due to a non-selective filter and a specific data distribution that was not anticipated by an application developer. + +Transactions that create a large number of [write intents]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#write-intents) could have a negative effect on the workload's performance. These transactions may create locking contention, thus limiting concurrency. This would reduce throughput, and in extreme cases, lead to stalled workloads. + +**Metric** +
`intentcount` + +**Rule** +
WARNING: `intentcount` greater than 10,000,000 for 2 minutes +
CRITICAL: `intentcount` greater than 10,000,000 for 5 minutes +
For tighter transaction scope scrutiny, lower the `intentcount` threshold that triggers an alert. + +**Action** + +- Identify the large scope transactions that acquire a lot of locks. Consider reducing the scope of large transactions, implementing them as several smaller scope transactions. For example, if the alert is triggered by a large scope `DELETE`, consider "paging" `DELETE`s that target thousands of records instead of millions. This is often the most effective resolution, however it generally means an application level [refactoring]({% link {{ page.version.version }}/bulk-update-data.md %}). +- After reviewing the workload, you may conclude that a possible performance impact of allowing transactions to take a large number of intents is not a concern. For example, a large delete of obsolete, not-in-use data may create no concurrency implications and the elapsed time to execute that transaction may not be impactful. In that case, no response could be a valid way to handle this alert. +{% endif %} + +{% if include.deployment == 'self-hosted' %} +## KV replication + +### Unavailable ranges + +Send an alert when the number of ranges with fewer live replicas than needed for quorum is non-zero for too long. + +**Metric** +
[`ranges.unavailable`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#ranges-unavailable) + +**Rule** +
WARNING: `ranges.unavailable` greater than `0` for `10 minutes` + +**Action** + +- Refer to [Replication issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#replication-issues). + +### Tripped replica circuit breakers + +Send an alert when a replica stops serving traffic due to other replicas being offline for too long. + +**Metric** +
`kv.replica_circuit_breaker.num_tripped_replicas` + +**Rule** +
WARNING: `kv.replica_circuit_breaker.num_tripped_replicas` greater than `0` for `10 minutes` + +**Action** + +- Refer to [Per-replica circuit breakers]({% link {{ page.version.version }}/architecture/replication-layer.md %}#per-replica-circuit-breakers) and [Replication issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#replication-issues). + +### Under-replicated ranges + +Send an alert when the number of ranges with replication below the replication factor is non-zero for too long. + +**Metric** +
[`ranges.underreplicated`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#ranges-underreplicated) + +**Rule** +
WARNING: `ranges.underreplicated` greater than `0` for `1 hour` + +**Action** + +- Refer to [Replication issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#replication-issues). + +### Requests stuck in raft + +Send an alert when requests are taking a very long time in replication. An (evaluated) request has to pass through the replication layer, notably the quota pool and raft. If it fails to do so within a highly permissive duration, the gauge is incremented (and decremented again once the request is either applied or returns an error). A nonzero value indicates range or replica unavailability, and should be investigated. + +**Metric** +
`requests.slow.raft` + +**Rule** +
WARNING: `requests.slow.raft` greater than `0` for `10 minutes` + +**Action** + +- Refer to [Raft]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft) and [Replication issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#replication-issues). +{% endif %} + +## SQL + +### Node not executing SQL + +Send an alert when a node is not executing SQL despite having connections. `sql.conns` shows the number of connections as well as the distribution, or balancing, of connections across cluster nodes. An imbalance can lead to nodes becoming overloaded. + +**Metric** +
[`sql.conns`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sql-conns) +
`sql.query.count` + +**Rule** +
Set alerts for each node: +
WARNING: `sql.conns` greater than `0` while `sql.query.count` equals `0` + +**Action** + +- Refer to [Connection Pooling]({% link {{ page.version.version }}/connection-pooling.md %}). + +### SQL query failure + +Send an alert when the query failure count exceeds a user-determined threshold based on their application's SLA. + +**Metric** +
[`sql.failure.count`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sql-failure-count) + +**Rule** +
WARNING: `sql.failure.count` is greater than a threshold (based on the user’s application SLA) + +**Action** + +- Use the [**Insights** page]({% link {{ page.version.version }}/ui-insights-page.md %}) to find failed executions with their error code to troubleshoot or use application-level logs, if instrumented, to determine the cause of error. + +### SQL queries experiencing high latency + +Send an alert when the query latency exceeds a user-determined threshold based on their application’s SLA. + +**Metric** +
[`sql.service.latency`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sql-service-latency) +
[`sql.conn.latency`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#sql-conn-latency) + +**Rule** +
WARNING: (p99 or p90 of `sql.service.latency` plus average of `sql.conn.latency`) is greater than a threshold (based on the user’s application SLA) + +**Action** + +- Apply the time range of the alert to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate. Use the [**Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}) P90 Latency and P99 latency columns to correlate [statement fingerprints]({% link {{ page.version.version }}/ui-statements-page.md %}#sql-statement-fingerprints) with this alert. + +{% if include.deployment == 'self-hosted' %} +## Backup + +### Backup failure + +While CockroachDB is a distributed product, there is always a need to ensure backups complete. + +**Metric** +
[`schedules.BACKUP.failed`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#schedules-BACKUP-failed) + +**Rule** +
Set alerts for each node: +
WARNING: `schedules.BACKUP.failed` is greater than `0` + +**Action** + +- Refer to [Backup and Restore Monitoring]({% link {{ page.version.version }}/backup-and-restore-monitoring.md %}). +{% endif %} + +## Changefeeds + +{{site.data.alerts.callout_info}} +During [rolling maintenance]({% link {{ page.version.version }}/upgrade-cockroach-version.md %}), [changefeed jobs]({% link {{ page.version.version }}/change-data-capture-overview.md %}) restart following node restarts. **Mute changefeed alerts described in the following sections during routine maintenance procedures** to avoid unnecessary distractions. +{{site.data.alerts.end}} + +### Changefeed failure + +Changefeeds can suffer permanent failures (that the [jobs system]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}) will not try to restart). Any increase in this metric counter should prompt investigative action. + +**Metric** +
[`changefeed.failures`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#changefeed-failures) + +**Rule** +
CRITICAL: If `changefeed.failures` is greater than `0` + +**Action** + +1. If the alert is triggered during cluster maintenance, mute it. Otherwise start investigation with the following query: + + {% include_cached copy-clipboard.html %} + ```sql + SELECT job_id, status, ((high_water_timestamp/1000000000)::INT::TIMESTAMP) - NOW() AS "changefeed latency", created, LEFT(description, 60), high_water_timestamp FROM crdb_internal.jobs WHERE job_type = 'CHANGEFEED' AND status IN ('running', 'paused', 'pause-requested') ORDER BY created DESC; + ``` + +2. If the cluster is not undergoing maintenance, check the health of [sink]({% link {{ page.version.version }}/changefeed-sinks.md %}) endpoints. If the sink is [Kafka]({% link {{ page.version.version }}/changefeed-sinks.md %}#kafka), check for sink connection errors such as `ERROR: connecting to kafka: path.to.cluster:port: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)`. + +### Frequent changefeed restarts + +Changefeeds automatically restart in case of transient errors. However too many restarts outside of a routine maintenance procedure may be due to a systemic condition and should be investigated. + +**Metric** +
[`changefeed.error_retries`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#changefeed-error-retries) + +**Rule** +
WARNING: If `changefeed.error_retries` is greater than `50` for more than `15 minutes` + +**Action** + +- Follow the action for a [changefeed failure](#changefeed-failure). + +### Changefeed falling behind + +Changefeed has fallen behind. This is determined by the end-to-end lag between a committed change and that change applied at the destination. This can be due to cluster capacity or changefeed sink availability. + +**Metric** +
[`changefeed.commit_latency`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#changefeed-commit-latency) + +**Rule** +
WARNING: `changefeed.commit_latency` is greater than `10 minutes` +
CRITICAL: `changefeed.commit_latency` is greater than `15 minutes` + +**Action** + +1. In the DB Console, navigate to **Metrics**, [**Changefeeds** dashboard]({% link {{ page.version.version }}/ui-cdc-dashboard.md %}) for the cluster and check the maximum values on the [**Commit Latency** graph]({% link {{ page.version.version }}/ui-cdc-dashboard.md %}#commit-latency). Alternatively, individual changefeed latency can be verified by using the following SQL query: + + {% include_cached copy-clipboard.html %} + ```sql + SELECT job_id, status, ((high_water_timestamp/1000000000)::INT::TIMESTAMP) - NOW() AS "changefeed latency", created, LEFT(description, 60), high_water_timestamp FROM crdb_internal.jobs WHERE job_type = 'CHANGEFEED' AND status IN ('running', 'paused', 'pause-requested') ORDER BY created DESC; + ``` + +2. Copy the `job_id` for the changefeed job with highest `changefeed latency` and pause the job: + + {% include_cached copy-clipboard.html %} + ```sql + PAUSE JOB 681491311976841286; + ``` + +3. Check the status of the pause request by running the query from step 1. If the job status is `pause-requested`, check again in a few minutes. + +4. After the job is `paused`, resume the job. + + {% include_cached copy-clipboard.html %} + ```sql + RESUME JOB 681491311976841286; + ``` + +5. If the changefeed latency does not progress after these steps due to lack of cluster resources or availability of the changefeed sink, [contact Support](https://support.cockroachlabs.com). + +### Changefeed has been paused a long time + +Changefeed jobs should not be paused for a long time because [the protected timestamp prevents garbage collection]({% link {{ page.version.version }}/protect-changefeed-data.md %}). To protect against an operational error, this alert guards against an inadvertently forgotten pause. + +**Metric** +
[`jobs.changefeed.currently_paused`]({% link {{ page.version.version }}/essential-metrics-{{ include.deployment }}.md %}#changefeed-currently-paused) + +**Rule** +
WARNING: `jobs.changefeed.currently_paused` is greater than `0` for more than `15 minutes` +
CRITICAL: `jobs.changefeed.currently_paused` is greater than `0` for more than `60 minutes` + +**Action** + +1. Check the status of each changefeed using the following SQL query: + + {% include_cached copy-clipboard.html %} + ```sql + SELECT job_id, status, ((high_water_timestamp/1000000000)::INT::TIMESTAMP) - NOW() AS "changefeed latency",created, LEFT(description, 60), high_water_timestamp FROM crdb_internal.jobs WHERE job_type = 'CHANGEFEED' AND status IN ('running', 'paused','pause-requested') ORDER BY created DESC; + ``` + +2. If all the changefeeds have status as `running`, one or more changefeeds may have run into an error and recovered. In the DB Console, navigate to **Metrics**, [**Changefeeds** dashboard]({% link {{ page.version.version }}/ui-cdc-dashboard.md %}) for the cluster and check the [**Changefeed Restarts** graph]({% link {{ page.version.version }}/ui-cdc-dashboard.md %}#changefeed-restarts). + +3. Resume paused changefeed(s) with the `job_id` using: + + {% include_cached copy-clipboard.html %} + ```sql + RESUME JOB 681491311976841286; + ``` + +### Changefeed experiencing high latency + +Send an alert when the maximum latency of any running changefeed exceeds a specified threshold, which is less than the [`gc.ttlseconds`]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-zone-variables) variable set in the cluster. This alert ensures that the changefeed progresses faster than the garbage collection TTL, preventing a changefeed's protected timestamp from delaying garbage collection. + +**Metric** +
[`changefeed.checkpoint_progress`]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#metrics) + +**Rule** +
WARNING: (current time minus `changefeed.checkpoint_progress`) is greater than a threshold (that is less than `gc.ttlseconds` variable) + +**Action** + +- Refer to [Monitor and Debug Changefeeds]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#recommended-changefeed-metrics-to-track). + +## See also + +- [Events to alert on]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#events-to-alert-on) +- [Common Issues to Monitor]({% link {{ page.version.version }}/common-issues-to-monitor.md %}) +{% if include.deployment == 'self-hosted' %} +- [Essential Metrics for CockroachDB Self-Hosted Deployments]({% link {{ page.version.version }}/essential-metrics-self-hosted.md %}) +{% elsif include.deployment == 'advanced' %} +- [Essential Metrics for CockroachDB Advanced Deployments]({% link {{ page.version.version }}/essential-metrics-advanced.md %}) +{% endif %} + diff --git a/src/current/_includes/v25.1/essential-metrics.md b/src/current/_includes/v25.1/essential-metrics.md new file mode 100644 index 00000000000..41039d31a75 --- /dev/null +++ b/src/current/_includes/v25.1/essential-metrics.md @@ -0,0 +1,197 @@ +These essential CockroachDB metrics enable you to build custom dashboards with the following tools: +{% if include.deployment == 'self-hosted' %} +* [Grafana]({% link {{ page.version.version }}/monitor-cockroachdb-with-prometheus.md %}#step-5-visualize-metrics-in-grafana) +* [Datadog Integration]({% link {{ page.version.version }}/datadog.md %}) - The [**Datadog Integration Metric Name**](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics) column lists the corresponding Datadog metric which requires the `cockroachdb.` prefix. +{% elsif include.deployment == 'advanced' %} +* [Datadog integration]({% link cockroachcloud/tools-page.md %}#monitor-cockroachdb-cloud-with-datadog) - The [**Datadog Integration Metric Name**](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics) column lists the corresponding Datadog metric which requires the `crdb_dedicated.` prefix. +* [Metrics export]({% link cockroachcloud/export-metrics-advanced.md %}) +{% endif %} + +The **Usage** column explains why each metric is important to visualize in a custom dashboard and how to make both practical and actionable use of the metric in a production deployment. + +## Platform + +|
CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| sys.cpu.combined.percent-normalized | sys.cpu.combined.percent.normalized | Current user+system CPU percentage consumed by the CRDB process, normalized by number of cores | This metric gives the CPU utilization percentage by the CockroachDB process. If it is equal to 1 (or 100%), then the CPU is overloaded. The CockroachDB process should not be running with over 80% utilization for extended periods of time (hours). This metric is used in the DB Console [**CPU Percent** graph]({% link {{ page.version.version }}/ui-hardware-dashboard.md %}#cpu-percent). |
+| sys.cpu.host.combined.percent-normalized | NOT AVAILABLE | Current user+system CPU percentage consumed by all processes on the host OS, normalized by number of cores. If the CRDB process is run in a containerized environment, the host OS is the container since the CRDB process cannot inspect CPU usage beyond the container. | This metric gives the CPU utilization percentage of the underlying server, virtual server or container hosting the CockroachDB process. It includes CockroachDB process and non-CockroachDB process usage. If it is equal to 1 (or 100%), then the CPU is overloaded. The CockroachDB process should not be running in an environment with an overloaded state for extended periods of time (hours). This metric is used in the DB Console **Host CPU Percent** graph. |
+| sys.cpu.user.percent | sys.cpu.user.percent | Current user CPU percentage consumed by the CRDB process | This metric gives the CPU usage percentage at the user level by the CockroachDB process only. This is similar to the Linux `top` command output. The metric value can be more than 1 (or 100%) on multi-core systems. It is best to combine user and system metrics. |
+| sys.cpu.sys.percent | sys.cpu.sys.percent | Current system CPU percentage consumed by the CRDB process | This metric gives the CPU usage percentage at the system (Linux kernel) level by the CockroachDB process only. This is similar to the Linux `top` command output. The metric value can be more than 1 (or 100%) on multi-core systems. It is best to combine user and system metrics. |
+| sys.rss | sys.rss | Current process memory (RSS) | This metric gives the amount of RAM used by the CockroachDB process. Persistently low values over an extended period of time suggest there is underutilized memory that can be put to work with adjusted [settings for `--cache` or `--max_sql_memory`]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size) or both. Conversely, a high utilization, even if a temporary spike, indicates an increased risk of [Out-of-memory (OOM) crash]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#out-of-memory-oom-crash) (particularly since the [swap is generally disabled]({% link {{ page.version.version }}/recommended-production-settings.md %}#memory)). |
+| sql.mem.root.current | {% if include.deployment == 'self-hosted' %}sql.mem.root.current |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Current sql statement memory usage for root | This metric shows how memory set aside for temporary materializations, such as hash tables and intermediary result sets, is utilized. Use this metric to optimize memory allocations based on long term observations. The maximum amount is set with [`--max_sql_memory`]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size). If the utilization of sql memory is persistently low, perhaps some portion of this memory allocation can be shifted to [`--cache`]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size). |
+| sys.host.disk.write.bytes | {% if include.deployment == 'self-hosted' %}sys.host.disk.write.bytes |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Bytes written to all disks since this process started | This metric reports the effective storage device write throughput (MB/s) rate. To confirm that storage is sufficiently provisioned, assess the I/O performance rates (IOPS and MBPS) in the context of the sys.host.disk.iopsinprogress metric. |
+| sys.host.disk.write.count | {% if include.deployment == 'self-hosted' %}sys.host.disk.write |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Disk write operations across all disks since this process started | This metric reports the effective storage device write IOPS rate. To confirm that storage is sufficiently provisioned, assess the I/O performance rates (IOPS and MBPS) in the context of the sys.host.disk.iopsinprogress metric. |
+| sys.host.disk.read.bytes | {% if include.deployment == 'self-hosted' %}sys.host.disk.read.bytes |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Bytes read from all disks since this process started | This metric reports the effective storage device read throughput (MB/s) rate. To confirm that storage is sufficiently provisioned, assess the I/O performance rates (IOPS and MBPS) in the context of the sys.host.disk.iopsinprogress metric. |
+| sys.host.disk.read.count | {% if include.deployment == 'self-hosted' %}sys.host.disk.read |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Disk read operations across all disks since this process started | This metric reports the effective storage device read IOPS rate. To confirm that storage is sufficiently provisioned, assess the I/O performance rates (IOPS and MBPS) in the context of the sys.host.disk.iopsinprogress metric. |
+| sys.host.disk.iopsinprogress | {% if include.deployment == 'self-hosted' %}sys.host.disk.iopsinprogress |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} IO operations currently in progress on this host | This metric gives the average queue length of the storage device. It characterizes the storage device's performance capability. All I/O performance metrics are Linux counters and correspond to the `avgqu-sz` in the Linux `iostat` command output. You need to view the device queue graph in the context of the actual read/write IOPS and MBPS metrics that show the actual device utilization. If the device is not keeping up, the queue will grow. Values over 10 are bad. Values around 5 mean the device is working hard trying to keep up. For internal (on chassis) [NVMe](https://www.wikipedia.org/wiki/NVM_Express) devices, the queue values are typically 0. For network connected devices, such as [AWS EBS volumes](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html), the normal operating range of values is 1 to 2. Spikes in values are OK. They indicate an I/O spike where the device fell behind and then caught up. End users may experience inconsistent response times, but there should be no cluster stability issues. If the queue is greater than 5 for an extended period of time and IOPS or MBPS are low, then the storage is most likely not provisioned per Cockroach Labs guidance. In AWS EBS, it is commonly an EBS type, such as gp2, not suitable as database primary storage. If I/O is low and the queue is low, the most likely scenario is that the CPU is lacking and not driving I/O. One such case is a cluster with nodes with only 2 vcpus which is not supported [sizing]({% link {{ page.version.version }}/recommended-production-settings.md %}#sizing) for production deployments. There are quite a few background processes in the database that take CPU away from the workload, so the workload is just not getting the CPU. Review [storage and disk I/O]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#storage-and-disk-i-o). |
+| sys.host.net.recv.bytes | sys.host.net.recv.bytes | Bytes received on all network interfaces since this process started | This metric gives the node's ingress/egress network transfer rates for flat sections which may indicate insufficiently provisioned networking or high error rates. CockroachDB is using a reliable TCP/IP protocol, so errors result in delivery retries that create a "slow network" effect. |
+| sys.host.net.send.bytes | sys.host.net.send.bytes | Bytes sent on all network interfaces since this process started | This metric gives the node's ingress/egress network transfer rates for flat sections which may indicate insufficiently provisioned networking or high error rates. CockroachDB is using a reliable TCP/IP protocol, so errors result in delivery retries that create a "slow network" effect. |
+| clock-offset.meannanos | clock.offset.meannanos | Mean clock offset with other nodes | This metric gives the node's clock skew. In a well-configured environment, the actual clock skew would be in the sub-millisecond range. A skew exceeding 5 ms is likely due to a NTP service mis-configuration. Reducing the actual clock skew reduces the probability of uncertainty related conflicts and corresponding retires which has a positive impact on workload performance. Conversely, a larger actual clock skew increases the probability of retries due to uncertainty conflicts, with potentially measurable adverse effects on workload performance. |
+
+## Storage
+
+CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| capacity | {% if include.deployment == 'self-hosted' %}capacity.total |{% elsif include.deployment == 'advanced' %}capacity |{% endif %} Total storage capacity | This metric gives total storage capacity. Measurements should comply with the following rule: CockroachDB storage volumes should not be utilized more than 60% (40% free space). |
+| capacity.available | capacity.available | Available storage capacity | This metric gives available storage capacity. Measurements should comply with the following rule: CockroachDB storage volumes should not be utilized more than 60% (40% free space). |
+| capacity.used | capacity.used | Used storage capacity | This metric gives used storage capacity. Measurements should comply with the following rule: CockroachDB storage volumes should not be utilized more than 60% (40% free space). |
+| storage.write-stalls | {% if include.deployment == 'self-hosted' %}storage.write.stalls |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of instances of intentional write stalls to backpressure incoming writes | This metric reports actual disk stall events. Ideally, investigate all reports of disk stalls. As a pratical guideline, one stall per minute is not likely to have a material impact on workload beyond an occasional increase in response time. However one stall per second should be viewed as problematic and investigated actively. It is particularly problematic if the rate persists over an extended period of time, and worse, if it is increasing. |
+| rocksdb.compactions | rocksdb.compactions.total | Number of SST compactions | This metric reports the number of a node's [LSM compactions]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#lsm-health). If the number of compactions remains elevated while the LSM health does not improve, compactions are not keeping up with the workload. If the condition persists for an extended period, the cluster will initially exhibit performance issues that will eventually escalate into stability issues. |
+| rocksdb.block.cache.hits | rocksdb.block.cache.hits | Count of block cache hits | This metric gives hits to block cache which is reserved memory. It is allocated upon the start of a node process by the [`--cache` flag]({% link {{ page.version.version }}/cockroach-start.md %}#general) and never shrinks. By observing block cache hits and misses, you can fine-tune memory allocations in the node process for the demands of the workload. |
+| rocksdb.block.cache.misses | rocksdb.block.cache.misses | Count of block cache misses | This metric gives misses to block cache which is reserved memory. It is allocated upon the start of a node process by the [`--cache` flag]({% link {{ page.version.version }}/cockroach-start.md %}#general) and never shrinks. By observing block cache hits and misses, you can fine-tune memory allocations in the node process for the demands of the workload. |
+
+## Health
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| sys.uptime | sys.uptime | Process uptime | This metric measures the length of time, in seconds, that the CockroachDB process has been running. Monitor this metric to detect events such as node restarts, which may require investigation or intervention. |
+| admission.io.overload | {% if include.deployment == 'self-hosted' %}admission.io.overload |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} 1-normalized float indicating whether IO admission control considers the store as overloaded with respect to compaction out of L0 (considers sub-level and file counts). | If the value of this metric exceeds 1, then it indicates overload. You can also look at the metrics `storage.l0-num-files`, `storage.l0-sublevels` or `rocksdb.read-amplification` directly. A healthy LSM shape is defined as “read-amp < 20” and “L0-files < 1000”, looking at [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}) `admission.l0_sub_level_count_overload_threshold` and `admission.l0_file_count_overload_threshold` respectively. |
+| admission.wait_durations.kv-p75 | {% if include.deployment == 'self-hosted' %}admission.wait.durations.kv |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Wait time durations for requests that waited | This metric shows if CPU utilization-based admission control feature is working effectively or potentially overaggressive. This is a latency histogram of how much delay was added to the workload due to throttling by CPU control. If observing over 100ms waits for over 5 seconds while there was excess CPU capacity available, then the admission control is overly aggressive. |
+| admission.wait_durations.kv-stores-p75 | {% if include.deployment == 'self-hosted' %}admission.wait.durations.kv_stores |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Wait time durations for requests that waited | This metric shows if I/O utilization-based admission control feature is working effectively or potentially overaggressive. This is a latency histogram of how much delay was added to the workload due to throttling by I/O control. If observing over 100ms waits for over 5 seconds while there was excess I/O capacity available, then the admission control is overly aggressive. |
+| sys.runnable.goroutines.per.cpu | {% if include.deployment == 'self-hosted' %}sys.runnable.goroutines.per_cpu |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Average number of goroutines that are waiting to run, normalized by number of cores | If this metric has a value over 30, it indicates a CPU overload. If the condition lasts a short period of time (a few seconds), the database users are likely to experience inconsistent response times. If the condition persists for an extended period of time (tens of seconds, or minutes) the cluster may start developing stability issues. Review [CPU planning]({% link {{ page.version.version }}/common-issues-to-monitor.md %}#cpu).
+
+{% if include.deployment == 'self-hosted' %}
+## Network
+
+| CockroachDB Metric Name
| [Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|(add `cockroachdb.` prefix)
Description
| Usage |
+| ------------------------------------------------------ | --------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
+| rpc.connection.avg_round_trip_latency | rpc.connection.avg_round_trip_latency | Sum of exponentially weighted moving average of round-trip latencies, as measured through a gRPC RPC. Dividing this gauge by `rpc.connection.healthy` gives an approximation of average latency, but the top-level round-trip-latency histogram is more useful. Instead, users should consult the label families of this metric if they are available (which requires Prometheus and the cluster setting `server.child_metrics.enabled`); these provide per-peer moving averages. This metric does not track failed connection. A failed connection's contribution is reset to zero. | This metric is helpful in understanding general network issues outside of CockroachDB that could be impacting the user’s workload. |
+| rpc.connection.failures | rpc.connection.failures.count | Counter of failed connections. This includes both the event in which a healthy connection terminates as well as unsuccessful reconnection attempts. Connections that are terminated as part of local node shutdown are excluded. Decommissioned peers are excluded. | See Description. |
+| rpc.connection.healthy | rpc.connection.healthy | Gauge of current connections in a healthy state (i.e., bidirectionally connected and heartbeating). | See Description. |
+| rpc.connection.healthy_nanos | rpc.connection.healthy_nanos | Gauge of nanoseconds of healthy connection time. On the Prometheus endpoint scraped when the cluster setting `server.child_metrics.enabled` is set, this gauge allows you to see the duration for which a given peer has been connected in a healthy state. | This can be useful for monitoring the stability and health of connections within your CockroachDB cluster. |
+| rpc.connection.heartbeats | rpc.connection.heartbeats.count | Counter of successful heartbeats. | See Description. |
+| rpc.connection.unhealthy | rpc.connection.unhealthy | Gauge of current connections in an unhealthy state (not bidirectionally connected or heartbeating). | If the value of this metric is greater than 0, this could indicate a network partition. |
+| rpc.connection.unhealthy_nanos | rpc.connection.unhealthy_nanos | Gauge of nanoseconds of unhealthy connection time. On the Prometheus endpoint scraped when the cluster setting `server.child_metrics.enabled` is set, this gauge allows you to see the duration for which a given peer has been unreachable. | If this duration is greater than 0, this could indicate how long a network partition has been occurring. |
+{% endif %}
+
+{% if include.deployment == 'self-hosted' %}
+## Expiration of license and certificates
+
+| CockroachDB Metric Name
| [Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|(add `cockroachdb.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | ---------------------------------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ |
+| seconds.until.enterprise.license.expiry | seconds.until.enterprise.license.expiry | Seconds until enterprise license expiry (0 if no license present or running without enterprise features) | See Description. |
+| security.certificate.expiration.ca | security.certificate_expiration.ca | Expiration for the CA certificate. 0 means no certificate or error | See Description. |
+| security.certificate.expiration.client-ca | security.certificate_expiration.client_ca | Expiration for the client CA certificate. 0 means no certificate or error| See Description. |
+| security.certificate.expiration.ui | security.certificate_expiration.ui | Expiration for the UI certificate. 0 means no certificate or error| See Description. |
+| security.certificate.expiration.ui-ca | security.certificate_expiration.ui_ca | Expiration for the UI CA certificate. 0 means no certificate or error| See Description. |
+| security.certificate.expiration.node | security.certificate_expiration.node | Expiration for the node certificate. 0 means no certificate or error| See Description. |
+| security.certificate.expiration.node-client | security.certificate_expiration.node_client | Expiration for the node's client certificate. 0 means no certificate or error| See Description. |
+{% endif %}
+
+## KV distributed
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| liveness.heartbeatlatency | {% if include.deployment == 'self-hosted' %}liveness.heartbeatlatency-p90 |{% elsif include.deployment == 'advanced' %}liveness.heartbeatlatency |{% endif %} Node liveness heartbeat latency | If this metric exceeds 1 second, it is a sign of cluster instability. |
+| liveness.livenodes | liveness.livenodes | Number of live nodes in the cluster (will be 0 if this node is not itself live) | This is a critical metric that tracks the live nodes in the cluster. |
+| distsender.rpc.sent.nextreplicaerror | distsender.rpc.sent.nextreplicaerror | Number of replica-addressed RPCs sent due to per-replica errors | [RPC](architecture/overview.html#overview) errors do not necessarily indicate a problem. This metric tracks remote procedure calls that return a status value other than "success". A non-success status of an RPC should not be misconstrued as a network transport issue. It is database code logic executed on another cluster node. The non-success status is a result of an orderly execution of an RPC that reports a specific logical condition. |
+| distsender.errors.notleaseholder | distsender.errors.notleaseholder | Number of NotLeaseHolderErrors encountered from replica-addressed RPCs | Errors of this type are normal during elastic cluster topology changes when leaseholders are actively rebalancing. They are automatically retried. However they may create occasional response time spikes. In that case, this metric may provide the explanation of the cause. |
+
+## KV replication
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| leases.transfers.success | leases.transfers.success | Number of successful lease transfers | A high number of [lease](architecture/replication-layer.html#leases) transfers is not a negative or positive signal, rather it is a reflection of the elastic cluster activities. For example, this metric is high during cluster topology changes. A high value is often the reason for NotLeaseHolderErrors which are normal and expected during rebalancing. Observing this metric may provide a confirmation of the cause of such errors. |
+| rebalancing.queriespersecond | {% if include.deployment == 'self-hosted' %}rebalancing.queriespersecond |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of kv-level requests received per second by the store, considering the last 30 minutes, as used in rebalancing decisions. | This metric shows hotspots along the queries per second (QPS) dimension. It provides insights into the ongoing rebalancing activities. |
+| ranges | ranges | Number of ranges | This metric provides a measure of the scale of the data size. |
+| replicas | {% if include.deployment == 'self-hosted' %}replicas.total |{% elsif include.deployment == 'advanced' %}replicas |{% endif %} Number of replicas | This metric provides an essential characterization of the data distribution across cluster nodes. |
+| replicas.leaseholders | replicas.leaseholders | Number of lease holders | This metric provides an essential characterization of the data processing points across cluster nodes. |
+| ranges.underreplicated | ranges.underreplicated | Number of ranges with fewer live replicas than the replication target | This metric is an indicator of [replication issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#replication-issues). It shows whether the cluster has data that is not conforming to resilience goals. The next step is to determine the corresponding database object, such as the table or index, of these under-replicated ranges and whether the under-replication is temporarily expected. Use the statement `SELECT table_name, index_name FROM [SHOW RANGES WITH INDEXES] WHERE range_id = {id of under-replicated range};`|
+| ranges.unavailable | ranges.unavailable | Number of ranges with fewer live replicas than needed for quorum | This metric is an indicator of [replication issues]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#replication-issues). It shows whether the cluster is unhealthy and can impact workload. If an entire range is unavailable, then it will be unable to process queries. |
+| queue.replicate.replacedecommissioningreplica.error | {% if include.deployment == 'self-hosted' %}queue.replicate.replacedecommissioningreplica.error.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of failed decommissioning replica replacements processed by the replicate queue | Refer to [Decommission the node]({% link {{ page.version.version }}/node-shutdown.md %}?filters=decommission#decommission-the-node). |
+| range.splits | {% if include.deployment == 'self-hosted' %}range.splits.total |{% elsif include.deployment == 'advanced' %}range.splits |{% endif %} Number of range splits | This metric indicates how fast a workload is scaling up. Spikes can indicate resource hot spots since the [split heuristic is based on QPS]({% link {{ page.version.version }}/load-based-splitting.md %}#control-load-based-splitting-threshold). To understand whether hot spots are an issue and with which tables and indexes they are occurring, correlate this metric with other metrics such as CPU usage, such as `sys.cpu.combined.percent-normalized`, or use the [**Hot Ranges** page]({% link {{ page.version.version }}/ui-hot-ranges-page.md %}). |
+| range.merges | {% if include.deployment == 'self-hosted' %}range.merges.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of range merges | This metric indicates how fast a workload is scaling down. Merges are Cockroach's [optimization for performance](architecture/distribution-layer.html#range-merges). This metric indicates that there have been deletes in the workload. |
+
+## SQL
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| sql.conns | sql.conns | Number of active SQL connections | This metric shows the number of connections as well as the distribution, or balancing, of connections across cluster nodes. An imbalance can lead to nodes becoming overloaded. Review [Connection Pooling]({% link {{ page.version.version }}/connection-pooling.md %}). |
+| sql.new_conns | {% if include.deployment == 'self-hosted' %}sql.new_conns.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of new connection attempts. | The rate of this metric shows how frequently new connections are being established. This can be useful in determining if a high rate of incoming new connections is causing additional load on the server due to a misconfigured application. |
+| sql.txns.open | sql.txns.open | Number of currently open user SQL transactions | This metric should roughly correspond to the number of cores * 4. If this metric is consistently larger, scale out the cluster. |
+| sql.statements.active | sql.statements.active | Number of currently active user SQL statements | This high-level metric reflects workload volume. |
+| sql.failure.count | {% if include.deployment == 'self-hosted' %}sql.failure |{% elsif include.deployment == 'advanced' %}sql.failure.count |{% endif %} Number of statements resulting in a planning or runtime error | This metric is a high-level indicator of workload and application degradation with query failures. Use the [Insights page]({% link {{ page.version.version }}/ui-insights-page.md %}) to find failed executions with their error code to troubleshoot or use application-level logs, if instrumented, to determine the cause of error. |
+| sql.full.scan.count | {% if include.deployment == 'self-hosted' %}sql.full.scan |{% elsif include.deployment == 'advanced' %}sql.full.scan.count |{% endif %} Number of full table or index scans | This metric is a high-level indicator of potentially suboptimal query plans in the workload that may require index tuning and maintenance. To identify the [statements with a full table scan]({% link {{ page.version.version }}/performance-recipes.md %}#statements-with-full-table-scans), use `SHOW FULL TABLE SCAN` or the [**SQL Activity Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}) with the corresponding metric time frame. The **Statements** page also includes [explain plans]({% link {{ page.version.version }}/ui-statements-page.md %}#explain-plans) and [index recommendations]({% link {{ page.version.version }}/ui-statements-page.md %}#insights). Not all full scans are necessarily bad especially over smaller tables. |
+| sql.insert.count | sql.insert.count | Number of SQL INSERT statements successfully executed | This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate interesting outliers or patterns. For example, on the [**Transactions** page]({% link {{ page.version.version }}/ui-transactions-page.md %}) and the [**Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}), sort on the Execution Count column. To find problematic sessions, on the [**Sessions** page]({% link {{ page.version.version }}/ui-sessions-page.md %}), sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application. |
+| sql.update.count | sql.update.count | Number of SQL UPDATE statements successfully executed | This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate interesting outliers or patterns. For example, on the [**Transactions** page]({% link {{ page.version.version }}/ui-transactions-page.md %}) and the [**Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}), sort on the Execution Count column. To find problematic sessions, on the [**Sessions** page]({% link {{ page.version.version }}/ui-sessions-page.md %}), sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application. |
+| sql.delete.count | sql.delete.count | Number of SQL DELETE statements successfully executed | This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate interesting outliers or patterns. For example, on the [**Transactions** page]({% link {{ page.version.version }}/ui-transactions-page.md %}) and the [**Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}), sort on the Execution Count column. To find problematic sessions, on the [**Sessions** page]({% link {{ page.version.version }}/ui-sessions-page.md %}), sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application. |
+| sql.select.count | sql.select.count | Number of SQL SELECT statements successfully executed | This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate interesting outliers or patterns. For example, on the [**Transactions** page]({% link {{ page.version.version }}/ui-transactions-page.md %}) and the [**Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}), sort on the Execution Count column. To find problematic sessions, on the [**Sessions** page]({% link {{ page.version.version }}/ui-sessions-page.md %}), sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application. |
+| sql.ddl.count | sql.ddl.count | Number of SQL DDL statements successfully executed | This high-level metric reflects workload volume. Monitor this metric to identify abnormal application behavior or patterns over time. If abnormal patterns emerge, apply the metric's time range to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate interesting outliers or patterns. For example, on the [**Transactions** page]({% link {{ page.version.version }}/ui-transactions-page.md %}) and the [**Statements** page]({% link {{ page.version.version }}/ui-statements-page.md %}), sort on the Execution Count column. To find problematic sessions, on the [**Sessions** page]({% link {{ page.version.version }}/ui-sessions-page.md %}), sort on the Transaction Count column. Find the sessions with high transaction counts and trace back to a user or application. |
+| sql.txn.begin.count | sql.txn.begin.count | Number of SQL transaction BEGIN statements successfully executed | This metric reflects workload volume by counting explicit [transactions]({% link {{ page.version.version }}/transactions.md %}). Use this metric to determine whether explicit transactions can be refactored as implicit transactions (individual statements). |
+| sql.txn.commit.count | sql.txn.commit.count | Number of SQL transaction COMMIT statements successfully executed | This metric shows the number of [transactions]({% link {{ page.version.version }}/transactions.md %}) that completed successfully. This metric can be used as a proxy to measure the number of successful explicit transactions. |
+| sql.txn.rollback.count | sql.txn.rollback.count | Number of SQL transaction ROLLBACK statements successfully executed | This metric shows the number of orderly transaction [rollbacks]({% link {{ page.version.version }}/rollback-transaction.md %}). A persistently high number of rollbacks may negatively impact the workload performance and needs to be investigated. |
+| sql.txn.abort.count | sql.txn.abort.count | Number of SQL transaction abort errors | This high-level metric reflects workload performance. A persistently high number of SQL transaction abort errors may negatively impact the workload performance and needs to be investigated. |
+| sql.service.latency-p90, sql.service.latency-p99 | sql.service.latency | Latency of SQL request execution | These high-level metrics reflect workload performance. Monitor these metrics to understand latency over time. If abnormal patterns emerge, apply the metric's time range to the [**SQL Activity** pages]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#sql-activity-pages) to investigate interesting outliers or patterns. The [**Statements page**]({% link {{ page.version.version }}/ui-statements-page.md %}) has P90 Latency and P99 latency columns to enable correlation with this metric. |
+| sql.txn.latency-p90, sql.txn.latency-p99 | sql.txn.latency | Latency of SQL transactions | These high-level metrics provide a latency histogram of all executed SQL transactions. These metrics provide an overview of the current SQL workload. |
+| txnwaitqueue.deadlocks_total | {% if include.deployment == 'self-hosted' %}txnwaitqueue.deadlocks.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of deadlocks detected by the transaction wait queue | Alert on this metric if its value is greater than zero, especially if transaction throughput is lower than expected. Applications should be able to detect and recover from deadlock errors. However, transaction performance and throughput can be maximized if the application logic avoids deadlock conditions in the first place, for example, by keeping transactions as short as possible. |
+| sql.distsql.contended_queries.count | {% if include.deployment == 'self-hosted' %}sql.distsql.contended.queries |{% elsif include.deployment == 'advanced' %} sql.distsql.contended.queries |{% endif %} Number of SQL queries that experienced contention | This metric is incremented whenever there is a non-trivial amount of contention experienced by a statement whether read-write or write-write conflicts. Monitor this metric to correlate possible workload performance issues to contention conflicts. |
+| sql.conn.latency-p90, sql.conn.latency-p99 | sql.conn.latency | Latency to establish and authenticate a SQL connection | These metrics characterize the database connection latency which can affect the application performance, for example, by having slow startup times. |
+| txn.restarts.serializable | txn.restarts.serializable | Number of restarts due to a forwarded commit timestamp and isolation=SERIALIZABLE | This metric is one measure of the impact of contention conflicts on workload performance. For guidance on contention conflicts, review [transaction contention best practices]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention) and [performance tuning recipes]({% link {{ page.version.version }}/performance-recipes.md %}#transaction-contention). Tens of restarts per minute may be a high value, a signal of an elevated degree of contention in the workload, which should be investigated. |
+| txn.restarts.writetooold | txn.restarts.writetooold | Number of restarts due to a concurrent writer committing first | This metric is one measure of the impact of contention conflicts on workload performance. For guidance on contention conflicts, review [transaction contention best practices]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention) and [performance tuning recipes]({% link {{ page.version.version }}/performance-recipes.md %}#transaction-contention). Tens of restarts per minute may be a high value, a signal of an elevated degree of contention in the workload, which should be investigated. |
+| txn.restarts.writetoooldmulti | {% if include.deployment == 'self-hosted' %}txn.restarts.writetoooldmulti.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of restarts due to multiple concurrent writers committing first | This metric is one measure of the impact of contention conflicts on workload performance. For guidance on contention conflicts, review [transaction contention best practices]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention) and [performance tuning recipes]({% link {{ page.version.version }}/performance-recipes.md %}#transaction-contention). Tens of restarts per minute may be a high value, a signal of an elevated degree of contention in the workload, which should be investigated. |
+| txn.restarts.unknown | {% if include.deployment == 'self-hosted' %}txn.restarts.unknown.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of restarts due to a unknown reasons | This metric is one measure of the impact of contention conflicts on workload performance. For guidance on contention conflicts, review [transaction contention best practices]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention) and [performance tuning recipes]({% link {{ page.version.version }}/performance-recipes.md %}#transaction-contention). Tens of restarts per minute may be a high value, a signal of an elevated degree of contention in the workload, which should be investigated. |
+| txn.restarts.txnpush | {% if include.deployment == 'self-hosted' %}txn.restarts.txnpush.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of restarts due to a transaction push failure | This metric is one measure of the impact of contention conflicts on workload performance. For guidance on contention conflicts, review [transaction contention best practices]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention) and [performance tuning recipes]({% link {{ page.version.version }}/performance-recipes.md %}#transaction-contention). Tens of restarts per minute may be a high value, a signal of an elevated degree of contention in the workload, which should be investigated. |
+| txn.restarts.txnaborted | {% if include.deployment == 'self-hosted' %}txn.restarts.txnaborted.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of restarts due to an abort by a concurrent transaction | The errors tracked by this metric are generally due to deadlocks. Deadlocks can often be prevented with a considered transaction design. Identify the conflicting transactions involved in the deadlocks, then, if possible, redesign the business logic implementation prone to deadlocks. |
+
+## Table Statistics
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| jobs.auto_create_stats.resume_failed | {% if include.deployment == 'self-hosted' %}jobs.auto.create.stats.resume_failed.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of auto_create_stats jobs which failed with a non-retryable error | This metric is a high-level indicator that automatically generated [table statistics]({% link {{ page.version.version }}/cost-based-optimizer.md %}#table-statistics) is failing. Failed statistic creation can lead to the query optimizer running with stale statistics. Stale statistics can cause suboptimal query plans to be selected leading to poor query performance. |
+| jobs.auto_create_stats.currently_running | {% if include.deployment == 'self-hosted' %}jobs.auto.create.stats.currently_running |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of auto_create_stats jobs currently running | This metric tracks the number of active automatically generated statistics jobs that could also be consuming resources. Ensure that foreground SQL traffic is not impacted by correlating this metric with SQL latency and query volume metrics. |
+| jobs.auto_create_stats.currently_paused | {% if include.deployment == 'self-hosted' %}jobs.auto.create.stats.currently_paused |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of auto_create_stats jobs currently considered Paused | This metric is a high-level indicator that automatically generated statistics jobs are paused which can lead to the query optimizer running with stale statistics. Stale statistics can cause suboptimal query plans to be selected leading to poor query performance. |
+| jobs.create_stats.currently_running | {% if include.deployment == 'self-hosted' %}jobs.create.stats.currently_running |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of create_stats jobs currently running | This metric tracks the number of active create statistics jobs that may be consuming resources. Ensure that foreground SQL traffic is not impacted by correlating this metric with SQL latency and query volume metrics. |
+
+## Backup and Restore
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| jobs.backup.currently_running | {% if include.deployment == 'self-hosted' %}jobs.backup.currently_running |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of backup jobs currently running | See Description. |
+| jobs.backup.currently_paused | {% if include.deployment == 'self-hosted' %}jobs.backup.currently_paused |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of backup jobs currently considered Paused | Monitor and alert on this metric to safeguard against an inadvertent operational error of leaving a backup job in a paused state for an extended period of time. In functional areas, a paused job can hold resources or have concurrency impact or some other negative consequence. Paused backup may break the [recovery point objective (RPO)]({% link {{ page.version.version }}/backup.md %}#performance). |
+| schedules.BACKUP.failed | {% if include.deployment == 'self-hosted' %}schedules.backup.failed |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of BACKUP jobs failed | Monitor this metric and investigate backup job failures. |
+| schedules.BACKUP.last-completed-time | {% if include.deployment == 'self-hosted' %}schedules.backup.last_completed_time |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} The Unix timestamp of the most recently completed backup by a schedule specified as maintaining this metric | Monitor this metric to ensure that backups are meeting the [recovery point objective (RPO)]({% link {{ page.version.version }}/disaster-recovery-overview.md %}). Each node exports the time that it last completed a backup on behalf of the schedule. If a node is restarted, it will report `0` until it completes a backup. If all nodes are restarted, `max()` is `0` until a node completes a backup.To make use of this metric, first, from each node, take the maximum over a rolling window equal to or greater than the backup frequency, and then take the maximum of those values across nodes. For example with a backup frequency of 60 minutes, monitor `time() - max_across_nodes(max_over_time(schedules_BACKUP_last_completed_time, 60min))`. | + +## Changefeeds + +If [changefeeds]({% link {{ page.version.version }}/change-data-capture-overview.md %}) are created in a CockroachDB cluster, monitor these additional metrics in your custom dashboards: + +|
CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| changefeed.running | changefeed.running | Number of currently running changefeeds, including sinkless | This metric tracks the total number of all running changefeeds. |
+| jobs.changefeed.currently_paused | {% if include.deployment == 'self-hosted' %}jobs.changefeed.currently_paused |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of changefeed jobs currently considered Paused | Monitor and alert on this metric to safeguard against an inadvertent operational error of leaving a changefeed job in a paused state for an extended period of time. Changefeed jobs should not be paused for a long time because the [protected timestamp prevents garbage collection]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#protected-timestamp-and-garbage-collection-monitoring). |
+| changefeed.failures | changefeed.failures | Total number of changefeed jobs which have failed | This metric tracks the permanent changefeed job failures that the jobs system will not try to restart. Any increase in this counter should be investigated. An alert on this metric is recommended. |
+| changefeed.error_retries | changefeed.error.retries | Total retryable errors encountered by all changefeeds | This metric tracks transient changefeed errors. Alert on "too many" errors, such as 50 retries in 15 minutes. For example, during a rolling upgrade this counter will increase because the changefeed jobs will restart following node restarts. There is an exponential backoff, up to 10 minutes. But if there is no rolling upgrade in process or other cluster maintenance, and the error rate is high, investigate the changefeed job.
+| changefeed.emitted_messages | changefeed.emitted.messages | Messages emitted by all feeds | This metric provides a useful context when assessing the state of changefeeds. This metric characterizes the rate of changes being streamed from the CockroachDB cluster. |
+| changefeed.emitted_bytes | {% if include.deployment == 'self-hosted' %}changefeed.emitted_bytes.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Bytes emitted by all feeds | This metric provides a useful context when assessing the state of changefeeds. This metric characterizes the throughput bytes being streamed from the CockroachDB cluster. |
+| changefeed.commit_latency | changefeed.commit.latency | The difference between the event MVCC timestamp and the time it was acknowledged by the downstream sink. If the sink batches events, then the difference between the oldest event in the batch and acknowledgement is recorded. Latency during backfill is excluded.| This metric provides a useful context when assessing the state of changefeeds. This metric characterizes the end-to-end lag between a committed change and that change applied at the destination. |
+| jobs.changefeed.protected_age_sec | {% if include.deployment == 'self-hosted' %}jobs.changefeed.protected_age_sec |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} The age of the oldest PTS record protected by changefeed jobs | [Changefeeds use protected timestamps to protect the data from being garbage collected]({% link {{ page.version.version }}/monitor-and-debug-changefeeds.md %}#protected-timestamp-and-garbage-collection-monitoring). Ensure the protected timestamp age does not significantly exceed the [GC TTL zone configuration]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-zone-variables). Alert on this metric if the protected timestamp age is greater than 3 times the GC TTL. |
+
+## Row-Level TTL
+
+If [Row-Level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) is configured for any table in a CockroachDB cluster, monitor these additional metrics in your custom dashboards:
+
+| CockroachDB Metric Name
| {% if include.deployment == 'self-hosted' %}[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb/?tab=host#metrics)
(add `cockroachdb.` prefix)
|{% elsif include.deployment == 'advanced' %}(add `cockroachdb.` prefix)
[Datadog Integration Metric Name](https://docs.datadoghq.com/integrations/cockroachdb_dedicated/#metrics)
(add `crdb_dedicated.` prefix)
|{% endif %}(add `crdb_dedicated.` prefix)
Description
| Usage |
+| ----------------------------------------------------- | {% if include.deployment == 'self-hosted' %}------ |{% elsif include.deployment == 'advanced' %}---- |{% endif %} ------------------------------------------------------------ | ------------------------------------------------------------ |
+| jobs.row_level_ttl.resume_completed | {% if include.deployment == 'self-hosted' %}jobs.row.level.ttl.resume_completed.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of row_level_ttl jobs which successfully resumed to completion | If Row Level TTL is enabled, this metric should be nonzero and correspond to the `ttl_cron` setting that was chosen. If this metric is zero, it means the job is not running |
+| jobs.row_level_ttl.resume_failed | {% if include.deployment == 'self-hosted' %}jobs.row.level.ttl.resume_failed.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of row_level_ttl jobs which failed with a non-retryable error | This metric should remain at zero. Repeated errors means the Row Level TTL job is not deleting data. |
+| jobs.row_level_ttl.rows_selected | {% if include.deployment == 'self-hosted' %}jobs.row.level.ttl.rows_selected.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of rows selected for deletion by the row level TTL job. | Correlate this metric with the metric `jobs.row_level_ttl.rows_deleted` to ensure all the rows that should be deleted are actually getting deleted. |
+| jobs.row_level_ttl.rows_deleted | {% if include.deployment == 'self-hosted' %}jobs.row.level.ttl.rows_deleted.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of rows deleted by the row level TTL job. | Correlate this metric with the metric `jobs.row_level_ttl.rows_selected` to ensure all the rows that should be deleted are actually getting deleted. |
+| jobs.row_level_ttl.currently_paused | {% if include.deployment == 'self-hosted' %}jobs.row.level.ttl.currently_paused |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of row_level_ttl jobs currently considered Paused | Monitor this metric to ensure the Row Level TTL job does not remain paused inadvertently for an extended period. |
+| jobs.row_level_ttl.currently_running | {% if include.deployment == 'self-hosted' %}jobs.row.level.ttl.currently_running |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of row_level_ttl jobs currently running | Monitor this metric to ensure there are not too many Row Level TTL jobs running at the same time. Generally, this metric should be in the low single digits. |
+| schedules.scheduled-row-level-ttl-executor.failed | {% if include.deployment == 'self-hosted' %}schedules.scheduled.row.level.ttl.executor_failed.count |{% elsif include.deployment == 'advanced' %}NOT AVAILABLE |{% endif %} Number of scheduled-row-level-ttl-executor jobs failed | Monitor this metric to ensure the Row Level TTL job is running. If it is non-zero, it means the job could not be created. |
+| jobs.row_level_ttl.span_total_duration | NOT AVAILABLE | Duration for processing a span during row level TTL. | See Description. |
+| jobs.row_level_ttl.select_duration | NOT AVAILABLE | Duration for select requests during row level TTL. | See Description. |
+| jobs.row_level_ttl.delete_duration | NOT AVAILABLE | Duration for delete requests during row level TTL. | See Description. |
+| jobs.row_level_ttl.num_active_spans | NOT AVAILABLE | Number of active spans the TTL job is deleting from. | See Description. |
+| jobs.row_level_ttl.total_rows | NOT AVAILABLE | Approximate number of rows on the TTL table. | See Description. |
+| jobs.row_level_ttl.total_expired_rows | NOT AVAILABLE | Approximate number of rows that have expired the TTL on the TTL table. | See Description. |
+
+## See also
+
+- [Available Metrics]({% link {{ page.version.version }}/metrics.md %}#available-metrics)
+- [Monitor CockroachDB with Prometheus]({% link {{ page.version.version }}/monitor-cockroachdb-with-prometheus.md %})
+- [Visualize metrics in Grafana]({% link {{ page.version.version }}/monitor-cockroachdb-with-prometheus.md %}#step-5-visualize-metrics-in-grafana)
+- [Custom Chart Debug Page]({% link {{ page.version.version }}/ui-custom-chart-debug-page.md %})
+- [Cluster API]({% link {{ page.version.version }}/cluster-api.md %})
+- [Essential Alerts]({% link {{ page.version.version }}/essential-alerts-{{ include.deployment}}.md %})
+- [CockroachDB Source Code - DB Console metrics to graphs mappings (in *.tsx files)](https://github.com/cockroachdb/cockroach/tree/master/pkg/ui/workspaces/db-console/src/views/cluster/containers/nodeGraphs/dashboards)
diff --git a/src/current/_includes/v25.1/faq/auto-generate-unique-ids.md b/src/current/_includes/v25.1/faq/auto-generate-unique-ids.md
new file mode 100644
index 00000000000..ebe3262e6df
--- /dev/null
+++ b/src/current/_includes/v25.1/faq/auto-generate-unique-ids.md
@@ -0,0 +1,111 @@
+To auto-generate unique row identifiers, you can use the `gen_random_uuid()`, `uuid_v4()`, or `unique_rowid()` [functions]({% link {{ page.version.version }}/functions-and-operators.md %}#id-generation-functions).
+
+To use the [`UUID`]({% link {{ page.version.version }}/uuid.md %}) column with the `gen_random_uuid()` [function]({% link {{ page.version.version }}/functions-and-operators.md %}#id-generation-functions) as the [default value]({% link {{ page.version.version }}/default-value.md %}):
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+CREATE TABLE users (
+ id UUID NOT NULL DEFAULT gen_random_uuid(),
+ city STRING NOT NULL,
+ name STRING NULL,
+ address STRING NULL,
+ credit_card STRING NULL,
+ CONSTRAINT "primary" PRIMARY KEY (city ASC, id ASC),
+ FAMILY "primary" (id, city, name, address, credit_card)
+);
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+INSERT INTO users (name, city) VALUES ('Petee', 'new york'), ('Eric', 'seattle'), ('Dan', 'seattle');
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+SELECT * FROM users;
+~~~
+
+~~~
+ id | city | name | address | credit_card
++--------------------------------------+----------+-------+---------+-------------+
+ cf8ee4e2-cd74-449a-b6e6-a0fb2017baa4 | new york | Petee | NULL | NULL
+ 2382564e-702f-42d9-a139-b6df535ae00a | seattle | Eric | NULL | NULL
+ 7d27e40b-263a-4891-b29b-d59135e55650 | seattle | Dan | NULL | NULL
+(3 rows)
+~~~
+
+Alternatively, you can use the [`BYTES`]({% link {{ page.version.version }}/bytes.md %}) column with the `uuid_v4()` function as the default value:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+CREATE TABLE users2 (
+ id BYTES DEFAULT uuid_v4(),
+ city STRING NOT NULL,
+ name STRING NULL,
+ address STRING NULL,
+ credit_card STRING NULL,
+ CONSTRAINT "primary" PRIMARY KEY (city ASC, id ASC),
+ FAMILY "primary" (id, city, name, address, credit_card)
+);
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+INSERT INTO users2 (name, city) VALUES ('Anna', 'new york'), ('Jonah', 'seattle'), ('Terry', 'chicago');
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+SELECT * FROM users;
+~~~
+
+~~~
+ id | city | name | address | credit_card
++------------------------------------------------+----------+-------+---------+-------------+
+ 4\244\277\323/\261M\007\213\275*\0060\346\025z | chicago | Terry | NULL | NULL
+ \273*t=u.F\010\274f/}\313\332\373a | new york | Anna | NULL | NULL
+ \004\\\364nP\024L)\252\364\222r$\274O0 | seattle | Jonah | NULL | NULL
+(3 rows)
+~~~
+
+In either case, generated IDs will be 128-bit, sufficiently large to generate unique values. Once the table grows beyond a single key-value range's [default size]({% link {{ page.version.version }}/configure-replication-zones.md %}#range-max-bytes), new IDs will be scattered across all of the table's ranges and, therefore, likely across different nodes. This means that multiple nodes will share in the load.
+
+This approach has the disadvantage of creating a primary key that may not be useful in a query directly, which can require a join with another table or a secondary index.
+
+If it is important for generated IDs to be stored in the same key-value range, you can use an [integer type]({% link {{ page.version.version }}/int.md %}) with the `unique_rowid()` [function]({% link {{ page.version.version }}/functions-and-operators.md %}#id-generation-functions) as the default value, either explicitly or via the [`SERIAL` pseudo-type]({% link {{ page.version.version }}/serial.md %}):
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+CREATE TABLE users3 (
+ id INT DEFAULT unique_rowid(),
+ city STRING NOT NULL,
+ name STRING NULL,
+ address STRING NULL,
+ credit_card STRING NULL,
+ CONSTRAINT "primary" PRIMARY KEY (city ASC, id ASC),
+ FAMILY "primary" (id, city, name, address, credit_card)
+);
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+INSERT INTO users3 (name, city) VALUES ('Blake', 'chicago'), ('Hannah', 'seattle'), ('Bobby', 'seattle');
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+SELECT * FROM users3;
+~~~
+
+~~~
+ id | city | name | address | credit_card
++--------------------+---------+--------+---------+-------------+
+ 469048192112197633 | chicago | Blake | NULL | NULL
+ 469048192112263169 | seattle | Hannah | NULL | NULL
+ 469048192112295937 | seattle | Bobby | NULL | NULL
+(3 rows)
+~~~
+
+Upon insert or upsert, the `unique_rowid()` function generates a default value from the timestamp and ID of the node executing the insert. Such time-ordered values are likely to be globally unique except in cases where a very large number of IDs (100,000+) are generated per node per second. Also, there can be gaps and the order is not completely guaranteed.
+
+To understand the differences between the `UUID` and `unique_rowid()` options, see the [SQL FAQs]({% link {{ page.version.version }}/sql-faqs.md %}#what-are-the-differences-between-uuid-sequences-and-unique_rowid). For further background on UUIDs, see [What is a UUID, and Why Should You Care?](https://www.cockroachlabs.com/blog/what-is-a-uuid/).
diff --git a/src/current/_includes/v25.1/faq/clock-synchronization-effects.md b/src/current/_includes/v25.1/faq/clock-synchronization-effects.md
new file mode 100644
index 00000000000..e335a97fc3e
--- /dev/null
+++ b/src/current/_includes/v25.1/faq/clock-synchronization-effects.md
@@ -0,0 +1,31 @@
+CockroachDB requires moderate levels of clock synchronization to preserve data consistency. For this reason, when a node detects that its clock is out of sync with at least half of the other nodes in the cluster by 80% of the maximum offset allowed, it spontaneously shuts down. This offset defaults to 500ms but can be changed via the [`--max-offset`]({% link {{ page.version.version }}/cockroach-start.md %}#flags-max-offset) flag when starting each node.
+
+Regardless of clock skew, [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) and [`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %}) transactions both serve globally consistent ("non-stale") reads and [commit atomically]({% link {{ page.version.version }}/developer-basics.md %}#how-transactions-work-in-cockroachdb). However, skew outside the configured clock offset bounds can result in violations of single-key linearizability between causally dependent transactions. It's therefore important to prevent clocks from drifting too far by running [NTP](http://www.ntp.org/) or other clock synchronization software on each node.
+
+In very rare cases, CockroachDB can momentarily run with a stale clock. This can happen when using vMotion, which can suspend a VM running CockroachDB, migrate it to different hardware, and resume it. This will cause CockroachDB to be out of sync for a short period before it jumps to the correct time. During this window, it would be possible for a client to read stale data and write data derived from stale reads. By enabling the `server.clock.forward_jump_check_enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}), you can be alerted when the CockroachDB clock jumps forward, indicating it had been running with a stale clock. To protect against this on vMotion, however, use the [`--clock-device`](cockroach-start.html#general) flag to specify a [PTP hardware clock](https://www.kernel.org/doc/html/latest/driver-api/ptp.html) for CockroachDB to use when querying the current time. When doing so, you should not enable `server.clock.forward_jump_check_enabled` because forward jumps will be expected and harmless. For more information on how `--clock-device` interacts with vMotion, see [this blog post](https://core.vmware.com/blog/cockroachdb-vmotion-support-vsphere-7-using-precise-timekeeping).
+
+{{site.data.alerts.callout_danger}}
+In CockroachDB versions prior to v22.2.13, and in v23.1 versions prior to v23.1.9, the [`--clock-device`](cockroach-start.html#general) flag had a bug that could cause it to generate timestamps in the far future. This could cause nodes to crash due to incorrect timestamps, or in the worst case irreversibly advance the cluster's HLC clock into the far future. This bug is fixed in CockroachDB v23.2.
+{{site.data.alerts.end}}
+
+### Considerations
+
+When setting up clock synchronization:
+
+- All nodes in the cluster must be synced to the same time source, or to different sources that implement leap second smearing in the same way. For example, Google and Amazon have time sources that are compatible with each other (they implement [leap second smearing](https://developers.google.com/time/smear) in the same way), but are incompatible with the default NTP pool (which does not implement leap second smearing).
+- For nodes running in AWS, we recommend [Amazon Time Sync Service](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html#configure-amazon-time-service). For nodes running in GCP, we recommend [Google's internal NTP service](https://cloud.google.com/compute/docs/instances/configure-ntp#configure_ntp_for_your_instances). For nodes running elsewhere, we recommend [Google Public NTP](https://developers.google.com/time/). Note that the Google and Amazon time services can be mixed with each other, but they cannot be mixed with other time services (unless you have verified leap second behavior). Either all of your nodes should use the Google and Amazon services, or none of them should.
+- If you do not want to use the Google or Amazon time sources, you can use [`chrony`](https://chrony.tuxfamily.org/index.html) and enable client-side leap smearing, unless the time source you're using already does server-side smearing. In most cases, we recommend the Google Public NTP time source because it handles smearing the leap second. If you use a different NTP time source that doesn't smear the leap second, you must configure client-side smearing manually and do so in the same way on each machine.
+- Do not run more than one clock sync service on VMs where `cockroach` is running.
+- {% include {{ page.version.version }}/misc/multiregion-max-offset.md %}
+
+### Tutorials
+
+For guidance on synchronizing clocks, see the tutorial for your deployment environment:
+
+Environment | Featured Approach
+------------|---------------------
+[On-Premises]({% link {{ page.version.version }}/deploy-cockroachdb-on-premises.md %}#step-1-synchronize-clocks) | Use NTP with Google's external NTP service.
+[AWS]({% link {{ page.version.version }}/deploy-cockroachdb-on-aws.md %}#step-3-synchronize-clocks) | Use the Amazon Time Sync Service.
+[Azure]({% link {{ page.version.version }}/deploy-cockroachdb-on-microsoft-azure.md %}#step-3-synchronize-clocks) | Disable Hyper-V time synchronization and use NTP with Google's external NTP service.
+[Digital Ocean]({% link {{ page.version.version }}/deploy-cockroachdb-on-digital-ocean.md %}#step-2-synchronize-clocks) | Use NTP with Google's external NTP service.
+[GCE]({% link {{ page.version.version }}/deploy-cockroachdb-on-google-cloud-platform.md %}#step-3-synchronize-clocks) | Use NTP with Google's internal NTP service.
diff --git a/src/current/_includes/v25.1/faq/clock-synchronization-monitoring.md b/src/current/_includes/v25.1/faq/clock-synchronization-monitoring.md
new file mode 100644
index 00000000000..c3022ad1a32
--- /dev/null
+++ b/src/current/_includes/v25.1/faq/clock-synchronization-monitoring.md
@@ -0,0 +1,8 @@
+As explained in more detail [in our monitoring documentation]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#prometheus-endpoint), each CockroachDB node exports a wide variety of metrics at `http://| Tag | +Example | +Description | +
| An exact patch | +`{{ page.version.name }}` | +Pins a cluster to an exact patch. The cluster is upgraded to a newer patch or major version only when you pull a newer tag. | +
| Latest patch within a major version | +`latest-{{ page.version.version }}` | +Automatically updates a cluster to the latest patch of the version you specify. This tag is recommended in production, because it keeps your cluster updated within a major version but does not automatically upgrade your cluster to a new major version. | +
| `latest` | +The latest patch within the latest major version. + | This is the default if you do not specify a tag. It updates your cluster automatically to each new patch and major version, and is not recommended in production. | +
+
+
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/ldr/immediate-description.md b/src/current/_includes/v25.1/ldr/immediate-description.md
new file mode 100644
index 00000000000..eb87361a009
--- /dev/null
+++ b/src/current/_includes/v25.1/ldr/immediate-description.md
@@ -0,0 +1 @@
+Attempts to replicate the changed row directly into the destination table, without re-running constraint validations. It does not support writing into tables with [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) constraints.
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/ldr/multiple-tables.md b/src/current/_includes/v25.1/ldr/multiple-tables.md
new file mode 100644
index 00000000000..0522fb605be
--- /dev/null
+++ b/src/current/_includes/v25.1/ldr/multiple-tables.md
@@ -0,0 +1 @@
+There are some tradeoffs between enabling one table per LDR job versus multiple tables in one LDR job. Multiple tables in one LDR job can be easier to operate. For example, if you pause and resume the single job, LDR will stop and resume for all the tables. However, the most granular level observability will be at the job level. One table in one LDR job will allow for table-level observability.
diff --git a/src/current/_includes/v25.1/ldr/show-logical-replication-responses.md b/src/current/_includes/v25.1/ldr/show-logical-replication-responses.md
new file mode 100644
index 00000000000..0316fe9e6d9
--- /dev/null
+++ b/src/current/_includes/v25.1/ldr/show-logical-replication-responses.md
@@ -0,0 +1,9 @@
+Field | Response
+---------+----------
+`job_id` | The job's ID. Use with [`CANCEL JOB`]({% link {{ page.version.version }}/cancel-job.md %}), [`PAUSE JOB`]({% link {{ page.version.version }}/pause-job.md %}), [`RESUME JOB`]({% link {{ page.version.version }}/resume-job.md %}), [`SHOW JOB`]({% link {{ page.version.version }}/show-jobs.md %}).
+`status` | The job's current state. Possible values: `pending`, `paused`, `pause-requested`, `failed`, `succeeded`, `canceled`, `cancel-requested`, `running`, `retry-running`, `retry-reverting`, `reverting`, `revert-failed`.Refer to [Jobs status]({% link {{ page.version.version }}/show-jobs.md %}#job-status) for a description of each status. +`targets` | The fully qualified name of the table(s) that are part of the LDR job. +`replicated_time` | The latest [timestamp]({% link {{ page.version.version }}/timestamp.md %}) at which the destination cluster has consistent data. This time advances automatically as long as the LDR job proceeds without error. `replicated_time` is updated periodically (every 30s). +`replication_start_time` | The start time of the LDR job. +`conflict_resolution_type` | The type of [conflict resolution]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#conflict-resolution): `LWW` last write wins. +`description` | Description of the job including the replicating table(s) and the source cluster connection. diff --git a/src/current/_includes/v25.1/ldr/show_logical_replication_jobs_stmt.html b/src/current/_includes/v25.1/ldr/show_logical_replication_jobs_stmt.html new file mode 100644 index 00000000000..50cef32cc22 --- /dev/null +++ b/src/current/_includes/v25.1/ldr/show_logical_replication_jobs_stmt.html @@ -0,0 +1,55 @@ + +
+
+
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/ldr/validated-description.md b/src/current/_includes/v25.1/ldr/validated-description.md
new file mode 100644
index 00000000000..4b7bb9a8b18
--- /dev/null
+++ b/src/current/_includes/v25.1/ldr/validated-description.md
@@ -0,0 +1 @@
+Attempts to apply the write in a similar way to a user-run query, which would re-run all constraint validations relevant to the destination table(s). If the change violates foreign key dependencies, unique constraints, or other constraints, the row will be put in the [dead letter queue (DLQ)]({% link {{ page.version.version }}/manage-logical-data-replication.md %}#dead-letter-queue-dlq) instead. Like the [SQL layer]({% link {{ page.version.version }}/architecture/sql-layer.md %}), `validated` mode does not recognize deletion tombstones. As a result, an update to the same key from cluster A will successfully apply on cluster B, even if that key was deleted on cluster B before the LDR job streamed the cluster A update to the key.
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/leader-leases-intro.md b/src/current/_includes/v25.1/leader-leases-intro.md
new file mode 100644
index 00000000000..e2e62cc0ccb
--- /dev/null
+++ b/src/current/_includes/v25.1/leader-leases-intro.md
@@ -0,0 +1 @@
+CockroachDB offers an improved leasing system rebuilt atop a stronger form of [Raft]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft) leadership that ensures that the Raft leader is **always** the range's leaseholder. This new type of lease is called a _Leader lease_, and supersedes [epoch-based leases]({% link {{ page.version.version }}/architecture/replication-layer.md %}#epoch-based-leases-table-data) and [expiration-based leases]({% link {{ page.version.version }}/architecture/replication-layer.md %}#expiration-based-leases-meta-and-system-ranges) leases while combining the performance of the former with the resilience of the latter. **Leader leases are not enabled by default.**
diff --git a/src/current/_includes/v25.1/metric-names-serverless.md b/src/current/_includes/v25.1/metric-names-serverless.md
new file mode 100644
index 00000000000..33fab10e2a8
--- /dev/null
+++ b/src/current/_includes/v25.1/metric-names-serverless.md
@@ -0,0 +1,244 @@
+Name | Description
+-----|-----
+`addsstable.applications` | Number of SSTable ingestions applied (i.e., applied by Replicas)
+`addsstable.copies` | number of SSTable ingestions that required copying files during application
+`addsstable.proposals` | Number of SSTable ingestions proposed (i.e., sent to Raft by lease holders)
+`admission.wait_sum.kv-stores` | Total wait time in micros
+`admission.wait_sum.kv` | Total wait time in micros
+`admission.wait_sum.sql-kv-response` | Total wait time in micros
+`admission.wait_sum.sql-sql-response` | Total wait time in micros
+`capacity.available` | Available storage capacity
+`capacity.reserved` | Capacity reserved for snapshots
+`capacity.used` | Used storage capacity
+`capacity` | Total storage capacity
+`changefeed.backfill_count` | Number of changefeeds currently executing backfill
+`changefeed.backfill_pending_ranges` | Number of ranges in an ongoing backfill that are yet to be fully emitted
+`changefeed.commit_latency` | Event commit latency: a difference between event MVCC timestamp and the time it was acknowledged by the downstream sink. If the sink batches events, then the difference between the earliest event in the batch and acknowledgement is recorded; Excludes latency during backfill
+`changefeed.emitted_messages` | Messages emitted by all feeds
+`changefeed.error_retries` | Total retryable errors encountered by all changefeeds
+`changefeed.failures` | Total number of changefeed jobs which have failed
+`changefeed.max_behind_nanos` | Largest commit-to-emit duration of any running feed
+`changefeed.message_size_hist` | Message size histogram
+`changefeed.running` | Number of currently running changefeeds, including sinkless
+`clock-offset.meannanos` | Mean clock offset with other nodes
+`clock-offset.stddevnanos` | Stddev clock offset with other nodes
+`distsender.batches.partial` | Number of partial batches processed after being divided on range boundaries
+`distsender.batches` | Number of batches processed
+`distsender.errors.notleaseholder` | Number of NotLeaseHolderErrors encountered from replica-addressed RPCs
+`distsender.rpc.sent.local` | Number of replica-addressed RPCs sent through the local-server optimization
+`distsender.rpc.sent.nextreplicaerror` | Number of replica-addressed RPCs sent due to per-replica errors
+`distsender.rpc.sent` | Number of replica-addressed RPCs sent
+`exec.error` | Number of batch KV requests that failed to execute on this node. This count excludes transaction restart/abort errors. However, it will include other errors expected during normal operation, such as ConditionFailedError. This metric is thus not an indicator of KV health.
+`exec.latency` | Latency of batch KV requests (including errors) executed on this node. This measures requests already addressed to a single replica, from the moment at which they arrive at the internal gRPC endpoint to the moment at which the response (or an error) is returned. This latency includes in particular commit waits, conflict resolution and replication, and end-users can easily produce high measurements via long-running transactions that conflict with foreground traffic. This metric thus does not provide a good signal for understanding the health of the KV layer.
+`exec.success` | Number of batch KV requests executed successfully on this node. A request is considered to have executed 'successfully' if it either returns a result or a transaction restart/abort error.
+`gcbytesage` | Cumulative age of non-live data
+`gossip.bytes.received` | Number of received gossip bytes
+`gossip.bytes.sent` | Number of sent gossip bytes
+`gossip.connections.incoming` | Number of active incoming gossip connections
+`gossip.connections.outgoing` | Number of active outgoing gossip connections
+`gossip.connections.refused` | Number of refused incoming gossip connections
+`gossip.infos.received` | Number of received gossip Info objects
+`gossip.infos.sent` | Number of sent gossip Info objects
+`intentage` | Cumulative age of intents
+`intentbytes` | Number of bytes in intent KV pairs
+`intentcount` | Count of intent keys
+`jobs.changefeed.resume_retry_error` | Number of changefeed jobs which failed with a retryable error
+`keybytes` | Number of bytes taken up by keys
+`keycount` | Count of all keys
+`leases.epoch` | Number of replica leaseholders using epoch-based leases
+`leases.error` | Number of failed lease requests
+`leases.expiration` | Number of replica leaseholders using expiration-based leases
+`leases.success` | Number of successful lease requests
+`leases.transfers.error` | Number of failed lease transfers
+`leases.transfers.success` | Number of successful lease transfers
+`livebytes` | Number of bytes of live data (keys plus values)
+`livecount` | Count of live keys
+`liveness.epochincrements` | Number of times this node has incremented its liveness epoch
+`liveness.heartbeatfailures` | Number of failed node liveness heartbeats from this node
+`liveness.heartbeatlatency` | Node liveness heartbeat latency
+`liveness.heartbeatsuccesses` | Number of successful node liveness heartbeats from this node
+`liveness.livenodes` | Number of live nodes in the cluster (will be 0 if this node is not itself live)
+`queue.consistency.pending` | Number of pending replicas in the consistency checker queue
+`queue.consistency.process.failure` | Number of replicas which failed processing in the consistency checker queue
+`queue.consistency.process.success` | Number of replicas successfully processed by the consistency checker queue
+`queue.consistency.processingnanos` | Nanoseconds spent processing replicas in the consistency checker queue
+`queue.gc.info.abortspanconsidered` | Number of AbortSpan entries eligible for removal based on their ages
+`queue.gc.info.abortspangcnum` | Number of AbortSpan entries fit for removal
+`queue.gc.info.abortspanscanned` | Number of transactions present in the AbortSpan scanned from the engine
+`queue.gc.info.intentsconsidered` | Number of intents eligible to be considered because they are at least two hours old
+`queue.gc.info.intenttxns` | Number of associated distinct transactions
+`queue.gc.info.numkeysaffected` | Number of keys with data that is eligible for garbage collection
+`queue.gc.info.pushtxn` | Number of attempted pushes
+`queue.gc.info.resolvesuccess` | Number of successful intent resolutions
+`queue.gc.info.resolvetotal` | Number of attempted intent resolutions
+`queue.gc.info.transactionspangcaborted` | Number of entries eligible for garbage collection that correspond to aborted txns
+`queue.gc.info.transactionspangccommitted` | Number of entries eligible for garbage collection that correspond to committed txns
+`queue.gc.info.transactionspangcpending` | Number of entries eligible for garbage collection that correspond to pending txns
+`queue.gc.info.transactionspanscanned` | Number of entries in transaction spans scanned from the engine
+`queue.gc.pending` | Number of pending replicas in the MVCC garbage collection queue
+`queue.gc.process.failure` | Number of replicas which failed processing in the MVCC garbage collection queue
+`queue.gc.process.success` | Number of replicas successfully processed by the MVCC garbage collection queue
+`queue.gc.processingnanos` | Nanoseconds spent processing replicas in the MVCC garbage collection queue
+`queue.raftlog.pending` | Number of pending replicas in the Raft log queue
+`queue.raftlog.process.failure` | Number of replicas which failed processing in the Raft log queue
+`queue.raftlog.process.success` | Number of replicas successfully processed by the Raft log queue
+`queue.raftlog.processingnanos` | Nanoseconds spent processing replicas in the Raft log queue
+`queue.raftsnapshot.pending` | Number of pending replicas in the Raft repair queue
+`queue.raftsnapshot.process.failure` | Number of replicas which failed processing in the Raft repair queue
+`queue.raftsnapshot.process.success` | Number of replicas successfully processed by the Raft repair queue
+`queue.raftsnapshot.processingnanos` | Nanoseconds spent processing replicas in the Raft repair queue
+`queue.replicagc.pending` | Number of pending replicas in the replica queue
+`queue.replicagc.process.failure` | Number of replicas which failed processing in the replica garbage collection queue
+`queue.replicagc.process.success` | Number of replicas successfully processed by the replica garbage collection queue
+`queue.replicagc.processingnanos` | Nanoseconds spent processing replicas in the replica garbage collection queue
+`queue.replicagc.removereplica` | Number of replica removals attempted by the replica garbage collection queue
+`queue.replicate.addreplica` | Number of replica additions attempted by the replicate queue
+`queue.replicate.pending` | Number of pending replicas in the replicate queue
+`queue.replicate.process.failure` | Number of replicas which failed processing in the replicate queue
+`queue.replicate.process.success` | Number of replicas successfully processed by the replicate queue
+`queue.replicate.processingnanos` | Nanoseconds spent processing replicas in the replicate queue
+`queue.replicate.purgatory` | Number of replicas in the replicate queue's purgatory, awaiting allocation options
+`queue.replicate.rebalancereplica` | Number of replica rebalancer-initiated additions attempted by the replicate queue
+`queue.replicate.removedeadreplica` | Number of dead replica removals attempted by the replicate queue (typically in response to a node outage)
+`queue.replicate.removereplica` | Number of replica removals attempted by the replicate queue (typically in response to a rebalancer-initiated addition)
+`queue.replicate.transferlease` | Number of range lease transfers attempted by the replicate queue
+`queue.split.pending` | Number of pending replicas in the split queue
+`queue.split.process.failure` | Number of replicas which failed processing in the split queue
+`queue.split.process.success` | Number of replicas successfully processed by the split queue
+`queue.split.processingnanos` | Nanoseconds spent processing replicas in the split queue
+`queue.tsmaintenance.pending` | Number of pending replicas in the time series maintenance queue
+`queue.tsmaintenance.process.failure` | Number of replicas which failed processing in the time series maintenance queue
+`queue.tsmaintenance.process.success` | Number of replicas successfully processed by the time series maintenance queue
+`queue.tsmaintenance.processingnanos` | Nanoseconds spent processing replicas in the time series maintenance queue
+`raft.commandsapplied` | Count of Raft commands applied. This measurement is taken on the Raft apply loops of all Replicas (leaders and followers alike), meaning that it does not measure the number of Raft commands *proposed* (in the hypothetical extreme case, all Replicas may apply all commands through snapshots, thus not increasing this metric at all). Instead, it is a proxy for how much work is being done advancing the Replica state machines on this node.
+`raft.heartbeats.pending` | Number of pending heartbeats and responses waiting to be coalesced
+`raft.process.commandcommit.latency` | Latency histogram for applying a batch of Raft commands to the state machine. This metric is misnamed: it measures the latency for *applying* a batch of committed Raft commands to a Replica state machine. This requires only non-durable I/O (except for replication configuration changes). Note that a "batch" in this context is really a sub-batch of the batch received for application during Raft ready handling. The 'raft.process.applycommitted.latency' histogram is likely more suitable in most cases, as it measures the total latency across all sub-batches (i.e., the sum of commandcommit.latency for a complete batch).
+`raft.process.logcommit.latency` | Latency histogram for committing Raft log entries to stable storage. This measures the latency of durably committing a group of newly received Raft entries as well as the HardState entry to disk. This excludes any data processing, i.e., we measure purely the commit latency of the resulting Engine write. Homogeneous bands of p50-p99 latencies (in the presence of regular Raft traffic), make it likely that the storage layer is healthy. Spikes in the latency bands can either hint at the presence of large sets of Raft entries being received, or at performance issues at the storage layer.
+`raft.process.tickingnanos` | Nanoseconds spent in store.processRaft() processing replica.Tick()
+`raft.process.workingnanos` | Nanoseconds spent in store.processRaft() working. This is the sum of the measurements passed to the raft.process.handleready.latency histogram.
+`raft.rcvd.app` | Number of MsgApp messages received by this store
+`raft.rcvd.appresp` | Number of MsgAppResp messages received by this store
+`raft.rcvd.dropped` | Number of incoming Raft messages dropped (due to queue length or size)
+`raft.rcvd.heartbeat` | Number of (coalesced, if enabled) MsgHeartbeat messages received by this store
+`raft.rcvd.heartbeatresp` | Number of (coalesced, if enabled) MsgHeartbeatResp messages received by this store
+`raft.rcvd.prevote` | Number of MsgPreVote messages received by this store
+`raft.rcvd.prevoteresp` | Number of MsgPreVoteResp messages received by this store
+`raft.rcvd.prop` | Number of MsgProp messages received by this store
+`raft.rcvd.snap` | Number of MsgSnap messages received by this store
+`raft.rcvd.timeoutnow` | Number of MsgTimeoutNow messages received by this store
+`raft.rcvd.transferleader` | Number of MsgTransferLeader messages received by this store
+`raft.rcvd.vote` | Number of MsgVote messages received by this store
+`raft.rcvd.voteresp` | Number of MsgVoteResp messages received by this store
+`raft.ticks` | Number of Raft ticks queued
+`raftlog.behind` | Number of Raft log entries followers on other stores are behind. This gauge provides a view of the aggregate number of log entries the Raft leaders on this node think the followers are behind. Since a Raft leader may not always have a good estimate for this information for all of its followers, and since followers are expected to be behind (when they are not required as part of a quorum) *and* the aggregate thus scales like the count of such followers, it is difficult to meaningfully interpret this metric.
+`raftlog.truncated` | Number of Raft log entries truncated
+`range.adds` | Number of range additions
+`range.raftleadertransfers` | Number of Raft leader transfers
+`range.removes` | Number of range removals
+`range.snapshots.generated` | Number of generated snapshots
+`range.snapshots.recv-in-progress` | Number of non-empty snapshots in progress on a receiver store
+`range.snapshots.recv-queue` | Number of queued non-empty snapshots on a receiver store
+`range.snapshots.recv-total-in-progress` | Number of empty and non-empty snapshots in progress on a receiver store
+`range.snapshots.send-in-progress` | Number of non-empty snapshots in progress on a sender store
+`range.snapshots.send-queue` | Number of queued non-empty snapshots on a sender store
+`range.snapshots.send-total-in-progress` | Number of empty and non-empty in-progress on a sender store
+`range.splits` | Number of range splits
+`ranges.overreplicated` | Number of ranges with more live replicas than the replication target
+`ranges.unavailable` | Number of ranges with fewer live replicas than needed for quorum
+`ranges.underreplicated` | Number of ranges with fewer live replicas than the replication target
+`ranges` | Number of ranges
+`rebalancing.writespersecond` | Number of keys written (i.e., applied by raft) per second to the store, averaged over a large time period as used in rebalancing decisions
+`replicas.leaders_not_leaseholders` | Number of replicas that are Raft leaders whose range lease is held by another store
+`replicas.leaders` | Number of Raft leaders
+`replicas.leaseholders` | Number of lease holders
+`replicas.quiescent` | Number of quiesced replicas
+`replicas.reserved` | Number of replicas reserved for snapshots
+`replicas` | Number of replicas
+`requests.backpressure.split` | Number of backpressured writes waiting on a range split. A range will backpressure (roughly) non-system traffic when the range is above the configured size until the range splits. When the rate of this metric is nonzero over extended periods of time, it should be investigated why splits are not occurring.
+`requests.slow.distsender` | Number of replica-bound RPCs currently stuck or retrying for a long time. Note that this is not a good signal for KV health. The remote side of the RPCs tracked here may experience contention, so an end user can easily cause values for this metric to be emitted by leaving a transaction open for a long time and contending with it using a second transaction.
+`requests.slow.lease` | Number of requests that have been stuck for a long time acquiring a lease. A nonzero value usually indicates range or replica unavailability, and should be investigated. Commonly, `requests.slow.raft` is also a nonzero value, which indicates that the lease requests are not getting a timely response from the replication layer.
+`requests.slow.raft` | Number of requests that have been stuck for a long time in the replication layer. An (evaluated) request has to pass through the replication layer, notably the quota pool and Raft. If it fails to do so within a highly permissive duration, this metric is incremented (and decremented again once the request is either applied or returns an error). A nonzero value indicates range or replica unavailability, and should be investigated.
+`rocksdb.block.cache.hits` | Count of block cache hits
+`rocksdb.block.cache.misses` | Count of block cache misses
+`rocksdb.block.cache.pinned-usage` | Bytes pinned by the block cache
+`rocksdb.block.cache.usage` | Bytes used by the block cache
+`rocksdb.bloom.filter.prefix.checked` | Number of times the bloom filter was checked
+`rocksdb.bloom.filter.prefix.useful` | Number of times the bloom filter helped avoid iterator creation
+`rocksdb.compactions` | Number of table compactions
+`rocksdb.flushes` | Number of table flushes
+`rocksdb.memtable.total-size` | Current size of memtable in bytes
+`rocksdb.num-sstables` | Number of storage engine SSTables
+`rocksdb.read-amplification` | Number of disk reads per query
+`rocksdb.table-readers-mem-estimate` | Memory used by index and filter blocks
+`round-trip-latency` | Distribution of round-trip latencies with other nodes
+`sql.bytesin` | Number of sql bytes received
+`sql.bytesout` | Number of sql bytes sent
+`sql.conn.latency` | Latency to establish and authenticate a SQL connection
+`sql.conns` | Number of active sql connections
+`sql.ddl.count` | Number of SQL DDL statements successfully executed
+`sql.delete.count` | Number of SQL DELETE statements successfully executed
+`sql.distsql.contended_queries.count` | Number of SQL queries that experienced contention
+`sql.distsql.exec.latency` | Latency of DistSQL statement execution
+`sql.distsql.flows.active` | Number of distributed SQL flows currently active
+`sql.distsql.flows.total` | Number of distributed SQL flows executed
+`sql.distsql.queries.active` | Number of SQL queries currently active
+`sql.distsql.queries.total` | Number of SQL queries executed
+`sql.distsql.select.count` | Number of DistSQL SELECT statements
+`sql.distsql.service.latency` | Latency of DistSQL request execution
+`sql.exec.latency` | Latency of SQL statement execution
+`sql.failure.count` | Number of statements resulting in a planning or runtime error
+`sql.full.scan.count` | Number of full table or index scans
+`sql.insert.count` | Number of SQL INSERT statements successfully executed
+`sql.mem.distsql.current` | Current sql statement memory usage for distsql
+`sql.mem.distsql.max` | Memory usage per sql statement for distsql
+`sql.mem.internal.session.current` | Current sql session memory usage for internal
+`sql.mem.internal.session.max` | Memory usage per sql session for internal
+`sql.mem.internal.txn.current` | Current sql transaction memory usage for internal
+`sql.mem.internal.txn.max` | Memory usage per sql transaction for internal
+`sql.misc.count` | Number of other SQL statements successfully executed
+`sql.query.count` | Number of SQL queries executed
+`sql.select.count` | Number of SQL SELECT statements successfully executed
+`sql.service.latency` | Latency of SQL request execution
+`sql.statements.active` | Number of currently active user SQL statements
+`sql.txn.abort.count` | Number of SQL transaction abort errors
+`sql.txn.begin.count` | Number of SQL transaction BEGIN statements successfully executed
+`sql.txn.commit.count` | Number of SQL transaction COMMIT statements successfully executed
+`sql.txn.latency` | Latency of SQL transactions
+`sql.txn.rollback.count` | Number of SQL transaction ROLLBACK statements successfully executed
+`sql.txns.open` | Number of currently open user SQL transactions
+`sql.update.count` | Number of SQL UPDATE statements successfully executed
+`sys.cgo.allocbytes` | Current bytes of memory allocated by cgo
+`sys.cgo.totalbytes` | Total bytes of memory allocated by cgo, but not released
+`sys.cgocalls` | Total number of cgo calls
+`sys.cpu.combined.percent-normalized` | Current user+system cpu percentage, normalized 0-1 by number of cores
+`sys.cpu.sys.ns` | Total system cpu time
+`sys.cpu.sys.percent` | Current system cpu percentage
+`sys.cpu.user.ns` | Total user cpu time
+`sys.cpu.user.percent` | Current user cpu percentage
+`sys.fd.open` | Process open file descriptors
+`sys.fd.softlimit` | Process open FD soft limit
+`sys.gc.count` | Total number of garbage collection runs
+`sys.gc.pause.ns` | Total garbage collection pause
+`sys.gc.pause.percent` | Current garbage collection pause percentage
+`sys.go.allocbytes` | Current bytes of memory allocated by go
+`sys.go.totalbytes` | Total bytes of memory allocated by go, but not released
+`sys.goroutines` | Current number of goroutines
+`sys.host.net.recv.bytes` | Bytes received on all network interfaces since this process started
+`sys.host.net.send.bytes` | Bytes sent on all network interfaces since this process started
+`sys.rss` | Current process RSS
+`sys.uptime` | Process uptime
+`sysbytes` | Number of bytes in system KV pairs
+`syscount` | Count of system KV pairs
+`timeseries.write.bytes` | Total size in bytes of metric samples written to disk
+`timeseries.write.errors` | Total errors encountered while attempting to write metrics to disk
+`timeseries.write.samples` | Total number of metric samples written to disk
+`totalbytes` | Total number of bytes taken up by keys and values including non-live data
+`txn.aborts` | Number of aborted KV transactions
+`txn.commits1PC` | Number of KV transaction one-phase commit attempts
+`txn.commits` | Number of committed KV transactions (including 1PC)
+`txn.durations` | KV transaction durations
+`txn.restarts.serializable` | Number of restarts due to a forwarded commit timestamp and isolation=SERIALIZABLE
+`txn.restarts.writetooold` | Number of restarts due to a concurrent writer committing first
+`txn.restarts` | Number of restarted KV transactions
+`valbytes` | Number of bytes taken up by values
+`valcount` | Count of all values
diff --git a/src/current/_includes/v25.1/metric-names.md b/src/current/_includes/v25.1/metric-names.md
new file mode 100644
index 00000000000..c72864fd149
--- /dev/null
+++ b/src/current/_includes/v25.1/metric-names.md
@@ -0,0 +1,29 @@
+{% assign list1 = site.data.metrics.available-metrics-in-metrics-list %}
+{% assign list2 = site.data.metrics.available-metrics-not-in-metrics-list %}
+
+{% assign available_metrics_combined = list1 | concat: list2 %}
+{% assign available_metrics_sorted = available_metrics_combined | sort: "metric_id" %}
+
+| CockroachDB Metric Name | +Description | +Type | +Unit | +
{{ m.metric_id }} |
+ {% comment %} Use the value from the metrics-list, if any, followed by the value in the available-metrics-not-in-metrics-list, if any. {% endcomment %}
+ {{ metrics-list[0].description }}{{ m.description }} | +{{ metrics-list[0].type }}{{ m.type }} | +{{ metrics-list[0].unit }}{{ m.unit }} | +
`external_id`: Use as a value to `ASSUME_ROLE` to specify the [external ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html) for third-party access to your S3 bucket. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/auth-intro-examples.md b/src/current/_includes/v25.1/misc/auth-intro-examples.md new file mode 100644 index 00000000000..27b7fb4484c --- /dev/null +++ b/src/current/_includes/v25.1/misc/auth-intro-examples.md @@ -0,0 +1,3 @@ +These examples use the **default** `AUTH=specified` parameter. For more detail on how to use `implicit` authentication with Amazon S3 buckets, read [Use Cloud Storage for Bulk Operations — Authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}). + +CockroachDB supports assume role authentication. This allows you to limit the control specific users have over your storage buckets. See [Assume role authentication]({% link {{ page.version.version }}/cloud-storage-authentication.md %}) for more information. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/available-capacity-metric.md b/src/current/_includes/v25.1/misc/available-capacity-metric.md new file mode 100644 index 00000000000..d3bb8ffae1e --- /dev/null +++ b/src/current/_includes/v25.1/misc/available-capacity-metric.md @@ -0,0 +1 @@ +If you are testing your deployment locally with multiple CockroachDB nodes running on a single machine (this is [not recommended in production]({% link {{ page.version.version }}/recommended-production-settings.md %}#topology)), you must explicitly [set the store size]({% link {{ page.version.version }}/cockroach-start.md %}#store) per node in order to display the correct capacity. Otherwise, the machine's actual disk capacity will be counted as a separate store for each node, thus inflating the computed capacity. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/aws-locations.md b/src/current/_includes/v25.1/misc/aws-locations.md new file mode 100644 index 00000000000..8b073c1f230 --- /dev/null +++ b/src/current/_includes/v25.1/misc/aws-locations.md @@ -0,0 +1,18 @@ +| Location | SQL Statement | +| ------ | ------ | +| US East (N. Virginia) | `INSERT into system.locations VALUES ('region', 'us-east-1', 37.478397, -76.453077)`| +| US East (Ohio) | `INSERT into system.locations VALUES ('region', 'us-east-2', 40.417287, -76.453077)` | +| US West (N. California) | `INSERT into system.locations VALUES ('region', 'us-west-1', 38.837522, -120.895824)` | +| US West (Oregon) | `INSERT into system.locations VALUES ('region', 'us-west-2', 43.804133, -120.554201)` | +| Canada (Central) | `INSERT into system.locations VALUES ('region', 'ca-central-1', 56.130366, -106.346771)` | +| EU (Frankfurt) | `INSERT into system.locations VALUES ('region', 'eu-central-1', 50.110922, 8.682127)` | +| EU (Ireland) | `INSERT into system.locations VALUES ('region', 'eu-west-1', 53.142367, -7.692054)` | +| EU (London) | `INSERT into system.locations VALUES ('region', 'eu-west-2', 51.507351, -0.127758)` | +| EU (Paris) | `INSERT into system.locations VALUES ('region', 'eu-west-3', 48.856614, 2.352222)` | +| Asia Pacific (Tokyo) | `INSERT into system.locations VALUES ('region', 'ap-northeast-1', 35.689487, 139.691706)` | +| Asia Pacific (Seoul) | `INSERT into system.locations VALUES ('region', 'ap-northeast-2', 37.566535, 126.977969)` | +| Asia Pacific (Osaka-Local) | `INSERT into system.locations VALUES ('region', 'ap-northeast-3', 34.693738, 135.502165)` | +| Asia Pacific (Singapore) | `INSERT into system.locations VALUES ('region', 'ap-southeast-1', 1.352083, 103.819836)` | +| Asia Pacific (Sydney) | `INSERT into system.locations VALUES ('region', 'ap-southeast-2', -33.86882, 151.209296)` | +| Asia Pacific (Mumbai) | `INSERT into system.locations VALUES ('region', 'ap-south-1', 19.075984, 72.877656)` | +| South America (São Paulo) | `INSERT into system.locations VALUES ('region', 'sa-east-1', -23.55052, -46.633309)` | diff --git a/src/current/_includes/v25.1/misc/azure-blob.md b/src/current/_includes/v25.1/misc/azure-blob.md new file mode 100644 index 00000000000..3449e639f7a --- /dev/null +++ b/src/current/_includes/v25.1/misc/azure-blob.md @@ -0,0 +1 @@ +For [changefeeds]({% link {{ page.version.version }}/changefeed-sinks.md %}), you must use the `azure://` scheme. For all other jobs, the `azure://` and `azure-storage://` schemes are also supported for backward compatibility, though `azure-blob://` is recommended. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/azure-env-param.md b/src/current/_includes/v25.1/misc/azure-env-param.md new file mode 100644 index 00000000000..62b6b01293e --- /dev/null +++ b/src/current/_includes/v25.1/misc/azure-env-param.md @@ -0,0 +1 @@ +The [Azure environment](https://learn.microsoft.com/azure/deployment-environments/concept-environments-key-concepts#environments) that the storage account belongs to. The accepted values are: `AZURECHINACLOUD`, `AZUREGERMANCLOUD`, `AZUREPUBLICCLOUD`, and [`AZUREUSGOVERNMENTCLOUD`](https://learn.microsoft.com/azure/azure-government/documentation-government-developer-guide). These are cloud environments that meet security, compliance, and data privacy requirements for the respective instance of Azure cloud. If the parameter is not specified, it will default to `AZUREPUBLICCLOUD`. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/azure-locations.md b/src/current/_includes/v25.1/misc/azure-locations.md new file mode 100644 index 00000000000..7119ff8b7cb --- /dev/null +++ b/src/current/_includes/v25.1/misc/azure-locations.md @@ -0,0 +1,30 @@ +| Location | SQL Statement | +| -------- | ------------- | +| eastasia (East Asia) | `INSERT into system.locations VALUES ('region', 'eastasia', 22.267, 114.188)` | +| southeastasia (Southeast Asia) | `INSERT into system.locations VALUES ('region', 'southeastasia', 1.283, 103.833)` | +| centralus (Central US) | `INSERT into system.locations VALUES ('region', 'centralus', 41.5908, -93.6208)` | +| eastus (East US) | `INSERT into system.locations VALUES ('region', 'eastus', 37.3719, -79.8164)` | +| eastus2 (East US 2) | `INSERT into system.locations VALUES ('region', 'eastus2', 36.6681, -78.3889)` | +| westus (West US) | `INSERT into system.locations VALUES ('region', 'westus', 37.783, -122.417)` | +| northcentralus (North Central US) | `INSERT into system.locations VALUES ('region', 'northcentralus', 41.8819, -87.6278)` | +| southcentralus (South Central US) | `INSERT into system.locations VALUES ('region', 'southcentralus', 29.4167, -98.5)` | +| northeurope (North Europe) | `INSERT into system.locations VALUES ('region', 'northeurope', 53.3478, -6.2597)` | +| westeurope (West Europe) | `INSERT into system.locations VALUES ('region', 'westeurope', 52.3667, 4.9)` | +| japanwest (Japan West) | `INSERT into system.locations VALUES ('region', 'japanwest', 34.6939, 135.5022)` | +| japaneast (Japan East) | `INSERT into system.locations VALUES ('region', 'japaneast', 35.68, 139.77)` | +| brazilsouth (Brazil South) | `INSERT into system.locations VALUES ('region', 'brazilsouth', -23.55, -46.633)` | +| australiaeast (Australia East) | `INSERT into system.locations VALUES ('region', 'australiaeast', -33.86, 151.2094)` | +| australiasoutheast (Australia Southeast) | `INSERT into system.locations VALUES ('region', 'australiasoutheast', -37.8136, 144.9631)` | +| southindia (South India) | `INSERT into system.locations VALUES ('region', 'southindia', 12.9822, 80.1636)` | +| centralindia (Central India) | `INSERT into system.locations VALUES ('region', 'centralindia', 18.5822, 73.9197)` | +| westindia (West India) | `INSERT into system.locations VALUES ('region', 'westindia', 19.088, 72.868)` | +| canadacentral (Canada Central) | `INSERT into system.locations VALUES ('region', 'canadacentral', 43.653, -79.383)` | +| canadaeast (Canada East) | `INSERT into system.locations VALUES ('region', 'canadaeast', 46.817, -71.217)` | +| uksouth (UK South) | `INSERT into system.locations VALUES ('region', 'uksouth', 50.941, -0.799)` | +| ukwest (UK West) | `INSERT into system.locations VALUES ('region', 'ukwest', 53.427, -3.084)` | +| westcentralus (West Central US) | `INSERT into system.locations VALUES ('region', 'westcentralus', 40.890, -110.234)` | +| westus2 (West US 2) | `INSERT into system.locations VALUES ('region', 'westus2', 47.233, -119.852)` | +| koreacentral (Korea Central) | `INSERT into system.locations VALUES ('region', 'koreacentral', 37.5665, 126.9780)` | +| koreasouth (Korea South) | `INSERT into system.locations VALUES ('region', 'koreasouth', 35.1796, 129.0756)` | +| francecentral (France Central) | `INSERT into system.locations VALUES ('region', 'francecentral', 46.3772, 2.3730)` | +| francesouth (France South) | `INSERT into system.locations VALUES ('region', 'francesouth', 43.8345, 2.1972)` | diff --git a/src/current/_includes/v25.1/misc/basic-terms.md b/src/current/_includes/v25.1/misc/basic-terms.md new file mode 100644 index 00000000000..f168f878c63 --- /dev/null +++ b/src/current/_includes/v25.1/misc/basic-terms.md @@ -0,0 +1,36 @@ +### Cluster +A group of interconnected CockroachDB nodes that function as a single distributed SQL database server. Nodes collaboratively organize transactions, and rebalance workload and data storage to optimize performance and fault-tolerance. + +Each cluster has its own authorization hierarchy, meaning that users and roles must be defined on that specific cluster. + +A CockroachDB cluster can be run in CockroachDB Cloud, within a customer [Organization]({% link {{ page.version.version }}/architecture/glossary.md %}#organization), or can be self-hosted. + +### Node +An individual instance of CockroachDB. One or more nodes form a cluster. + +### Range + +CockroachDB stores all user data (tables, indexes, etc.) and almost all system data in a sorted map of key-value pairs. This keyspace is divided into contiguous chunks called _ranges_, such that every key is found in one range. + +From a SQL perspective, a table and its secondary indexes initially map to a single range, where each key-value pair in the range represents a single row in the table (also called the _primary index_ because the table is sorted by the primary key) or a single row in a secondary index. As soon as the size of a range reaches [the default range size]({% link {{ page.version.version }}/configure-replication-zones.md %}#range-max-bytes), it is [split into two ranges]({% link {{ page.version.version }}/architecture/distribution-layer.md %}#range-splits). This process continues for these new ranges as the table and its indexes continue growing. + +### Replica + +A copy of a range stored on a node. By default, there are three [replicas]({% link {{ page.version.version }}/configure-replication-zones.md %}#num_replicas) of each range on different nodes. + +### Leaseholder + +The replica that holds the "range lease." This replica receives and coordinates all read and write requests for the range. + +For most types of tables and queries, the leaseholder is the only replica that can serve consistent reads (reads that return "the latest" data). + +### Raft protocol + +The [consensus protocol]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft) employed in CockroachDB that ensures that your data is safely stored on multiple nodes and that those nodes agree on the current state even if some of them are temporarily disconnected. + +### Raft leader + +For each range, the replica that is the "leader" for write requests. The leader uses the Raft protocol to ensure that a majority of replicas (the leader and enough followers) agree, based on their Raft logs, before committing the write. The Raft leader is almost always the same replica as the leaseholder. + +### Raft log +A time-ordered log of writes to a range that its replicas have agreed on. This log exists on-disk with each replica and is the range's source of truth for consistent replication. diff --git a/src/current/_includes/v25.1/misc/beta-release-warning.md b/src/current/_includes/v25.1/misc/beta-release-warning.md new file mode 100644 index 00000000000..c228f650d04 --- /dev/null +++ b/src/current/_includes/v25.1/misc/beta-release-warning.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_danger}} +Beta releases are intended for testing and experimentation only. Beta releases are not recommended for production use, as they can lead to data corruption, cluster unavailability, performance issues, etc. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/beta-warning.md b/src/current/_includes/v25.1/misc/beta-warning.md new file mode 100644 index 00000000000..4a5b9e3c6ae --- /dev/null +++ b/src/current/_includes/v25.1/misc/beta-warning.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_danger}} +**This is a beta feature.** It is currently undergoing continued testing. Please [file a Github issue]({% link {{ page.version.version }}/file-an-issue.md %}) with us if you identify a bug. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/bulk-permission-note.md b/src/current/_includes/v25.1/misc/bulk-permission-note.md new file mode 100644 index 00000000000..cb008696def --- /dev/null +++ b/src/current/_includes/v25.1/misc/bulk-permission-note.md @@ -0,0 +1 @@ +We recommend using [cloud storage]({% link {{ page.version.version }}/use-cloud-storage.md %}). You also need to ensure that the permissions at your storage destination are configured for the operation. See [Storage Permissions]({% link {{ page.version.version }}/use-cloud-storage.md %}#storage-permissions) for a list of the necessary permissions that each bulk operation requires. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/cert-auth-using-x509-subject.md b/src/current/_includes/v25.1/misc/cert-auth-using-x509-subject.md new file mode 100644 index 00000000000..281feb1986f --- /dev/null +++ b/src/current/_includes/v25.1/misc/cert-auth-using-x509-subject.md @@ -0,0 +1 @@ +If you manage your own Certificate Authority (CA) infrastructure, CockroachDB supports mapping between the Subject field of your [X.509 certificates](https://en.wikipedia.org/wiki/X.509) and SQL [roles]({% link {{ page.version.version }}/security-reference/authorization.md %}#roles). For more information, see [Certificate-based authentication using multiple values from the X.509 Subject field]({% link {{page.version.version}}/certificate-based-authentication-using-the-x509-subject-field.md %}). diff --git a/src/current/_includes/v25.1/misc/chrome-localhost.md b/src/current/_includes/v25.1/misc/chrome-localhost.md new file mode 100644 index 00000000000..d794ff339d0 --- /dev/null +++ b/src/current/_includes/v25.1/misc/chrome-localhost.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +If you are using Google Chrome, and you are getting an error about not being able to reach `localhost` because its certificate has been revoked, go to chrome://flags/#allow-insecure-localhost, enable "Allow invalid certificates for resources loaded from localhost", and then restart the browser. Enabling this Chrome feature degrades security for all sites running on `localhost`, not just CockroachDB's DB Console, so be sure to enable the feature only temporarily. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/csv-import-callout.md b/src/current/_includes/v25.1/misc/csv-import-callout.md new file mode 100644 index 00000000000..60555c5d0b6 --- /dev/null +++ b/src/current/_includes/v25.1/misc/csv-import-callout.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +The column order in your schema must match the column order in the file being imported. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/customizing-the-savepoint-name.md b/src/current/_includes/v25.1/misc/customizing-the-savepoint-name.md new file mode 100644 index 00000000000..6a00f8f6d8c --- /dev/null +++ b/src/current/_includes/v25.1/misc/customizing-the-savepoint-name.md @@ -0,0 +1,5 @@ +Set the `force_savepoint_restart` [session variable]({% link {{ page.version.version }}/set-vars.md %}#supported-variables) to `true` to enable using a custom name for the [retry savepoint]({% link {{ page.version.version }}/advanced-client-side-transaction-retries.md %}#retry-savepoints). + +Once this variable is set, the [`SAVEPOINT`]({% link {{ page.version.version }}/savepoint.md %}) statement will accept any name for the retry savepoint, not just `cockroach_restart`. In addition, it causes every savepoint name to be equivalent to `cockroach_restart`, therefore disallowing the use of [nested transactions]({% link {{ page.version.version }}/transactions.md %}#nested-transactions). + +This feature exists to support applications that want to use the [advanced client-side transaction retry protocol]({% link {{ page.version.version }}/advanced-client-side-transaction-retries.md %}), but cannot customize the name of savepoints to be `cockroach_restart`. For example, this may be necessary because you are using an ORM that requires its own names for savepoints. diff --git a/src/current/_includes/v25.1/misc/database-terms.md b/src/current/_includes/v25.1/misc/database-terms.md new file mode 100644 index 00000000000..78663985607 --- /dev/null +++ b/src/current/_includes/v25.1/misc/database-terms.md @@ -0,0 +1,30 @@ +### Consistency +The requirement that a transaction must change affected data only in allowed ways. CockroachDB uses "consistency" in both the sense of [ACID semantics](https://en.wikipedia.org/wiki/ACID) and the [CAP theorem](https://wikipedia.org/wiki/CAP_theorem), albeit less formally than either definition. + +### Isolation +The degree to which a transaction may be affected by other transactions running at the same time. CockroachDB provides the [`SERIALIZABLE`](https://wikipedia.org/wiki/Serializability) and `READ COMMITTED` isolation levels. For more information, see [Isolation levels]({% link {{ page.version.version }}/transactions.md %}#isolation-levels). + +### Consensus + The process of reaching agreement on whether a transaction is committed or aborted. CockroachDB uses the [Raft consensus protocol](#architecture-raft). In CockroachDB, when a range receives a write, a quorum of nodes containing replicas of the range acknowledge the write. This means your data is safely stored and a majority of nodes agree on the database's current state, even if some of the nodes are offline. + +When a write does not achieve consensus, forward progress halts to maintain consistency within the cluster. + +### Replication +The process of creating and distributing copies of data, as well as ensuring that those copies remain consistent. CockroachDB requires all writes to propagate to a [quorum](https://wikipedia.org/wiki/Quorum_%28distributed_computing%29) of copies of the data before being considered committed. This ensures the consistency of your data. + +### Transaction +A set of operations performed on a database that satisfy the requirements of [ACID semantics](https://en.wikipedia.org/wiki/ACID). This is a crucial feature for a consistent system to ensure developers can trust the data in their database. For more information about how transactions work in CockroachDB, see [Transaction Layer]({% link {{ page.version.version }}/architecture/transaction-layer.md %}). + +### Transaction contention + A [state of conflict]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention) that occurs when: + +- A [transaction]({% link {{ page.version.version }}/transactions.md %}) is unable to complete due to another concurrent or recent transaction attempting to write to the same data. This is also called *lock contention*. +- A transaction is [automatically retried]({% link {{ page.version.version }}/transactions.md %}#automatic-retries) because it could not be placed into a [serializable ordering]({% link {{ page.version.version }}/demo-serializable.md %}) among all of the currently executing transactions. This is also called a *serialization conflict*. If the automatic retry is not possible or fails, a [*transaction retry error*](../transaction-retry-error-reference.html) is emitted to the client, requiring a client application running under `SERIALIZABLE` isolation to [retry the transaction](../transaction-retry-error-reference.html#client-side-retry-handling). + +Steps should be taken to [reduce transaction contention]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#reduce-transaction-contention) in the first place. + +### Multi-active availability +A consensus-based notion of high availability that lets each node in the cluster handle reads and writes for a subset of the stored data (on a per-range basis). This is in contrast to _active-passive replication_, in which the active node receives 100% of request traffic, and _active-active_ replication, in which all nodes accept requests but typically cannot guarantee that reads are both up-to-date and fast. + +### User +A SQL user is an identity capable of executing SQL statements and performing other cluster actions against CockroachDB clusters. SQL users must authenticate with an option permitted on the cluster (username/password, single sign-on (SSO), or certificate). Note that a SQL/cluster user is distinct from a CockroachDB {{ site.data.products.cloud }} organization user. diff --git a/src/current/_includes/v25.1/misc/debug-subcommands.md b/src/current/_includes/v25.1/misc/debug-subcommands.md new file mode 100644 index 00000000000..25feb08481b --- /dev/null +++ b/src/current/_includes/v25.1/misc/debug-subcommands.md @@ -0,0 +1,5 @@ +While the `cockroach debug` command has a few subcommands, users are expected to use only the [`zip`]({% link {{ page.version.version }}/cockroach-debug-zip.md %}), [`encryption-active-key`]({% link {{ page.version.version }}/cockroach-debug-encryption-active-key.md %}), [`merge-logs`]({% link {{ page.version.version }}/cockroach-debug-merge-logs.md %}), [`list-files`](cockroach-debug-list-files.html), [`tsdump`](cockroach-debug-tsdump.html), and [`ballast`](cockroach-debug-ballast.html) subcommands. + +We recommend using the [`encryption-decrypt`]({% link {{ page.version.version }}/cockroach-debug-encryption-decrypt.md %}) and [`job-trace`]({% link {{ page.version.version }}/cockroach-debug-job-trace.md %}) subcommands only when directed by the [Cockroach Labs support team]({% link {{ page.version.version }}/support-resources.md %}). + +The other `debug` subcommands are useful only to Cockroach Labs. Output of `debug` commands may contain sensitive or secret information. diff --git a/src/current/_includes/v25.1/misc/declarative-schema-changer-note.md b/src/current/_includes/v25.1/misc/declarative-schema-changer-note.md new file mode 100644 index 00000000000..fb1374e2bee --- /dev/null +++ b/src/current/_includes/v25.1/misc/declarative-schema-changer-note.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_danger}} +`{{ page.title }}` now uses the [declarative schema changer]({% link {{ page.version.version }}/online-schema-changes.md %}#declarative-schema-changer) by default. Declarative schema changer statements and legacy schema changer statements operating on the same objects cannot exist within the same transaction. Either split the transaction into multiple transactions, or disable either the `sql.defaults.use_declarative_schema_changer` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) or the `use_declarative_schema_changer` [session variable]({% link {{ page.version.version }}/set-vars.md %}). +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/delete-statistics.md b/src/current/_includes/v25.1/misc/delete-statistics.md new file mode 100644 index 00000000000..6954194fc75 --- /dev/null +++ b/src/current/_includes/v25.1/misc/delete-statistics.md @@ -0,0 +1,15 @@ +To delete statistics for all tables in all databases: + +{% include_cached copy-clipboard.html %} +~~~ sql +DELETE FROM system.table_statistics WHERE true; +~~~ + +To delete a named set of statistics (e.g, one named "users_stats"), run a query like the following: + +{% include_cached copy-clipboard.html %} +~~~ sql +DELETE FROM system.table_statistics WHERE name = 'users_stats'; +~~~ + +For more information about the `DELETE` statement, see [`DELETE`]({% link {{ page.version.version }}/delete.md %}). diff --git a/src/current/_includes/v25.1/misc/diagnostics-callout.html b/src/current/_includes/v25.1/misc/diagnostics-callout.html new file mode 100644 index 00000000000..a969a8cf152 --- /dev/null +++ b/src/current/_includes/v25.1/misc/diagnostics-callout.html @@ -0,0 +1 @@ +{{site.data.alerts.callout_info}}By default, each node of a CockroachDB cluster periodically shares anonymous usage details with Cockroach Labs. For an explanation of the details that get shared and how to opt-out of reporting, see Diagnostics Reporting.{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/enterprise-features.md b/src/current/_includes/v25.1/misc/enterprise-features.md new file mode 100644 index 00000000000..5f9193e1ca9 --- /dev/null +++ b/src/current/_includes/v25.1/misc/enterprise-features.md @@ -0,0 +1,32 @@ +## Cluster optimization + +Feature | Description +--------+------------------------- +[Read Committed isolation]({% link {{ page.version.version }}/read-committed.md %}) | Achieve predictable query performance at high workload concurrencies, but without guaranteed transaction serializability. +[Follower Reads]({% link {{ page.version.version }}/follower-reads.md %}) | Reduce read latency in multi-region deployments by using the closest replica at the expense of reading slightly historical data. +[Multi-Region Capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}) | Row-level control over where your data is stored to help you reduce read and write latency and meet regulatory requirements. +[PL/pgSQL]({% link {{ page.version.version }}/plpgsql.md %}) | Use a procedural language in [user-defined functions]({% link {{ page.version.version }}/user-defined-functions.md %}) and [stored procedures]({% link {{ page.version.version }}/stored-procedures.md %}) to improve performance and enable more complex queries. +[Node Map]({% link {{ page.version.version }}/enable-node-map.md %}) | Visualize the geographical distribution of a cluster by plotting its node localities on a world map. +[Generic query plans]({% link {{ page.version.version }}/cost-based-optimizer.md %}#query-plan-type) | Improve performance for prepared statements by enabling generic plans that eliminate most of the query latency attributed to planning. +[`VECTOR` type]({% link {{ page.version.version }}/vector.md %}) | Represent data points in multi-dimensional space, using fixed-length arrays of floating-point numbers. + +## Recovery and streaming + +Feature | Description +--------+------------------------- +[`BACKUP`]({% link {{ page.version.version }}/backup.md %}) and restore capabilities | Taking and restoring [full backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}), [incremental backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}), [backups with revision history]({% link {{ page.version.version }}/take-backups-with-revision-history-and-restore-from-a-point-in-time.md %}), [locality-aware backups](take-and-restore-locality-aware-backups.html), and [encrypted backups](take-and-restore-encrypted-backups.html). +[Changefeeds into a Configurable Sink]({% link {{ page.version.version }}/create-changefeed.md %}) | For every change in a configurable allowlist of tables, configure a changefeed to emit a record to a configurable sink: Apache Kafka, cloud storage, Google Cloud Pub/Sub, or a webhook sink. These records can be processed by downstream systems for reporting, caching, or full-text indexing. +[Change Data Capture Queries]({% link {{ page.version.version }}/cdc-queries.md %}) | Use `SELECT` queries to filter and modify change data before sending it to a changefeed's sink. +[Physical Cluster Replication]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}) | Send all data at the byte level from a primary cluster to an independent standby cluster. Existing data and ongoing changes on the active primary cluster, which is serving application data, replicate asynchronously to the passive standby cluster. + +## Security and IAM + +Feature | Description +--------+------------------------- +[Encryption at Rest]({% link {{ page.version.version }}/security-reference/encryption.md %}#encryption-at-rest-enterprise) | Enable automatic transparent encryption of a node's data on the local disk using AES in counter mode, with all key sizes allowed. This feature works together with CockroachDB's automatic encryption of data in transit. +[Column-level encryption]({% link {{ page.version.version }}/column-level-encryption.md %}) | Encrypt specific columns within a table. +[GSSAPI with Kerberos Authentication]({% link {{ page.version.version }}/gssapi_authentication.md %}) | Authenticate to your cluster using identities stored in an external enterprise directory system that supports Kerberos, such as Active Directory. +[Cluster Single Sign-on (SSO)]({% link {{ page.version.version }}/sso-sql.md %}) | Grant SQL access to a cluster using JSON Web Tokens (JWTs) issued by an external identity provider (IdP) or custom JWT issuer. +[Single Sign-on (SSO) for DB Console]({% link {{ page.version.version }}/sso-db-console.md %}) | Grant access to a cluster's DB Console interface using SSO through an IdP that supports OIDC. +[Role-based SQL Audit Logs]({% link {{ page.version.version }}/role-based-audit-logging.md %}) | Enable logging of queries being executed against your system by specific users or roles. +[Certificate-based authentication using multiple values from the X.509 Subject field]({% link {{ page.version.version }}/certificate-based-authentication-using-the-x509-subject-field.md %}) | Map SQL user [roles]({% link {{ page.version.version }}/security-reference/authorization.md %}#roles) to values in the Subject field of the [X.509 certificate](https://en.wikipedia.org/wiki/X.509) used for [TLS authentication]({% link {{ page.version.version }}/security-reference/transport-layer-security.md %}#what-is-transport-layer-security-tls). diff --git a/src/current/_includes/v25.1/misc/explore-benefits-see-also.md b/src/current/_includes/v25.1/misc/explore-benefits-see-also.md new file mode 100644 index 00000000000..2ad7178c808 --- /dev/null +++ b/src/current/_includes/v25.1/misc/explore-benefits-see-also.md @@ -0,0 +1,7 @@ +- [Replication & Rebalancing]({% link {{ page.version.version }}/demo-replication-and-rebalancing.md %}) +- [CockroachDB Resilience]({% link {{ page.version.version }}/demo-cockroachdb-resilience.md %}) +- [Low Latency Multi-Region Deployment]({% link {{ page.version.version }}/demo-low-latency-multi-region-deployment.md %}) +- [Serializable Transactions]({% link {{ page.version.version }}/demo-serializable.md %}) +- [Cross-Cloud Migration]({% link {{ page.version.version }}/demo-automatic-cloud-migration.md %}) +- [Orchestration]({% link {{ page.version.version }}/orchestrate-a-local-cluster-with-kubernetes-insecure.md %}) +- [JSON Support]({% link {{ page.version.version }}/demo-json-support.md %}) diff --git a/src/current/_includes/v25.1/misc/external-connection-kafka.md b/src/current/_includes/v25.1/misc/external-connection-kafka.md new file mode 100644 index 00000000000..2ffa2b599a1 --- /dev/null +++ b/src/current/_includes/v25.1/misc/external-connection-kafka.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +You can create an external connection to represent a Kafka sink URI. This allows you to specify the external connection's name in statements rather than the provider-specific URI. For detail on using external connections, see the [`CREATE EXTERNAL CONNECTION`]({% link {{ page.version.version }}/create-external-connection.md %}) page. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/external-connection-note.md b/src/current/_includes/v25.1/misc/external-connection-note.md new file mode 100644 index 00000000000..f9bc7914ed8 --- /dev/null +++ b/src/current/_includes/v25.1/misc/external-connection-note.md @@ -0,0 +1 @@ +You can create an external connection to represent an external storage or sink URI. This allows you to specify the external connection's name in statements rather than the provider-specific URI. For detail on using external connections, see the [`CREATE EXTERNAL CONNECTION`]({% link {{ page.version.version }}/create-external-connection.md %}) page. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/external-io-privilege.md b/src/current/_includes/v25.1/misc/external-io-privilege.md new file mode 100644 index 00000000000..c3f92f8e24e --- /dev/null +++ b/src/current/_includes/v25.1/misc/external-io-privilege.md @@ -0,0 +1 @@ +You can grant a user the `EXTERNALIOIMPLICITACCESS` [system-level privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges) to interact with external resources that require implicit access. \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/force-index-selection.md b/src/current/_includes/v25.1/misc/force-index-selection.md new file mode 100644 index 00000000000..a2f70c98ee6 --- /dev/null +++ b/src/current/_includes/v25.1/misc/force-index-selection.md @@ -0,0 +1,157 @@ +By using the explicit index annotation, you can override [CockroachDB's index selection](https://www.cockroachlabs.com/blog/index-selection-cockroachdb-2/) and use a specific [index]({% link {{ page.version.version }}/indexes.md %}) when reading from a named table. + +{{site.data.alerts.callout_info}} +Index selection can impact [performance]({% link {{ page.version.version }}/performance-best-practices-overview.md %}), but does not change the result of a query. +{{site.data.alerts.end}} + +##### Force index scan + +To force a scan of a specific index: + +{% include_cached copy-clipboard.html %} +~~~ sql +SELECT * FROM table@my_idx; +~~~ + +This is equivalent to the longer expression: + +{% include_cached copy-clipboard.html %} +~~~ sql +SELECT * FROM table@{FORCE_INDEX=my_idx}; +~~~ + +##### Force reverse scan + +To force a reverse scan of a specific index: + +{% include_cached copy-clipboard.html %} +~~~ sql +SELECT * FROM table@{FORCE_INDEX=my_idx,DESC}; +~~~ + +Forcing a reverse scan can help with [performance tuning]({% link {{ page.version.version }}/performance-best-practices-overview.md %}). To choose an index and its scan direction: + +{% include_cached copy-clipboard.html %} +~~~ sql +SELECT * FROM table@{FORCE_INDEX=idx[,DIRECTION]}; +~~~ + +where the optional `DIRECTION` is either `ASC` (ascending) or `DESC` (descending). + +When a direction is specified, that scan direction is forced; otherwise the [cost-based optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) is free to choose the direction it calculates will result in the best performance. + +You can verify that the optimizer is choosing your desired scan direction using [`EXPLAIN (OPT)`]({% link {{ page.version.version }}/explain.md %}#opt-option). For example, given the table + +{% include_cached copy-clipboard.html %} +~~~ sql +CREATE TABLE kv (K INT PRIMARY KEY, v INT); +~~~ + +you can check the scan direction with: + +{% include_cached copy-clipboard.html %} +~~~ sql +EXPLAIN (opt) SELECT * FROM users@{FORCE_INDEX=primary,DESC}; +~~~ + +~~~ + text ++-------------------------------------+ + scan users,rev + └── flags: force-index=primary,rev +(2 rows) +~~~ + +#### Force inverted index scan + +To force a scan of any [inverted index]({% link {{ page.version.version }}/inverted-indexes.md %}) of the hinted table: + +{% include_cached copy-clipboard.html %} +~~~ sql +SELECT * FROM table@{FORCE_INVERTED_INDEX}; +~~~ + +The `FORCE_INVERTED_INDEX` hint does not allow specifying an inverted index. If no query plan can be generated, the query will result in the error: + +~~~ +ERROR: could not produce a query plan conforming to the FORCE_INVERTED_INDEX hint +~~~ + +##### Force partial index scan + +To force a [partial index scan]({% link {{ page.version.version }}/partial-indexes.md %}), your statement must have a `WHERE` clause that implies the partial index filter. + +{% include_cached copy-clipboard.html %} +~~~ sql +CREATE TABLE t ( + a INT, + INDEX idx (a) WHERE a > 0); +INSERT INTO t(a) VALUES (5); +SELECT * FROM t@idx WHERE a > 0; +~~~ + +~~~ +CREATE TABLE + +Time: 13ms total (execution 12ms / network 0ms) + +INSERT 1 + +Time: 22ms total (execution 21ms / network 0ms) + + a +----- + 5 +(1 row) + +Time: 1ms total (execution 1ms / network 0ms) +~~~ + +##### Force partial GIN index scan + +To force a [partial GIN index]({% link {{ page.version.version }}/inverted-indexes.md %}#partial-gin-indexes) scan, your statement must have a `WHERE` clause that: + +- Implies the partial index. +- Constrains the GIN index scan. + +{% include_cached copy-clipboard.html %} +~~~ sql +DROP TABLE t; +CREATE TABLE t ( + j JSON, + INVERTED INDEX idx (j) WHERE j->'a' = '1'); +INSERT INTO t(j) + VALUES ('{"a": 1}'), + ('{"a": 3, "b": 2}'), + ('{"a": 1, "b": 2}'); +SELECT * FROM t@idx WHERE j->'a' = '1' AND j->'b' = '2'; +~~~ + +~~~ +DROP TABLE + +Time: 68ms total (execution 22ms / network 45ms) + +CREATE TABLE + +Time: 10ms total (execution 10ms / network 0ms) + +INSERT 3 + +Time: 22ms total (execution 22ms / network 0ms) + + j +-------------------- + {"a": 1, "b": 2} +(1 row) + +Time: 1ms total (execution 1ms / network 0ms) +~~~ + +##### Prevent full scan + +{% include {{ page.version.version }}/sql/no-full-scan.md %} + +{{site.data.alerts.callout_success}} +For other ways to prevent full scans, refer to [Prevent the optimizer from planning full scans]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#prevent-the-optimizer-from-planning-full-scans). +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/gce-locations.md b/src/current/_includes/v25.1/misc/gce-locations.md new file mode 100644 index 00000000000..22122aae78d --- /dev/null +++ b/src/current/_includes/v25.1/misc/gce-locations.md @@ -0,0 +1,18 @@ +| Location | SQL Statement | +| ------ | ------ | +| us-east1 (South Carolina) | `INSERT into system.locations VALUES ('region', 'us-east1', 33.836082, -81.163727)` | +| us-east4 (N. Virginia) | `INSERT into system.locations VALUES ('region', 'us-east4', 37.478397, -76.453077)` | +| us-central1 (Iowa) | `INSERT into system.locations VALUES ('region', 'us-central1', 42.032974, -93.581543)` | +| us-west1 (Oregon) | `INSERT into system.locations VALUES ('region', 'us-west1', 43.804133, -120.554201)` | +| northamerica-northeast1 (Montreal) | `INSERT into system.locations VALUES ('region', 'northamerica-northeast1', 56.130366, -106.346771)` | +| europe-west1 (Belgium) | `INSERT into system.locations VALUES ('region', 'europe-west1', 50.44816, 3.81886)` | +| europe-west2 (London) | `INSERT into system.locations VALUES ('region', 'europe-west2', 51.507351, -0.127758)` | +| europe-west3 (Frankfurt) | `INSERT into system.locations VALUES ('region', 'europe-west3', 50.110922, 8.682127)` | +| europe-west4 (Netherlands) | `INSERT into system.locations VALUES ('region', 'europe-west4', 53.4386, 6.8355)` | +| europe-west6 (Zürich) | `INSERT into system.locations VALUES ('region', 'europe-west6', 47.3769, 8.5417)` | +| asia-east1 (Taiwan) | `INSERT into system.locations VALUES ('region', 'asia-east1', 24.0717, 120.5624)` | +| asia-northeast1 (Tokyo) | `INSERT into system.locations VALUES ('region', 'asia-northeast1', 35.689487, 139.691706)` | +| asia-southeast1 (Singapore) | `INSERT into system.locations VALUES ('region', 'asia-southeast1', 1.352083, 103.819836)` | +| australia-southeast1 (Sydney) | `INSERT into system.locations VALUES ('region', 'australia-southeast1', -33.86882, 151.209296)` | +| asia-south1 (Mumbai) | `INSERT into system.locations VALUES ('region', 'asia-south1', 19.075984, 72.877656)` | +| southamerica-east1 (São Paulo) | `INSERT into system.locations VALUES ('region', 'southamerica-east1', -23.55052, -46.633309)` | diff --git a/src/current/_includes/v25.1/misc/geojson_geometry_note.md b/src/current/_includes/v25.1/misc/geojson_geometry_note.md new file mode 100644 index 00000000000..a023f205c20 --- /dev/null +++ b/src/current/_includes/v25.1/misc/geojson_geometry_note.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +The screenshots in these examples were generated using [geojson.io](http://geojson.io), but they are designed to showcase the shapes, not the map. Representing `GEOMETRY` data in GeoJSON can lead to unexpected results if using geometries with [SRIDs]({% link {{ page.version.version }}/architecture/glossary.md %}#srid) other than 4326 (as shown below). +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/haproxy.md b/src/current/_includes/v25.1/misc/haproxy.md new file mode 100644 index 00000000000..c94bd654466 --- /dev/null +++ b/src/current/_includes/v25.1/misc/haproxy.md @@ -0,0 +1,39 @@ +By default, the generated configuration file is called `haproxy.cfg` and looks as follows, with the `server` addresses pre-populated correctly: + + ~~~ + global + maxconn 4096 + + defaults + mode tcp + # Timeout values should be configured for your specific use. + # See: https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#4-timeout%20connect + timeout connect 10s + timeout client 1m + timeout server 1m + # TCP keep-alive on client side. Server already enables them. + option clitcpka + + listen psql + bind :26257 + mode tcp + balance roundrobin + option httpchk GET /health?ready=1 + server cockroach1
`timeout client`
`timeout server` | Timeout values that should be suitable for most deployments. + `bind` | The port that HAProxy listens on. This is the port clients will connect to and thus needs to be allowed by your network configuration.
This tutorial assumes HAProxy is running on a separate machine from CockroachDB nodes. If you run HAProxy on the same machine as a node (not recommended), you'll need to change this port, as `26257` is likely already being used by the CockroachDB node. + `balance` | The balancing algorithm. This is set to `roundrobin` to ensure that connections get rotated amongst nodes (connection 1 on node 1, connection 2 on node 2, etc.). Check the [HAProxy Configuration Manual](http://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4-balance) for details about this and other balancing algorithms. + `option httpchk` | The HTTP endpoint that HAProxy uses to check node health. [`/health?ready=1`]({% link {{ page.version.version }}/monitoring-and-alerting.md %}#health-ready-1) ensures that HAProxy doesn't direct traffic to nodes that are live but not ready to receive requests. + `server` | For each included node, this field specifies the address the node advertises to other nodes in the cluster, i.e., the addressed pass in the [`--advertise-addr` flag]({% link {{ page.version.version }}/cockroach-start.md %}#networking) on node startup. Make sure hostnames are resolvable and IP addresses are routable from HAProxy. + + {{site.data.alerts.callout_info}} + For full details on these and other configuration settings, see the [HAProxy Configuration Manual](http://cbonte.github.io/haproxy-dconv/1.7/configuration.html). + {{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/htpp-import-only.md b/src/current/_includes/v25.1/misc/htpp-import-only.md new file mode 100644 index 00000000000..e69de29bb2d diff --git a/src/current/_includes/v25.1/misc/import-perf.md b/src/current/_includes/v25.1/misc/import-perf.md new file mode 100644 index 00000000000..34bee9acdb4 --- /dev/null +++ b/src/current/_includes/v25.1/misc/import-perf.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_success}} +For best practices for optimizing import performance in CockroachDB, see [Import Performance Best Practices]({% link {{ page.version.version }}/import-performance-best-practices.md %}). +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/misc/index-storage-parameters.md b/src/current/_includes/v25.1/misc/index-storage-parameters.md new file mode 100644 index 00000000000..e2aa6d76301 --- /dev/null +++ b/src/current/_includes/v25.1/misc/index-storage-parameters.md @@ -0,0 +1,14 @@ +| Parameter name | Description | Data type | Default value +|---------------------+----------------------|-----|------| +| `bucket_count` | The number of buckets into which a [hash-sharded index]({% link {{ page.version.version }}/hash-sharded-indexes.md %}) will split. | Integer | The value of the `sql.defaults.default_hash_sharded_index_bucket_count` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}). | +| `geometry_max_x` | The maximum X-value of the [spatial reference system]({% link {{ page.version.version }}/architecture/glossary.md %}#spatial-reference-system) for the object(s) being covered. This only needs to be set if you are using a custom [SRID]({% link {{ page.version.version }}/architecture/glossary.md %}#srid). | | Derived from SRID bounds, else `(1 << 31) -1`. | +| `geometry_max_y` | The maximum Y-value of the [spatial reference system]({% link {{ page.version.version }}/architecture/glossary.md %}#spatial-reference-system) for the object(s) being covered. This only needs to be set if you are using a custom [SRID]({% link {{ page.version.version }}/architecture/glossary.md %}#srid). | | Derived from SRID bounds, else `(1 << 31) -1`. | +| `geometry_min_x` | The minimum X-value of the [spatial reference system]({% link {{ page.version.version }}/architecture/glossary.md %}#spatial-reference-system) for the object(s) being covered. This only needs to be set if the default bounds of the SRID are too large/small for the given data, or SRID = 0 and you wish to use a smaller range (unfortunately this is currently not exposed, but is viewable on
-
+
The CockroachDB binary for Linux requires glibc, libncurses, and tzdata, which are found by default on nearly all Linux distributions, with Alpine as the notable exception.
`--log-config-file` can also be used.
**Note:** The logging flags below cannot be combined with `--log`, but can be defined instead in the YAML payload. +`--log-config-file` | Configure logging parameters by specifying a path to a YAML file. For details, see [Configure logs]({% link {{ page.version.version }}/configure-logs.md %}#flag). If a YAML configuration is not specified, the [default configuration]({% link {{ page.version.version }}/configure-logs.md %}#default-logging-configuration) is used.
`--log` can also be used.
**Note:** The logging flags below cannot be combined with `--log-config-file`, but can be defined instead in the YAML file. +`--log-dir` | An alias for the [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag) flag, for configuring the log directory where log files are stored and written to. Specifically, `--log-dir=XXX` is an alias for `--log='file-defaults: {dir: XXX}'`.
Setting `--log-dir` to a blank directory (`--log-dir=`) disables logging to files. Do not use `--log-dir=""`; this creates a new directory named `""` and stores log files in that directory. +`--log-group-max-size` | An alias for the [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag) flag, for configuring the maximum size for a logging group (for example, `cockroach`, `cockroach-sql-audit`, `cockroach-auth`, `cockroach-sql-exec`, `cockroach-pebble`), after which the oldest log file is deleted. `--log-group-max-size=XXX` is an alias for `--log='file-defaults: {max-group-size: XXX}'`. Accepts a valid file size, such as `--log-group-max-size=1GiB`.
**Default:** `100MiB` +`--log-file-max-size` | An alias for [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag), used to specify the maximum size that a log file can grow before a new log file is created. `--log-file-max-size=XXX` is an alias for `--log='file-defaults: {max-file-size: XXX}'`. Accepts a valid file size, such as `--log-file-max-size=2MiB`. **Requires** logging to files.
**Default:** `10MiB` +`--log-file-verbosity` | An alias for [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag), used to specify the minimum [severity level]({% link {{ page.version.version }}/logging.md %}#logging-levels-severities) of messages that are logged. `--log-file-verbosity=XXX` is an alias for `--log='file-defaults: {filter: XXX}'`. When a severity is specified, such as `--log-file-verbosity=WARNING`, log messages that are below the specified severity level are not written to the target log file. **Requires** logging to files.
**Default:** `INFO` +`--logtostderr` | An alias for [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag), to optionally output log messages at or above the configured [severity level]({% link {{ page.version.version }}/logging.md %}#logging-levels-severities) to the `stderr` sink. `--logtostderr=XXX` is an alias for `--log='sinks: {stderr: {filter: XXX}}'`. Accepts a valid [severity level]({% link {{ page.version.version }}/logging.md %}#logging-levels-severities). If no value is specified, by default messages related to server commands are logged to `stderr` at `INFO` severity and above, and messages related to client commands are logged to `stderr` at `WARNING` severity and above.
Setting `--logtostderr=NONE` disables logging to `stderr`.
**Default:** `UNKNOWN` +`--no-color` | An alias for [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag) flag, used to control whether log output to the `stderr` sinc is colorized. `--no-color=XXX` is an alias for `--log='sinks: {stderr: {no-color: XXX}}'`. Accepts either `true` or `false`.
When set to `false`, messages logged to `stderr` are colorized based on [severity level]({% link {{ page.version.version }}/logging.md %}#logging-levels-severities).
**Default:** `false` +`--redactable-logs` | An alias for [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag) flag, used to whether [redaction markers]({% link {{ page.version.version }}/configure-logs.md %}#redact-logs) are used in place of secret or sensitive information in log messages. `--redactable-logs=XXX` is an alias for `--log='file-defaults: {redactable: XXX}'`. Accepts `true` or `false`.
**Default:** `false` +`--sql-audit-dir` | An alias for [`--log`]({% link {{ page.version.version }}/configure-logs.md %}#flag), used to optionally confine log output of the `SENSITIVE_ACCESS` [logging channel]({% link {{ page.version.version }}/logging-overview.md %}#logging-channels) to a separate directory. `--sql-audit-dir=XXX` is an alias for `--log='sinks: {file-groups: {sql-audit: {channels: SENSITIVE_ACCESS, dir: ...}}}'`.
Enabling `SENSITIVE_ACCESS` logs can negatively impact performance. As a result, we recommend using the `SENSITIVE_ACCESS` channel for security purposes only. For more information, refer to [Security and Audit Monitoring]({% link {{ page.version.version }}/logging-use-cases.md %}#security-and-audit-monitoring). diff --git a/src/current/_includes/v25.1/misc/movr-live-demo.md b/src/current/_includes/v25.1/misc/movr-live-demo.md new file mode 100644 index 00000000000..f8cfb24cb21 --- /dev/null +++ b/src/current/_includes/v25.1/misc/movr-live-demo.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_success}} +For a live demo of the deployed example application, see [https://movr.cloud](https://movr.cloud). +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/movr-schema.md b/src/current/_includes/v25.1/misc/movr-schema.md new file mode 100644 index 00000000000..e838bcf4572 --- /dev/null +++ b/src/current/_includes/v25.1/misc/movr-schema.md @@ -0,0 +1,12 @@ +The six tables in the `movr` database store user, vehicle, and ride data for MovR: + +Table | Description +--------|---------------------------- +`users` | People registered for the service. +`vehicles` | The pool of vehicles available for the service. +`rides` | When and where users have rented a vehicle. +`promo_codes` | Promotional codes for users. +`user_promo_codes` | Promotional codes in use by users. +`vehicle_location_histories` | Vehicle location history. + +
diff --git a/src/current/_includes/v25.1/misc/movr-workflow.md b/src/current/_includes/v25.1/misc/movr-workflow.md
new file mode 100644
index 00000000000..a682c099b70
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/movr-workflow.md
@@ -0,0 +1,76 @@
+The workflow for MovR is as follows:
+
+1. A user loads the app and sees the 25 closest vehicles.
+
+ For example:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > SELECT id, city, status FROM vehicles WHERE city='amsterdam' limit 25;
+ ~~~
+
+1. The user signs up for the service.
+
+ For example:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > INSERT INTO users (id, name, address, city, credit_card)
+ VALUES ('66666666-6666-4400-8000-00000000000f', 'Mariah Lam', '88194 Angela Gardens Suite 60', 'amsterdam', '123245696');
+ ~~~
+
+ {{site.data.alerts.callout_info}}Usually for Universally Unique Identifier (UUID) you would need to generate it automatically but for the sake of this follow up we will use predetermined UUID to keep track of them in our examples.{{site.data.alerts.end}}
+
+1. In some cases, the user adds their own vehicle to share.
+
+ For example:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > INSERT INTO vehicles (id, city, type, owner_id,creation_time,status, current_location, ext)
+ VALUES ('ffffffff-ffff-4400-8000-00000000000f', 'amsterdam', 'skateboard', '66666666-6666-4400-8000-00000000000f', current_timestamp(), 'available', '88194 Angela Gardens Suite 60', '{"color": "blue"}');
+ ~~~
+1. More often, the user reserves a vehicle and starts a ride, applying a promo code, if available and valid.
+
+ For example:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > SELECT code FROM user_promo_codes WHERE user_id ='66666666-6666-4400-8000-00000000000f';
+ ~~~
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > UPDATE vehicles SET status = 'in_use' WHERE id='bbbbbbbb-bbbb-4800-8000-00000000000b';
+ ~~~
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > INSERT INTO rides (id, city, vehicle_city, rider_id, vehicle_id, start_address,end_address, start_time, end_time, revenue)
+ VALUES ('cd032f56-cf1a-4800-8000-00000000066f', 'amsterdam', 'amsterdam', '66666666-6666-4400-8000-00000000000f', 'bbbbbbbb-bbbb-4800-8000-00000000000b', '70458 Mary Crest', '', TIMESTAMP '2020-10-01 10:00:00.123456', NULL, 0.0);
+ ~~~
+
+1. During the ride, MovR tracks the location of the vehicle.
+
+ For example:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > INSERT INTO vehicle_location_histories (city, ride_id, timestamp, lat, long)
+ VALUES ('amsterdam', 'cd032f56-cf1a-4800-8000-00000000066f', current_timestamp(), -101, 60);
+ ~~~
+
+1. The user ends the ride and releases the vehicle.
+
+ For example:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > UPDATE vehicles SET status = 'available' WHERE id='bbbbbbbb-bbbb-4800-8000-00000000000b';
+ ~~~
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ sql
+ > UPDATE rides SET end_address ='33862 Charles Junctions Apt. 49', end_time=TIMESTAMP '2020-10-01 10:30:00.123456', revenue=88.6
+ WHERE id='cd032f56-cf1a-4800-8000-00000000066f';
+ ~~~
diff --git a/src/current/_includes/v25.1/misc/multiregion-max-offset.md b/src/current/_includes/v25.1/misc/multiregion-max-offset.md
new file mode 100644
index 00000000000..794ed12ca84
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/multiregion-max-offset.md
@@ -0,0 +1 @@
+For new clusters using the [multi-region SQL abstractions]({% link {{ page.version.version }}/multiregion-overview.md %}), Cockroach Labs recommends lowering the [`--max-offset`]({% link {{ page.version.version }}/cockroach-start.md %}#flags-max-offset) setting to `250ms`. This setting is especially helpful for lowering the write latency of [global tables]({% link {{ page.version.version }}/table-localities.md %}#global-tables). Nodes can run with different values for `--max-offset`, but only for the purpose of updating the setting across the cluster using a rolling upgrade.
diff --git a/src/current/_includes/v25.1/misc/note-egress-perimeter-cdc-backup.md b/src/current/_includes/v25.1/misc/note-egress-perimeter-cdc-backup.md
new file mode 100644
index 00000000000..9b80cec66f6
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/note-egress-perimeter-cdc-backup.md
@@ -0,0 +1,3 @@
+{{site.data.alerts.callout_info}}
+Cockroach Labs recommends enabling Egress Perimeter Controls on CockroachDB {{ site.data.products.advanced }} clusters to mitigate the risk of data exfiltration when accessing external resources, such as cloud storage for change data capture or backup and restore operations. See [Egress Perimeter Controls]({% link cockroachcloud/egress-perimeter-controls.md %}) for detail and setup instructions.
+{{site.data.alerts.end}}
diff --git a/src/current/_includes/v25.1/misc/remove-user-callout.html b/src/current/_includes/v25.1/misc/remove-user-callout.html
new file mode 100644
index 00000000000..925f83d779d
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/remove-user-callout.html
@@ -0,0 +1 @@
+Removing a user does not remove that user's privileges. Therefore, to prevent a future user with an identical username from inheriting an old user's privileges, it's important to revoke a user's privileges before or after removing the user.
diff --git a/src/current/_includes/v25.1/misc/s3-compatible-warning.md b/src/current/_includes/v25.1/misc/s3-compatible-warning.md
new file mode 100644
index 00000000000..5ad4966ad94
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/s3-compatible-warning.md
@@ -0,0 +1,3 @@
+{{site.data.alerts.callout_danger}}
+While Cockroach Labs actively tests Amazon S3, Google Cloud Storage, and Azure Storage, we **do not** test [S3-compatible services]({% link {{ page.version.version }}/cloud-storage-authentication.md %}) (e.g., [MinIO](https://min.io/), [Red Hat Ceph](https://docs.ceph.com/en/pacific/radosgw/s3/)).
+{{site.data.alerts.end}}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/misc/schema-change-stmt-note.md b/src/current/_includes/v25.1/misc/schema-change-stmt-note.md
new file mode 100644
index 00000000000..792cd2b9e51
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/schema-change-stmt-note.md
@@ -0,0 +1,3 @@
+{{site.data.alerts.callout_info}}
+The `{{ page.title }}` statement performs a schema change. For more information about how online schema changes work in CockroachDB, see [Online Schema Changes]({% link {{ page.version.version }}/online-schema-changes.md %}).
+{{site.data.alerts.end}}
diff --git a/src/current/_includes/v25.1/misc/schema-change-view-job.md b/src/current/_includes/v25.1/misc/schema-change-view-job.md
new file mode 100644
index 00000000000..37e46feb4d8
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/schema-change-view-job.md
@@ -0,0 +1 @@
+This schema change statement is registered as a job. You can view long-running jobs with [`SHOW JOBS`]({% link {{ page.version.version }}/show-jobs.md %}).
diff --git a/src/current/_includes/v25.1/misc/session-vars.md b/src/current/_includes/v25.1/misc/session-vars.md
new file mode 100644
index 00000000000..e4f9e1eee6b
--- /dev/null
+++ b/src/current/_includes/v25.1/misc/session-vars.md
@@ -0,0 +1,109 @@
+| Variable name | Description | Initial value | Modify with [`SET`]({% link {{ page.version.version }}/set-vars.md %})? | View with [`SHOW`]({% link {{ page.version.version }}/show-vars.md %})? |
+|---|---|---|---|---|
+| `application_name` | The current application name for statistics collection. | Empty string, or `cockroach` for sessions from the [built-in SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}). | Yes | Yes |
+| `autocommit_before_ddl` | When the [`autocommit_before_ddl` session setting]({% link {{page.version.version}}/set-vars.md %}#autocommit-before-ddl) is set to `on`, any schema change statement that is sent during an [explicit transaction]({% link {{page.version.version}}/transactions.md %}) will cause the transaction to [commit]({% link {{page.version.version}}/commit-transaction.md %}) before executing the schema change. | `off` | Yes | Yes |
+| `bytea_output` | The [mode for conversions from `STRING` to `BYTES`]({% link {{ page.version.version }}/bytes.md %}#supported-conversions). | hex | Yes | Yes |
+| `client_min_messages` | The severity level of notices displayed in the [SQL shell]({% link {{ page.version.version }}/cockroach-sql.md %}). Accepted values include `debug5`, `debug4`, `debug3`, `debug2`, `debug1`, `log`, `notice`, `warning`, and `error`. | `notice` | Yes | Yes |
+| `copy_from_atomic_enabled` | If set to `on`, [`COPY FROM`]({% link {{ page.version.version }}/copy-from.md %}) statements are committed atomically, matching PostgreSQL behavior. If set to `off`, `COPY FROM` statements are segmented into batches of 100 rows unless issued within an explicit transaction, matching the CockroachDB behavior in versions prior to v22.2. | `on` | Yes | Yes |
+| `copy_transaction_quality_of_service` | The default quality of service for [`COPY`]({% link {{ page.version.version }}/copy-from.md %}) statements in the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#copy-qos). | `background` | Yes | Yes |
+| `cost_scans_with_default_col_size` | Whether to prevent the optimizer from considering column size when costing plans. | `false` | Yes | Yes |
+| `crdb_version` | The version of CockroachDB. | CockroachDB OSS version | No | Yes |
+| `database` | The [current database]({% link {{ page.version.version }}/sql-name-resolution.md %}#current-database). | Database in connection string, or empty if not specified. | Yes | Yes |
+| `datestyle` | The input string format for [`DATE`]({% link {{ page.version.version }}/date.md %}) and [`TIMESTAMP`]({% link {{ page.version.version }}/timestamp.md %}) values. Accepted values include `ISO,MDY`, `ISO,DMY`, and `ISO,YMD`. | The value set by the `sql.defaults.datestyle` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`ISO,MDY`, by default). | Yes | Yes |
+| `default_int_size` | The size, in bytes, of an [`INT`]({% link {{ page.version.version }}/int.md %}) type. | `8` | Yes | Yes |
+| `default_text_search_config` | The dictionary used to normalize tokens and eliminate stop words when calling a [full-text search function]({% link {{ page.version.version }}/functions-and-operators.md %}#full-text-search-functions) without a configuration parameter. See [Full-Text Search]({% link {{ page.version.version }}/full-text-search.md %}). | `english` | Yes | Yes |
+| `default_transaction_isolation` | The isolation level at which transactions in the session execute ([`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) or [`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %})). See [Isolation levels]({% link {{ page.version.version }}/transactions.md %}#isolation-levels). | `SERIALIZABLE` | Yes | Yes |
+| `default_transaction_priority` | The default transaction priority for the current session. The supported options are `low`, `normal`, and `high`. | `normal` | Yes | Yes |
+| `default_transaction_quality_of_service` | The default transaction quality of service for the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#set-quality-of-service-level-for-a-session). | `regular` | Yes | Yes |
+| `default_transaction_read_only` | The default transaction access mode for the current session. If set to `on`, only read operations are allowed in transactions in the current session; if set to `off`, both read and write operations are allowed. See [`SET TRANSACTION`]({% link {{ page.version.version }}/set-transaction.md %}) for more details. | `off` | Yes | Yes | +| `default_transaction_use_follower_reads` | If set to on, all read-only transactions use [`AS OF SYSTEM TIME follower_read_timestamp()`]({% link {{ page.version.version }}/as-of-system-time.md %}) to allow the transaction to use follower reads.
If set to `off`, read-only transactions will only use follower reads if an `AS OF SYSTEM TIME` clause is specified in the statement, with an interval of at least 4.8 seconds. | `off` | Yes | Yes | +| `disable_changefeed_replication` | When `true`, [changefeeds]({% link {{ page.version.version }}/changefeed-messages.md %}#filtering-changefeed-messages) will not emit messages for any changes (e.g., `INSERT`, `UPDATE`) issued to watched tables during that session. | `false` | Yes | Yes | +| `disallow_full_table_scans` | If set to `on`, queries on "large" tables with a row count greater than [`large_full_scan_rows`](#large-full-scan-rows) will not use full table or index scans. If no other query plan is possible, queries will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes || `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes | +| `enable_auto_rehoming` | When enabled, the [home regions]({% link {{ page.version.version }}/alter-table.md %}#crdb_region) of rows in [`REGIONAL BY ROW`]({% link {{ page.version.version }}/alter-table.md %}#set-the-table-locality-to-regional-by-row) tables are automatically set to the region of the [gateway node]({% link {{ page.version.version }}/ui-sessions-page.md %}#session-details-gateway-node) from which any [`UPDATE`]({% link {{ page.version.version }}/update.md %}) or [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements that operate on those rows originate. | `off` | Yes | Yes | +| `enable_durable_locking_for_serializable` | Indicates whether CockroachDB replicates [`FOR UPDATE` and `FOR SHARE`]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) locks via [Raft]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft), allowing locks to be preserved when leases are transferred. Note that replicating `FOR UPDATE` and `FOR SHARE` locks will add latency to those statements. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes | +| `enable_experimental_alter_column_type_general` | If `on`, it is possible to [alter column data types]({% link {{ page.version.version }}/alter-table.md %}#alter-column-data-types). | `off` | Yes | Yes | +| `enable_implicit_fk_locking_for_serializable` | Indicates whether CockroachDB uses [shared locks]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) to perform [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) checks. To take effect, the [`enable_shared_locking_for_serializable`](#enable-shared-locking-for-serializable) setting must also be enabled. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes | +| `enable_implicit_select_for_update` | Indicates whether [`UPDATE`]({% link {{ page.version.version }}/update.md %}) and [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements acquire locks using the `FOR UPDATE` locking mode during their initial row scan, which improves performance for contended workloads.
For more information about how `FOR UPDATE` locking works, see the documentation for [`SELECT FOR UPDATE`]({% link {{ page.version.version }}/select-for-update.md %}). | `on` | Yes | Yes | +| `enable_implicit_transaction_for_batch_statements` | Indicates whether multiple statements in a single query (a "batch statement") will all run in the same implicit transaction, which matches the PostgreSQL wire protocol. | `on` | Yes | Yes | +| `enable_insert_fast_path` | Indicates whether CockroachDB will use a specialized execution operator for inserting into a table. We recommend leaving this setting `on`. | `on` | Yes | Yes | +| `enable_shared_locking_for_serializable` | Indicates whether [shared locks]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) are enabled for `SERIALIZABLE` transactions. When `off`, `SELECT` statements using `FOR SHARE` are still permitted under `SERIALIZABLE` isolation, but silently do not lock. | `off` | Yes | Yes | +| `enable_super_regions` | When enabled, you can define a super region: a set of [database regions]({% link {{ page.version.version }}/multiregion-overview.md %}#super-regions) on a multi-region cluster such that your [schema objects]({% link {{ page.version.version }}/schema-design-overview.md %}#database-schema-objects) will have all of their [replicas]({% link {{ page.version.version }}/architecture/overview.md %}#architecture-replica) stored _only_ in regions that are members of the super region. | `off` | Yes | Yes | +| `enable_zigzag_join` | Indicates whether the [cost-based optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will plan certain queries using a [zigzag merge join algorithm]({% link {{ page.version.version }}/cost-based-optimizer.md %}#zigzag-joins), which searches for the desired intersection by jumping back and forth between the indexes based on the fact that after constraining indexes, they share an ordering. | `on` | Yes | Yes | +| `enforce_home_region` | If set to `on`, queries return an error and in some cases a suggested resolution if they cannot run entirely in their home region. This can occur if a query has no home region (for example, if it reads from different home regions in a [regional by row table]({% link {{ page.version.version }}/table-localities.md %}#regional-by-row-tables)) or a query's home region differs from the [gateway]({% link {{ page.version.version }}/architecture/life-of-a-distributed-transaction.md %}#gateway) region. Note that only tables with `ZONE` [survivability]({% link {{ page.version.version }}/multiregion-survival-goals.md %}#when-to-use-zone-vs-region-survival-goals) can be scanned without error when this is enabled. For more information about home regions, see [Table localities]({% link {{ page.version.version }}/multiregion-overview.md %}#table-localities).
This feature is in preview. It is subject to change. | `off` | Yes | Yes | +| `enforce_home_region_follower_reads_enabled` | If `on` while the [`enforce_home_region`]({% link {{ page.version.version }}/cost-based-optimizer.md %}#control-whether-queries-are-limited-to-a-single-region) setting is `on`, allows `enforce_home_region` to perform `AS OF SYSTEM TIME` [follower reads]({% link {{ page.version.version }}/follower-reads.md %}) to detect and report a query's [home region]({% link {{ page.version.version }}/multiregion-overview.md %}#table-localities), if any.
This feature is in preview. It is subject to change. | `off` | Yes | Yes | +| `expect_and_ignore_not_visible_columns_in_copy` | If `on`, [`COPY FROM`]({% link {{ page.version.version }}/copy-from.md %}) with no column specifiers will assume that hidden columns are in the copy data, but will ignore them when applying `COPY FROM`. | `off` | Yes | Yes | +| `extra_float_digits` | The number of digits displayed for floating-point values. Only values between `-15` and `3` are supported. | `0` | Yes | Yes | +| `force_savepoint_restart` | When set to `true`, allows the [`SAVEPOINT`]({% link {{ page.version.version }}/savepoint.md %}) statement to accept any name for a savepoint. | `off` | Yes | Yes | +| `foreign_key_cascades_limit` | Limits the number of [cascading operations]({% link {{ page.version.version }}/foreign-key.md %}#use-a-foreign-key-constraint-with-cascade) that run as part of a single query. | `10000` | Yes | Yes | +| `idle_in_session_timeout` | Automatically terminates sessions that idle past the specified threshold.
When set to `0`, the session will not timeout. | The value set by the `sql.defaults.idle_in_session_timeout` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`0s`, by default). | Yes | Yes | +| `idle_in_transaction_session_timeout` | Automatically terminates sessions that are idle in a transaction past the specified threshold.
When set to `0`, the session will not timeout. | The value set by the `sql.defaults.idle_in_transaction_session_timeout` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (0s, by default). | Yes | Yes | +| `index_recommendations_enabled` | If `true`, display recommendations to create indexes required to eliminate full table scans.
For more details, see [Default statement plans]({% link {{ page.version.version }}/explain.md %}#default-statement-plans). | `true` | Yes | Yes | +| `inject_retry_errors_enabled` | If `true`, any statement executed inside of an explicit transaction (with the exception of [`SET`]({% link {{ page.version.version }}/set-vars.md %}) statements) will return a transaction retry error. If the client retries the transaction using the special [`cockroach_restart SAVEPOINT` name]({% link {{ page.version.version }}/savepoint.md %}#savepoints-for-client-side-transaction-retries), after the 3rd retry error, the transaction will proceed as normal. Otherwise, the errors will continue until `inject_retry_errors_enabled` is set to `false`. For more details, see [Test transaction retry logic]({% link {{ page.version.version }}/transaction-retry-error-example.md %}#test-transaction-retry-logic). | `false` | Yes | Yes | +| `intervalstyle` | The input string format for [`INTERVAL`]({% link {{ page.version.version }}/interval.md %}) values. Accepted values include `postgres`, `iso_8601`, and `sql_standard`. | The value set by the `sql.defaults.intervalstyle` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`postgres`, by default). | Yes | Yes | +| `is_superuser` | If `on` or `true`, the current user is a member of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role). | User-dependent | No | Yes | +| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` rejects full table or index scans of "large" tables. The default value is `1000`. To reject all full table or index scans, set to `0`. | User-dependent | No | Yes | +| `locality` | The location of the node.
For more information, see [Locality]({% link {{ page.version.version }}/cockroach-start.md %}#locality). | Node-dependent | No | Yes | +| `lock_timeout` | The amount of time a query can spend acquiring or waiting for a single [row-level lock]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#concurrency-control).
In CockroachDB, unlike in PostgreSQL, non-locking reads wait for conflicting locks to be released. As a result, the `lock_timeout` configuration applies to writes, and to locking and non-locking reads in read-write and read-only transactions.
If `lock_timeout = 0`, queries do not timeout due to lock acquisitions. | The value set by the `sql.defaults.lock_timeout` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`0`, by default) | Yes | Yes | +| `multiple_active_portals_enabled` | Whether to enable the [multiple active portals]({% link {{ page.version.version }}/postgresql-compatibility.md %}#multiple-active-portals) pgwire feature. | `false` | Yes | Yes | +| `node_id` | The ID of the node currently connected to.
This variable is particularly useful for verifying load balanced connections. | Node-dependent | No | Yes | +| `null_ordered_last` | Set the default ordering of `NULL`s. The default order is `NULL`s first for ascending order and `NULL`s last for descending order. | `false` | Yes | Yes | +| `optimizer_merge_joins_enabled` | If `on`, the optimizer will explore query plans with merge joins. | `on` | Yes | Yes | +| `optimizer_push_offset_into_index_join` | If `on`, the optimizer will attempt to push offset expressions into index join expressions to produce more efficient query plans. | `on` | Yes | Yes | +| `optimizer_use_forecasts` | If `on`, the optimizer uses forecasted statistics for query planning. | `on` | Yes | Yes | +| `optimizer_use_histograms` | If `on`, the optimizer uses collected histograms for cardinality estimation. | `on` | No | Yes | +| `optimizer_use_improved_multi_column_selectivity_estimate` | If `on`, the optimizer uses an improved selectivity estimate for multi-column predicates. | `on` | Yes | Yes | +| `optimizer_use_improved_zigzag_join_costing` | If `on`, the cost of [zigzag joins]({% link {{ page.version.version }}/cost-based-optimizer.md %}#zigzag-joins) is updated so they will be never be chosen over scans unless they produce fewer rows. To take effect, the [`enable_zigzag_join`](#enable-zigzag-join) setting must also be enabled. | `on` | Yes | Yes | +| `optimizer_use_lock_op_for_serializable` | If `on`, the optimizer uses a `Lock` operator to construct query plans for `SELECT` statements using the [`FOR UPDATE` and `FOR SHARE`]({% link {{ page.version.version }}/select-for-update.md %}) clauses. This setting only affects `SERIALIZABLE` transactions. `READ COMMITTED` transactions are evaluated with the `Lock` operator regardless of the setting. | `off` | Yes | Yes | +| `optimizer_use_multicol_stats` | If `on`, the optimizer uses collected multi-column statistics for cardinality estimation. | `on` | No | Yes | +| `optimizer_use_not_visible_indexes` | If `on`, the optimizer uses not visible indexes for planning. | `off` | No | Yes | +| `optimizer_use_virtual_computed_column_stats` | If `on`, the optimizer uses table statistics on [virtual computed columns]({% link {{ page.version.version }}/computed-columns.md %}#virtual-computed-columns). | `on` | Yes | Yes | +| `plan_cache_mode` | The type of plan that is cached in the [query plan cache]({% link {{ page.version.version }}/cost-based-optimizer.md %}#query-plan-cache): `auto`, `force_generic_plan`, or `force_custom_plan`.
For more information, refer to [Query plan type]({% link {{ page.version.version }}/cost-based-optimizer.md %}#query-plan-type). | `force_custom_plan` | Yes | Yes +| `plpgsql_use_strict_into` | If `on`, PL/pgSQL [`SELECT ... INTO` and `RETURNING ... INTO` statements]({% link {{ page.version.version }}/plpgsql.md %}#assign-a-result-to-a-variable) behave as though the `STRICT` option is specified. This causes the SQL statement to error if it does not return exactly one row. | `off` | Yes | Yes | +| `pg_trgm.similarity_threshold` | The threshold above which a [`%`]({% link {{ page.version.version }}/functions-and-operators.md %}#operators) string comparison returns `true`. The value must be between `0` and `1`. For more information, see [Trigram Indexes]({% link {{ page.version.version }}/trigram-indexes.md %}). | `0.3` | Yes | Yes | +| `prefer_lookup_joins_for_fks` | If `on`, the optimizer prefers [`lookup joins`]({% link {{ page.version.version }}/joins.md %}#lookup-joins) to [`merge joins`]({% link {{ page.version.version }}/joins.md %}#merge-joins) when performing [`foreign key`]({% link {{ page.version.version }}/foreign-key.md %}) checks. | `off` | Yes | Yes | +| `reorder_joins_limit` | Maximum number of joins that the optimizer will attempt to reorder when searching for an optimal query execution plan.
For more information, see [Join reordering]({% link {{ page.version.version }}/cost-based-optimizer.md %}#join-reordering). | `8` | Yes | Yes | +| `require_explicit_primary_keys` | If `on`, CockroachDB throws an error for all tables created without an explicit primary key defined. | `off` | Yes | Yes | +| `search_path` | A list of schemas that will be searched to resolve unqualified table or function names.
For more details, see [SQL name resolution]({% link {{ page.version.version }}/sql-name-resolution.md %}). | `public` | Yes | Yes | +| `serial_normalization` | Specifies the default handling of [`SERIAL`]({% link {{ page.version.version }}/serial.md %}) in table definitions. Valid options include `'rowid'`, `'virtual_sequence'`, `sql_sequence`, `sql_sequence_cached`, and `unordered_rowid`.
If set to `'virtual_sequence'`, the `SERIAL` type auto-creates a sequence for [better compatibility with Hibernate sequences](https://forum.cockroachlabs.com/t/hibernate-sequence-generator-returns-negative-number-and-ignore-unique-rowid/1885).
If set to `sql_sequence_cached`, you can use the `sql.defaults.serial_sequences_cache_size` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) to control the number of values to cache in a user's session, with a default of 256.
If set to `unordered_rowid`, the `SERIAL` type generates a globally unique 64-bit integer (a combination of the insert timestamp and the ID of the node executing the statement) that does not have unique ordering. | `'rowid'` | Yes | Yes | +| `server_version` | The version of PostgreSQL that CockroachDB emulates. | Version-dependent | No | Yes | +| `server_version_num` | The version of PostgreSQL that CockroachDB emulates. | Version-dependent | Yes | Yes | +| `session_id` | The ID of the current session. | Session-dependent | No | Yes | +| `session_user` | The user connected for the current session. | User in connection string | No | Yes | +| `sql_safe_updates` | If `true`, the following potentially unsafe SQL statements are disallowed: [`DROP DATABASE`]({% link {{ page.version.version }}/drop-database.md %}) of a non-empty database and all dependent objects; [`DELETE`]({% link {{ page.version.version }}/delete.md %}) and [`UPDATE`]({% link {{ page.version.version }}/update.md %}) without a `WHERE` clause, unless a [`LIMIT`]({% link {{ page.version.version }}/limit-offset.md %}) clause is included; [`SELECT ... FOR UPDATE`]({% link {{ page.version.version }}/select-for-update.md %}) and [`SELECT ... FOR SHARE`]({% link {{ page.version.version }}/select-for-update.md %}) without a `WHERE` or [`LIMIT`]({% link {{ page.version.version }}/limit-offset.md %}) clause; and [`ALTER TABLE ... DROP COLUMN`]({% link {{ page.version.version }}/alter-table.md %}#drop-column).
For more details, refer to [Allow potentially unsafe SQL statements]({% link {{ page.version.version }}/cockroach-sql.md %}#allow-potentially-unsafe-sql-statements). | `true` for interactive sessions from the [built-in SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}),
`false` for sessions from other clients | Yes | Yes | +| `statement_timeout` | The amount of time a statement can run before being stopped.
This value can be an `int` (e.g., `10`) and will be interpreted as milliseconds. It can also be an interval or string argument, where the string can be parsed as a valid interval (e.g., `'4s'`).
A value of `0` turns it off. | The value set by the `sql.defaults.statement_timeout` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`0s`, by default). | Yes | Yes | +| `stub_catalog_tables` | If `off`, querying an unimplemented, empty [`pg_catalog`]({% link {{ page.version.version }}/pg-catalog.md %}) table will result in an error, as is the case in v20.2 and earlier. If `on`, querying an unimplemented, empty `pg_catalog` table simply returns no rows. | `on` | Yes | Yes | +| `timezone` | The default time zone for the current session. | `UTC` | Yes | Yes | +| `tracing` | The trace recording state. | `off` | Yes | Yes | +| `transaction_isolation` | The isolation level at which the transaction executes ([`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) or [`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %})). See [Isolation levels]({% link {{ page.version.version }}/transactions.md %}#isolation-levels). | `SERIALIZABLE` | Yes | Yes | +| `transaction_priority` | The priority of the current transaction. See Transactions: Transaction priorities for more details. This session variable was called transaction priority (with a space) in CockroachDB 1.x. It has been renamed for compatibility with PostgreSQL. | `NORMAL` | Yes | Yes | +| `transaction_read_only` | The access mode of the current transaction. See [`SET TRANSACTION`]({% link {{ page.version.version }}/set-transaction.md %}) for more details. | `off` | Yes | Yes | +| `transaction_rows_read_err` | The limit for the number of rows read by a SQL transaction. If this value is exceeded the transaction will fail (or the event will be logged to `SQL_INTERNAL_PERF` for internal transactions). | `0` | Yes | Yes | +| `transaction_rows_read_log` | The threshold for the number of rows read by a SQL transaction. If this value is exceeded, the event will be logged to `SQL_PERF` (or `SQL_INTERNAL_PERF` for internal transactions). | `0` | Yes | Yes | +| `transaction_rows_written_err` | The limit for the number of rows written by a SQL transaction. If this value is exceeded the transaction will fail (or the event will be logged to `SQL_INTERNAL_PERF` for internal transactions). | `0` | Yes | Yes | +| `transaction_rows_written_log` | The threshold for the number of rows written by a SQL transaction. If this value is exceeded, the event will be logged to `SQL_PERF` (or `SQL_INTERNAL_PERF` for internal transactions). | `0` | Yes | Yes | +| `transaction_status` | The state of the current transaction. See [Transactions]({% link {{ page.version.version }}/transactions.md %}) for more details. | `NoTxn` | No | Yes | +| `transaction_timeout` | Aborts an explicit [transaction]({% link {{ page.version.version }}/transactions.md %}) when it runs longer than the configured duration. Stored in milliseconds; can be expressed in milliseconds or as an [`INTERVAL`]({% link {{ page.version.version }}/interval.md %}). | `0` | Yes | Yes | +| `troubleshooting_mode_enabled` | When enabled, avoid performing additional work on queries, such as collecting and emitting telemetry data. This session variable is particularly useful when the cluster is experiencing issues, unavailability, or failure. | `off` | Yes | Yes | +| `use_declarative_schema_changer` | Whether to use the declarative schema changer for supported statements. See [Declarative schema changer]({% link {{ page.version.version }}/online-schema-changes.md %}#declarative-schema-changer) for more details. | `on` | Yes | Yes | +| `vectorize` | The vectorized execution engine mode. Options include `on` and `off`. For more details, see [Configure vectorized execution for CockroachDB]({% link {{ page.version.version }}/vectorized-execution.md %}#configure-vectorized-execution). | `on` | Yes | Yes | +| `virtual_cluster_name` | The name of the virtual cluster that the SQL client is connected to. | Session-dependent | No | Yes | + +The following session variables are exposed only for backwards compatibility with earlier CockroachDB releases and have no impact on how CockroachDB runs: + +| Variable name | Initial value | Modify with [`SET`]({% link {{ page.version.version }}/set-vars.md %})? | View with [`SHOW`]({% link {{ page.version.version }}/show-vars.md %})? | +|---|---|---|---| +| `backslash_quote` | `safe_encoding` | No | Yes | +| `client_encoding` | `UTF8` | No | Yes | +| `default_tablespace` | | No | Yes | +| `enable_drop_enum_value` | `off` | Yes | Yes | +| `enable_seqscan` | `on` | Yes | Yes | +| `escape_string_warning` | `on` | No | Yes | +| `experimental_enable_hash_sharded_indexes` | `off` | Yes | Yes | +| `integer_datetimes` | `on` | No | Yes | +| `max_identifier_length` | `128` | No | Yes | +| `max_index_keys` | `32` | No | Yes | +| `row_security` | `off` | No | Yes | +| `standard_conforming_strings` | `on` | No | Yes | +| `server_encoding` | `UTF8` | Yes | Yes | +| `synchronize_seqscans` | `on` | No | Yes | +| `synchronous_commit` | `on` | Yes | Yes | diff --git a/src/current/_includes/v25.1/misc/sorting-delete-output.md b/src/current/_includes/v25.1/misc/sorting-delete-output.md new file mode 100644 index 00000000000..b48a138a279 --- /dev/null +++ b/src/current/_includes/v25.1/misc/sorting-delete-output.md @@ -0,0 +1,9 @@ +To sort the output of a `DELETE` statement, use: + +{% include_cached copy-clipboard.html %} +~~~ sql +> WITH a AS (DELETE ... RETURNING ...) + SELECT ... FROM a ORDER BY ... +~~~ + +For an example, see [Sort and return deleted rows]({% link {{ page.version.version }}/delete.md %}#sort-and-return-deleted-rows). diff --git a/src/current/_includes/v25.1/misc/source-privileges.md b/src/current/_includes/v25.1/misc/source-privileges.md new file mode 100644 index 00000000000..543801a7201 --- /dev/null +++ b/src/current/_includes/v25.1/misc/source-privileges.md @@ -0,0 +1,12 @@ +The source file URL does _not_ require the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role) in the following scenarios: + +- S3 and GS using `SPECIFIED` (and not `IMPLICIT`) credentials. Azure is always `SPECIFIED` by default. +- [Userfile]({% link {{ page.version.version }}/use-userfile-storage.md %}) + +The source file URL _does_ require the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role) in the following scenarios: + +- S3 or GS using `IMPLICIT` credentials +- Use of a [custom endpoint](https://docs.aws.amazon.com/sdk-for-go/api/aws/endpoints/) on S3 +- [Nodelocal]({% link {{ page.version.version }}/cockroach-nodelocal-upload.md %}), [HTTP]({% link {{ page.version.version }}/use-a-local-file-server.md %}) or [HTTPS] ({% link {{ page.version.version }}/use-a-local-file-server.md %}) + +We recommend using [cloud storage]({% link {{ page.version.version }}/use-cloud-storage.md %}). diff --git a/src/current/_includes/v25.1/misc/storage-class-glacier-incremental.md b/src/current/_includes/v25.1/misc/storage-class-glacier-incremental.md new file mode 100644 index 00000000000..9daebd72c14 --- /dev/null +++ b/src/current/_includes/v25.1/misc/storage-class-glacier-incremental.md @@ -0,0 +1,5 @@ +{{site.data.alerts.callout_danger}} +[Incremental backups]({% link {{ page.version.version }}/take-full-and-incremental-backups.md %}#incremental-backups) are **not** compatible with the [S3 Glacier Flexible Retrieval or Glacier Deep Archive storage classes](https://docs.aws.amazon.com/AmazonS3/latest/userguide//storage-class-intro.html#sc-glacier). Incremental backups require the reading of previous backups on an ad-hoc basis, which is not possible with backup files already in Glacier Flexible Retrieval or Glacier Deep Archive. This is because these storage classes do not allow immediate access to an S3 object without first [restoring the archived objects](https://docs.aws.amazon.com/AmazonS3/latest/userguide/restoring-objects.html) to its S3 bucket. + +Refer to [Incremental backups and storage classes]({% link {{ page.version.version }}/use-cloud-storage.md %}#incremental-backups-and-archive-storage-classes) for more detail. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/storage-classes.md b/src/current/_includes/v25.1/misc/storage-classes.md new file mode 100644 index 00000000000..c4dafce941e --- /dev/null +++ b/src/current/_includes/v25.1/misc/storage-classes.md @@ -0,0 +1 @@ +Use the parameter to set one of these [storage classes](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html#AmazonS3-PutObject-request-header-StorageClass) listed in Amazon's documentation. For more general usage information, see Amazon's [Using Amazon S3 storage classes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-class-intro.html) documentation. diff --git a/src/current/_includes/v25.1/misc/table-storage-parameters.md b/src/current/_includes/v25.1/misc/table-storage-parameters.md new file mode 100644 index 00000000000..3ca7f601648 --- /dev/null +++ b/src/current/_includes/v25.1/misc/table-storage-parameters.md @@ -0,0 +1,15 @@ +| Parameter name | Description | Data type | Default value | +|------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|-----------|---------------| +| `exclude_data_from_backup` | Exclude the data in this table from any future backups. | Boolean | `false` | +| `schema_locked` | Disallow [schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}) on this table. Enabling `schema_locked` can help [improve performance of changefeeds]({% link {{ page.version.version }}/create-changefeed.md %}#disallow-schema-changes-on-tables-to-improve-changefeed-performance) running on this table. | Boolean | `false` | +| `sql_stats_automatic_collection_enabled` | Enable [automatic statistics collection]({% link {{ page.version.version }}/cost-based-optimizer.md %}#enable-and-disable-automatic-statistics-collection-for-tables) for this table. | Boolean | `true` | +| `sql_stats_automatic_collection_min_stale_rows` | Minimum number of stale rows in this table that will trigger a statistics refresh. | Integer | 500 | +| `sql_stats_automatic_collection_fraction_stale_rows` | Fraction of stale rows in this table that will trigger a statistics refresh. | Float | 0.2 | +| `sql_stats_forecasts_enabled` | Enable [forecasted statistics]({% link {{ page.version.version }}/show-statistics.md %}#display-forecasted-statistics) collection for this table. | Boolean | `true` | + +The following parameters are included for PostgreSQL compatibility and do not affect how CockroachDB runs: + +- `autovacuum_enabled` +- `fillfactor` + +For the list of storage parameters that affect how [Row-Level TTL]({% link {{ page.version.version }}/row-level-ttl.md %}) works, see the list of [TTL storage parameters]({% link {{ page.version.version }}/row-level-ttl.md %}#ttl-storage-parameters). \ No newline at end of file diff --git a/src/current/_includes/v25.1/misc/tooling.md b/src/current/_includes/v25.1/misc/tooling.md new file mode 100644 index 00000000000..f587b47babf --- /dev/null +++ b/src/current/_includes/v25.1/misc/tooling.md @@ -0,0 +1,107 @@ +## Support levels + +Cockroach Labs has partnered with open-source projects, vendors, and individuals to offer the following levels of support with third-party tools: + +- **Full support** indicates that Cockroach Labs is committed to maintaining compatibility with the vast majority of the tool's features. CockroachDB is regularly tested against the latest version documented in the table below. +- **Partial support** indicates that Cockroach Labs is working towards full support for the tool. The primary features of the tool are compatible with CockroachDB (e.g., connecting and basic database operations), but full integration may require additional steps, lack support for all features, or exhibit unexpected behavior. +- **Partner supported** indicates that Cockroach Labs has a partnership with a third-party vendor that provides support for the CockroachDB integration with their tool. + +{{site.data.alerts.callout_danger}} +Tools, drivers, or frameworks are considered **unsupported** if: + +- The tool, driver, or framework is not listed on this page. +- The version of a supported tool, driver, or framework is not listed on this page. + +If you encounter issues when using unsupported tools, drivers, or frameworks, contact the maintainer directly. + +Cockroach Labs provides "best effort" support for tools, drivers, and frameworks that are not officially supported. This means that while we will do our best to assist you, we may not be able to fully troubleshoot errors in your deployment. + +Customers should contact their account team before moving production workloads to CockroachDB that use unsupported drivers. +{{site.data.alerts.end}} + +{{site.data.alerts.callout_info}} +Unless explicitly stated, support for a [driver](#drivers) or [data access framework](#data-access-frameworks-e-g-orms) does not include [automatic, client-side transaction retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). For client-side transaction retry handling samples, see [Example Apps]({% link {{ page.version.version }}/example-apps.md %}). +{{site.data.alerts.end}} + +If you encounter problems using CockroachDB with any of the tools listed on this page, please [open an issue](https://github.com/cockroachdb/cockroach/issues/new) with details to help us make progress toward better support. + +For a list of tools supported by the CockroachDB community, see [Third-Party Tools Supported by the Community]({% link {{ page.version.version }}/community-tooling.md %}). + +## Drivers + +| Language | Driver | Latest tested version | Support level | CockroachDB adapter | Tutorial | +|----------+--------+-----------------------+---------------------+---------------------+----------| +| C | [libpq](http://www.postgresql.org/docs/13/static/libpq.html)| PostgreSQL 13 | Partial | N/A | N/A | +| C# (.NET) | [Npgsql](https://www.nuget.org/packages/Npgsql/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/npgsql.go ||var npgsqlSupportedTag = "v||"\n\n %} | Full | N/A | [Build a C# App with CockroachDB (Npgsql)](build-a-csharp-app-with-cockroachdb.html) | +| Go | [pgx](https://github.com/jackc/pgx/releases)
[pq](https://github.com/lib/pq) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgx.go ||var supportedPGXTag = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/libpq.go ||var libPQSupportedTag = "||"\n\n %} | Full
Full | [`crdbpgx`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbpgx)
(includes client-side transaction retry handling)
N/A | [Build a Go App with CockroachDB (pgx)](build-a-go-app-with-cockroachdb.html)
[Build a Go App with CockroachDB (pq)](build-a-go-app-with-cockroachdb-pq.html) | +| Java | [JDBC](https://jdbc.postgresql.org/download/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/pgjdbc.go ||var supportedPGJDBCTag = "||"\n\n %} | Full | N/A | [Build a Java App with CockroachDB (JDBC)](build-a-java-app-with-cockroachdb.html) | +| JavaScript | [pg](https://www.npmjs.com/package/pg) | 8.2.1 | Full | N/A | [Build a Node.js App with CockroachDB (pg)](build-a-nodejs-app-with-cockroachdb.html) | +| Python | [psycopg3](https://www.psycopg.org/psycopg3/docs/)
[psycopg2](https://www.psycopg.org/docs/install.html)
[asyncpg](https://magicstack.github.io/asyncpg/current/index.html) | 3.0.16
2.8.6
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/asyncpg.go || var asyncpgSupportedTag = "||"\n\n %} | Full
Full
Partial | N/A
N/A
N/A | [Build a Python App with CockroachDB (psycopg3)](build-a-python-app-with-cockroachdb-psycopg3.html)
[Build a Python App with CockroachDB (psycopg2)](build-a-python-app-with-cockroachdb.html)
[Build a Python App with CockroachDB (asyncpg)](build-a-python-app-with-cockroachdb-asyncpg.html) | +| Ruby | [pg](https://rubygems.org/gems/pg) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/ruby_pg.go ||var rubyPGVersion = "||"\n\n %} | Full | N/A | [Build a Ruby App with CockroachDB (pg)](build-a-ruby-app-with-cockroachdb.html) | +| Rust | [rust-postgres](https://github.com/sfackler/rust-postgres) | 0.19.2 | Partial | N/A | [Build a Rust App with CockroachDB]({% link {{ page.version.version }}/build-a-rust-app-with-cockroachdb.md %}) | + +## Data access frameworks (e.g., ORMs) + +| Language | Framework | Latest tested version | Support level | CockroachDB adapter | Tutorial | +|----------+-----------+-----------------------+---------------+---------------------+----------| +| Go | [GORM](https://github.com/jinzhu/gorm/releases)
[go-pg](https://github.com/go-pg/pg)
[upper/db](https://github.com/upper/db) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gorm.go ||var gormSupportedTag = "||"\n\n %}
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/gopg.go ||var gopgSupportedTag = "||"\n\n %}
v4 | Full
Full
Full | [`crdbgorm`](https://pkg.go.dev/github.com/cockroachdb/cockroach-go/crdb/crdbgorm)
(includes client-side transaction retry handling)
N/A
N/A | [Build a Go App with CockroachDB (GORM)](build-a-go-app-with-cockroachdb-gorm.html)
N/A
[Build a Go App with CockroachDB (upper/db)](build-a-go-app-with-cockroachdb-upperdb.html) | +| Java | [Hibernate](https://hibernate.org/orm/)
(including [Hibernate Spatial](https://docs.jboss.org/hibernate/orm/current/userguide/html_single/Hibernate_User_Guide.html#spatial))
[jOOQ](https://www.jooq.org/)
[MyBatis](https://mybatis.org/mybatis-3/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/hibernate.go ||var supportedHibernateTag = "||"\n\n %} (must be at least 5.4.19)
3.13.2 (must be at least 3.13.0)
3.5.5| Full
Full
Full | N/A
N/A
N/A | [Build a Java App with CockroachDB (Hibernate)](build-a-java-app-with-cockroachdb-hibernate.html)
[Build a Java App with CockroachDB (jOOQ)](build-a-java-app-with-cockroachdb-jooq.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) | +| JavaScript/TypeScript | [Sequelize](https://www.npmjs.com/package/sequelize)
[Knex.js](https://knexjs.org/)
[Prisma](https://prisma.io)
[TypeORM](https://www.npmjs.com/package/typeorm) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/sequelize.go ||var supportedSequelizeCockroachDBRelease = "||"\n\n %}
(use latest version of CockroachDB adapter)
{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/knex.go ||const supportedKnexTag = "||"\n\n %}
3.14.0
0.3.17 {% comment %}{% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/typeorm.go ||const supportedTypeORMRelease = "||"\n %}{% endcomment %} | Full
Full
Full
Full | [`sequelize-cockroachdb`](https://www.npmjs.com/package/sequelize-cockroachdb)
N/A
N/A
N/A | [Build a Node.js App with CockroachDB (Sequelize)](build-a-nodejs-app-with-cockroachdb-sequelize.html)
[Build a Node.js App with CockroachDB (Knex.js)](build-a-nodejs-app-with-cockroachdb-knexjs.html)
[Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html)
[Build a TypeScript App with CockroachDB (TypeORM)](build-a-typescript-app-with-cockroachdb.html) | +| Ruby | [ActiveRecord](https://rubygems.org/gems/activerecord)
[RGeo/RGeo-ActiveRecord](https://github.com/cockroachdb/activerecord-cockroachdb-adapter#working-with-spatial-data) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/activerecord.go ||var supportedRailsVersion = "||"\nvar %}
(use latest version of CockroachDB adapter) | Full | [`activerecord-cockroachdb-adapter`](https://rubygems.org/gems/activerecord-cockroachdb-adapter)
(includes client-side transaction retry handling) | [Build a Ruby App with CockroachDB (ActiveRecord)](build-a-ruby-app-with-cockroachdb-activerecord.html) | +| Python | [Django](https://pypi.org/project/Django/)
(including [GeoDjango](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/))
[peewee](https://github.com/coleifer/peewee/)
[SQLAlchemy](https://www.sqlalchemy.org/) | {% remote_include https://raw.githubusercontent.com/cockroachdb/cockroach/master/pkg/cmd/roachtest/tests/django.go ||var djangoSupportedTag = "cockroach-||"\nvar %}
(use latest version of CockroachDB adapter)
3.13.3
0.7.13
1.4.17
(use latest version of CockroachDB adapter) | Full
Full
Full
Full | [`django-cockroachdb`](https://pypi.org/project/django-cockroachdb/)
N/A
N/A
[`sqlalchemy-cockroachdb`](https://pypi.org/project/sqlalchemy-cockroachdb)
(includes client-side transaction retry handling) | [Build a Python App with CockroachDB (Django)](build-a-python-app-with-cockroachdb-django.html)
N/A (See [peewee docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#cockroach-database).)
[Build a Python App with CockroachDB (SQLAlchemy)](build-a-python-app-with-cockroachdb-sqlalchemy.html) | + +## Application frameworks + +| Framework | Data access | Latest tested version | Support level | Tutorial | +|-----------+-------------+-----------------------+---------------+----------| +| Spring | [JDBC]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-jdbc.md %})
[JPA (Hibernate)](build-a-spring-app-with-cockroachdb-jpa.html)
[MyBatis]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) | See individual Java ORM or [driver](#drivers) for data access version support. | See individual Java ORM or [driver](#drivers) for data access support level. | [Build a Spring App with CockroachDB (JDBC)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-jdbc.md %})
[Build a Spring App with CockroachDB (JPA)](build-a-spring-app-with-cockroachdb-jpa.html)
[Build a Spring App with CockroachDB (MyBatis)]({% link {{ page.version.version }}/build-a-spring-app-with-cockroachdb-mybatis.md %}) + +## Graphical user interfaces (GUIs) + +| GUI | Latest tested version | Support level | Tutorial | +|-----+-----------------------+---------------+----------| +| [DBeaver](https://dbeaver.com/) | 5.2.3 | Full | [Visualize CockroachDB Schemas with DBeaver]({% link {{ page.version.version }}/dbeaver.md %}) + +## Integrated development environments (IDEs) + +| IDE | Latest tested version | Support level | Tutorial | +|-----+-----------------------+---------------+----------| +| [DataGrip](https://www.jetbrains.com/datagrip/) | 2024.1 | Full | N/A +| [IntelliJ IDEA](https://www.jetbrains.com/idea/) | 2024.1 | Full | [Use IntelliJ IDEA with CockroachDB]({% link {{ page.version.version }}/intellij-idea.md %}) + +## Enhanced data security tools + +| Tool | Support level | Integration | +|-----+---------------+----------| +| [Satori](https://satoricyber.com/) | Partner supported | [Satori Integration]({% link {{ page.version.version }}/satori-integration.md %}) | +| [HashiCorp Vault](https://www.vaultproject.io/) | Partner supported | [HashiCorp Vault Integration]({% link {{ page.version.version }}/hashicorp-integration.md %}) | + +## Schema migration tools + +| Tool | Latest tested version | Support level | Tutorial | +|-----+------------------------+----------------+----------| +| [Alembic](https://alembic.sqlalchemy.org/en/latest/) | 1.7 | Full | [Migrate CockroachDB Schemas with Alembic]({% link {{ page.version.version }}/alembic.md %}) +| [Flyway](https://flywaydb.org/documentation/commandline/#download-and-installation) | 7.1.0 | Full | [Migrate CockroachDB Schemas with Flyway]({% link {{ page.version.version }}/flyway.md %}) +| [Liquibase](https://www.liquibase.org/download) | 4.2.0 | Full | [Migrate CockroachDB Schemas with Liquibase]({% link {{ page.version.version }}/liquibase.md %}) +| [Prisma](https://prisma.io) | 3.14.0 | Full | [Build a Node.js App with CockroachDB (Prisma)](build-a-nodejs-app-with-cockroachdb-prisma.html) + +## Data migration tools + +| Tool | Latest tested version | Support level | Documentation | +|-----+------------------------+----------------+----------| +| [AWS DMS](https://aws.amazon.com/dms/) | 3.4.6 | Full | [Migrate with AWS Database Migration Service (DMS)](aws-dms.html) +| [Qlik Replicate](https://www.qlik.com/us/products/qlik-replicate) | November 2022 | Full | [Migrate and Replicate Data with Qlik Replicate]({% link {{ page.version.version }}/qlik.md %}) +| [Striim](https://www.striim.com) | 4.1.2 | Full | [Migrate and Replicate Data with Striim]({% link {{ page.version.version }}/striim.md %}) +| [Oracle GoldenGate](https://www.oracle.com/integration/goldengate/) | 21.3 | Partial | [Migrate and Replicate Data with Oracle GoldenGate]({% link {{ page.version.version }}/goldengate.md %}) +| [Debezium](https://debezium.io/) | 2.4 | Full | [Migrate Data with Debezium]({% link {{ page.version.version }}/debezium.md %}) + +## Provisioning tools +| Tool | Latest tested version | Support level | Documentation | +|------+-----------------------+---------------+---------------| +| [Terraform](https://terraform.io/) | 1.3.2 | Partial | [Terraform provider for CockroachDB Cloud](https://github.com/cockroachdb/terraform-provider-cockroach#get-started) | + +## Other tools + +| Tool | Latest tested version | Support level | Tutorial | +|-----+------------------------+---------------+----------| +| [Flowable](https://github.com/flowable/flowable-engine) | 6.4.2 | Full | [Getting Started with Flowable and CockroachDB (external)](https://blog.flowable.org/2019/07/11/getting-started-with-flowable-and-cockroachdb/) diff --git a/src/current/_includes/v25.1/misc/userfile.md b/src/current/_includes/v25.1/misc/userfile.md new file mode 100644 index 00000000000..dbeb640a84b --- /dev/null +++ b/src/current/_includes/v25.1/misc/userfile.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} + CockroachDB now supports uploading files to a [user-scoped file storage]({% link {{ page.version.version }}/use-userfile-storage.md %}) using a SQL connection. We recommend using `userfile` instead of `nodelocal`, as it is user-scoped and more secure. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/orchestration/apply-custom-resource.md b/src/current/_includes/v25.1/orchestration/apply-custom-resource.md new file mode 100644 index 00000000000..e7aacf41a1e --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/apply-custom-resource.md @@ -0,0 +1,6 @@ +Apply the new settings to the cluster: + +{% include_cached copy-clipboard.html %} +~~~ shell +$ kubectl apply -f example.yaml +~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/orchestration/apply-helm-values.md b/src/current/_includes/v25.1/orchestration/apply-helm-values.md new file mode 100644 index 00000000000..90f9c8783f8 --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/apply-helm-values.md @@ -0,0 +1,6 @@ +Apply the custom values to override the default Helm chart [values](https://github.com/cockroachdb/helm-charts/blob/master/cockroachdb/values.yaml): + +{% include_cached copy-clipboard.html %} +~~~ shell +$ helm upgrade {release-name} --values {custom-values}.yaml cockroachdb/cockroachdb +~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/orchestration/apply-statefulset-manifest.md b/src/current/_includes/v25.1/orchestration/apply-statefulset-manifest.md new file mode 100644 index 00000000000..0236903c497 --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/apply-statefulset-manifest.md @@ -0,0 +1,6 @@ +Apply the new settings to the cluster: + +{% include_cached copy-clipboard.html %} +~~~ shell +$ kubectl apply -f {statefulset-manifest}.yaml +~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/orchestration/kubernetes-basic-sql.md b/src/current/_includes/v25.1/orchestration/kubernetes-basic-sql.md new file mode 100644 index 00000000000..341c9bca23b --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/kubernetes-basic-sql.md @@ -0,0 +1,44 @@ +1. Run some basic [CockroachDB SQL statements]({% link {{ page.version.version }}/learn-cockroachdb-sql.md %}): + + {% include_cached copy-clipboard.html %} + ~~~ sql + > CREATE DATABASE bank; + ~~~ + + {% include_cached copy-clipboard.html %} + ~~~ sql + > CREATE TABLE bank.accounts (id INT PRIMARY KEY, balance DECIMAL); + ~~~ + + {% include_cached copy-clipboard.html %} + ~~~ sql + > INSERT INTO bank.accounts VALUES (1, 1000.50); + ~~~ + + {% include_cached copy-clipboard.html %} + ~~~ sql + > SELECT * FROM bank.accounts; + ~~~ + + ~~~ + id | balance + +----+---------+ + 1 | 1000.50 + (1 row) + ~~~ + +1. [Create a user with a password]({% link {{ page.version.version }}/create-user.md %}#create-a-user-with-a-password): + + {% include_cached copy-clipboard.html %} + ~~~ sql + > CREATE USER roach WITH PASSWORD 'Q7gc8rEdS'; + ~~~ + + You will need this username and password to access the DB Console later. + +1. Exit the SQL shell and pod: + + {% include_cached copy-clipboard.html %} + ~~~ sql + > \q + ~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/orchestration/kubernetes-cockroach-cert.md b/src/current/_includes/v25.1/orchestration/kubernetes-cockroach-cert.md new file mode 100644 index 00000000000..12fa4d9783f --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/kubernetes-cockroach-cert.md @@ -0,0 +1,90 @@ +{{site.data.alerts.callout_info}} +The below steps use [`cockroach cert` commands]({% link {{ page.version.version }}/cockroach-cert.md %}) to quickly generate and sign the CockroachDB node and client certificates. Read our [Authentication]({% link {{ page.version.version }}/authentication.md %}#using-digital-certificates-with-cockroachdb) docs to learn about other methods of signing certificates. +{{site.data.alerts.end}} + +1. Create two directories: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ mkdir certs my-safe-directory + ~~~ + + Directory | Description + ----------|------------ + `certs` | You'll generate your CA certificate and all node and client certificates and keys in this directory. + `my-safe-directory` | You'll generate your CA key in this directory and then reference the key when generating node and client certificates. + +1. Create the CA certificate and key pair: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-ca \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +1. Create a client certificate and key pair for the root user: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-client \ + root \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +1. Upload the client certificate and key to the Kubernetes cluster as a secret: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.client.root \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.client.root created + ~~~ + +1. Create the certificate and key pair for your CockroachDB nodes: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach cert create-node \ + localhost 127.0.0.1 \ + cockroachdb-public \ + cockroachdb-public.default \ + cockroachdb-public.default.svc.cluster.local \ + *.cockroachdb \ + *.cockroachdb.default \ + *.cockroachdb.default.svc.cluster.local \ + --certs-dir=certs \ + --ca-key=my-safe-directory/ca.key + ~~~ + +1. Upload the node certificate and key to the Kubernetes cluster as a secret: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ kubectl create secret \ + generic cockroachdb.node \ + --from-file=certs + ~~~ + + ~~~ + secret/cockroachdb.node created + ~~~ + +1. Check that the secrets were created on the cluster: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ kubectl get secrets + ~~~ + + ~~~ + NAME TYPE DATA AGE + cockroachdb.client.root Opaque 3 41m + cockroachdb.node Opaque 5 14s + default-token-6qjdb kubernetes.io/service-account-token 3 4m + ~~~ \ No newline at end of file diff --git a/src/current/_includes/v25.1/orchestration/kubernetes-expand-disk-helm.md b/src/current/_includes/v25.1/orchestration/kubernetes-expand-disk-helm.md new file mode 100644 index 00000000000..4ec3d2f171f --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/kubernetes-expand-disk-helm.md @@ -0,0 +1,118 @@ +You can expand certain [types of persistent volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#types-of-persistent-volumes +) (including GCE Persistent Disk and Amazon Elastic Block Store) by editing their persistent volume claims. + +{{site.data.alerts.callout_info}} +These steps assume you followed the tutorial [Deploy CockroachDB on Kubernetes](deploy-cockroachdb-with-kubernetes.html?filters=helm). +{{site.data.alerts.end}} + +1. Get the persistent volume claims for the volumes: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ kubectl get pvc + ~~~ + + ~~~ + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE + datadir-my-release-cockroachdb-0 Bound pvc-75dadd4c-01a1-11ea-b065-42010a8e00cb 100Gi RWO standard 17m + datadir-my-release-cockroachdb-1 Bound pvc-75e143ca-01a1-11ea-b065-42010a8e00cb 100Gi RWO standard 17m + datadir-my-release-cockroachdb-2 Bound pvc-75ef409a-01a1-11ea-b065-42010a8e00cb 100Gi RWO standard 17m + ~~~ + +1. In order to expand a persistent volume claim, `AllowVolumeExpansion` in its storage class must be `true`. Examine the storage class: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ kubectl describe storageclass standard + ~~~ + + ~~~ + Name: standard + IsDefaultClass: Yes + Annotations: storageclass.kubernetes.io/is-default-class=true + Provisioner: kubernetes.io/gce-pd + Parameters: type=pd-standard + AllowVolumeExpansion: False + MountOptions:
When using `minikube`, persistent volumes are external temporary directories that endure until they are manually deleted or until the entire Kubernetes cluster is deleted. +[persistent volume claim](http://kubernetes.io/docs/user-guide/persistent-volumes/#persistentvolumeclaims) | When e pod is created, it requests a persistent volume claim to claim durable storage for its node. + +## Step 1. Start Kubernetes + +1. Follow the [Minikube documentation](https://kubernetes.io/docs/tasks/tools/install-minikube/) to install the latest version of `minikube`, a hypervisor, and the `kubectl` command-line tool. + +1. Start a local Kubernetes cluster: + + {% include_cached copy-clipboard.html %} + ~~~ shell + minikube start + ~~~ diff --git a/src/current/_includes/v25.1/orchestration/monitor-cluster.md b/src/current/_includes/v25.1/orchestration/monitor-cluster.md new file mode 100644 index 00000000000..171dbb24914 --- /dev/null +++ b/src/current/_includes/v25.1/orchestration/monitor-cluster.md @@ -0,0 +1,106 @@ +To access the cluster's [DB Console]({% link {{ page.version.version }}/ui-overview.md %}): + +{% if page.secure == true %} + +1. On secure clusters, [certain pages of the DB Console]({% link {{ page.version.version }}/ui-overview.md %}#db-console-access) can only be accessed by `admin` users. + + Get a shell into the pod and start the CockroachDB [built-in SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}): + +
port-forward command must be run on the same machine as the web browser in which you want to view the DB Console. If you have been running these commands from a cloud instance or other non-local shell, you will not be able to view the UI without configuring kubectl locally and running the above port-forward command on your local machine.{{site.data.alerts.end}}
+
+{% if page.secure == true %}
+
+1. Go to https://localhost:8080 and log in with the username and password you created earlier.
+
+ {% include {{ page.version.version }}/misc/chrome-localhost.md %}
+
+{% else %}
+
+1. Go to http://localhost:8080.
+
+{% endif %}
+
+1. In the UI, verify that the cluster is running as expected:
+ - View the [Node List]({% link {{ page.version.version }}/ui-cluster-overview-page.md %}#node-list) to ensure that all nodes successfully joined the cluster.
+ - Click the **Databases** tab on the left to verify that `bank` is listed.
diff --git a/src/current/_includes/v25.1/orchestration/operator-check-namespace.md b/src/current/_includes/v25.1/orchestration/operator-check-namespace.md
new file mode 100644
index 00000000000..bc37c6e1681
--- /dev/null
+++ b/src/current/_includes/v25.1/orchestration/operator-check-namespace.md
@@ -0,0 +1,3 @@
+{{site.data.alerts.callout_info}}
+All `kubectl` steps should be performed in the [namespace where you installed the Operator]({% link {{ page.version.version }}/deploy-cockroachdb-with-kubernetes.md %}#install-the-operator). By default, this is `cockroach-operator-system`.
+{{site.data.alerts.end}}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/orchestration/start-cockroachdb-helm-insecure.md b/src/current/_includes/v25.1/orchestration/start-cockroachdb-helm-insecure.md
new file mode 100644
index 00000000000..db3916f2fa9
--- /dev/null
+++ b/src/current/_includes/v25.1/orchestration/start-cockroachdb-helm-insecure.md
@@ -0,0 +1,111 @@
+1. [Install the Helm client](https://helm.sh/docs/intro/install) (version 3.0 or higher) and add the `cockroachdb` chart repository:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ helm repo add cockroachdb https://charts.cockroachdb.com/
+ ~~~
+
+ ~~~
+ "cockroachdb" has been added to your repositories
+ ~~~
+
+1. Update your Helm chart repositories to ensure that you're using the [latest CockroachDB chart](https://github.com/cockroachdb/helm-charts/blob/master/cockroachdb/Chart.yaml):
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ helm repo update
+ ~~~
+
+1. Modify our Helm chart's [`values.yaml`](https://github.com/cockroachdb/helm-charts/blob/master/cockroachdb/values.yaml) parameters for your deployment scenario.
+
+ Create a `my-values.yaml` file to override the defaults in `values.yaml`, substituting your own values in this example based on the guidelines below.
+
+ {% include_cached copy-clipboard.html %}
+ ~~~
+ statefulset:
+ resources:
+ limits:
+ memory: "8Gi"
+ requests:
+ memory: "8Gi"
+ conf:
+ cache: "2Gi"
+ max-sql-memory: "2Gi"
+ ~~~
+
+ 1. To avoid running out of memory when CockroachDB is not the only pod on a Kubernetes node, you *must* set memory limits explicitly. This is because CockroachDB does not detect the amount of memory allocated to its pod when run in Kubernetes. We recommend setting `conf.cache` and `conf.max-sql-memory` each to 1/4 of the `memory` allocation specified in `statefulset.resources.requests` and `statefulset.resources.limits`.
+
+ {{site.data.alerts.callout_success}}
+ For example, if you are allocating 8Gi of `memory` to each CockroachDB node, allocate 2Gi to `cache` and 2Gi to `max-sql-memory`.
+ {{site.data.alerts.end}}
+
+1. For an insecure deployment, set `tls.enabled` to `false`. For clarity, this example includes the example configuration from the previous steps.
+
+ {% include_cached copy-clipboard.html %}
+ ~~~
+ statefulset:
+ resources:
+ limits:
+ memory: "8Gi"
+ requests:
+ memory: "8Gi"
+ conf:
+ cache: "2Gi"
+ max-sql-memory: "2Gi"
+ tls:
+ enabled: false
+ ~~~
+
+ 1. You may want to modify `storage.persistentVolume.size` and `storage.persistentVolume.storageClass` for your use case. This chart defaults to 100Gi of disk space per pod. For more details on customizing disks for performance, see [these instructions]({% link {{ page.version.version }}/kubernetes-performance.md %}#disk-type).
+
+ {{site.data.alerts.callout_info}}
+ If necessary, you can [expand disk size](/docs/{{ page.version.version }}/configure-cockroachdb-kubernetes.html?filters=helm#expand-disk-size) after the cluster is live.
+ {{site.data.alerts.end}}
+
+1. Install the CockroachDB Helm chart.
+
+ Provide a "release" name to identify and track this particular deployment of the chart, and override the default values with those in `my-values.yaml`.
+
+ {{site.data.alerts.callout_info}}
+ This tutorial uses `my-release` as the release name. If you use a different value, be sure to adjust the release name in subsequent commands.
+ {{site.data.alerts.end}}
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ helm install my-release --values my-values.yaml cockroachdb/cockroachdb
+ ~~~
+
+ Behind the scenes, this command uses our `cockroachdb-statefulset.yaml` file to create the StatefulSet that automatically creates 3 pods, each with a CockroachDB node running inside it, where each pod has distinguishable network identity and always binds back to the same persistent storage on restart.
+
+1. Confirm that CockroachDB cluster initialization has completed successfully, with the pods for CockroachDB showing `1/1` under `READY` and the pod for initialization showing `COMPLETED` under `STATUS`:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ kubectl get pods
+ ~~~
+
+ ~~~
+ NAME READY STATUS RESTARTS AGE
+ my-release-cockroachdb-0 1/1 Running 0 8m
+ my-release-cockroachdb-1 1/1 Running 0 8m
+ my-release-cockroachdb-2 1/1 Running 0 8m
+ my-release-cockroachdb-init-hxzsc 0/1 Completed 0 1h
+ ~~~
+
+1. Confirm that the persistent volumes and corresponding claims were created successfully for all three pods:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ kubectl get pv
+ ~~~
+
+ ~~~
+ NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
+ pvc-71019b3a-fc67-11e8-a606-080027ba45e5 100Gi RWO Delete Bound default/datadir-my-release-cockroachdb-0 standard 11m
+ pvc-7108e172-fc67-11e8-a606-080027ba45e5 100Gi RWO Delete Bound default/datadir-my-release-cockroachdb-1 standard 11m
+ pvc-710dcb66-fc67-11e8-a606-080027ba45e5 100Gi RWO Delete Bound default/datadir-my-release-cockroachdb-2 standard 11m
+ ~~~
+
+{{site.data.alerts.callout_success}}
+The StatefulSet configuration sets all CockroachDB nodes to log to `stderr`, so if you ever need access to logs for a pod, use `kubectl logs 
+
+To check at a more granular level, SSH to one of the instances not running CockroachDB and run the `SHOW EXPERIMENTAL_RANGES` statement on the `vehicles` table:
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+$ cockroach sql \
+{{page.certs}} \
+--host= \
+--database=movr \
+--execute="SELECT * FROM \
+[SHOW EXPERIMENTAL_RANGES FROM TABLE vehicles] \
+WHERE \"start_key\" IS NOT NULL \
+ AND \"start_key\" NOT LIKE '%Prefix%';"
+~~~
+
+~~~
+ start_key | end_key | range_id | replicas | lease_holder
++------------------+----------------------------+----------+----------+--------------+
+ /"boston" | /"boston"/PrefixEnd | 105 | {1,2,3} | 3
+ /"los angeles" | /"los angeles"/PrefixEnd | 121 | {7,8,9} | 8
+ /"new york" | /"new york"/PrefixEnd | 101 | {1,2,3} | 3
+ /"san francisco" | /"san francisco"/PrefixEnd | 117 | {7,8,9} | 8
+ /"seattle" | /"seattle"/PrefixEnd | 113 | {4,5,6} | 5
+ /"washington dc" | /"washington dc"/PrefixEnd | 109 | {1,2,3} | 1
+(6 rows)
+~~~
+
+For reference, here's how the nodes map to zones:
+
+Node IDs | Zone
+---------|-----
+1-3 | `us-east1-b` (South Carolina)
+4-6 | `us-west1-a` (Oregon)
+7-9 | `us-west2-a` (Los Angeles)
+
+We can see that, after partitioning, the replicas for New York, Boston, and Washington DC are located on nodes 1-3 in `us-east1-b`, replicas for Seattle are located on nodes 4-6 in `us-west1-a`, and replicas for San Francisco and Los Angeles are located on nodes 7-9 in `us-west2-a`.
diff --git a/src/current/_includes/v25.1/performance/check-rebalancing.md b/src/current/_includes/v25.1/performance/check-rebalancing.md
new file mode 100644
index 00000000000..32e3d98f8f1
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/check-rebalancing.md
@@ -0,0 +1,33 @@
+Since you started each node with the `--locality` flag set to its GCE zone, over the next minutes, CockroachDB will rebalance data evenly across the zones.
+
+To check this, access the DB Console on any node at `
+
+For reference, here's how the nodes map to zones:
+
+Node IDs | Zone
+---------|-----
+1-3 | `us-east1-b` (South Carolina)
+4-6 | `us-west1-a` (Oregon)
+7-9 | `us-west2-a` (Los Angeles)
+
+To verify even balancing at range level, SSH to one of the instances not running CockroachDB and run the `SHOW EXPERIMENTAL_RANGES` statement:
+
+{% include_cached copy-clipboard.html %}
+~~~ shell
+$ cockroach sql \
+{{page.certs}} \
+--host= \
+--database=movr \
+--execute="SHOW EXPERIMENTAL_RANGES FROM TABLE vehicles;"
+~~~
+
+~~~
+ start_key | end_key | range_id | replicas | lease_holder
++-----------+---------+----------+----------+--------------+
+ NULL | NULL | 33 | {3,4,7} | 7
+(1 row)
+~~~
+
+In this case, we can see that, for the single range containing `vehicles` data, one replica is in each zone, and the leaseholder is in the `us-west2-a` zone.
diff --git a/src/current/_includes/v25.1/performance/configure-network.md b/src/current/_includes/v25.1/performance/configure-network.md
new file mode 100644
index 00000000000..e9abeb94df3
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/configure-network.md
@@ -0,0 +1,18 @@
+CockroachDB requires TCP communication on two ports:
+
+- **26257** (`tcp:26257`) for inter-node communication (i.e., working as a cluster)
+- **8080** (`tcp:8080`) for accessing the DB Console
+
+Since GCE instances communicate on their internal IP addresses by default, you do not need to take any action to enable inter-node communication. However, to access the DB Console from your local network, you must [create a firewall rule for your project](https://cloud.google.com/vpc/docs/using-firewalls):
+
+Field | Recommended Value
+------|------------------
+Name | **cockroachweb**
+Source filter | IP ranges
+Source IP ranges | Your local network's IP ranges
+Allowed protocols | **tcp:8080**
+Target tags | `cockroachdb`
+
+{{site.data.alerts.callout_info}}
+The **tag** feature will let you easily apply the rule to your instances.
+{{site.data.alerts.end}}
diff --git a/src/current/_includes/v25.1/performance/create-index-hash-sharded-secondary-index.md b/src/current/_includes/v25.1/performance/create-index-hash-sharded-secondary-index.md
new file mode 100644
index 00000000000..05f66896541
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/create-index-hash-sharded-secondary-index.md
@@ -0,0 +1,62 @@
+Let's assume the `events` table already exists:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> CREATE TABLE events (
+ product_id INT8,
+ owner UUID,
+ serial_number VARCHAR,
+ event_id UUID,
+ ts TIMESTAMP,
+ data JSONB,
+ PRIMARY KEY (product_id, owner, serial_number, ts, event_id)
+);
+~~~
+
+You can create a hash-sharded index on an existing table:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> CREATE INDEX ON events(ts) USING HASH;
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SHOW INDEX FROM events;
+~~~
+
+~~~
+ table_name | index_name | non_unique | seq_in_index | column_name | direction | storing | implicit
+-------------+---------------+------------+--------------+---------------------------+-----------+---------+-----------
+ events | events_pkey | false | 1 | product_id | ASC | false | false
+ events | events_pkey | false | 2 | owner | ASC | false | false
+ events | events_pkey | false | 3 | serial_number | ASC | false | false
+ events | events_pkey | false | 4 | ts | ASC | false | false
+ events | events_pkey | false | 5 | event_id | ASC | false | false
+ events | events_pkey | false | 6 | data | N/A | true | false
+ events | events_ts_idx | true | 1 | crdb_internal_ts_shard_16 | ASC | false | true
+ events | events_ts_idx | true | 2 | ts | ASC | false | false
+ events | events_ts_idx | true | 3 | product_id | ASC | false | true
+ events | events_ts_idx | true | 4 | owner | ASC | false | true
+ events | events_ts_idx | true | 5 | serial_number | ASC | false | true
+ events | events_ts_idx | true | 6 | event_id | ASC | false | true
+(12 rows)
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SHOW COLUMNS FROM events;
+~~~
+
+~~~
+ column_name | data_type | is_nullable | column_default | generation_expression | indices | is_hidden
+----------------------------+-----------+-------------+----------------+---------------------------------------------------+-----------------------------+------------
+ product_id | INT8 | false | NULL | | {events_pkey,events_ts_idx} | false
+ owner | UUID | false | NULL | | {events_pkey,events_ts_idx} | false
+ serial_number | VARCHAR | false | NULL | | {events_pkey,events_ts_idx} | false
+ event_id | UUID | false | NULL | | {events_pkey,events_ts_idx} | false
+ ts | TIMESTAMP | false | NULL | | {events_pkey,events_ts_idx} | false
+ data | JSONB | true | NULL | | {events_pkey} | false
+ crdb_internal_ts_shard_16 | INT8 | false | NULL | mod(fnv32(crdb_internal.datums_to_bytes(ts)), 16) | {events_ts_idx} | true
+(7 rows)
+~~~
diff --git a/src/current/_includes/v25.1/performance/create-table-hash-sharded-primary-index.md b/src/current/_includes/v25.1/performance/create-table-hash-sharded-primary-index.md
new file mode 100644
index 00000000000..40ba79a096a
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/create-table-hash-sharded-primary-index.md
@@ -0,0 +1,37 @@
+Let's create the `products` table and add a hash-sharded primary key on the `ts` column:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> CREATE TABLE products (
+ ts DECIMAL PRIMARY KEY USING HASH,
+ product_id INT8
+ );
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SHOW INDEX FROM products;
+~~~
+
+~~~
+ table_name | index_name | non_unique | seq_in_index | column_name | direction | storing | implicit
+-------------+---------------+------------+--------------+---------------------------+-----------+---------+-----------
+ products | products_pkey | false | 1 | crdb_internal_ts_shard_16 | ASC | false | true
+ products | products_pkey | false | 2 | ts | ASC | false | false
+ products | products_pkey | false | 3 | product_id | N/A | true | false
+(3 rows)
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SHOW COLUMNS FROM products;
+~~~
+
+~~~
+ column_name | data_type | is_nullable | column_default | generation_expression | indices | is_hidden
+----------------------------+-----------+-------------+----------------+---------------------------------------------------+-----------------+------------
+ crdb_internal_ts_shard_16 | INT8 | false | NULL | mod(fnv32(crdb_internal.datums_to_bytes(ts)), 16) | {products_pkey} | true
+ ts | DECIMAL | false | NULL | | {products_pkey} | false
+ product_id | INT8 | true | NULL | | {products_pkey} | false
+(3 rows)
+~~~
diff --git a/src/current/_includes/v25.1/performance/create-table-hash-sharded-secondary-index.md b/src/current/_includes/v25.1/performance/create-table-hash-sharded-secondary-index.md
new file mode 100644
index 00000000000..dc0e164a0fb
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/create-table-hash-sharded-secondary-index.md
@@ -0,0 +1,56 @@
+Let's now create the `events` table and add a secondary index on the `ts` column in a single statement:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> CREATE TABLE events (
+ product_id INT8,
+ owner UUID,
+ serial_number VARCHAR,
+ event_id UUID,
+ ts TIMESTAMP,
+ data JSONB,
+ PRIMARY KEY (product_id, owner, serial_number, ts, event_id),
+ INDEX (ts) USING HASH
+);
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SHOW INDEX FROM events;
+~~~
+
+~~~
+ table_name | index_name | non_unique | seq_in_index | column_name | direction | storing | implicit
+-------------+---------------+------------+--------------+---------------------------+-----------+---------+-----------
+ events | events_pkey | false | 1 | product_id | ASC | false | false
+ events | events_pkey | false | 2 | owner | ASC | false | false
+ events | events_pkey | false | 3 | serial_number | ASC | false | false
+ events | events_pkey | false | 4 | ts | ASC | false | false
+ events | events_pkey | false | 5 | event_id | ASC | false | false
+ events | events_pkey | false | 6 | data | N/A | true | false
+ events | events_ts_idx | true | 1 | crdb_internal_ts_shard_16 | ASC | false | true
+ events | events_ts_idx | true | 2 | ts | ASC | false | false
+ events | events_ts_idx | true | 3 | product_id | ASC | false | true
+ events | events_ts_idx | true | 4 | owner | ASC | false | true
+ events | events_ts_idx | true | 5 | serial_number | ASC | false | true
+ events | events_ts_idx | true | 6 | event_id | ASC | false | true
+(12 rows)
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SHOW COLUMNS FROM events;
+~~~
+
+~~~
+ column_name | data_type | is_nullable | column_default | generation_expression | indices | is_hidden
+----------------------------+-----------+-------------+----------------+---------------------------------------------------+-----------------------------+------------
+ product_id | INT8 | false | NULL | | {events_pkey,events_ts_idx} | false
+ owner | UUID | false | NULL | | {events_pkey,events_ts_idx} | false
+ serial_number | VARCHAR | false | NULL | | {events_pkey,events_ts_idx} | false
+ event_id | UUID | false | NULL | | {events_pkey,events_ts_idx} | false
+ ts | TIMESTAMP | false | NULL | | {events_pkey,events_ts_idx} | false
+ data | JSONB | true | NULL | | {events_pkey} | false
+ crdb_internal_ts_shard_16 | INT8 | false | NULL | mod(fnv32(crdb_internal.datums_to_bytes(ts)), 16) | {events_ts_idx} | true
+(7 rows)
+~~~
diff --git a/src/current/_includes/v25.1/performance/increase-server-side-retries.md b/src/current/_includes/v25.1/performance/increase-server-side-retries.md
new file mode 100644
index 00000000000..95f2c2a9647
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/increase-server-side-retries.md
@@ -0,0 +1,5 @@
+- [Send statements in transactions as a single batch]({% link {{ page.version.version }}/transactions.md %}#batched-statements). Batching allows CockroachDB to [automatically retry]({% link {{ page.version.version }}/transactions.md %}#automatic-retries) a transaction when [previous reads are invalidated]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#read-refreshing) at a [pushed timestamp]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#timestamp-cache). When a multi-statement transaction is not batched, and takes more than a single round trip, CockroachDB cannot automatically retry the transaction. For an example showing how to break up large transactions in an application, see [Break up large transactions into smaller units of work](build-a-python-app-with-cockroachdb-sqlalchemy.html#break-up-large-transactions-into-smaller-units-of-work).
+
+
+
+- Limit the size of the result sets of your transactions to under 16KB, so that CockroachDB is more likely to [automatically retry]({% link {{ page.version.version }}/transactions.md %}#automatic-retries) when [previous reads are invalidated]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#read-refreshing) at a [pushed timestamp]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#timestamp-cache). When a transaction returns a result set over 16KB, even if that transaction has been sent as a single batch, CockroachDB cannot automatically retry the transaction. You can change the results buffer size for all new sessions using the `sql.defaults.results_buffer.size` [cluster setting](cluster-settings.html), or for a specific session using the `results_buffer_size` [connection parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters).
diff --git a/src/current/_includes/v25.1/performance/lease-preference-system-database.md b/src/current/_includes/v25.1/performance/lease-preference-system-database.md
new file mode 100644
index 00000000000..9661aef0e2d
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/lease-preference-system-database.md
@@ -0,0 +1,10 @@
+To reduce latency while making {% if page.name == "online-schema-changes.md" %}online schema changes{% else %}[online schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}){% endif %}, we recommend specifying a `lease_preference` [zone configuration]({% link {{ page.version.version }}/configure-replication-zones.md %}) on the `system` database to a single region and running all subsequent schema changes from a node within that region. For example, if the majority of online schema changes come from machines that are geographically close to `us-east1`, run the following:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+ALTER DATABASE system CONFIGURE ZONE USING constraints = '{"+region=us-east1": 1}', lease_preferences = '[[+region=us-east1]]';
+~~~
+
+Run all subsequent schema changes from a node in the specified region.
+
+If you do not intend to run more schema changes from that region, you can safely [remove the lease preference from the zone configuration]({% link {{ page.version.version }}/alter-database.md %}#remove-a-replication-zone) for the system database.
diff --git a/src/current/_includes/v25.1/performance/partition-by-city.md b/src/current/_includes/v25.1/performance/partition-by-city.md
new file mode 100644
index 00000000000..226c2d1d5f3
--- /dev/null
+++ b/src/current/_includes/v25.1/performance/partition-by-city.md
@@ -0,0 +1,419 @@
+For this service, the most effective technique for improving read and write latency is to [geo-partition]({% link {{ page.version.version }}/partitioning.md %}) the data by city. In essence, this means changing the way data is mapped to ranges. Instead of an entire table and its indexes mapping to a specific range or set of ranges, all rows in the table and its indexes with a given city will map to a range or set of ranges. Once ranges are defined in this way, we can then use the [replication zone]({% link {{ page.version.version }}/configure-replication-zones.md %}) feature to pin partitions to specific locations, ensuring that read and write requests from users in a specific city do not have to leave that region.
+
+1. Partitioning is an enterprise feature, so start off by [registering for a 30-day trial license](https://www.cockroachlabs.com/get-cockroachdb/enterprise/).
+
+1. Once you've received the trial license, SSH to any node in your cluster and [apply the license]({% link {{ page.version.version }}/licensing-faqs.md %}#set-a-license):
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ cockroach sql \
+ {{page.certs}} \
+ --host= \
+ --execute="SET CLUSTER SETTING cluster.organization = '
+
+
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/create-virtual-cluster-diagram.html b/src/current/_includes/v25.1/physical-replication/create-virtual-cluster-diagram.html
new file mode 100644
index 00000000000..2f0eff15eb9
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/create-virtual-cluster-diagram.html
@@ -0,0 +1,171 @@
+
+
+
+
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/drop-virtual-cluster-diagram.html b/src/current/_includes/v25.1/physical-replication/drop-virtual-cluster-diagram.html
new file mode 100644
index 00000000000..df3fc0eb983
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/drop-virtual-cluster-diagram.html
@@ -0,0 +1,65 @@
+
+
+
+
diff --git a/src/current/_includes/v25.1/physical-replication/failover-read-virtual-cluster.md b/src/current/_includes/v25.1/physical-replication/failover-read-virtual-cluster.md
new file mode 100644
index 00000000000..7430d38f9a9
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/failover-read-virtual-cluster.md
@@ -0,0 +1 @@
+If you started the PCR stream with the `READ VIRTUAL CLUSTER` option, failing over with `SYSTEM TIME` will destroy the `readonly` virtual cluster. If you fail over with `LATEST`, the `readonly` virtual cluster will remain on the original standby cluster, but will **not** update with new writes.
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/fast-failback-latest-timestamp.md b/src/current/_includes/v25.1/physical-replication/fast-failback-latest-timestamp.md
new file mode 100644
index 00000000000..b5c5234fbb4
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/fast-failback-latest-timestamp.md
@@ -0,0 +1 @@
+When you [fail back]({% link {{ page.version.version }}/failover-replication.md %}#fail-back-to-the-primary-cluster) to a cluster that was previously the primary cluster, you should fail over to the `LATEST` timestamp. Using a [historical timestamp]({% link {{ page.version.version }}/as-of-system-time.md %}) may lead to the failback failing. {% if page.name == "failover-replication.md" %} Refer to the [PCR known limitations]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}#known-limitations).{% endif %}
diff --git a/src/current/_includes/v25.1/physical-replication/fast-failback-syntax.md b/src/current/_includes/v25.1/physical-replication/fast-failback-syntax.md
new file mode 100644
index 00000000000..6bf2a8f03bc
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/fast-failback-syntax.md
@@ -0,0 +1,8 @@
+To {% if page.name == "alter-virtual-cluster.md" %} [fail back]({% link {{ page.version.version }}/failover-replication.md %}#fail-back-to-the-primary-cluster) {% else %} fail back {% endif %} to a cluster that was previously the primary cluster, use the {% if page.name == "alter-virtual-cluster.md" %} `ALTER VIRTUAL CLUSTER` {% else %} [`ALTER VIRTUAL CLUSTER`]({% link {{ page.version.version }}/alter-virtual-cluster.md %}) {% endif %} syntax:
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+ALTER VIRTUAL CLUSTER {original_primary_vc} START REPLICATION OF {promoted_standby_vc} ON {connection_string_standby};
+~~~
+
+The original primary virtual cluster may be almost up to date with the promoted standby's virtual cluster. The difference in data between the two virtual clusters will include only the writes that have been applied to the promoted standby after failover from the primary cluster.
diff --git a/src/current/_includes/v25.1/physical-replication/interface-virtual-cluster.md b/src/current/_includes/v25.1/physical-replication/interface-virtual-cluster.md
new file mode 100644
index 00000000000..02890c3fc83
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/interface-virtual-cluster.md
@@ -0,0 +1,2 @@
+- The system virtual cluster manages the cluster's control plane and the replication of the cluster's data. Admins connect to the system virtual cluster to configure and manage the underlying CockroachDB cluster, set up PCR, create and manage a virtual cluster, and observe metrics and logs for the CockroachDB cluster and each virtual cluster.
+- Each other virtual cluster manages its own data plane. Users connect to a virtual cluster by default, rather than the system virtual cluster. To connect to the system virtual cluster, the connection string must be modified. Virtual clusters contain user data and run application workloads. When PCR is enabled, the non-system virtual cluster on both primary and secondary clusters is named `main`.
diff --git a/src/current/_includes/v25.1/physical-replication/like-description.md b/src/current/_includes/v25.1/physical-replication/like-description.md
new file mode 100644
index 00000000000..5922c4a6463
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/like-description.md
@@ -0,0 +1 @@
+Including the `LIKE template` parameter ensures that the virtual cluster on the standby is created with the correct capabilities, which manage what the virtual cluster can do. `LIKE` will refer to a virtual cluster on the CockroachDB cluster you're running the statement from.
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/phys-rep-sql-pages.md b/src/current/_includes/v25.1/physical-replication/phys-rep-sql-pages.md
new file mode 100644
index 00000000000..562905ff97e
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/phys-rep-sql-pages.md
@@ -0,0 +1,5 @@
+PCR happens between an _active_ primary cluster and a _passive_ standby cluster that accepts updates from the primary cluster. The unit of replication is a _virtual cluster_, which is part of the underlying infrastructure in the primary and standby clusters. The CockroachDB cluster has:
+
+{% include {{ page.version.version }}/physical-replication/interface-virtual-cluster.md %}
+
+For more detail, refer to the [Physical Cluster Replication Overview]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}).
diff --git a/src/current/_includes/v25.1/physical-replication/reference-links-replication.md b/src/current/_includes/v25.1/physical-replication/reference-links-replication.md
new file mode 100644
index 00000000000..5d7b017b9fb
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/reference-links-replication.md
@@ -0,0 +1,4 @@
+{% comment %}
+- Cluster virtualization: The primary and standby clusters are started as virtualized clusters.
+- [Physical Cluster Replication Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-overview.md %}) page.
+{% endcomment %}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/retention.md b/src/current/_includes/v25.1/physical-replication/retention.md
new file mode 100644
index 00000000000..303fe6ebc79
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/retention.md
@@ -0,0 +1 @@
+We do not recommend setting `RETENTION` much higher than the 24-hour default on the standby cluster. Accumulated data from an excessive [retention (failover) window]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}#failover-and-promotion-process) could affect queries running on the standby cluster that is active following a [failover]({% link {{ page.version.version }}/failover-replication.md %}).
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-data-state.md b/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-data-state.md
new file mode 100644
index 00000000000..16f858ef621
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-data-state.md
@@ -0,0 +1,10 @@
+State | Description
+-----------+----------------
+`add` | ([**Preview**]({% link {{ page.version.version }}/cockroachdb-feature-availability.md %}#features-in-preview)) The [`readonly` virtual cluster]({% link {{ page.version.version }}/create-virtual-cluster.md %}#start-a-pcr-stream-with-read-from-standby) is waiting for the PCR job's initial scan to complete, then `readonly` will be available for read queries.
+`initializing replication` | The replication job is completing the initial scan of data from the primary cluster before it starts replicating data in real time.
+`ready` | A virtual cluster's data is ready for use. The `readonly` virtual cluster is ready to serve read queries.
+`replicating` | The replication job has started and is replicating data.
+`replication paused` | The replication job is paused due to an error or a manual request with [`ALTER VIRTUAL CLUSTER ... PAUSE REPLICATION`]({% link {{ page.version.version }}/alter-virtual-cluster.md %}).
+`replication pending failover` | The replication job is running and the failover time has been set. Once the the replication reaches the failover time, the failover will begin automatically.
+`replication failing over` | The job has started failing over. The failover time can no longer be changed. Once failover is complete, a virtual cluster will be available for use with [`ALTER VIRTUAL CLUSTER ... START SERVICE SHARED`]({% link {{ page.version.version }}/alter-virtual-cluster.md %}).
+`replication error` | An error has occurred. You can find more detail in the error message and the [logs]({% link {{ page.version.version }}/configure-logs.md %}). **Note:** A PCR job will retry for 3 minutes before failing.
diff --git a/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-diagram.html b/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-diagram.html
new file mode 100644
index 00000000000..f9d31ede888
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-diagram.html
@@ -0,0 +1,89 @@
+
+
+
+
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-responses.md b/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-responses.md
new file mode 100644
index 00000000000..97c962a2547
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/show-virtual-cluster-responses.md
@@ -0,0 +1,15 @@
+Field | Response
+---------+----------
+`id` | The ID of a virtual cluster.
+`name` | The name of the standby (destination) virtual cluster.
+`data_state` | The state of the data on a virtual cluster. This can show one of the following: `initializing replication`, `ready`, `replicating`, `replication paused`, `replication pending failover`, `replication failing over`, `replication error`. Refer to [Data state](#data-state) for more detail on each response.
+`service_mode` | The service mode shows whether a virtual cluster is ready to accept SQL requests. This can show `none` or `shared`. When `shared`, a virtual cluster's SQL connections will be served by the same nodes that are serving the system virtual cluster.
+`source_tenant_name` | The name of the primary (source) virtual cluster.
+`source_cluster_uri` | The URI of the primary (source) cluster. The standby cluster connects to the primary cluster using this URI when [starting a replication stream]({% link {{ page.version.version }}/set-up-physical-cluster-replication.md %}#step-4-start-replication).
+`replicated_time` | The latest timestamp at which the standby cluster has consistent data — that is, the latest time you can fail over to. This time advances automatically as long as the replication proceeds without error. `replicated_time` is updated periodically (every `30s`).
+`retained_time` | The earliest timestamp at which the standby cluster has consistent data — that is, the earliest time you can fail over to.
+`replication_lag` | The time between the most up-to-date replicated time and the actual time. Refer to the [Technical Overview]({% link {{ page.version.version }}/physical-cluster-replication-technical-overview.md %}) for more detail.
+`failover_time` | The time at which the failover will begin. This can be in the past or the future. Refer to [Fail over to a point in time]({% link {{ page.version.version }}/failover-replication.md %}#fail-over-to-a-point-in-time).
+`status` | The status of the replication stream. This can show one of the following: `initializing replication`, `ready`, `replicating`, `replication paused`, `replication pending failover`, `replication failing over`, `replication error`. Refer to [Data state](#data-state) for more detail on each response.
+`capability_name` | The [capability]({% link {{ page.version.version }}/create-virtual-cluster.md %}#capabilities) name.
+`capability_value` | Whether the [capability]({% link {{ page.version.version }}/create-virtual-cluster.md %}#capabilities) is enabled for a virtual cluster.
diff --git a/src/current/_includes/v25.1/physical-replication/template-description.md b/src/current/_includes/v25.1/physical-replication/template-description.md
new file mode 100644
index 00000000000..233b31f99b1
--- /dev/null
+++ b/src/current/_includes/v25.1/physical-replication/template-description.md
@@ -0,0 +1 @@
+The [configuration profile](#start-the-standby-cluster) included at startup creates the `template` virtual cluster with the same set of _capabilities_ per CockroachDB version. When you start a replication stream, you can specify the `template` VC with `LIKE` to ensure other virtual clusters on the standby cluster will work in the same way. Refer to [Step 4: Start replication](#step-4-start-replication) for syntax details.
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/prod-deployment/advertise-addr-join.md b/src/current/_includes/v25.1/prod-deployment/advertise-addr-join.md
new file mode 100644
index 00000000000..2c8d39660fb
--- /dev/null
+++ b/src/current/_includes/v25.1/prod-deployment/advertise-addr-join.md
@@ -0,0 +1,4 @@
+Flag | Description
+-----|------------
+`--advertise-addr` | Specifies the IP address/hostname and port to tell other nodes to use. The port number can be omitted, in which case it defaults to `26257`.This value must route to an IP address the node is listening on (with `--listen-addr` unspecified, the node listens on all IP addresses).
In some networking scenarios, you may need to use `--advertise-addr` and/or `--listen-addr` differently. For more details, see [Networking]({% link {{ page.version.version }}/recommended-production-settings.md %}#networking). +`--join` | Identifies the address of 3-5 of the initial nodes of the cluster. These addresses should match the addresses that the target nodes are advertising. diff --git a/src/current/_includes/v25.1/prod-deployment/aws-inbound-rules.md b/src/current/_includes/v25.1/prod-deployment/aws-inbound-rules.md new file mode 100644 index 00000000000..8be748205a6 --- /dev/null +++ b/src/current/_includes/v25.1/prod-deployment/aws-inbound-rules.md @@ -0,0 +1,31 @@ +#### Inter-node and load balancer-node communication + + Field | Value +-------|------------------- + Port Range | **26257** + Source | The ID of your security group (e.g., *sg-07ab277a*) + +#### Application data + + Field | Value +-------|------------------- + Port Range | **26257** + Source | Your application's IP ranges + +#### DB Console + + Field | Value +-------|------------------- + Port Range | **8080** + Source | Your network's IP ranges + +You can set your network IP by selecting "My IP" in the Source field. + +#### Load balancer-health check communication + + Field | Value +-------|------------------- + Port Range | **8080** + Source | The IP range of your VPC in CIDR notation (e.g., 10.12.0.0/16) + + To get the IP range of a VPC, open the [Amazon VPC console](https://console.aws.amazon.com/vpc/) and find the VPC listed in the section called Your VPCs. \ No newline at end of file diff --git a/src/current/_includes/v25.1/prod-deployment/backup.sh b/src/current/_includes/v25.1/prod-deployment/backup.sh new file mode 100644 index 00000000000..efcbd4c7041 --- /dev/null +++ b/src/current/_includes/v25.1/prod-deployment/backup.sh @@ -0,0 +1,21 @@ +#!/bin/bash + +set -euo pipefail + +# This script creates full backups when run on the configured +# day of the week and incremental backups when run on other days, and tracks +# recently created backups in a file to pass as the base for incremental backups. + +what="" # Leave empty for cluster backup, or add "DATABASE database_name" to backup a database. +base="
+
+
+
+
+
+
+
+
+
+
+
+This value must route to an IP address the node is listening on (with `--listen-addr` unspecified, the node listens on all IP addresses).
In some networking scenarios, you may need to use `--advertise-addr` and/or `--listen-addr` differently. For more details, see [Networking]({% link {{ page.version.version }}/recommended-production-settings.md %}#networking). + `--join` | Identifies the address of 3-5 of the initial nodes of the cluster. These addresses should match the addresses that the target nodes are advertising. + `--cache`
`--max-sql-memory` | Increases the node's cache size to 25% of available system memory to improve read performance. The capacity for in-memory SQL processing defaults to 25% of system memory but can be raised, if necessary, to increase the number of simultaneous client connections allowed by the node as well as the node's capacity for in-memory processing of rows when using `ORDER BY`, `GROUP BY`, `DISTINCT`, joins, and window functions. For more details, see [Cache and SQL Memory Size]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size). + `--background` | Starts the node in the background so you gain control of the terminal to issue more commands. + + When deploying across multiple datacenters, or when there is otherwise high latency between nodes, it is recommended to set `--locality` as well. It is also required to use certain enterprise features. For more details, see [Locality]({% link {{ page.version.version }}/cockroach-start.md %}#locality). + + For other flags not explicitly set, the command uses default values. For example, the node stores data in `--store=cockroach-data` and binds DB Console HTTP requests to `--http-addr=localhost:8080`. To set these options manually, see [Start a Node]({% link {{ page.version.version }}/cockroach-start.md %}). + +1. Repeat these steps for each additional node that you want in your cluster. + +
(2 * --max-sql-memory) + --cache <= 80% of system RAM
+
+
+
+
+
+
+
+
+
+
+
+
+This value must route to an IP address the node is listening on (with `--listen-addr` unspecified, the node listens on all IP addresses).
In some networking scenarios, you may need to use `--advertise-addr` and/or `--listen-addr` differently. For more details, see [Networking]({% link {{ page.version.version }}/recommended-production-settings.md %}#networking). + `--join` | Identifies the address of 3-5 of the initial nodes of the cluster. These addresses should match the addresses that the target nodes are advertising. + `--cache`
`--max-sql-memory` | Increases the node's cache size to 25% of available system memory to improve read performance. The capacity for in-memory SQL processing defaults to 25% of system memory but can be raised, if necessary, to increase the number of simultaneous client connections allowed by the node as well as the node's capacity for in-memory processing of rows when using `ORDER BY`, `GROUP BY`, `DISTINCT`, joins, and window functions. For more details, see [Cache and SQL Memory Size]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size). + `--background` | Starts the node in the background so you gain control of the terminal to issue more commands. + + When deploying across multiple datacenters, or when there is otherwise high latency between nodes, it is recommended to set `--locality` as well. For more details, see [Locality]({% link {{ page.version.version }}/cockroach-start.md %}#locality). + + For other flags not explicitly set, the command uses default values. For example, the node stores data in `--store=cockroach-data` and binds DB Console HTTP requests to `--http-addr=
| + | Point-in-time backup & restore | +Physical cluster replication (asynchronous) | +
|---|---|---|
| + RPO + | +>=5 minutes | +10s of seconds | +
| + RTO + | +Minutes to hours, depending on data size and number of nodes | +Seconds to minutes, depending on cluster size, and time of failover | +
| + Write latency + | +No impact | +No impact | +
| + Recovery + | +Manual restore | +Manual failover | +
| + Fault tolerance + | +Not applicable | +Zero RPO node, availability zone within a cluster, region failures with loss up to RPO in a two-region (or two-datacenter) setup | +
| + Minimum regions to achieve fault tolerance + | +1 | +2 | +
+
+
+
+
+
+
+
+
+
+Choose your installation method
+ +You can create a CockroachDB {{ site.data.products.serverless }} cluster using either the CockroachDB Cloud Console, a web-based graphical user interface (GUI) tool, orccloud, a command-line interface (CLI) tool.
+
+
+
+
+
+
+
+
+### Create a free cluster
+
+{% include cockroachcloud/quickstart/create-a-free-cluster.md %}
+
+### Create a SQL user
+
+{% include {{ page.version.version }}/setup/create-first-sql-user.md %}
+
+### Get the root certificate
+
+The **Connect to cluster** dialog shows information about how to connect to your cluster.
+
+1. Select **General connection string** from the **Select option** dropdown.
+1. Open a new terminal on your local machine, and run the **CA Cert download command** provided in the **Download CA Cert** section. The client driver used in this tutorial requires this certificate to connect to CockroachDB {{ site.data.products.cloud }}.
+
+### Get the connection string
+
+Open the **General connection string** section, then copy the connection string provided and save it in a secure location.
+
+{{site.data.alerts.callout_info}}
+The connection string is pre-populated with your username, password, cluster name, and other details. Your password, in particular, will be provided *only once*. Save it in a secure place (Cockroach Labs recommends a password manager) to connect to your cluster in the future. If you forget your password, you can reset it by going to the **SQL Users** page for the cluster, found at `https://cockroachlabs.cloud/cluster//users`.
+{{site.data.alerts.end}}
+
+
+
+
+
+Follow these steps to create a CockroachDB {{ site.data.products.serverless }} cluster using the Install
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **General connection string**, then copy the connection string displayed and save it in a secure location. The connection string is the line starting `postgresql://`.
+
+~~~
+? How would you like to connect? General connection string
+Retrieving cluster info: succeeded
+ Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
+postgresql://maxroach:ThisIsNotAGoodPassword@blue-dog-147.6wr.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt
+~~~
+
+
+ccloud CLI tool.
+
+{{site.data.alerts.callout_info}}
+The ccloud CLI tool is in Preview.
+{{site.data.alerts.end}}
+
+Install ccloud
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **General connection string**, then copy the connection string displayed and save it in a secure location. The connection string is the line starting `postgresql://`.
+
+~~~
+? How would you like to connect? General connection string
+Retrieving cluster info: succeeded
+ Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
+postgresql://maxroach:ThisIsNotAGoodPassword@blue-dog-147.6wr.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt
+~~~
+
+
+
+
+
+
+
+
+
+Choose your installation method
+ +You can create a CockroachDB {{ site.data.products.serverless }} cluster using either the CockroachDB Cloud Console, a web-based graphical user interface (GUI) tool, orccloud, a command-line interface (CLI) tool.
+
+
+
+
+
+
+
+
+### Create a free cluster
+
+{% include cockroachcloud/quickstart/create-a-free-cluster.md %}
+
+### Create a SQL user
+
+{% include {{ page.version.version }}/setup/create-first-sql-user.md %}
+
+### Get the connection string
+
+The **Connect to cluster** dialog shows information about how to connect to your cluster.
+
+1. Select **Java** from the **Select option/language** dropdown.
+1. Select **JDBC** from the **Select tool** dropdown.
+1. Copy the command provided to set the `JDBC_DATABASE_URL` environment variable.
+
+ {{site.data.alerts.callout_info}}
+ The JDBC connection URL is pre-populated with your username, password, cluster name, and other details. Your password, in particular, will be provided *only once*. Save it in a secure place (Cockroach Labs recommends a password manager) to connect to your cluster in the future. If you forget your password, you can reset it by going to the **SQL Users** page for the cluster, found at `https://cockroachlabs.cloud/cluster//users`.
+ {{site.data.alerts.end}}
+
+
+
+
+
+Follow these steps to create a CockroachDB {{ site.data.products.serverless }} cluster using the Install
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **General connection string**, then copy the connection string displayed and save it in a secure location. The connection string is the line starting `postgresql://`.
+
+~~~
+? How would you like to connect? General connection string
+Retrieving cluster info: succeeded
+ Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
+postgresql://maxroach:ThisIsNotAGoodPassword@blue-dog-147.6wr.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt
+~~~
+
+
+ccloud CLI tool.
+
+{{site.data.alerts.callout_info}}
+The ccloud CLI tool is in Preview.
+{{site.data.alerts.end}}
+
+Install ccloud
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **General connection string**, then copy the connection string displayed and save it in a secure location. The connection string is the line starting `postgresql://`.
+
+~~~
+? How would you like to connect? General connection string
+Retrieving cluster info: succeeded
+ Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
+postgresql://maxroach:ThisIsNotAGoodPassword@blue-dog-147.6wr.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt
+~~~
+
+
+
+
+
+
+
+
+
+
+Choose your installation method
+ +You can install a CockroachDB {{ site.data.products.serverless }} cluster using either the CockroachDB Cloud Console, a web-based graphical user interface (GUI) tool, orccloud, a command-line interface (CLI) tool.
+
+
+
+
+
+
+
+
+### Create a free cluster
+
+{% include cockroachcloud/quickstart/create-a-free-cluster.md %}
+
+### Create a SQL user
+
+{% include {{ page.version.version }}/setup/create-first-sql-user.md %}
+
+### Get the root certificate
+
+The **Connect to cluster** dialog shows information about how to connect to your cluster.
+
+1. Select **General connection string** from the **Select option** dropdown.
+1. Open a new terminal on your local machine, and run the **CA Cert download command** provided in the **Download CA Cert** section. The client driver used in this tutorial requires this certificate to connect to CockroachDB {{ site.data.products.cloud }}.
+
+### Get the connection information
+
+1. Select **Parameters only** from the **Select option** dropdown.
+1. Copy the connection information for each parameter displayed and save it in a secure location.
+
+
+
+
+
+Follow these steps to create a CockroachDB {{ site.data.products.serverless }} cluster using the Install
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **Parameters only** then copy the connection parameters displayed and save them in a secure location.
+
+~~~
+? How would you like to connect? Parameters only
+Looking up cluster ID: succeeded
+Creating SQL user: succeeded
+Success! Created SQL user
+ name: maxroach
+ cluster: 37174250-b944-461f-b1c1-3a99edb6af32
+Retrieving cluster info: succeeded
+Connection parameters
+ Database: defaultdb
+ Host: blue-dog-147.6wr.cockroachlabs.cloud
+ Password: ThisIsNotAGoodPassword
+ Port: 26257
+ Username: maxroach
+~~~
+
+
+
+ccloud CLI tool.
+
+{{site.data.alerts.callout_info}}
+The ccloud CLI tool is in Preview.
+{{site.data.alerts.end}}
+
+Install ccloud
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **Parameters only** then copy the connection parameters displayed and save them in a secure location.
+
+~~~
+? How would you like to connect? Parameters only
+Looking up cluster ID: succeeded
+Creating SQL user: succeeded
+Success! Created SQL user
+ name: maxroach
+ cluster: 37174250-b944-461f-b1c1-3a99edb6af32
+Retrieving cluster info: succeeded
+Connection parameters
+ Database: defaultdb
+ Host: blue-dog-147.6wr.cockroachlabs.cloud
+ Password: ThisIsNotAGoodPassword
+ Port: 26257
+ Username: maxroach
+~~~
+
+
+
+
+
+
+
+
+
+
+Choose your installation method
+ +You can install a CockroachDB {{ site.data.products.serverless }} cluster using either the CockroachDB Cloud Console, a web-based graphical user interface (GUI) tool, orccloud, a command-line interface (CLI) tool.
+
+
+
+
+
+
+
+
+### Create a free cluster
+
+{% include cockroachcloud/quickstart/create-a-free-cluster.md %}
+
+### Create a SQL user
+
+{% include {{ page.version.version }}/setup/create-first-sql-user.md %}
+
+### Get the connection information
+
+The **Connect to cluster** dialog shows information about how to connect to your cluster.
+
+1. Select **Parameters only** from the **Select option** dropdown.
+1. Copy the connection information for each parameter displayed and save it in a secure location.
+
+
+
+
+
+Follow these steps to create a CockroachDB {{ site.data.products.serverless }} cluster using the Install
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **Parameters only** then copy the connection parameters displayed and save them in a secure location.
+
+~~~
+? How would you like to connect? Parameters only
+Looking up cluster ID: succeeded
+Creating SQL user: succeeded
+Success! Created SQL user
+ name: maxroach
+ cluster: 37174250-b944-461f-b1c1-3a99edb6af32
+Retrieving cluster info: succeeded
+Connection parameters
+ Database: defaultdb
+ Host: blue-dog-147.6wr.cockroachlabs.cloud
+ Password: ThisIsNotAGoodPassword
+ Port: 26257
+ Username: maxroach
+~~~
+
+
+
+ccloud CLI tool.
+
+{{site.data.alerts.callout_info}}
+The ccloud CLI tool is in Preview.
+{{site.data.alerts.end}}
+
+Install ccloud
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **Parameters only** then copy the connection parameters displayed and save them in a secure location.
+
+~~~
+? How would you like to connect? Parameters only
+Looking up cluster ID: succeeded
+Creating SQL user: succeeded
+Success! Created SQL user
+ name: maxroach
+ cluster: 37174250-b944-461f-b1c1-3a99edb6af32
+Retrieving cluster info: succeeded
+Connection parameters
+ Database: defaultdb
+ Host: blue-dog-147.6wr.cockroachlabs.cloud
+ Password: ThisIsNotAGoodPassword
+ Port: 26257
+ Username: maxroach
+~~~
+
+
+
+
+
+
+
+
+
+
+Choose your installation method
+ +You can install a CockroachDB {{ site.data.products.serverless }} cluster using either the CockroachDB Cloud Console, a web-based graphical user interface (GUI) tool, orccloud, a command-line interface (CLI) tool.
+
+
+
+
+
+
+
+
+### Create a free cluster
+
+{% include cockroachcloud/quickstart/create-a-free-cluster.md %}
+
+### Create a SQL user
+
+{% include {{ page.version.version }}/setup/create-first-sql-user.md %}
+
+### Get the connection string
+
+The **Connect to cluster** dialog shows information about how to connect to your cluster.
+
+1. Select **General connection string** from the **Select option** dropdown.
+1. Open the **General connection string** section, then copy the connection string provided and save it in a secure location.
+
+ The sample application used in this tutorial uses system CA certificates for server certificate verification, so you can skip the **Download CA Cert** instructions.
+
+ {{site.data.alerts.callout_info}}
+ The connection string is pre-populated with your username, password, cluster name, and other details. Your password, in particular, will be provided *only once*. Save it in a secure place (Cockroach Labs recommends a password manager) to connect to your cluster in the future. If you forget your password, you can reset it by going to the **SQL Users** page for the cluster, found at `https://cockroachlabs.cloud/cluster//users`.
+ {{site.data.alerts.end}}
+
+
+
+
+
+Follow these steps to create a CockroachDB {{ site.data.products.serverless }} cluster using the Install
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **General connection string**, then copy the connection string displayed and save it in a secure location. The connection string is the line starting `postgresql://`.
+
+~~~
+? How would you like to connect? General connection string
+Retrieving cluster info: succeeded
+ Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
+postgresql://maxroach:ThisIsNotAGoodPassword@blue-dog-147.6wr.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt
+~~~
+
+
+ccloud CLI tool.
+
+{{site.data.alerts.callout_info}}
+The ccloud CLI tool is in Preview.
+{{site.data.alerts.end}}
+
+Install ccloud
+
+{% include cockroachcloud/ccloud/install-ccloud.md %}
+
+### Run `ccloud quickstart` to create a new cluster, create a SQL user, and retrieve the connection string.
+
+{% include cockroachcloud/ccloud/quickstart.md %}
+
+Select **General connection string**, then copy the connection string displayed and save it in a secure location. The connection string is the line starting `postgresql://`.
+
+~~~
+? How would you like to connect? General connection string
+Retrieving cluster info: succeeded
+ Downloading cluster cert to /Users/maxroach/.postgresql/root.crt: succeeded
+postgresql://maxroach:ThisIsNotAGoodPassword@blue-dog-147.6wr.cockroachlabs.cloud:26257/defaultdb?sslmode=verify-full&sslrootcert=%2FUsers%2Fmaxroach%2F.postgresql%2Froot.crt
+~~~
+ccloud CLI",
+ "urls": [
+ "/cockroachcloud/ccloud-get-started.html"
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/connect-to-cockroachdb.json b/src/current/_includes/v25.1/sidebar-data/connect-to-cockroachdb.json
new file mode 100644
index 00000000000..640358304ad
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/connect-to-cockroachdb.json
@@ -0,0 +1,30 @@
+{
+ "title": "Connect to an Application",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Install a Driver or ORM Framework",
+ "urls": [
+ "/${VERSION}/install-client-drivers.html"
+ ]
+ },
+ {
+ "title": "Connect to a Cluster",
+ "urls": [
+ "/${VERSION}/connect-to-the-database.html"
+ ]
+ },
+ {
+ "title": "Client Connection Parameters",
+ "urls": [
+ "/${VERSION}/connection-parameters.html"
+ ]
+ },
+ {
+ "title": "Connection Pooling",
+ "urls": [
+ "/${VERSION}/connection-pooling.html"
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/sidebar-data/cross-cluster-replication.json b/src/current/_includes/v25.1/sidebar-data/cross-cluster-replication.json
new file mode 100644
index 00000000000..925e3e2aca9
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/cross-cluster-replication.json
@@ -0,0 +1,99 @@
+{
+ "title": "Cross-Cluster Replication",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Logical Data Replication",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/logical-data-replication-overview.html"
+ ]
+ },
+ {
+ "title": "Set Up Logical Data Replication",
+ "urls": [
+ "/${VERSION}/set-up-logical-data-replication.html"
+ ]
+ },
+ {
+ "title": "Manage Logical Data Replication",
+ "urls": [
+ "/${VERSION}/manage-logical-data-replication.html"
+ ]
+ },
+ {
+ "title": "Monitor Logical Data Replication",
+ "urls": [
+ "/${VERSION}/logical-data-replication-monitoring.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Physical Cluster Replication",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/physical-cluster-replication-overview.html"
+ ]
+ },
+ {
+ "title": "Set Up Physical Cluster Replication",
+ "urls": [
+ "/${VERSION}/set-up-physical-cluster-replication.html"
+ ]
+ },
+ {
+ "title": "Fail Over from a Primary to a Standby Cluster",
+ "urls": [
+ "/${VERSION}/failover-replication.html"
+ ]
+ },
+ {
+ "title": "Monitor a Replication Stream",
+ "urls": [
+ "/${VERSION}/physical-cluster-replication-monitoring.html"
+ ]
+ },
+ {
+ "title": "Technical Overview",
+ "urls": [
+ "/${VERSION}/physical-cluster-replication-technical-overview.html"
+ ]
+ },
+ {
+ "title": "Cluster Virtualization",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/cluster-virtualization-overview.html"
+ ]
+ },
+ {
+ "title": "Work with Virtual Clusters",
+ "urls": [
+ "/${VERSION}/work-with-virtual-clusters.html"
+ ]
+ },
+ {
+ "title": "Setting Scopes",
+ "urls": [
+ "/${VERSION}/cluster-virtualization-setting-scopes.html"
+ ]
+ },
+ {
+ "title": "Metric Scopes",
+ "urls": [
+ "/${VERSION}/cluster-virtualization-metric-scopes.html"
+ ]
+ }
+ ]
+ }
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/sidebar-data/faqs.json b/src/current/_includes/v25.1/sidebar-data/faqs.json
new file mode 100644
index 00000000000..01887487d4b
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/faqs.json
@@ -0,0 +1,24 @@
+{
+ "title": "FAQs",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "CockroachDB FAQs",
+ "urls": [
+ "/${VERSION}/frequently-asked-questions.html"
+ ]
+ },
+ {
+ "title": "Operational FAQs",
+ "urls": [
+ "/${VERSION}/operational-faqs.html"
+ ]
+ },
+ {
+ "title": "Licensing FAQs",
+ "urls": [
+ "/${VERSION}/licensing-faqs.html"
+ ]
+ }
+ ]
+ }
diff --git a/src/current/_includes/v25.1/sidebar-data/feature-overview.json b/src/current/_includes/v25.1/sidebar-data/feature-overview.json
new file mode 100644
index 00000000000..fc2d25a13af
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/feature-overview.json
@@ -0,0 +1,71 @@
+{
+ "title": "Feature Overview",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "CockroachDB Basics",
+ "items": [
+ {
+ "title": "Why CockroachDB?",
+ "urls": [
+ "/${VERSION}/why-cockroachdb.html"
+ ]
+ },
+ {
+ "title": "Replication & Rebalancing",
+ "urls": [
+ "/${VERSION}/demo-replication-and-rebalancing.html"
+ ]
+ },
+ {
+ "title": "CockroachDB Resilience",
+ "urls": [
+ "/${VERSION}/demo-cockroachdb-resilience.html"
+ ]
+ },
+ {
+ "title": "Serializable Transactions",
+ "urls": [
+ "/${VERSION}/demo-serializable.html"
+ ]
+ },
+ {
+ "title": "Multi-Active Availability",
+ "urls": [
+ "/${VERSION}/multi-active-availability.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "CockroachDB SQL",
+ "items": [
+ {
+ "title": "PostgreSQL Compatibility",
+ "urls": [
+ "/${VERSION}/postgresql-compatibility.html"
+ ]
+ },
+ {
+ "title": "SQL Features",
+ "urls": [
+ "/${VERSION}/sql-feature-support.html"
+ ]
+ },
+ {
+ "title": "SQL FAQs",
+ "urls": [
+ "/${VERSION}/sql-faqs.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "CockroachDB in Comparison",
+ "urls": [
+ "/${VERSION}/cockroachdb-in-comparison.html"
+ ]
+ }
+ ]
+}
+
diff --git a/src/current/_includes/v25.1/sidebar-data/get-started.json b/src/current/_includes/v25.1/sidebar-data/get-started.json
new file mode 100644
index 00000000000..4a48e9e48b5
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/get-started.json
@@ -0,0 +1,218 @@
+{
+ "title": "Get Started",
+ "is_top_level": true,
+ "items": [{
+ "title": "Quickstart",
+ "urls": [
+ "/cockroachcloud/quickstart.html"
+ ]
+ },
+ {
+ "title": "Install CockroachDB",
+ "urls": [
+ "/${VERSION}/install-cockroachdb.html",
+ "/${VERSION}/install-cockroachdb-mac.html",
+ "/${VERSION}/install-cockroachdb-linux.html",
+ "/${VERSION}/fips.html",
+ "/${VERSION}/install-cockroachdb-windows.html"
+ ]
+ },
+ {
+ "title": "Learn CockroachDB SQL",
+ "urls": [
+ "/cockroachcloud/learn-cockroachdb-sql.html",
+ "/${VERSION}/learn-cockroachdb-sql.html"
+ ]
+ },
+ {
+ "title": "Develop with CockroachDB",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/developer-guide-overview.html"
+ ]
+ },
+ {
+ "title": "Developer Basics",
+ "urls": [
+ "/${VERSION}/developer-basics.html"
+ ]
+ },
+ {
+ "title": "JavaScript/TypeScript",
+ "urls": [
+ "/${VERSION}/build-a-nodejs-app-with-cockroachdb.html",
+ "/${VERSION}/build-a-nodejs-app-with-cockroachdb-sequelize.html",
+ "/${VERSION}/build-a-nodejs-app-with-cockroachdb-knexjs.html",
+ "/${VERSION}/build-a-nodejs-app-with-cockroachdb-prisma.html",
+ "/${VERSION}/build-a-typescript-app-with-cockroachdb.html"
+ ]
+ },
+ {
+ "title": "Python",
+ "urls": [
+ "/${VERSION}/build-a-python-app-with-cockroachdb-psycopg3.html",
+ "/${VERSION}/build-a-python-app-with-cockroachdb.html",
+ "/${VERSION}/build-a-python-app-with-cockroachdb-sqlalchemy.html",
+ "/${VERSION}/build-a-python-app-with-cockroachdb-django.html",
+ "/${VERSION}/build-a-python-app-with-cockroachdb-asyncpg.html"
+ ]
+ },
+ {
+ "title": "Golang",
+ "urls": [
+ "/${VERSION}/build-a-go-app-with-cockroachdb.html",
+ "/${VERSION}/build-a-go-app-with-cockroachdb-gorm.html",
+ "/${VERSION}/build-a-go-app-with-cockroachdb-pq.html",
+ "/${VERSION}/build-a-go-app-with-cockroachdb-upperdb.html"
+ ]
+ },
+ {
+ "title": "Java",
+ "urls": [
+ "/${VERSION}/build-a-java-app-with-cockroachdb.html",
+ "/${VERSION}/build-a-java-app-with-cockroachdb-hibernate.html",
+ "/${VERSION}/build-a-java-app-with-cockroachdb-jooq.html",
+ "/${VERSION}/build-a-spring-app-with-cockroachdb-mybatis.html"
+ ]
+ },
+ {
+ "title": "Ruby",
+ "urls": [
+ "/${VERSION}/build-a-ruby-app-with-cockroachdb.html",
+ "/${VERSION}/build-a-ruby-app-with-cockroachdb-activerecord.html"
+ ]
+ },
+ {
+ "title": "C# (.NET)",
+ "urls": [
+ "/${VERSION}/build-a-csharp-app-with-cockroachdb.html"
+ ]
+ },
+ {
+ "title": "Rust",
+ "urls": [
+ "/${VERSION}/build-a-rust-app-with-cockroachdb.html"
+ ]
+ },
+ {
+ "title": "Hasura (GraphQL)",
+ "urls": [
+ "/${VERSION}/hasura-getting-started.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Standard Examples",
+ "items": [
+ {
+ "title": "AWS Lambda",
+ "urls": [
+ "/${VERSION}/deploy-lambda-function.html"
+ ]
+ },
+ {
+ "title": "Google Cloud Run",
+ "urls": [
+ "/${VERSION}/deploy-app-gcr.html"
+ ]
+ },
+ {
+ "title": "Netlify",
+ "urls": [
+ "/${VERSION}/deploy-app-netlify.html"
+ ]
+ },
+ {
+ "title": "Vercel",
+ "urls": [
+ "/${VERSION}/deploy-app-vercel.html"
+ ]
+ },
+ {
+ "title": "Serverless Function Best Practices",
+ "urls": [
+ "/${VERSION}/serverless-function-best-practices.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Advanced Example Applications",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/example-apps.html"
+ ]
+ },
+ {
+ "title": "Spring Boot",
+ "items": [
+ {
+ "title": "Spring Boot with JDBC",
+ "urls": [
+ "/${VERSION}/build-a-spring-app-with-cockroachdb-jdbc.html"
+ ]
+ },
+ {
+ "title": "Spring Boot with JPA",
+ "urls": [
+ "/${VERSION}/build-a-spring-app-with-cockroachdb-jpa.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "MovR",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/movr.html"
+ ]
+ },
+ {
+ "title": "Global Application Use Case",
+ "urls": [
+ "/${VERSION}/movr-flask-use-case.html"
+ ]
+ },
+ {
+ "title": "Multi-region Database Schema",
+ "urls": [
+ "/${VERSION}/movr-flask-database.html"
+ ]
+ },
+ {
+ "title": "Set up a Development Environment",
+ "urls": [
+ "/${VERSION}/movr-flask-setup.html"
+ ]
+ },
+ {
+ "title": "Develop a Global Application",
+ "urls": [
+ "/${VERSION}/movr-flask-application.html"
+ ]
+ },
+ {
+ "title": "Deploy a Global Application",
+ "urls": [
+ "/${VERSION}/movr-flask-deployment.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Deploy a Python To-Do App with Flask, Kubernetes, and CockroachDB Cloud",
+ "urls": [
+ "/cockroachcloud/deploy-a-python-to-do-app-with-flask-kubernetes-and-cockroachcloud.html"
+ ]
+ }
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/latest-releases.json b/src/current/_includes/v25.1/sidebar-data/latest-releases.json
new file mode 100644
index 00000000000..e25f8c0fa7a
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/latest-releases.json
@@ -0,0 +1,7 @@
+{
+ "title": "Latest Releases",
+ "is_top_level": true,
+ "items": [
+ {% include_cached sidebar-latest-releases.json %}
+ ]
+ }
diff --git a/src/current/_includes/v25.1/sidebar-data/migrate.json b/src/current/_includes/v25.1/sidebar-data/migrate.json
new file mode 100644
index 00000000000..48e1670082a
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/migrate.json
@@ -0,0 +1,147 @@
+{
+ "title": "Migrate",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/migration-overview.html"
+ ]
+ },
+ {
+ "title": "MOLT Tools",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/molt/molt-overview.html"
+ ]
+ },
+ {
+ "title": "Schema Conversion Tool",
+ "urls": [
+ "/cockroachcloud/migrations-page.html"
+ ]
+ },
+ {
+ "title": "Fetch",
+ "urls": [
+ "/molt/molt-fetch.html"
+ ]
+ },
+ {
+ "title": "Verify",
+ "urls": [
+ "/molt/molt-verify.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Third-Party Migration Tools",
+ "items": [
+ {
+ "title": "AWS DMS",
+ "urls": [
+ "/${VERSION}/aws-dms.html"
+ ]
+ },
+ {
+ "title": "Qlik Replicate",
+ "urls": [
+ "/${VERSION}/qlik.html"
+ ]
+ },
+ {
+ "title": "Striim",
+ "urls": [
+ "/${VERSION}/striim.html"
+ ]
+ },
+ {
+ "title": "Oracle GoldenGate",
+ "urls": [
+ "/${VERSION}/goldengate.html"
+ ]
+ },
+ {
+ "title": "Debezium",
+ "urls": [
+ "/${VERSION}/debezium.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Migrate Data Types",
+ "items": [
+ {
+ "title": "Migrate from CSV",
+ "urls": [
+ "/${VERSION}/migrate-from-csv.html"
+ ]
+ },
+ {
+ "title": "Migrate from Avro",
+ "urls": [
+ "/${VERSION}/migrate-from-avro.html"
+ ]
+ },
+ {
+ "title": "Migrate from Shapefiles",
+ "urls": [
+ "/${VERSION}/migrate-from-shapefiles.html"
+ ]
+ },
+ {
+ "title": "Migrate from OpenStreetMap",
+ "urls": [
+ "/${VERSION}/migrate-from-openstreetmap.html"
+ ]
+ },
+ {
+ "title": "Migrate from GeoJSON",
+ "urls": [
+ "/${VERSION}/migrate-from-geojson.html"
+ ]
+ },
+ {
+ "title": "Migrate from GeoPackage",
+ "urls": [
+ "/${VERSION}/migrate-from-geopackage.html"
+ ]
+ },
+ {
+ "title": "Import Performance Best Practices",
+ "urls": [
+ "/${VERSION}/import-performance-best-practices.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Migrate from PostgreSQL",
+ "urls": [
+ "/${VERSION}/migrate-from-postgres.html"
+ ]
+ },
+ {
+ "title": "Migrate from MySQL",
+ "urls": [
+ "/${VERSION}/migrate-from-mysql.html"
+ ]
+ },
+ {
+ "title": "Migrate from Oracle",
+ "urls": [
+ "/${VERSION}/migrate-from-oracle.html"
+ ]
+ },
+ {
+ "title": "Migration Strategy: Lift and Shift",
+ "urls": [
+ "/${VERSION}/migration-strategy-lift-and-shift.html"
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/multi-region-capabilities.json b/src/current/_includes/v25.1/sidebar-data/multi-region-capabilities.json
new file mode 100644
index 00000000000..6ba988ae925
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/multi-region-capabilities.json
@@ -0,0 +1,76 @@
+{
+ "title": "Multi-Region Capabilities",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/multiregion-overview.html"
+ ]
+ },
+ {
+ "title": "Survival Goals",
+ "urls": [
+ "/${VERSION}/multiregion-survival-goals.html"
+ ]
+ },
+ {
+ "title": "Data Domiciling and Performance",
+ "items": [
+ {
+ "title": "Table Localities",
+ "urls": [
+ "/${VERSION}/table-localities.html"
+ ]
+ },
+ {
+ "title": "Table Partitioning",
+ "urls": [
+ "/${VERSION}/partitioning.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "How to Choose a Multi-Region Configuration",
+ "urls": [
+ "/${VERSION}/choosing-a-multi-region-configuration.html"
+ ]
+ },
+ {
+ "title": "Multi-Region Zone Config Extensions",
+ "urls": [
+ "/${VERSION}/zone-config-extensions.html"
+ ]
+ },
+ {
+ "title": "Multi-Region Tutorials",
+ "items": [
+ {
+ "title": "Low Latency Reads and Writes",
+ "urls": [
+ "/${VERSION}/demo-low-latency-multi-region-deployment.html"
+ ]
+ },
+ {
+ "title": "Data Domiciling with CockroachDB",
+ "urls": [
+ "/${VERSION}/data-domiciling.html"
+ ]
+ },
+ {
+ "title": "Migrate to Multi-Region SQL with Replication Zones",
+ "urls": [
+ "/${VERSION}/migrate-to-multiregion-sql.html"
+ ]
+ },
+ {
+ "title": "Using GeoServer with CockroachDB",
+ "urls": [
+ "/${VERSION}/geoserver.html"
+ ]
+ }
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/optimize-performance.json b/src/current/_includes/v25.1/sidebar-data/optimize-performance.json
new file mode 100644
index 00000000000..2d61b0dd507
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/optimize-performance.json
@@ -0,0 +1,90 @@
+{
+ "title": "Optimize Performance",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/make-queries-fast.html"
+ ]
+ },
+ {
+ "title": "SQL Performance Best Practices",
+ "urls": [
+ "/${VERSION}/performance-best-practices-overview.html"
+ ]
+ },
+
+ {
+ "title": "Indexes",
+ "urls": [
+ "/${VERSION}/indexes.html"
+ ]
+ },
+ {
+ "title": "Map SQL Activity using an Application Name",
+ "urls": [
+ "/${VERSION}/map-sql-activity-to-app.html"
+ ]
+ },
+ {
+ "title": "Cost-Based Optimizer",
+ "urls": [
+ "/${VERSION}/cost-based-optimizer.html"
+ ]
+ },
+ {
+ "title": "Vectorized Execution Engine",
+ "urls": [
+ "/${VERSION}/vectorized-execution.html"
+ ]
+ },
+ {
+ "title": "Load-Based Splitting",
+ "urls": [
+ "/${VERSION}/load-based-splitting.html"
+ ]
+ },
+ {
+ "title": "Replication Controls",
+ "urls": [
+ "/${VERSION}/configure-replication-zones.html"
+ ]
+ },
+ {
+ "title": "Admission Control",
+ "urls": [
+ "/${VERSION}/admission-control.html"
+ ]
+ },
+ {
+ "title": "Performance Tuning Recipes",
+ "urls": [
+ "/${VERSION}/performance-recipes.html"
+ ]
+ },
+ {
+ "title": "Performance Tuning Tutorials",
+ "items": [
+ {
+ "title": "Statement Tuning with EXPLAIN",
+ "urls": [
+ "/${VERSION}/sql-tuning-with-explain.html"
+ ]
+ },
+ {
+ "title": "Apply SQL Statement Performance Rules",
+ "urls": [
+ "/${VERSION}/apply-statement-performance-rules.html"
+ ]
+ },
+ {
+ "title": "Troubleshoot Lock Contention",
+ "urls": [
+ "/${VERSION}/troubleshoot-lock-contention.html"
+ ]
+ }
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/sidebar-data/reads-and-writes.json b/src/current/_includes/v25.1/sidebar-data/reads-and-writes.json
new file mode 100644
index 00000000000..5dd310e5f1d
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/reads-and-writes.json
@@ -0,0 +1,189 @@
+{
+ "title": "Reads and Writes",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/architecture/reads-and-writes-overview.html"
+ ]
+ },
+ {
+ "title": "Read Data",
+ "items": [
+ {
+ "title": "Query Data",
+ "urls": [
+ "/${VERSION}/query-data.html"
+ ]
+ },
+ {
+ "title": "Reusable Views",
+ "urls": [
+ "/${VERSION}/views.html"
+ ]
+ },
+ {
+ "title": "Subqueries",
+ "urls": [
+ "/${VERSION}/subqueries.html"
+ ]
+ },
+ {
+ "title": "Temporary Tables",
+ "urls": [
+ "/${VERSION}/temporary-tables.html"
+ ]
+ },
+ {
+ "title": "Paginate Results",
+ "urls": [
+ "/${VERSION}/pagination.html"
+ ]
+ },
+ {
+ "title": "Follower Reads",
+ "urls": [
+ "/${VERSION}/follower-reads.html"
+ ]
+ },
+ {
+ "title": "AS OF SYSTEM TIME",
+ "urls": [
+ "/${VERSION}/as-of-system-time.html"
+ ]
+ },
+ {
+ "title": "Query Spatial Data",
+ "urls": [
+ "/${VERSION}/query-spatial-data.html"
+ ]
+ },
+ {
+ "title": "Export Spatial Data",
+ "urls": [
+ "/${VERSION}/export-spatial-data.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Write Data",
+ "items": [
+ {
+ "title": "Insert Data",
+ "urls": [
+ "/${VERSION}/insert-data.html"
+ ]
+ },
+ {
+ "title": "Update Data",
+ "urls": [
+ "/${VERSION}/update-data.html"
+ ]
+ },
+ {
+ "title": "Bulk-update Data",
+ "urls": [
+ "/${VERSION}/bulk-update-data.html"
+ ]
+ },
+ {
+ "title": "Delete Data",
+ "urls": [
+ "/${VERSION}/delete-data.html"
+ ]
+ },
+ {
+ "title": "Bulk-delete Data",
+ "urls": [
+ "/${VERSION}/bulk-delete-data.html"
+ ]
+ },
+ {
+ "title": "Batch Delete Expired Data with Row-Level TTL",
+ "urls": [
+ "/${VERSION}/row-level-ttl.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Transactions",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/transactions.html"
+ ]
+ },
+ {
+ "title": "Advanced Client-side Transaction Retries",
+ "urls": [
+ "/${VERSION}/advanced-client-side-transaction-retries.html"
+ ]
+ },
+ {
+ "title": "Life of a Distributed Transaction",
+ "urls": [
+ "/${VERSION}/architecture/life-of-a-distributed-transaction.html"
+ ]
+ },
+ {
+ "title": "Read Committed Transactions",
+ "urls": [
+ "/${VERSION}/read-committed.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Test Your Application Locally",
+ "urls": [
+ "/${VERSION}/local-testing.html"
+ ]
+ },
+ {
+ "title": "JSON Support",
+ "urls": [
+ "/${VERSION}/demo-json-support.html"
+ ]
+ },
+ {
+ "title": "Spatial Data",
+ "urls": [
+ "/${VERSION}/spatial-tutorial.html"
+ ]
+ },
+ {
+ "title": "Cross-Cloud Migration",
+ "urls": [
+ "/${VERSION}/demo-automatic-cloud-migration.html"
+ ]
+ },
+ {
+ "title": "SQL Playground",
+ "is_top_level": true,
+ "urls": [
+ "https://www.cockroachlabs.com/docs/tutorials/sql-playground"
+ ]
+ },
+ {
+ "title": "Database Management Tools",
+ "items": [
+ {
+ "title": "DBeaver GUI",
+ "urls": [
+ "/${VERSION}/dbeaver.html"
+ ]
+ },
+ {
+ "title": "IntelliJ IDEA",
+ "urls": [
+ "/${VERSION}/intellij-idea.html"
+ ]
+ }
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/reference.json b/src/current/_includes/v25.1/sidebar-data/reference.json
new file mode 100644
index 00000000000..4691b0f8bbd
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/reference.json
@@ -0,0 +1,429 @@
+{
+ "title": "Reference",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Glossary",
+ "urls": [
+ "/${VERSION}/architecture/glossary.html"
+ ]
+ },
+ {
+ "title": "Architecture",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/architecture/overview.html"
+ ]
+ },
+ {
+ "title": "SQL Layer",
+ "urls": [
+ "/${VERSION}/architecture/sql-layer.html"
+ ]
+ },
+ {
+ "title": "Transaction Layer",
+ "urls": [
+ "/${VERSION}/architecture/transaction-layer.html"
+ ]
+ },
+ {
+ "title": "Distribution Layer",
+ "urls": [
+ "/${VERSION}/architecture/distribution-layer.html"
+ ]
+ },
+ {
+ "title": "Replication Layer",
+ "urls": [
+ "/${VERSION}/architecture/replication-layer.html"
+ ]
+ },
+ {
+ "title": "Storage Layer",
+ "urls": [
+ "/${VERSION}/architecture/storage-layer.html"
+ ]
+ },
+ {
+ "title": "Backups",
+ "urls": [
+ "/${VERSION}/backup-architecture.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Cloud API",
+ "urls": [
+ "https://www.cockroachlabs.com/docs/api/cloud/v1"
+ ]
+ },
+ {
+ "title": "Cluster API",
+ "urls": [
+ "https://www.cockroachlabs.com/docs/api/cluster/v2"
+ ]
+ },
+ {
+ "title": "Cluster Settings",
+ "urls": [
+ "/${VERSION}/cluster-settings.html"
+ ]
+ },
+ {
+ "title": "Cockroach Commands",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/cockroach-commands.html"
+ ]
+ },
+ {
+ "title": "cockroach start",
+ "urls": [
+ "/${VERSION}/cockroach-start.html"
+ ]
+ },
+ {
+ "title": "cockroach init",
+ "urls": [
+ "/${VERSION}/cockroach-init.html"
+ ]
+ },
+ {
+ "title": "cockroach start-single-node",
+ "urls": [
+ "/${VERSION}/cockroach-start-single-node.html"
+ ]
+ },
+ {
+ "title": "cockroach cert",
+ "urls": [
+ "/${VERSION}/cockroach-cert.html"
+ ]
+ },
+ {
+ "title": "cockroach sql",
+ "urls": [
+ "/${VERSION}/cockroach-sql.html"
+ ]
+ },
+ {
+ "title": "cockroach sqlfmt",
+ "urls": [
+ "/${VERSION}/cockroach-sqlfmt.html"
+ ]
+ },
+ {
+ "title": "cockroach node",
+ "urls": [
+ "/${VERSION}/cockroach-node.html"
+ ]
+ },
+ {
+ "title": "cockroach nodelocal upload",
+ "urls": [
+ "/${VERSION}/cockroach-nodelocal-upload.html"
+ ]
+ },
+ {
+ "title": "cockroach auth-session",
+ "urls": [
+ "/${VERSION}/cockroach-auth-session.html"
+ ]
+ },
+ {
+ "title": "cockroach demo",
+ "urls": [
+ "/${VERSION}/cockroach-demo.html"
+ ]
+ },
+ {
+ "title": "cockroach debug ballast",
+ "urls": [
+ "/${VERSION}/cockroach-debug-ballast.html"
+ ]
+ },
+ {
+ "title": "cockroach debug encryption-active-key",
+ "urls": [
+ "/${VERSION}/cockroach-debug-encryption-active-key.html"
+ ]
+ },
+ {
+ "title": "cockroach debug encryption-decrypt",
+ "urls": [
+ "/${VERSION}/cockroach-debug-encryption-decrypt.html"
+ ]
+ },
+ {
+ "title": "cockroach debug job-trace",
+ "urls": [
+ "/${VERSION}/cockroach-debug-job-trace.html"
+ ]
+ },
+ {
+ "title": "cockroach debug list-files",
+ "urls": [
+ "/${VERSION}/cockroach-debug-list-files.html"
+ ]
+ },
+ {
+ "title": "cockroach debug merge-logs",
+ "urls": [
+ "/${VERSION}/cockroach-debug-merge-logs.html"
+ ]
+ },
+ {
+ "title": "cockroach debug tsdump",
+ "urls": [
+ "/${VERSION}/cockroach-debug-tsdump.html"
+ ]
+ },
+ {
+ "title": "cockroach debug zip",
+ "urls": [
+ "/${VERSION}/cockroach-debug-zip.html"
+ ]
+ },
+ {
+ "title": "cockroach statement-diag",
+ "urls": [
+ "/${VERSION}/cockroach-statement-diag.html"
+ ]
+ },
+ {
+ "title": "cockroach gen",
+ "urls": [
+ "/${VERSION}/cockroach-gen.html"
+ ]
+ },
+ {
+ "title": "cockroach userfile upload",
+ "urls": [
+ "/${VERSION}/cockroach-userfile-upload.html"
+ ]
+ },
+ {
+ "title": "cockroach userfile list",
+ "urls": [
+ "/${VERSION}/cockroach-userfile-list.html"
+ ]
+ },
+ {
+ "title": "cockroach userfile get",
+ "urls": [
+ "/${VERSION}/cockroach-userfile-get.html"
+ ]
+ },
+ {
+ "title": "cockroach userfile delete",
+ "urls": [
+ "/${VERSION}/cockroach-userfile-delete.html"
+ ]
+ },
+ {
+ "title": "cockroach version",
+ "urls": [
+ "/${VERSION}/cockroach-version.html"
+ ]
+ },
+ {
+ "title": "cockroach workload",
+ "urls": [
+ "/${VERSION}/cockroach-workload.html"
+ ]
+ },
+ {
+ "title": "The cockroach-sql command",
+ "urls": [
+ "/${VERSION}/cockroach-sql-binary.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Diagnostics Reporting",
+ "urls": [
+ "/${VERSION}/diagnostics-reporting.html"
+ ]
+ },
+ {
+ "title": "Logs",
+ "items": [
+ {
+ "title": "Logging Levels and Channels",
+ "urls": [
+ "/${VERSION}/logging.html"
+ ]
+ },
+ {
+ "title": "Log Formats",
+ "urls": [
+ "/${VERSION}/log-formats.html"
+ ]
+ },
+ {
+ "title": "Notable Event Types",
+ "urls": [
+ "/${VERSION}/eventlog.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Metrics",
+ "items": [
+ {
+ "title": "Available Metrics",
+ "urls": [
+ "/${VERSION}/metrics.html"
+ ]
+ },
+ {
+ "title": "Child Metrics",
+ "urls": [
+ "/${VERSION}/child-metrics.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Policies",
+ "items": [
+ {
+ "title": "CockroachDB Feature Availability",
+ "urls": [
+ "/${VERSION}/cockroachdb-feature-availability.html"
+ ]
+ },
+ {
+ "title": "API Support Policy",
+ "urls": [
+ "/${VERSION}/api-support-policy.html"
+ ]
+ },
+ {
+ "title": "Telemetry Collected by CockroachDB",
+ "urls": [
+ "/${VERSION}/telemetry.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Third-Party Support",
+ "items": [
+ {
+ "title": "Supported Tools",
+ "urls": [
+ "/${VERSION}/third-party-database-tools.html"
+ ]
+ },
+ {
+ "title": "Monitoring Integrations",
+ "urls": [
+ "/${VERSION}/third-party-monitoring-tools.html"
+ ]
+ },
+ {
+ "title": "Community-supported Tools",
+ "urls": [
+ "/${VERSION}/community-tooling.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Security",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/security-reference/security-overview.html"
+ ]
+ },
+ {
+ "title": "Authentication",
+ "items": [
+ {
+ "title": "SQL Authentication",
+ "urls": [
+ "/${VERSION}/security-reference/authentication.html"
+ ]
+ },
+ {
+ "title": "SASL/SCRAM-SHA-256 Secure Password-based Authentication",
+ "urls": [
+ "/${VERSION}/security-reference/scram-authentication.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Authorization",
+ "urls": [
+ "/${VERSION}/security-reference/authorization.html"
+ ]
+ },
+ {
+ "title": "Encryption",
+ "urls": [
+ "/${VERSION}/security-reference/encryption.html"
+ ]
+ },
+ {
+ "title": "Column Level Encryption",
+ "urls": [
+ "/${VERSION}/column-level-encryption.html"
+ ]
+ },
+ {
+ "title": "PKI and TLS",
+ "urls": [
+ "/${VERSION}/security-reference/transport-layer-security.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "System Catalogs",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/system-catalogs.html"
+ ]
+ },
+ {
+ "title": "crdb_internal",
+ "urls": [
+ "/${VERSION}/crdb-internal.html"
+ ]
+ },
+ {
+ "title": "information_schema",
+ "urls": [
+ "/${VERSION}/information-schema.html"
+ ]
+ },
+ {
+ "title": "pg_catalog",
+ "urls": [
+ "/${VERSION}/pg-catalog.html"
+ ]
+ },
+ {
+ "title": "pg_extension",
+ "urls": [
+ "/${VERSION}/pg-extension.html"
+ ]
+ }
+ ]
+ }
+ ]
+ }
diff --git a/src/current/_includes/v25.1/sidebar-data/releases.json b/src/current/_includes/v25.1/sidebar-data/releases.json
new file mode 100644
index 00000000000..fb49f7c9acc
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/releases.json
@@ -0,0 +1,7 @@
+{
+ "title": "CockroachDB Releases",
+ "is_top_level": true,
+ "items": [
+ {% include_cached sidebar-all-releases.json %}
+ ]
+ }
diff --git a/src/current/_includes/v25.1/sidebar-data/resilience.json b/src/current/_includes/v25.1/sidebar-data/resilience.json
new file mode 100644
index 00000000000..df6d97fbd9b
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/resilience.json
@@ -0,0 +1,18 @@
+{
+ "title": "Data Resilience",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/data-resilience.html"
+ ]
+ },
+ {
+ "title": "Disaster Recovery",
+ "urls": [
+ "/${VERSION}/disaster-recovery-overview.html"
+ ]
+ }
+ ]
+}
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/sidebar-data/schema-design.json b/src/current/_includes/v25.1/sidebar-data/schema-design.json
new file mode 100644
index 00000000000..6af88390189
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/schema-design.json
@@ -0,0 +1,135 @@
+{
+ "title": "Schema Design",
+ "items": [
+ {
+ "title": "Database Schemas",
+ "urls": [
+ "/${VERSION}/schema-design-overview.html"
+ ]
+ },
+ {
+ "title": "Create a Database",
+ "urls": [
+ "/${VERSION}/schema-design-database.html"
+ ]
+ },
+ {
+ "title": "Create a User-defined Schema",
+ "urls": [
+ "/${VERSION}/schema-design-schema.html"
+ ]
+ },
+ {
+ "title": "Create a Table",
+ "urls": [
+ "/${VERSION}/schema-design-table.html"
+ ]
+ },
+ {
+ "title": "Computed Columns",
+ "urls": [
+ "/${VERSION}/computed-columns.html"
+ ]
+ },
+ {
+ "title": "Column Families",
+ "urls": [
+ "/${VERSION}/column-families.html"
+ ]
+ },
+ {
+ "title": "Scale to Multiple Regions",
+ "urls": [
+ "/${VERSION}/multiregion-scale-application.html"
+ ]
+ },
+ {
+ "title": "Indexes",
+ "items": [
+ {
+ "title": "Secondary Indexes",
+ "urls": [
+ "/${VERSION}/schema-design-indexes.html"
+ ]
+ },
+ {
+ "title": "Partial Indexes",
+ "urls": [
+ "/${VERSION}/partial-indexes.html"
+ ]
+ },
+ {
+ "title": "Hash-sharded Indexes",
+ "urls": [
+ "/${VERSION}/hash-sharded-indexes.html"
+ ]
+ },
+ {
+ "title": "Generalized Inverted Indexes",
+ "urls": [
+ "/${VERSION}/inverted-indexes.html"
+ ]
+ },
+ {
+ "title": "Full-Text Search",
+ "urls": [
+ "/${VERSION}/full-text-search.html"
+ ]
+ },
+ {
+ "title": "Trigram Indexes",
+ "urls": [
+ "/${VERSION}/trigram-indexes.html"
+ ]
+ },
+ {
+ "title": "Expression Indexes",
+ "urls": [
+ "/${VERSION}/expression-indexes.html"
+ ]
+ },
+ {
+ "title": "Spatial Indexes",
+ "urls": [
+ "/${VERSION}/spatial-indexes.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Update a Database Schema",
+ "items": [
+ {
+ "title": "Change and Remove Objects",
+ "urls": [
+ "/${VERSION}/schema-design-update.html"
+ ]
+ },
+ {
+ "title": "Online Schema Changes",
+ "urls": [
+ "/${VERSION}/online-schema-changes.html"
+ ]
+ },
+ {
+ "title": "Use Alembic",
+ "urls": [
+ "/${VERSION}/alembic.html"
+ ]
+ },
+ {
+ "title": "Use Flyway",
+ "urls": [
+ "/${VERSION}/flyway.html"
+ ]
+ },
+ {
+ "title": "Use Liquibase",
+ "urls": [
+ "/${VERSION}/liquibase.html"
+ ]
+ }
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/self-hosted-deployments.json b/src/current/_includes/v25.1/sidebar-data/self-hosted-deployments.json
new file mode 100644
index 00000000000..76608d8939a
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/self-hosted-deployments.json
@@ -0,0 +1,757 @@
+{
+ "title": "Self-Hosted Deployments",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Production Checklist",
+ "urls": [
+ "/${VERSION}/recommended-production-settings.html"
+ ]
+ },
+ {
+ "title": "Deploy Locally",
+ "items": [
+ {
+ "title": "Deploy from Binary",
+ "urls": [
+ "/${VERSION}/secure-a-cluster.html",
+ "/${VERSION}/start-a-local-cluster.html"
+ ]
+ },
+ {
+ "title": "Deploy in Kubernetes",
+ "urls": [
+ "/${VERSION}/orchestrate-a-local-cluster-with-kubernetes.html",
+ "/${VERSION}/orchestrate-a-local-cluster-with-kubernetes-insecure.html"
+ ]
+ },
+ {
+ "title": "Deploy in Docker",
+ "urls": [
+ "/${VERSION}/start-a-local-cluster-in-docker-mac.html",
+ "/${VERSION}/start-a-local-cluster-in-docker-linux.html",
+ "/${VERSION}/start-a-local-cluster-in-docker-windows.html"
+ ]
+ },
+ {
+ "title": "Simulate a Multi-Region Cluster Locally",
+ "urls": [
+ "/${VERSION}/simulate-a-multi-region-cluster-on-localhost.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Deploy Manually",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/manual-deployment.html"
+ ]
+ },
+ {
+ "title": "Deploy On-Premises",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-on-premises.html",
+ "/${VERSION}/deploy-cockroachdb-on-premises-insecure.html"
+ ]
+ },
+ {
+ "title": "Deploy on AWS",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-on-aws.html",
+ "/${VERSION}/deploy-cockroachdb-on-aws-insecure.html"
+ ]
+ },
+ {
+ "title": "Deploy on Azure",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-on-microsoft-azure.html",
+ "/${VERSION}/deploy-cockroachdb-on-microsoft-azure-insecure.html"
+ ]
+ },
+ {
+ "title": "Deploy on Google Cloud Platform GCE",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-on-google-cloud-platform.html",
+ "/${VERSION}/deploy-cockroachdb-on-google-cloud-platform-insecure.html"
+ ]
+ },
+ {
+ "title": "Deploy on Digital Ocean",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-on-digital-ocean.html",
+ "/${VERSION}/deploy-cockroachdb-on-digital-ocean-insecure.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Deploy on Kubernetes",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/kubernetes-overview.html"
+ ]
+ },
+ {
+ "title": "Single-Cluster Deployment",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-with-kubernetes.html",
+ "/${VERSION}/deploy-cockroachdb-with-kubernetes-insecure.html"
+ ]
+ },
+ {
+ "title": "OpenShift Deployment",
+ "urls": [
+ "/${VERSION}/deploy-cockroachdb-with-kubernetes-openshift.html"
+ ]
+ },
+ {
+ "title": "Multi-Cluster Deployment",
+ "urls": [
+ "/${VERSION}/orchestrate-cockroachdb-with-kubernetes-multi-cluster.html"
+ ]
+ },
+ {
+ "title": "Operate on Kubernetes",
+ "items": [
+ {
+ "title": "Pod Scheduling",
+ "urls": [
+ "/${VERSION}/schedule-cockroachdb-kubernetes.html"
+ ]
+ },
+ {
+ "title": "Resource Management",
+ "urls": [
+ "/${VERSION}/configure-cockroachdb-kubernetes.html"
+ ]
+ },
+ {
+ "title": "Certificate Management",
+ "urls": [
+ "/${VERSION}/secure-cockroachdb-kubernetes.html"
+ ]
+ },
+ {
+ "title": "Cluster Scaling",
+ "urls": [
+ "/${VERSION}/scale-cockroachdb-kubernetes.html"
+ ]
+ },
+ {
+ "title": "Cluster Monitoring",
+ "urls": [
+ "/${VERSION}/monitor-cockroachdb-kubernetes.html"
+ ]
+ },
+ {
+ "title": "Cluster Upgrades",
+ "urls": [
+ "/${VERSION}/upgrade-cockroachdb-kubernetes.html"
+ ]
+ },
+ {
+ "title": "Kubernetes Performance",
+ "urls": [
+ "/${VERSION}/kubernetes-performance.html"
+ ]
+ }
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Multi-Region for Self-Hosted Deployments",
+ "items": [
+ {
+ "title": "Topology Patterns Overview",
+ "urls": [
+ "/${VERSION}/topology-patterns.html"
+ ]
+ },
+ {
+ "title": "Development Topology",
+ "urls": [
+ "/${VERSION}/topology-development.html"
+ ]
+ },
+ {
+ "title": "Basic Production Topology",
+ "urls": [
+ "/${VERSION}/topology-basic-production.html"
+ ]
+ },
+ {
+ "title": "Follower Reads Topology",
+ "urls": [
+ "/${VERSION}/topology-follower-reads.html"
+ ]
+ },
+ {
+ "title": "Follow-the-Workload Topology",
+ "urls": [
+ "/${VERSION}/topology-follow-the-workload.html"
+ ]
+ },
+ {
+ "title": "Regional Tables",
+ "urls": [
+ "/${VERSION}/regional-tables.html"
+ ]
+ },
+ {
+ "title": "Global Tables",
+ "urls": [
+ "/${VERSION}/global-tables.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Security",
+ "items": [
+ {
+ "title": "Certificates",
+ "items": [
+ {
+ "title": "Use the CockroachDB CLI to provision a development cluster",
+ "urls": [
+ "/${VERSION}/manage-certs-cli.html"
+ ]
+ },
+ {
+ "title": "Manage PKI certificates with HashiCorp Vault",
+ "urls": [
+ "/${VERSION}/manage-certs-vault.html"
+ ]
+ },
+ {
+ "title": "Create Security Certificates using OpenSSL",
+ "urls": [
+ "/${VERSION}/create-security-certificates-openssl.html"
+ ]
+ },
+ {
+ "title": "Use Online Certificate Status Protocol (OCSP)",
+ "urls": [
+ "/${VERSION}/manage-certs-revoke-ocsp.html"
+ ]
+ },
+ {
+ "title": "Certificate-based authentication using multiple values from the X.509 SUBJECT field",
+ "urls": [
+ "/${VERSION}/certificate-based-authentication-using-the-x509-subject-field.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Authentication",
+ "urls": [
+ "/${VERSION}/authentication.html"
+ ]
+ },
+ {
+ "title": "Encryption",
+ "urls": [
+ "/${VERSION}/encryption.html"
+ ]
+ },
+ {
+ "title": "Authorization",
+ "urls": [
+ "/${VERSION}/authorization.html"
+ ]
+ },
+ {
+ "title": "Table-based SQL Audit Logging",
+ "urls": [
+ "/${VERSION}/sql-audit-logging.html"
+ ]
+ },
+ {
+ "title": "Role-based SQL Audit Logging",
+ "urls": [
+ "/${VERSION}/role-based-audit-logging.html"
+ ]
+ },
+ {
+ "title": "LDAP Authentication",
+ "urls": [
+ "/${VERSION}/ldap-authentication.html"
+ ]
+ },
+ {
+ "title": "LDAP Authorization",
+ "urls": [
+ "/${VERSION}/ldap-authorization.html"
+ ]
+ },
+ {
+ "title": "GSSAPI Authentication",
+ "urls": [
+ "/${VERSION}/gssapi_authentication.html"
+ ]
+ },
+ {
+ "title": "Cluster SSO using JWT",
+ "urls": [
+ "/${VERSION}/sso-sql.html"
+ ]
+ },
+ {
+ "title": "SSO for DB Console",
+ "urls": [
+ "/${VERSION}/sso-db-console.html"
+ ]
+ },
+ {
+ "title": "Rotate Security Certificates",
+ "urls": [
+ "/${VERSION}/rotate-certificates.html"
+ ]
+ },
+ {
+ "title": "Security Tutorials",
+ "items": [
+ {
+ "title": "Use Hashicorp Vault's Dynamic Secrets",
+ "urls": [
+ "/${VERSION}/vault-db-secrets-tutorial.html"
+ ]
+ }
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Monitoring and Alerting",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/monitoring-and-alerting.html"
+ ]
+ },
+ {
+ "title": "Common Issues to Monitor",
+ "urls": [
+ "/${VERSION}/common-issues-to-monitor.html"
+ ]
+ },
+ {
+ "title": "Enable the Node Map",
+ "urls": [
+ "/${VERSION}/enable-node-map.html"
+ ]
+ },
+ {
+ "title": "Use Prometheus and Alertmanager",
+ "urls": [
+ "/${VERSION}/monitor-cockroachdb-with-prometheus.html"
+ ]
+ },
+ {
+ "title": "Monitor CockroachDB {{ site.data.products.core }} with Datadog",
+ "urls": [
+ "/${VERSION}/datadog.html"
+ ]
+ },
+ {
+ "title": "Monitor CockroachDB {{ site.data.products.core }} with DBmarlin",
+ "urls": [
+ "/${VERSION}/dbmarlin.html"
+ ]
+ },
+ {
+ "title": "Monitor CockroachDB {{ site.data.products.core }} with Kibana",
+ "urls": [
+ "/${VERSION}/kibana.html"
+ ]
+ },
+ {
+ "title": "Essential Metrics for CockroachDB {{ site.data.products.core }} Deployments",
+ "urls": [
+ "/${VERSION}/essential-metrics-self-hosted.html"
+ ]
+ },
+ {
+ "title": "Essential Alerts for CockroachDB {{ site.data.products.core }} Deployments",
+ "urls": [
+ "/${VERSION}/essential-alerts-self-hosted.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Administration",
+ "items": [
+ {
+ "title": "DB Console Overview",
+ "urls": [
+ "/${VERSION}/ui-overview.html"
+ ]
+ },
+ {
+ "title": "Cluster Overview Page",
+ "urls": [
+ "/${VERSION}/ui-cluster-overview-page.html"
+ ]
+ },
+ {
+ "title": "Metrics Dashboards",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/ui-overview-dashboard.html"
+ ]
+ },
+ {
+ "title": "Hardware",
+ "urls": [
+ "/${VERSION}/ui-hardware-dashboard.html"
+ ]
+ },
+ {
+ "title": "Runtime",
+ "urls": [
+ "/${VERSION}/ui-runtime-dashboard.html"
+ ]
+ },
+ {
+ "title": "Networking",
+ "urls": [
+ "/${VERSION}/ui-networking-dashboard.html"
+ ]
+ },
+ {
+ "title": "SQL",
+ "urls": [
+ "/${VERSION}/ui-sql-dashboard.html"
+ ]
+ },
+ {
+ "title": "Storage",
+ "urls": [
+ "/${VERSION}/ui-storage-dashboard.html"
+ ]
+ },
+ {
+ "title": "Replication",
+ "urls": [
+ "/${VERSION}/ui-replication-dashboard.html"
+ ]
+ },
+ {
+ "title": "Distributed",
+ "urls": [
+ "/${VERSION}/ui-distributed-dashboard.html"
+ ]
+ },
+ {
+ "title": "Queues",
+ "urls": [
+ "/${VERSION}/ui-queues-dashboard.html"
+ ]
+ },
+ {
+ "title": "Slow Requests",
+ "urls": [
+ "/${VERSION}/ui-slow-requests-dashboard.html"
+ ]
+ },
+ {
+ "title": "Changefeeds",
+ "urls": [
+ "/${VERSION}/ui-cdc-dashboard.html"
+ ]
+ },
+ {
+ "title": "Overload",
+ "urls": [
+ "/${VERSION}/ui-overload-dashboard.html"
+ ]
+ },
+ {
+ "title": "TTL",
+ "urls": [
+ "/${VERSION}/ui-ttl-dashboard.html"
+ ]
+ },
+ {
+ "title": "Physical Cluster Replication",
+ "urls": [
+ "/${VERSION}/ui-physical-cluster-replication-dashboard.html"
+ ]
+ },
+ {
+ "title": "Logical Data Replication",
+ "urls": [
+ "/${VERSION}/ui-logical-data-replication-dashboard.html"
+ ]
+ },
+ {
+ "title": "Custom Chart",
+ "urls": [
+ "/${VERSION}/ui-custom-chart-debug-page.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Databases Page",
+ "urls": [
+ "/${VERSION}/ui-databases-page.html"
+ ]
+ },
+ {
+ "title": "Sessions Page",
+ "urls": [
+ "/${VERSION}/ui-sessions-page.html"
+ ]
+ },
+ {
+ "title": "Statements Page",
+ "urls": [
+ "/${VERSION}/ui-statements-page.html"
+ ]
+ },
+ {
+ "title": "Transactions Page",
+ "urls": [
+ "/${VERSION}/ui-transactions-page.html"
+ ]
+ },
+ {
+ "title": "Insights Page",
+ "urls": [
+ "/${VERSION}/ui-insights-page.html"
+ ]
+ },
+ {
+ "title": "Network Page",
+ "urls": [
+ "/${VERSION}/ui-network-latency-page.html"
+ ]
+ },
+ {
+ "title": "Hot Ranges Page",
+ "urls": [
+ "/${VERSION}/ui-hot-ranges-page.html"
+ ]
+ },
+ {
+ "title": "Jobs Page",
+ "urls": [
+ "/${VERSION}/ui-jobs-page.html"
+ ]
+ },
+ {
+ "title": "Schedules Page",
+ "urls": [
+ "/${VERSION}/ui-schedules-page.html"
+ ]
+ },
+ {
+ "title": "Advanced Debug Page",
+ "urls": [
+ "/${VERSION}/ui-debug-pages.html"
+ ]
+ },
+ {
+ "title": "Key Visualizer",
+ "urls": [
+ "/${VERSION}/ui-key-visualizer.html"
+ ]
+ },
+ {
+ "title": "WAL Failover",
+ "urls": [
+ "/${VERSION}/wal-failover.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Backups and Restores",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/backup-and-restore-overview.html"
+ ]
+ },
+ {
+ "title": "Full and Incremental Backups",
+ "urls": [
+ "/${VERSION}/take-full-and-incremental-backups.html"
+ ]
+ },
+ {
+ "title": "Backups with Revision History and Point-in-time Restore",
+ "urls": [
+ "/${VERSION}/take-backups-with-revision-history-and-restore-from-a-point-in-time.html"
+ ]
+ },
+ {
+ "title": "Encrypted Backup and Restore",
+ "urls": [
+ "/${VERSION}/take-and-restore-encrypted-backups.html"
+ ]
+ },
+ {
+ "title": "Locality-restricted Backup Execution",
+ "urls": [
+ "/${VERSION}/take-locality-restricted-backups.html"
+ ]
+ },
+ {
+ "title": "Locality-aware Backup and Restore",
+ "urls": [
+ "/${VERSION}/take-and-restore-locality-aware-backups.html"
+ ]
+ },
+ {
+ "title": "Scheduled Backups",
+ "urls": [
+ "/${VERSION}/manage-a-backup-schedule.html"
+ ]
+ },
+ {
+ "title": "Backup Validation",
+ "urls": [
+ "/${VERSION}/backup-validation.html"
+ ]
+ },
+ {
+ "title": "Expire Past Backups",
+ "urls": [
+ "/${VERSION}/expire-past-backups.html"
+ ]
+ },
+ {
+ "title": "Backup and Restore Monitoring",
+ "urls": [
+ "/${VERSION}/backup-and-restore-monitoring.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "File Storage",
+ "items": [
+ {
+ "title": "Cloud Storage",
+ "urls": [
+ "/${VERSION}/use-cloud-storage.html"
+ ]
+ },
+ {
+ "title": "Cloud Storage Authentication",
+ "urls": [
+ "/${VERSION}/cloud-storage-authentication.html"
+ ]
+ },
+ {
+ "title": "Userfile Storage",
+ "urls": [
+ "/${VERSION}/use-userfile-storage.html"
+ ]
+ },
+ {
+ "title": "Local File Server",
+ "urls": [
+ "/${VERSION}/use-a-local-file-server.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Cluster Maintenance",
+ "items": [
+ {
+ "title": "Upgrade CockroachDB",
+ "urls": [
+ "/${VERSION}/upgrade-cockroach-version.html"
+ ]
+ },
+ {
+ "title": "Disaster Recovery Planning",
+ "urls": [
+ "/${VERSION}/disaster-recovery-planning.html"
+ ]
+ },
+ {
+ "title": "Restoring Backups Across Versions",
+ "urls": [
+ "/${VERSION}/restoring-backups-across-versions.html"
+ ]
+ },
+ {
+ "title": "Manage Long-Running Queries",
+ "urls": [
+ "/${VERSION}/manage-long-running-queries.html"
+ ]
+ },
+ {
+ "title": "Node Shutdown",
+ "urls": [
+ "/${VERSION}/node-shutdown.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Logs",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/logging-overview.html"
+ ]
+ },
+ {
+ "title": "Configure Logs",
+ "urls": [
+ "/${VERSION}/configure-logs.html"
+ ]
+ },
+ {
+ "title": "Logging Use Cases",
+ "urls": [
+ "/${VERSION}/logging-use-cases.html"
+ ]
+ },
+ {
+ "title": "Log SQL Statistics to Datadog",
+ "urls": [
+ "/${VERSION}/log-sql-statistics-to-datadog.html"
+ ]
+ },
+ {
+ "title": "Logging Best Practices",
+ "urls": [
+ "/${VERSION}/logging-best-practices.html"
+ ]
+ },
+ {
+ "title": "Critical Log Messages",
+ "urls": [
+ "/${VERSION}/critical-log-messages.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Cluster API",
+ "urls": [
+ "/${VERSION}/cluster-api.html"
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/sql.json b/src/current/_includes/v25.1/sidebar-data/sql.json
new file mode 100644
index 00000000000..7ccd1a56aa7
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/sql.json
@@ -0,0 +1,1272 @@
+{
+ "title": "SQL",
+ "is_top_level": true,
+ "items": [{
+ "title": "Statements",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/sql-statements.html"
+ ]
+ },
+ {
+ "title": "ALTER BACKUP (Enterprise)",
+ "urls": [
+ "/${VERSION}/alter-backup.html"
+ ]
+ },
+ {
+ "title": "ALTER BACKUP SCHEDULE",
+ "urls": [
+ "/${VERSION}/alter-backup-schedule.html"
+ ]
+ },
+ {
+ "title": "ALTER CHANGEFEED (Enterprise)",
+ "urls": [
+ "/${VERSION}/alter-changefeed.html"
+ ]
+ },
+ {
+ "title": "ALTER DATABASE",
+ "urls": [
+ "/${VERSION}/alter-database.html"
+ ]
+ },
+ {
+ "title": "ALTER DEFAULT PRIVILEGES",
+ "urls": [
+ "/${VERSION}/alter-default-privileges.html"
+ ]
+ },
+ {
+ "title": "ALTER FUNCTION",
+ "urls": [
+ "/${VERSION}/alter-function.html"
+ ]
+ },
+ {
+ "title": "ALTER INDEX",
+ "urls": [
+ "/${VERSION}/alter-index.html"
+ ]
+ },
+ {
+ "title": "ALTER PARTITION (Enterprise)",
+ "urls": [
+ "/${VERSION}/alter-partition.html"
+ ]
+ },
+ {
+ "title": "ALTER PROCEDURE",
+ "urls": [
+ "/${VERSION}/alter-procedure.html"
+ ]
+ },
+ {
+ "title": "ALTER RANGE",
+ "urls": [
+ "/${VERSION}/alter-range.html"
+ ]
+ },
+ {
+ "title": "ALTER ROLE",
+ "urls": [
+ "/${VERSION}/alter-role.html"
+ ]
+ },
+ {
+ "title": "ALTER SCHEMA",
+ "urls": [
+ "/${VERSION}/alter-schema.html"
+ ]
+ },
+ {
+ "title": "ALTER SEQUENCE",
+ "urls": [
+ "/${VERSION}/alter-sequence.html"
+ ]
+ },
+ {
+ "title": "ALTER TABLE",
+ "urls": [
+ "/${VERSION}/alter-table.html"
+ ]
+ },
+ {
+ "title": "ALTER TYPE",
+ "urls": [
+ "/${VERSION}/alter-type.html"
+ ]
+ },
+ {
+ "title": "ALTER USER",
+ "urls": [
+ "/${VERSION}/alter-user.html"
+ ]
+ },
+ {
+ "title": "ALTER VIEW",
+ "urls": [
+ "/${VERSION}/alter-view.html"
+ ]
+ },
+ {
+ "title": "ALTER VIRTUAL CLUSTER",
+ "urls": [
+ "/${VERSION}/alter-virtual-cluster.html"
+ ]
+ },
+ {
+ "title": "BACKUP",
+ "urls": [
+ "/${VERSION}/backup.html"
+ ]
+ },
+ {
+ "title": "BEGIN",
+ "urls": [
+ "/${VERSION}/begin-transaction.html"
+ ]
+ },
+ {
+ "title": "CALL",
+ "urls": [
+ "/${VERSION}/call.html"
+ ]
+ },
+ {
+ "title": "CANCEL JOB",
+ "urls": [
+ "/${VERSION}/cancel-job.html"
+ ]
+ },
+ {
+ "title": "CANCEL QUERY",
+ "urls": [
+ "/${VERSION}/cancel-query.html"
+ ]
+ },
+ {
+ "title": "CANCEL SESSION",
+ "urls": [
+ "/${VERSION}/cancel-session.html"
+ ]
+ },
+ {
+ "title": "COMMENT ON",
+ "urls": [
+ "/${VERSION}/comment-on.html"
+ ]
+ },
+ {
+ "title": "COMMIT",
+ "urls": [
+ "/${VERSION}/commit-transaction.html"
+ ]
+ },
+ {
+ "title": "COPY FROM",
+ "urls": [
+ "/${VERSION}/copy-from.html"
+ ]
+ },
+ {
+ "title": "CREATE CHANGEFEED (Enterprise)",
+ "urls": [
+ "/${VERSION}/create-changefeed.html"
+ ]
+ },
+ {
+ "title": "CREATE DATABASE",
+ "urls": [
+ "/${VERSION}/create-database.html"
+ ]
+ },
+ {
+ "title": "CREATE EXTERNAL CONNECTION",
+ "urls": [
+ "/${VERSION}/create-external-connection.html"
+ ]
+ },
+ {
+ "title": "CREATE FUNCTION",
+ "urls": [
+ "/${VERSION}/create-function.html"
+ ]
+ },
+ {
+ "title": "CREATE INDEX",
+ "urls": [
+ "/${VERSION}/create-index.html"
+ ]
+ },
+ {
+ "title": "CREATE LOGICAL REPLICATION STREAM",
+ "urls": [
+ "/${VERSION}/create-logical-replication-stream.html"
+ ]
+ },
+ {
+ "title": "CREATE PROCEDURE",
+ "urls": [
+ "/${VERSION}/create-procedure.html"
+ ]
+ },
+ {
+ "title": "CREATE ROLE",
+ "urls": [
+ "/${VERSION}/create-role.html"
+ ]
+ },
+ {
+ "title": "CREATE SCHEDULE FOR BACKUP",
+ "urls": [
+ "/${VERSION}/create-schedule-for-backup.html"
+ ]
+ },
+ {
+ "title": "CREATE SCHEDULE FOR CHANGEFEED",
+ "urls": [
+ "/${VERSION}/create-schedule-for-changefeed.html"
+ ]
+ },
+ {
+ "title": "CREATE SCHEMA",
+ "urls": [
+ "/${VERSION}/create-schema.html"
+ ]
+ },
+ {
+ "title": "CREATE SEQUENCE",
+ "urls": [
+ "/${VERSION}/create-sequence.html"
+ ]
+ },
+ {
+ "title": "CREATE STATISTICS",
+ "urls": [
+ "/${VERSION}/create-statistics.html"
+ ]
+ },
+ {
+ "title": "CREATE TABLE",
+ "urls": [
+ "/${VERSION}/create-table.html"
+ ]
+ },
+ {
+ "title": "CREATE TABLE AS",
+ "urls": [
+ "/${VERSION}/create-table-as.html"
+ ]
+ },
+ {
+ "title": "CREATE TRIGGER",
+ "urls": [
+ "/${VERSION}/create-trigger.html"
+ ]
+ },
+ {
+ "title": "CREATE TYPE",
+ "urls": [
+ "/${VERSION}/create-type.html"
+ ]
+ },
+ {
+ "title": "CREATE USER",
+ "urls": [
+ "/${VERSION}/create-user.html"
+ ]
+ },
+ {
+ "title": "CREATE VIEW",
+ "urls": [
+ "/${VERSION}/create-view.html"
+ ]
+ },
+ {
+ "title": "CREATE VIRTUAL CLUSTER",
+ "urls": [
+ "/${VERSION}/create-virtual-cluster.html"
+ ]
+ },
+ {
+ "title": "DELETE",
+ "urls": [
+ "/${VERSION}/delete.html"
+ ]
+ },
+ {
+ "title": "DROP DATABASE",
+ "urls": [
+ "/${VERSION}/drop-database.html"
+ ]
+ },
+ {
+ "title": "DROP EXTERNAL CONNECTION",
+ "urls": [
+ "/${VERSION}/drop-external-connection.html"
+ ]
+ },
+ {
+ "title": "DROP FUNCTION",
+ "urls": [
+ "/${VERSION}/drop-function.html"
+ ]
+ },
+ {
+ "title": "DROP OWNED BY",
+ "urls": [
+ "/${VERSION}/drop-owned-by.html"
+ ]
+ },
+ {
+ "title": "DROP TRIGGER",
+ "urls": [
+ "/${VERSION}/drop-trigger.html"
+ ]
+ },
+ {
+ "title": "DROP TYPE",
+ "urls": [
+ "/${VERSION}/drop-type.html"
+ ]
+ },
+ {
+ "title": "DROP INDEX",
+ "urls": [
+ "/${VERSION}/drop-index.html"
+ ]
+ },
+ {
+ "title": "DROP PROCEDURE",
+ "urls": [
+ "/${VERSION}/drop-procedure.html"
+ ]
+ },
+ {
+ "title": "DROP ROLE",
+ "urls": [
+ "/${VERSION}/drop-role.html"
+ ]
+ },
+ {
+ "title": "DROP SCHEDULES",
+ "urls": [
+ "/${VERSION}/drop-schedules.html"
+ ]
+ },
+ {
+ "title": "DROP SCHEMA",
+ "urls": [
+ "/${VERSION}/drop-schema.html"
+ ]
+ },
+ {
+ "title": "DROP SEQUENCE",
+ "urls": [
+ "/${VERSION}/drop-sequence.html"
+ ]
+ },
+ {
+ "title": "DROP TABLE",
+ "urls": [
+ "/${VERSION}/drop-table.html"
+ ]
+ },
+ {
+ "title": "DROP USER",
+ "urls": [
+ "/${VERSION}/drop-user.html"
+ ]
+ },
+ {
+ "title": "DROP VIEW",
+ "urls": [
+ "/${VERSION}/drop-view.html"
+ ]
+ },
+ {
+ "title": "DROP VIRTUAL CLUSTER",
+ "urls": [
+ "/${VERSION}/drop-virtual-cluster.html"
+ ]
+ },
+ {
+ "title": "EXPERIMENTAL CHANGEFEED FOR",
+ "urls": [
+ "/${VERSION}/changefeed-for.html"
+ ]
+ },
+ {
+ "title": "EXPLAIN",
+ "urls": [
+ "/${VERSION}/explain.html"
+ ]
+ },
+ {
+ "title": "EXPLAIN ANALYZE",
+ "urls": [
+ "/${VERSION}/explain-analyze.html"
+ ]
+ },
+ {
+ "title": "EXPORT",
+ "urls": [
+ "/${VERSION}/export.html"
+ ]
+ },
+ {
+ "title": "GRANT",
+ "urls": [
+ "/${VERSION}/grant.html"
+ ]
+ },
+ {
+ "title": "IMPORT INTO",
+ "urls": [
+ "/${VERSION}/import-into.html"
+ ]
+ },
+ {
+ "title": "INSERT",
+ "urls": [
+ "/${VERSION}/insert.html"
+ ]
+ },
+ {
+ "title": "JOIN",
+ "urls": [
+ "/${VERSION}/joins.html"
+ ]
+ },
+ {
+ "title": "LIMIT/OFFSET",
+ "urls": [
+ "/${VERSION}/limit-offset.html"
+ ]
+ },
+ {
+ "title": "ORDER BY",
+ "urls": [
+ "/${VERSION}/order-by.html"
+ ]
+ },
+ {
+ "title": "PAUSE JOB",
+ "urls": [
+ "/${VERSION}/pause-job.html"
+ ]
+ },
+ {
+ "title": "PAUSE SCHEDULES",
+ "urls": [
+ "/${VERSION}/pause-schedules.html"
+ ]
+ },
+ {
+ "title": "REASSIGN OWNED",
+ "urls": [
+ "/${VERSION}/reassign-owned.html"
+ ]
+ },
+ {
+ "title": "REFRESH",
+ "urls": [
+ "/${VERSION}/refresh.html"
+ ]
+ },
+ {
+ "title": "RELEASE SAVEPOINT",
+ "urls": [
+ "/${VERSION}/release-savepoint.html"
+ ]
+ },
+ {
+ "title": "RESET CLUSTER SETTING",
+ "urls": [
+ "/${VERSION}/reset-cluster-setting.html"
+ ]
+ },
+ {
+ "title": "RESET {session variable}",
+ "urls": [
+ "/${VERSION}/reset-vars.html"
+ ]
+ },
+ {
+ "title": "RESTORE",
+ "urls": [
+ "/${VERSION}/restore.html"
+ ]
+ },
+ {
+ "title": "RESUME JOB",
+ "urls": [
+ "/${VERSION}/resume-job.html"
+ ]
+ },
+ {
+ "title": "RESUME SCHEDULES",
+ "urls": [
+ "/${VERSION}/resume-schedules.html"
+ ]
+ },
+ {
+ "title": "REVOKE",
+ "urls": [
+ "/${VERSION}/revoke.html"
+ ]
+ },
+ {
+ "title": "ROLLBACK",
+ "urls": [
+ "/${VERSION}/rollback-transaction.html"
+ ]
+ },
+ {
+ "title": "SAVEPOINT",
+ "urls": [
+ "/${VERSION}/savepoint.html"
+ ]
+ },
+ {
+ "title": "SELECT",
+ "urls": [
+ "/${VERSION}/select-clause.html"
+ ]
+ },
+ {
+ "title": "FOR UPDATE and FOR SHARE",
+ "urls": [
+ "/${VERSION}/select-for-update.html"
+ ]
+ },
+ {
+ "title": "SET CLUSTER SETTING",
+ "urls": [
+ "/${VERSION}/set-cluster-setting.html"
+ ]
+ },
+ {
+ "title": "SET {session variable}",
+ "urls": [
+ "/${VERSION}/set-vars.html"
+ ]
+ },
+ {
+ "title": "SET TRANSACTION",
+ "urls": [
+ "/${VERSION}/set-transaction.html"
+ ]
+ },
+ {
+ "title": "SHOW BACKUP",
+ "urls": [
+ "/${VERSION}/show-backup.html"
+ ]
+ },
+ {
+ "title": "SHOW CLUSTER SETTING",
+ "urls": [
+ "/${VERSION}/show-cluster-setting.html"
+ ]
+ },
+ {
+ "title": "SHOW COLUMNS",
+ "urls": [
+ "/${VERSION}/show-columns.html"
+ ]
+ },
+ {
+ "title": "SHOW CONSTRAINTS",
+ "urls": [
+ "/${VERSION}/show-constraints.html"
+ ]
+ },
+ {
+ "title": "SHOW CREATE",
+ "urls": [
+ "/${VERSION}/show-create.html"
+ ]
+ },
+ {
+ "title": "SHOW CREATE EXTERNAL CONNECTION",
+ "urls": [
+ "/${VERSION}/show-create-external-connection.html"
+ ]
+ },
+ {
+ "title": "SHOW CREATE SCHEDULE",
+ "urls": [
+ "/${VERSION}/show-create-schedule.html"
+ ]
+ },
+ {
+ "title": "SHOW DATABASES",
+ "urls": [
+ "/${VERSION}/show-databases.html"
+ ]
+ },
+ {
+ "title": "SHOW DEFAULT PRIVILEGES",
+ "urls": [
+ "/${VERSION}/show-default-privileges.html"
+ ]
+ },
+ {
+ "title": "SHOW DEFAULT SESSION VARIABLES FOR ROLE",
+ "urls": [
+ "/${VERSION}/show-default-session-variables-for-role.html"
+ ]
+ },
+ {
+ "title": "SHOW ENUMS",
+ "urls": [
+ "/${VERSION}/show-enums.html"
+ ]
+ },
+ {
+ "title": "SHOW EXTERNAL CONNECTION",
+ "urls": [
+ "/${VERSION}/show-external-connection.html"
+ ]
+ },
+ {
+ "title": "SHOW FULL TABLE SCANS",
+ "urls": [
+ "/${VERSION}/show-full-table-scans.html"
+ ]
+ },
+ {
+ "title": "SHOW GRANTS",
+ "urls": [
+ "/${VERSION}/show-grants.html"
+ ]
+ },
+ {
+ "title": "SHOW INDEX",
+ "urls": [
+ "/${VERSION}/show-index.html"
+ ]
+ },
+ {
+ "title": "SHOW JOBS",
+ "urls": [
+ "/${VERSION}/show-jobs.html"
+ ]
+ },
+ {
+ "title": "SHOW LOCALITY",
+ "urls": [
+ "/${VERSION}/show-locality.html"
+ ]
+ },
+ {
+ "title": "SHOW LOGICAL REPLICATION JOBS",
+ "urls": [
+ "/${VERSION}/show-logical-replication-jobs.html"
+ ]
+ },
+ {
+ "title": "SHOW PARTITIONS (Enterprise)",
+ "urls": [
+ "/${VERSION}/show-partitions.html"
+ ]
+ },
+ {
+ "title": "SHOW RANGES",
+ "urls": [
+ "/${VERSION}/show-ranges.html"
+ ]
+ },
+ {
+ "title": "SHOW RANGE FOR ROW",
+ "urls": [
+ "/${VERSION}/show-range-for-row.html"
+ ]
+ },
+ {
+ "title": "SHOW REGIONS",
+ "urls": [
+ "/${VERSION}/show-regions.html"
+ ]
+ },
+ {
+ "title": "SHOW {session variable}",
+ "urls": [
+ "/${VERSION}/show-vars.html"
+ ]
+ },
+ {
+ "title": "SHOW SUPER REGIONS",
+ "urls": [
+ "/${VERSION}/show-super-regions.html"
+ ]
+ },
+ {
+ "title": "SHOW SYSTEM GRANTS",
+ "urls": [
+ "/${VERSION}/show-system-grants.html"
+ ]
+ },
+ {
+ "title": "SHOW ROLES",
+ "urls": [
+ "/${VERSION}/show-roles.html"
+ ]
+ },
+ {
+ "title": "SHOW SCHEDULES",
+ "urls": [
+ "/${VERSION}/show-schedules.html"
+ ]
+ },
+ {
+ "title": "SHOW SCHEMAS",
+ "urls": [
+ "/${VERSION}/show-schemas.html"
+ ]
+ },
+ {
+ "title": "SHOW SEQUENCES",
+ "urls": [
+ "/${VERSION}/show-sequences.html"
+ ]
+ },
+ {
+ "title": "SHOW SESSIONS",
+ "urls": [
+ "/${VERSION}/show-sessions.html"
+ ]
+ },
+ {
+ "title": "SHOW STATEMENTS",
+ "urls": [
+ "/${VERSION}/show-statements.html"
+ ]
+ },
+ {
+ "title": "SHOW STATISTICS",
+ "urls": [
+ "/${VERSION}/show-statistics.html"
+ ]
+ },
+ {
+ "title": "SHOW SAVEPOINT STATUS",
+ "urls": [
+ "/${VERSION}/show-savepoint-status.html"
+ ]
+ },
+ {
+ "title": "SHOW TABLES",
+ "urls": [
+ "/${VERSION}/show-tables.html"
+ ]
+ },
+ {
+ "title": "SHOW TRACE FOR SESSION",
+ "urls": [
+ "/${VERSION}/show-trace.html"
+ ]
+ },
+ {
+ "title": "SHOW TRANSACTIONS",
+ "urls": [
+ "/${VERSION}/show-transactions.html"
+ ]
+ },
+ {
+ "title": "SHOW TYPES",
+ "urls": [
+ "/${VERSION}/show-types.html"
+ ]
+ },
+ {
+ "title": "SHOW USERS",
+ "urls": [
+ "/${VERSION}/show-users.html"
+ ]
+ },
+ {
+ "title": "SHOW VIRTUAL CLUSTER",
+ "urls": [
+ "/${VERSION}/show-virtual-cluster.html"
+ ]
+ },
+ {
+ "title": "SHOW ZONE CONFIGURATIONS",
+ "urls": [
+ "/${VERSION}/show-zone-configurations.html"
+ ]
+ },
+ {
+ "title": "TRUNCATE",
+ "urls": [
+ "/${VERSION}/truncate.html"
+ ]
+ },
+ {
+ "title": "UPDATE",
+ "urls": [
+ "/${VERSION}/update.html"
+ ]
+ },
+ {
+ "title": "UPSERT",
+ "urls": [
+ "/${VERSION}/upsert.html"
+ ]
+ },
+ {
+ "title": "WITH {storage parameter}",
+ "urls": [
+ "/${VERSION}/with-storage-parameter.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Syntax",
+ "items": [
+ {
+ "title": "Keywords & Identifiers",
+ "urls": [
+ "/${VERSION}/keywords-and-identifiers.html"
+ ]
+ },
+ {
+ "title": "Constants",
+ "urls": [
+ "/${VERSION}/sql-constants.html"
+ ]
+ },
+ {
+ "title": "Selection Queries",
+ "urls": [
+ "/${VERSION}/selection-queries.html"
+ ]
+ },
+ {
+ "title": "Cursors",
+ "urls": [
+ "/${VERSION}/cursors.html"
+ ]
+ },
+ {
+ "title": "Table Expressions",
+ "urls": [
+ "/${VERSION}/table-expressions.html"
+ ]
+ },
+ {
+ "title": "Common Table Expressions",
+ "urls": [
+ "/${VERSION}/common-table-expressions.html"
+ ]
+ },
+ {
+ "title": "Name Resolution",
+ "urls": [
+ "/${VERSION}/sql-name-resolution.html"
+ ]
+ },
+ {
+ "title": "NULL Handling",
+ "urls": [
+ "/${VERSION}/null-handling.html"
+ ]
+ },
+ {
+ "title": "Scalar Expressions",
+ "urls": [
+ "/${VERSION}/scalar-expressions.html"
+ ]
+ },
+ {
+ "title": "User-Defined Functions",
+ "urls": [
+ "/${VERSION}/user-defined-functions.html"
+ ]
+ },
+ {
+ "title": "Stored Procedures",
+ "urls": [
+ "/${VERSION}/stored-procedures.html"
+ ]
+ },
+ {
+ "title": "Triggers",
+ "urls": [
+ "/${VERSION}/triggers.html"
+ ]
+ },
+ {
+ "title": "Window Functions",
+ "urls": [
+ "/${VERSION}/window-functions.html"
+ ]
+ },
+ {
+ "title": "Full SQL Grammar",
+ "urls": [
+ "/${VERSION}/sql-grammar.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Data Types",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/data-types.html"
+ ]
+ },
+ {
+ "title": "ARRAY",
+ "urls": [
+ "/${VERSION}/array.html"
+ ]
+ },
+ {
+ "title": "BIT",
+ "urls": [
+ "/${VERSION}/bit.html"
+ ]
+ },
+ {
+ "title": "BOOL",
+ "urls": [
+ "/${VERSION}/bool.html"
+ ]
+ },
+ {
+ "title": "BYTES",
+ "urls": [
+ "/${VERSION}/bytes.html"
+ ]
+ },
+ {
+ "title": "COLLATE",
+ "urls": [
+ "/${VERSION}/collate.html"
+ ]
+ },
+ {
+ "title": "DATE",
+ "urls": [
+ "/${VERSION}/date.html"
+ ]
+ },
+ {
+ "title": "DECIMAL",
+ "urls": [
+ "/${VERSION}/decimal.html"
+ ]
+ },
+ {
+ "title": "ENUM",
+ "urls": [
+ "/${VERSION}/enum.html"
+ ]
+ },
+ {
+ "title": "FLOAT",
+ "urls": [
+ "/${VERSION}/float.html"
+ ]
+ },
+ {
+ "title": "INET",
+ "urls": [
+ "/${VERSION}/inet.html"
+ ]
+ },
+ {
+ "title": "INT",
+ "urls": [
+ "/${VERSION}/int.html"
+ ]
+ },
+ {
+ "title": "INTERVAL",
+ "urls": [
+ "/${VERSION}/interval.html"
+ ]
+ },
+ {
+ "title": "JSONB",
+ "urls": [
+ "/${VERSION}/jsonb.html"
+ ]
+ },
+ {
+ "title": "OID",
+ "urls": [
+ "/${VERSION}/oid.html"
+ ]
+ },
+ {
+ "title": "SERIAL",
+ "urls": [
+ "/${VERSION}/serial.html"
+ ]
+ },
+ {
+ "title": "STRING",
+ "urls": [
+ "/${VERSION}/string.html"
+ ]
+ },
+ {
+ "title": "TIME",
+ "urls": [
+ "/${VERSION}/time.html"
+ ]
+ },
+ {
+ "title": "TIMESTAMP",
+ "urls": [
+ "/${VERSION}/timestamp.html"
+ ]
+ },
+ {
+ "title": "TSQUERY",
+ "urls": [
+ "/${VERSION}/tsquery.html"
+ ]
+ },
+ {
+ "title": "TSVECTOR",
+ "urls": [
+ "/${VERSION}/tsvector.html"
+ ]
+ },
+ {
+ "title": "UUID",
+ "urls": [
+ "/${VERSION}/uuid.html"
+ ]
+ },
+ {
+ "title": "VECTOR",
+ "urls": [
+ "/${VERSION}/vector.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Spatial Data",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/spatial-data-overview.html"
+ ]
+ },
+ {
+ "title": "POINT",
+ "urls": [
+ "/${VERSION}/point.html"
+ ]
+ },
+ {
+ "title": "LINESTRING",
+ "urls": [
+ "/${VERSION}/linestring.html"
+ ]
+ },
+ {
+ "title": "POLYGON",
+ "urls": [
+ "/${VERSION}/polygon.html"
+ ]
+ },
+ {
+ "title": "MULTIPOINT",
+ "urls": [
+ "/${VERSION}/multipoint.html"
+ ]
+ },
+ {
+ "title": "MULTILINESTRING",
+ "urls": [
+ "/${VERSION}/multilinestring.html"
+ ]
+ },
+ {
+ "title": "MULTIPOLYGON",
+ "urls": [
+ "/${VERSION}/multipolygon.html"
+ ]
+ },
+ {
+ "title": "GEOMETRYCOLLECTION",
+ "urls": [
+ "/${VERSION}/geometrycollection.html"
+ ]
+ },
+ {
+ "title": "Well Known Text (WKT)",
+ "urls": [
+ "/${VERSION}/well-known-text.html"
+ ]
+ },
+ {
+ "title": "Well Known Binary (WKB)",
+ "urls": [
+ "/${VERSION}/well-known-binary.html"
+ ]
+ },
+ {
+ "title": "GeoJSON",
+ "urls": [
+ "/${VERSION}/geojson.html"
+ ]
+ },
+ {
+ "title": "SRID 4326 - longitude and latitude",
+ "urls": [
+ "/${VERSION}/srid-4326.html"
+ ]
+ },
+ {
+ "title": "ST_Contains",
+ "urls": [
+ "/${VERSION}/st_contains.html"
+ ]
+ },
+ {
+ "title": "ST_Within",
+ "urls": [
+ "/${VERSION}/st_within.html"
+ ]
+ },
+ {
+ "title": "ST_Intersects",
+ "urls": [
+ "/${VERSION}/st_intersects.html"
+ ]
+ },
+ {
+ "title": "ST_CoveredBy",
+ "urls": [
+ "/${VERSION}/st_coveredby.html"
+ ]
+ },
+ {
+ "title": "ST_Covers",
+ "urls": [
+ "/${VERSION}/st_covers.html"
+ ]
+ },
+ {
+ "title": "ST_Disjoint",
+ "urls": [
+ "/${VERSION}/st_disjoint.html"
+ ]
+ },
+ {
+ "title": "ST_Equals",
+ "urls": [
+ "/${VERSION}/st_equals.html"
+ ]
+ },
+ {
+ "title": "ST_Overlaps",
+ "urls": [
+ "/${VERSION}/st_overlaps.html"
+ ]
+ },
+ {
+ "title": "ST_Touches",
+ "urls": [
+ "/${VERSION}/st_touches.html"
+ ]
+ },
+ {
+ "title": "ST_ConvexHull",
+ "urls": [
+ "/${VERSION}/st_convexhull.html"
+ ]
+ },
+ {
+ "title": "ST_Union",
+ "urls": [
+ "/${VERSION}/st_union.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Constraints",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/constraints.html"
+ ]
+ },
+ {
+ "title": "Check",
+ "urls": [
+ "/${VERSION}/check.html"
+ ]
+ },
+ {
+ "title": "Default Value",
+ "urls": [
+ "/${VERSION}/default-value.html"
+ ]
+ },
+ {
+ "title": "Foreign Key",
+ "urls": [
+ "/${VERSION}/foreign-key.html"
+ ]
+ },
+ {
+ "title": "Not Null",
+ "urls": [
+ "/${VERSION}/not-null.html"
+ ]
+ },
+ {
+ "title": "Primary Key",
+ "urls": [
+ "/${VERSION}/primary-key.html"
+ ]
+ },
+ {
+ "title": "Unique",
+ "urls": [
+ "/${VERSION}/unique.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Functions & Operators",
+ "urls": [
+ "/${VERSION}/functions-and-operators.html"
+ ]
+ },
+ {
+ "title": "Session Variables",
+ "urls": [
+ "/${VERSION}/session-variables.html"
+ ]
+ },
+ {
+ "title": "PL/pgSQL",
+ "urls": [
+ "/${VERSION}/plpgsql.html"
+ ]
+ }
+ ]
+ }
diff --git a/src/current/_includes/v25.1/sidebar-data/stream-data.json b/src/current/_includes/v25.1/sidebar-data/stream-data.json
new file mode 100644
index 00000000000..d0637912232
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/stream-data.json
@@ -0,0 +1,150 @@
+{
+ "title": "Stream Data",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Change Data Capture Overview",
+ "urls": [
+ "/${VERSION}/change-data-capture-overview.html"
+ ]
+ },
+ {
+ "title": "Get Started with Changefeeds",
+ "items": [
+ {
+ "title": "Create and Configure Changefeeds",
+ "urls": [
+ "/${VERSION}/create-and-configure-changefeeds.html"
+ ]
+ },
+ {
+ "title": "Changefeed Best Practices",
+ "urls": [
+ "/${VERSION}/changefeed-best-practices.html"
+ ]
+ },
+ {
+ "title": "Changefeed Messages",
+ "urls": [
+ "/${VERSION}/changefeed-messages.html"
+ ]
+ },
+ {
+ "title": "Changefeed Sinks",
+ "urls": [
+ "/${VERSION}/changefeed-sinks.html"
+ ]
+ },
+ {
+ "title": "Changefeed Examples",
+ "urls": [
+ "/${VERSION}/changefeed-examples.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Monitor Changefeeds",
+ "items": [
+ {
+ "title": "Monitor and Debug Changefeeds",
+ "urls": [
+ "/${VERSION}/monitor-and-debug-changefeeds.html"
+ ]
+ },
+ {
+ "title": "Protect Changefeed Data",
+ "urls": [
+ "/${VERSION}/protect-changefeed-data.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Optimize Changefeeds",
+ "items": [
+ {
+ "title": "Change Data Capture Queries",
+ "urls": [
+ "/${VERSION}/cdc-queries.html"
+ ]
+ },
+ {
+ "title": "Changefeeds on Tables with Column Families",
+ "urls": [
+ "/${VERSION}/changefeeds-on-tables-with-column-families.html"
+ ]
+ },
+ {
+ "title": "Export Data with Changefeeds",
+ "urls": [
+ "/${VERSION}/export-data-with-changefeeds.html"
+ ]
+ },
+ {
+ "title": "Changefeeds in Multi-Region Deployments",
+ "urls": [
+ "/${VERSION}/changefeeds-in-multi-region-deployments.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Changefeed Tutorials",
+ "items": [
+ {
+ "title": "Stream a Changefeed to an Amazon MSK Cluster",
+ "items": [
+ {
+ "title": "Amazon MSK",
+ "urls": [
+ "/${VERSION}/stream-a-changefeed-to-amazon-msk.html"
+ ]
+ },
+ {
+ "title": "Amazon MSK Serverless",
+ "urls": [
+ "/${VERSION}/stream-a-changefeed-to-amazon-msk-serverless.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Connect to a Changefeed Kafka Sink with OAuth Using Okta",
+ "urls": [
+ "/${VERSION}/connect-to-a-changefeed-kafka-sink-with-oauth-using-okta.html"
+ ]
+ },
+ {
+ "title": "Stream a Changefeed from CockroachDB Cloud to Snowflake",
+ "urls": [
+ "/cockroachcloud/stream-changefeed-to-snowflake-aws.html"
+ ]
+ },
+ {
+ "title": "Stream a Changefeed to a Confluent Cloud Kafka Cluster",
+ "urls": [
+ "/${VERSION}/stream-a-changefeed-to-a-confluent-cloud-kafka-cluster.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Technical Overview",
+ "items": [
+ {
+ "title": "How Does an Enterprise Changefeed Work?",
+ "urls": [
+ "/${VERSION}/how-does-an-enterprise-changefeed-work.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Advanced Changefeed Configuration",
+ "urls": [
+ "/${VERSION}/advanced-changefeed-configuration.html"
+ ]
+ }
+ ]
+}
diff --git a/src/current/_includes/v25.1/sidebar-data/troubleshooting.json b/src/current/_includes/v25.1/sidebar-data/troubleshooting.json
new file mode 100644
index 00000000000..739e9cfa5ef
--- /dev/null
+++ b/src/current/_includes/v25.1/sidebar-data/troubleshooting.json
@@ -0,0 +1,99 @@
+{
+ "title": "Troubleshooting",
+ "is_top_level": true,
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/troubleshooting-overview.html"
+ ]
+ },
+ {
+ "title": "Common Errors and Solutions",
+ "urls": [
+ "/${VERSION}/common-errors.html"
+ ]
+ },
+ {
+ "title": "Troubleshoot Cloud Setup",
+ "urls": [
+ "/cockroachcloud/troubleshooting-page.html"
+ ]
+ },
+ {
+ "title": "Troubleshoot Self-Hosted Setup",
+ "urls": [
+ "/${VERSION}/cluster-setup-troubleshooting.html"
+ ]
+ },
+ {
+ "title": "Troubleshoot SQL Statements",
+ "urls": [
+ "/${VERSION}/query-behavior-troubleshooting.html"
+ ]
+ },
+ {
+ "title": "Transaction Retry Error Reference",
+ "urls": [
+ "/${VERSION}/transaction-retry-error-reference.html"
+ ]
+ },
+ {
+ "title": "Transaction Retry Error Example",
+ "urls": [
+ "/${VERSION}/transaction-retry-error-example.html"
+ ]
+ },
+ {
+ "title": "Differences in Metrics between Third-Party Monitoring Integrations and DB Console",
+ "urls": [
+ "/${VERSION}/differences-in-metrics-between-third-party-monitoring-integrations-and-db-console.html"
+ ]
+ },
+ {
+ "title": "Replication Reports",
+ "urls": [
+ "/${VERSION}/query-replication-reports.html"
+ ]
+ },
+ {
+ "title": "Benchmarking",
+ "items": [
+ {
+ "title": "Overview",
+ "urls": [
+ "/${VERSION}/performance.html"
+ ]
+ },
+ {
+ "title": "Benchmarking with TPC-C",
+ "urls": [
+ "/${VERSION}/performance-benchmarking-with-tpcc-local.html",
+ "/${VERSION}/performance-benchmarking-with-tpcc-local-multiregion.html",
+ "/${VERSION}/performance-benchmarking-with-tpcc-small.html",
+ "/${VERSION}/performance-benchmarking-with-tpcc-medium.html",
+ "/${VERSION}/performance-benchmarking-with-tpcc-large.html"
+ ]
+ }
+ ]
+ },
+ {
+ "title": "Support Resources",
+ "urls": [
+ "/${VERSION}/support-resources.html"
+ ]
+ },
+ {
+ "title": "File an Issue",
+ "urls": [
+ "/${VERSION}/file-an-issue.html"
+ ]
+ },
+ {
+ "title": "Automatic CPU Profiler",
+ "urls": [
+ "/${VERSION}/automatic-cpu-profiler.html"
+ ]
+ }
+ ]
+ }
diff --git a/src/current/_includes/v25.1/spatial/ogr2ogr-supported-version.md b/src/current/_includes/v25.1/spatial/ogr2ogr-supported-version.md
new file mode 100644
index 00000000000..ad444257227
--- /dev/null
+++ b/src/current/_includes/v25.1/spatial/ogr2ogr-supported-version.md
@@ -0,0 +1,3 @@
+{{site.data.alerts.callout_info}}
+An `ogr2ogr` version of 3.1.0 or higher is required to generate data that can be imported into CockroachDB.
+{{site.data.alerts.end}}
diff --git a/src/current/_includes/v25.1/spatial/zmcoords.md b/src/current/_includes/v25.1/spatial/zmcoords.md
new file mode 100644
index 00000000000..2eb86de3a89
--- /dev/null
+++ b/src/current/_includes/v25.1/spatial/zmcoords.md
@@ -0,0 +1,27 @@
+ You can also store a `{{page.title}}` with the following additional dimensions:
+
+- A third dimension coordinate `Z` (`{{page.title}}Z`).
+- A measure coordinate `M` (`{{page.title}}M`).
+- Both a third dimension and a measure coordinate (`{{page.title}}ZM`).
+
+The `Z` and `M` dimensions can be accessed or modified using a number of [built-in functions]({% link {{ page.version.version }}/functions-and-operators.md %}#spatial-functions), including:
+
+- `ST_Z`
+- `ST_M`
+- `ST_Affine`
+- `ST_Zmflag`
+- `ST_MakePoint`
+- `ST_MakePointM`
+- `ST_Force3D`
+- `ST_Force3DZ`
+- `ST_Force3DM`
+- `ST_Force4D`
+- `ST_Snap`
+- `ST_SnapToGrid`
+- `ST_RotateZ`
+- `ST_AddMeasure`
+
+Note that CockroachDB's [spatial indexing]({% link {{ page.version.version }}/spatial-indexes.md %}) is still based on the 2D coordinate system. This means that:
+
+- The Z/M dimension is not index accelerated when using spatial predicates.
+- Some spatial functions ignore the Z/M dimension, with transformations discarding the Z/M value.
diff --git a/src/current/_includes/v25.1/sql/add-size-limits-to-indexed-columns.md b/src/current/_includes/v25.1/sql/add-size-limits-to-indexed-columns.md
new file mode 100644
index 00000000000..98b04a36251
--- /dev/null
+++ b/src/current/_includes/v25.1/sql/add-size-limits-to-indexed-columns.md
@@ -0,0 +1,22 @@
+We **strongly recommend** adding size limits to all [indexed columns]({% link {{ page.version.version }}/indexes.md %}), which includes columns in [primary keys]({% link {{ page.version.version }}/primary-key.md %}).
+
+Values exceeding 1 MiB can lead to [storage layer write amplification]({% link {{ page.version.version }}/architecture/storage-layer.md %}#write-amplification) and cause significant performance degradation or even [crashes due to OOMs (out of memory errors)]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#out-of-memory-oom-crash).
+
+To add a size limit using [`CREATE TABLE`]({% link {{ page.version.version }}/create-table.md %}):
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+CREATE TABLE name (first STRING(100), last STRING(100));
+~~~
+
+To add a size limit using [`ALTER TABLE ... ALTER COLUMN`]({% link {{ page.version.version }}/alter-table.md %}#alter-column):
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+SET enable_experimental_alter_column_type_general = true;
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+ALTER TABLE name ALTER first TYPE STRING(99);
+~~~
diff --git a/src/current/_includes/v25.1/sql/begin-transaction-as-of-system-time-example.md b/src/current/_includes/v25.1/sql/begin-transaction-as-of-system-time-example.md
new file mode 100644
index 00000000000..7f2c11dac77
--- /dev/null
+++ b/src/current/_includes/v25.1/sql/begin-transaction-as-of-system-time-example.md
@@ -0,0 +1,19 @@
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> BEGIN AS OF SYSTEM TIME '2019-04-09 18:02:52.0+00:00';
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SELECT * FROM orders;
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> SELECT * FROM products;
+~~~
+
+{% include_cached copy-clipboard.html %}
+~~~ sql
+> COMMIT;
+~~~
diff --git a/src/current/_includes/v25.1/sql/connection-parameters.md b/src/current/_includes/v25.1/sql/connection-parameters.md
new file mode 100644
index 00000000000..b61bb3caf16
--- /dev/null
+++ b/src/current/_includes/v25.1/sql/connection-parameters.md
@@ -0,0 +1,9 @@
+Flag | Description
+-----|------------
+ `--url` | A [connection URL]({% link {{ page.version.version }}/connection-parameters.md %}#connect-using-a-url) to use instead of the other arguments. To convert a connection URL to the syntax that works with your client driver, run [`cockroach convert-url`]({% link {{ page.version.version }}/connection-parameters.md %}#convert-a-url-for-different-drivers).**Env Variable:** `COCKROACH_URL`
**Default:** no URL +`--host` | The server host and port number to connect to. This can be the address of any node in the cluster.
**Env Variable:** `COCKROACH_HOST`
**Default:** `localhost:26257` +`--port`
`-p` | The server port to connect to. Note: The port number can also be specified via `--host`.
**Env Variable:** `COCKROACH_PORT`
**Default:** `26257` +`--user`
`-u` | The [SQL user]({% link {{ page.version.version }}/create-user.md %}) that will own the client session.
**Env Variable:** `COCKROACH_USER`
**Default:** `root` +`--insecure` | Use an insecure connection.
**Env Variable:** `COCKROACH_INSECURE`
**Default:** `false` +`--cert-principal-map` | A comma-separated list of `
**Env Variable:** `COCKROACH_CERTS_DIR`
**Default:** `${HOME}/.cockroach-certs/` \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/covering-index.md b/src/current/_includes/v25.1/sql/covering-index.md new file mode 100644 index 00000000000..4ce5b00cf12 --- /dev/null +++ b/src/current/_includes/v25.1/sql/covering-index.md @@ -0,0 +1 @@ +An index that stores all the columns needed by a query is also known as a _covering index_ for that query. When a query has a covering index, CockroachDB can use that index directly instead of doing an "index join" with the primary index, which is likely to be slower. diff --git a/src/current/_includes/v25.1/sql/crdb-internal-is-not-supported-for-production-use.md b/src/current/_includes/v25.1/sql/crdb-internal-is-not-supported-for-production-use.md new file mode 100644 index 00000000000..475a8804b04 --- /dev/null +++ b/src/current/_includes/v25.1/sql/crdb-internal-is-not-supported-for-production-use.md @@ -0,0 +1 @@ +Many of the tables in the `crdb_internal` system catalog are **not supported for external use in production**. This output is provided **as a debugging aid only**. The output of particular `crdb_internal` facilities may change from patch release to patch release without advance warning. For more information, see [the `crdb_internal` documentation]({% link {{ page.version.version }}/crdb-internal.md %}). diff --git a/src/current/_includes/v25.1/sql/crdb-internal-partitions-example.md b/src/current/_includes/v25.1/sql/crdb-internal-partitions-example.md new file mode 100644 index 00000000000..680b0adf261 --- /dev/null +++ b/src/current/_includes/v25.1/sql/crdb-internal-partitions-example.md @@ -0,0 +1,43 @@ +## Querying partitions programmatically + +The `crdb_internal.partitions` internal table contains information about the partitions in your database. In testing, scripting, and other programmatic environments, we recommend querying this table for partition information instead of using the `SHOW PARTITIONS` statement. For example, to get all `us_west` partitions of in your database, you can run the following query: + +{% include_cached copy-clipboard.html %} +~~~ sql +> SELECT * FROM crdb_internal.partitions WHERE name='us_west'; +~~~ + +~~~ + table_id | index_id | parent_name | name | columns | column_names | list_value | range_value | zone_id | subzone_id ++----------+----------+-------------+---------+---------+--------------+-------------------------------------------------+-------------+---------+------------+ + 53 | 1 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 0 | 0 + 54 | 1 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 54 | 1 + 54 | 2 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 54 | 2 + 55 | 1 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 55 | 1 + 55 | 2 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 55 | 2 + 55 | 3 | NULL | us_west | 1 | vehicle_city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 55 | 3 + 56 | 1 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 56 | 1 + 58 | 1 | NULL | us_west | 1 | city | ('seattle'), ('san francisco'), ('los angeles') | NULL | 58 | 1 +(8 rows) +~~~ + +Other internal tables, like `crdb_internal.tables`, include information that could be useful in conjunction with `crdb_internal.partitions`. + +For example, if you want the output for your partitions to include the name of the table and database, you can perform a join of the two tables: + +{% include_cached copy-clipboard.html %} +~~~ sql +> SELECT + partitions.name AS partition_name, column_names, list_value, tables.name AS table_name, database_name + FROM crdb_internal.partitions JOIN crdb_internal.tables ON partitions.table_id=tables.table_id + WHERE tables.name='users'; +~~~ + +~~~ + partition_name | column_names | list_value | table_name | database_name ++----------------+--------------+-------------------------------------------------+------------+---------------+ + us_west | city | ('seattle'), ('san francisco'), ('los angeles') | users | movr + us_east | city | ('new york'), ('boston'), ('washington dc') | users | movr + europe_west | city | ('amsterdam'), ('paris'), ('rome') | users | movr +(3 rows) +~~~ diff --git a/src/current/_includes/v25.1/sql/crdb-internal-partitions.md b/src/current/_includes/v25.1/sql/crdb-internal-partitions.md new file mode 100644 index 00000000000..11faab704cd --- /dev/null +++ b/src/current/_includes/v25.1/sql/crdb-internal-partitions.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_success}} +In testing, scripting, and other programmatic environments, we recommend querying the `crdb_internal.partitions` internal table for partition information instead of using the `SHOW PARTITIONS` statement. For more information, see [Querying partitions programmatically]({% link {{ page.version.version }}/show-partitions.md %}#querying-partitions-programmatically). +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/sql/cursors-vs-keyset-pagination.md b/src/current/_includes/v25.1/sql/cursors-vs-keyset-pagination.md new file mode 100644 index 00000000000..ba5391b5ace --- /dev/null +++ b/src/current/_includes/v25.1/sql/cursors-vs-keyset-pagination.md @@ -0,0 +1,3 @@ +_Cursors_ are stateful objects that use more database resources than keyset pagination, since each cursor holds open a transaction. However, they are easier to use, and make it easier to get consistent results without having to write complex queries from your application logic. They do not require that the results be returned in a particular order (that is, you don't have to include an `ORDER BY` clause), which makes them more flexible. + +_Keyset pagination_ queries are usually much faster than cursors since they order by indexed columns. However, in order to get that performance they require that you return results in some defined order that can be calculated by your application's queries. Because that ordering involves calculating the start/end point of pages of results based on an indexed key, they require more care to write correctly. diff --git a/src/current/_includes/v25.1/sql/db-terms.md b/src/current/_includes/v25.1/sql/db-terms.md new file mode 100644 index 00000000000..5776ed951e0 --- /dev/null +++ b/src/current/_includes/v25.1/sql/db-terms.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +To avoid confusion with the general term "[database](https://en.wikipedia.org/wiki/Database)", throughout this guide we refer to the logical object as a *database*, to CockroachDB by name, and to a deployment of CockroachDB as a [*cluster*]({% link {{ page.version.version }}/architecture/glossary.md %}#cockroachdb-architecture-terms). +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/sql/dev-schema-change-limits.md b/src/current/_includes/v25.1/sql/dev-schema-change-limits.md new file mode 100644 index 00000000000..e6f10db0bc9 --- /dev/null +++ b/src/current/_includes/v25.1/sql/dev-schema-change-limits.md @@ -0,0 +1,3 @@ +Review the [limitations of online schema changes]({% link {{ page.version.version }}/online-schema-changes.md %}#known-limitations). CockroachDB [doesn't guarantee the atomicity of schema changes within transactions with multiple statements]({% link {{ page.version.version }}/online-schema-changes.md %}#schema-changes-within-transactions). + + Cockroach Labs recommends that you perform schema changes outside explicit transactions. When a database [schema management tool]({% link {{ page.version.version }}/third-party-database-tools.md %}#schema-migration-tools) manages transactions on your behalf, include one schema change operation per transaction. diff --git a/src/current/_includes/v25.1/sql/dev-schema-changes.md b/src/current/_includes/v25.1/sql/dev-schema-changes.md new file mode 100644 index 00000000000..9e42fd08614 --- /dev/null +++ b/src/current/_includes/v25.1/sql/dev-schema-changes.md @@ -0,0 +1 @@ +Use a [database schema migration tool]({% link {{ page.version.version }}/third-party-database-tools.md %}#schema-migration-tools) or the [CockroachDB SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}) instead of a [client library]({% link {{ page.version.version }}/third-party-database-tools.md %}#drivers) to execute [database schema changes](online-schema-changes.html). diff --git a/src/current/_includes/v25.1/sql/disallow-full-table-scans.md b/src/current/_includes/v25.1/sql/disallow-full-table-scans.md new file mode 100644 index 00000000000..57701770065 --- /dev/null +++ b/src/current/_includes/v25.1/sql/disallow-full-table-scans.md @@ -0,0 +1,12 @@ +- At the cluster level, set `disallow_full_table_scans` for some or all users and roles. For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET disallow_full_table_scans = true; + ~~~ + +- At the application level, add `disallow_full_table_scans` to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters). + +If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error. + +If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint and it is not possible to avoid a full scan, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated. \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/drop-role-considerations.md b/src/current/_includes/v25.1/sql/drop-role-considerations.md new file mode 100644 index 00000000000..585a062934b --- /dev/null +++ b/src/current/_includes/v25.1/sql/drop-role-considerations.md @@ -0,0 +1,4 @@ +- The `admin` user/role cannot be dropped, and `root` must always be a member of `admin`. +- A user/role cannot be dropped if it has privileges. Use [`REVOKE`]({% link {{ page.version.version }}/revoke.md %}) to remove privileges. +- Users/roles that [own objects]({% link {{ page.version.version }}/security-reference/authorization.md %}#object-ownership) (such as databases, tables, schemas, and types) cannot be dropped until the [ownership is transferred to another user/role]({% link {{ page.version.version }}/alter-database.md %}#change-a-databases-owner). +- If a user/role is logged in while a [different session]({% link {{ page.version.version }}/show-sessions.md %}) drops that user, CockroachDB checks that the user exists before allowing it to inherit privileges from [the `public` role]({% link {{ page.version.version }}/security-reference/authorization.md %}). In addition, any active [web]({% link {{ page.version.version }}/ui-overview.md %}#authentication) [sessions]({% link {{ page.version.version }}/cockroach-auth-session.md %}) are revoked when a user is dropped. diff --git a/src/current/_includes/v25.1/sql/enable-super-region-primary-region-changes.md b/src/current/_includes/v25.1/sql/enable-super-region-primary-region-changes.md new file mode 100644 index 00000000000..94920f7d481 --- /dev/null +++ b/src/current/_includes/v25.1/sql/enable-super-region-primary-region-changes.md @@ -0,0 +1,23 @@ +By default, you may not change the [primary region]({% link {{ page.version.version }}/alter-database.md %}#set-primary-region) of a [multi-region database]({% link {{ page.version.version }}/multiregion-overview.md %}) when that region is part of a super region. This is a safety setting designed to prevent you from accidentally moving the data for a [regional table]({% link {{ page.version.version }}/regional-tables.md %}) that is meant to be stored in the super region out of that super region, which could break your data domiciling setup. + +If you are sure about what you are doing, you can allow modifying the primary region by setting the `alter_primary_region_super_region_override` [session setting]({% link {{ page.version.version }}/set-vars.md %}) to `'on'`: + +{% include_cached copy-clipboard.html %} +~~~ sql +SET alter_primary_region_super_region_override = 'on'; +~~~ + +~~~ +SET +~~~ + +You can also accomplish this by setting the `sql.defaults.alter_primary_region_super_region_override.enable` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) to `true`: + +{% include_cached copy-clipboard.html %} +~~~ sql +SET CLUSTER SETTING sql.defaults.alter_primary_region_super_region_override.enable = true; +~~~ + +~~~ +SET CLUSTER SETTING +~~~ diff --git a/src/current/_includes/v25.1/sql/enable-super-regions.md b/src/current/_includes/v25.1/sql/enable-super-regions.md new file mode 100644 index 00000000000..0dd7ac26077 --- /dev/null +++ b/src/current/_includes/v25.1/sql/enable-super-regions.md @@ -0,0 +1,21 @@ +To enable super regions, set the `enable_super_regions` [session setting]({% link {{ page.version.version }}/set-vars.md %}) to `'on'`: + +{% include_cached copy-clipboard.html %} +~~~ sql +SET enable_super_regions = 'on'; +~~~ + +~~~ +SET +~~~ + +You can also set the `sql.defaults.super_regions.enabled` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) to `true`: + +{% include_cached copy-clipboard.html %} +~~~ sql +SET CLUSTER SETTING sql.defaults.super_regions.enabled = true; +~~~ + +~~~ +SET CLUSTER SETTING +~~~ diff --git a/src/current/_includes/v25.1/sql/export-csv-tsv.md b/src/current/_includes/v25.1/sql/export-csv-tsv.md new file mode 100644 index 00000000000..ea1e69968a0 --- /dev/null +++ b/src/current/_includes/v25.1/sql/export-csv-tsv.md @@ -0,0 +1,11 @@ +[`IMPORT INTO`](import-into.html) requires that you export one file per table with the following attributes: + +- Files must be in valid [CSV](https://tools.ietf.org/html/rfc4180) (comma-separated values) or [TSV](https://www.iana.org/assignments/media-types/text/tab-separated-values) (tab-separated values) format. +- The delimiter must be a single character. Use the [`delimiter` option](import-into.html#import-options) to set a character other than a comma (such as a tab, for TSV format). +- Files must be UTF-8 encoded. +- If one of the following characters appears in a field, the field must be enclosed by double quotes: + - Delimiter (`,` by default). + - Double quote (`"`). Because the field will be enclosed by double quotes, escape a double quote inside a field by preceding it with another double quote. For example: `"aaa","b""bb","ccc"`. + - Newline (`\n`). + - Carriage return (`\r`). +- If a column is of type [`BYTES`](bytes.html), it can either be a valid UTF-8 string or a [hex-encoded byte literal](sql-constants.html#hexadecimal-encoded-byte-array-literals) beginning with `\x`. For example, a field whose value should be the bytes `1`, `2` would be written as `\x0102`. \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/function-special-forms.md b/src/current/_includes/v25.1/sql/function-special-forms.md new file mode 100644 index 00000000000..b9ac987444a --- /dev/null +++ b/src/current/_includes/v25.1/sql/function-special-forms.md @@ -0,0 +1,29 @@ +| Special form | Equivalent to | +|-----------------------------------------------------------|---------------------------------------------| +| `AT TIME ZONE` | `timezone()` | +| `CURRENT_CATALOG` | `current_catalog()` | +| `COLLATION FOR` | `pg_collation_for()` | +| `CURRENT_DATE` | `current_date()` | +| `CURRENT_ROLE` | `current_user()` | +| `CURRENT_SCHEMA` | `current_schema()` | +| `CURRENT_TIMESTAMP` | `current_timestamp()` | +| `CURRENT_TIME` | `current_time()` | +| `CURRENT_USER` | `current_user()` | +| `EXTRACT(
+
+In **iTerm2**:
+
+1. Navigate to "Preferences", then "Profiles", then "Keys".
+1. Select the radio button "Esc+" for the behavior of the Left Option Key.
+
+
+
diff --git a/src/current/_includes/v25.1/sql/mixed-isolation-levels.md b/src/current/_includes/v25.1/sql/mixed-isolation-levels.md
new file mode 100644
index 00000000000..440e29f24bd
--- /dev/null
+++ b/src/current/_includes/v25.1/sql/mixed-isolation-levels.md
@@ -0,0 +1,12 @@
+{% if page.name == "transactions.md" %}### Mixed isolation levels{% else if page.name == "transaction-layer.md" %}#### Mixed isolation levels{% endif %}
+
+Regardless of the isolation levels of other transactions, transactions behave according to their respective isolation levels: Statements in `SERIALIZABLE` transactions see data that committed before the transaction began, whereas statements in `READ COMMITTED` transactions see data that committed before each **statement** began. Therefore:
+
+- If a `READ COMMITTED` transaction `R` commits before a `SERIALIZABLE` transaction `S`, every statement in `S` will observe all writes from `R`. Otherwise, `S` will not observe any writes from `R`.
+- If a `SERIALIZABLE` transaction `S` commits before a `READ COMMITTED` transaction `R`, every **subsequent** statement in `R` will observe all writes from `S`. Otherwise, `R` will not observe any writes from `S`.
+
+However, there is one difference in how `SERIALIZABLE` writes affect non-locking reads: While writes in a `SERIALIZABLE` transaction can block reads in concurrent `SERIALIZABLE` transactions, they will **not** block reads in concurrent `READ COMMITTED` transactions. Writes in a `READ COMMITTED` transaction will never block reads in concurrent transactions, regardless of their isolation levels. Therefore:
+
+- If a `READ COMMITTED` transaction `R` writes but does not commit before a `SERIALIZABLE` transaction `S`, no statement in `S` will observe or be blocked by any uncommitted writes from `R`.
+- If a `SERIALIZABLE` transaction `S` writes but does not commit before a `READ COMMITTED` transaction `R`, no statement in `R` will observe or be blocked by any uncommitted writes from `S`.
+- If a `SERIALIZABLE` transaction `S1` writes but does not commit before a `SERIALIZABLE` transaction `S2`, the first statement in `S2` that would observe an unwritten row from `S1` will be blocked until `S1` commits or aborts.
\ No newline at end of file
diff --git a/src/current/_includes/v25.1/sql/movr-start-nodes.md b/src/current/_includes/v25.1/sql/movr-start-nodes.md
new file mode 100644
index 00000000000..3af9ecfbf2b
--- /dev/null
+++ b/src/current/_includes/v25.1/sql/movr-start-nodes.md
@@ -0,0 +1,6 @@
+Run [`cockroach demo`]({% link {{ page.version.version }}/cockroach-demo.md %}) with the [`--nodes`]({% link {{ page.version.version }}/cockroach-demo.md %}#flags) and [`--demo-locality`]({% link {{ page.version.version }}/cockroach-demo.md %}#flags) flags This command opens an interactive SQL shell to a temporary, multi-node in-memory cluster with the `movr` database preloaded and set as the [current database]({% link {{ page.version.version }}/sql-name-resolution.md %}#current-database).
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ cockroach demo --nodes=3 --demo-locality=region=us-east1:region=us-central1:region=us-west1
+ ~~~
diff --git a/src/current/_includes/v25.1/sql/movr-start.md b/src/current/_includes/v25.1/sql/movr-start.md
new file mode 100644
index 00000000000..2c1bb50abf2
--- /dev/null
+++ b/src/current/_includes/v25.1/sql/movr-start.md
@@ -0,0 +1,62 @@
+- Run [`cockroach demo`]({% link {{ page.version.version }}/cockroach-demo.md %}) to start a temporary, in-memory cluster with the `movr` dataset preloaded:
+
+ {% include_cached copy-clipboard.html %}
+ ~~~ shell
+ $ cockroach demo
+ ~~~
+
+- Load the `movr` dataset into a persistent local cluster and open an interactive SQL shell:
+ 1. Start a [secure]({% link {{ page.version.version }}/secure-a-cluster.md %}) or [insecure]({% link {{ page.version.version }}/start-a-local-cluster.md %}) local cluster.
+ 1. Use [`cockroach workload`]({% link {{ page.version.version }}/cockroach-workload.md %}) to load the `movr` dataset:
+
+
+
+
+
+
+ By default, the role option is set to `NOCANCELQUERY` for all non-`admin` roles. +`CONTROLCHANGEFEED`/`NOCONTROLCHANGEFEED` | **Deprecated in v23.1: Use the `CHANGEFEED` [privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges).** Allow or disallow a role to run [`CREATE CHANGEFEED`]({% link {{ page.version.version }}/create-changefeed.md %}) on tables they have `SELECT` privileges on.
By default, the role option is set to `NOCONTROLCHANGEFEED` for all non-`admin` roles. +`CONTROLJOB`/`NOCONTROLJOB` | Allow or disallow a role to [pause]({% link {{ page.version.version }}/pause-job.md %}), [resume]({% link {{ page.version.version }}/resume-job.md %}), and [cancel]({% link {{ page.version.version }}/cancel-job.md %}) jobs. Non-`admin` roles cannot control jobs created by `admin` roles.
By default, the role option is set to `NOCONTROLJOB` for all non-`admin` roles. +`CREATEDB`/`NOCREATEDB` | Allow or disallow a role to [create]({% link {{ page.version.version }}/create-database.md %}) or [rename]({% link {{ page.version.version }}/alter-database.md %}#rename-to) a database. The role is assigned as the owner of the database.
By default, the role option is set to `NOCREATEDB` for all non-`admin` roles. +`CREATELOGIN`/`NOCREATELOGIN` | Allow or disallow a role to manage authentication using the `WITH PASSWORD`, `VALID UNTIL`, and `LOGIN/NOLOGIN` role options.
By default, the role option is set to `NOCREATELOGIN` for all non-`admin` roles. +`CREATEROLE`/`NOCREATEROLE` | Allow or disallow the new role to [create]({% link {{ page.version.version }}/create-role.md %}), alter, and [drop]({% link {{ page.version.version }}/drop-role.md %}) other non-`admin` roles.
By default, the role option is set to `NOCREATEROLE` for all non-`admin` roles. +`LOGIN`/`NOLOGIN` | Allow or disallow a role to log in with one of the [client authentication methods]({% link {{ page.version.version }}/authentication.md %}#client-authentication). Setting the role option to `NOLOGIN` prevents the role from logging in using any authentication method. +`MODIFYCLUSTERSETTING`/`NOMODIFYCLUSTERSETTING` | Allow or disallow a role to modify the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}) with the `sql.defaults` prefix.
By default, the role option is set to `NOMODIFYCLUSTERSETTING` for all non-`admin` roles. +`PASSWORD password`/`PASSWORD NULL` | The credential the role uses to [authenticate their access to a secure cluster]({% link {{ page.version.version }}/authentication.md %}#client-authentication). A password should be entered as a [string literal]({% link {{ page.version.version }}/sql-constants.md %}#string-literals). For compatibility with PostgreSQL, a password can also be entered as an identifier.
To prevent a role from using [password authentication]({% link {{ page.version.version }}/authentication.md %}#client-authentication) and to mandate [certificate-based client authentication]({% link {{ page.version.version }}/authentication.md %}#client-authentication), [set the password as `NULL`]({% link {{ page.version.version }}/create-role.md %}#prevent-a-role-from-using-password-authentication). +`SQLLOGIN`/`NOSQLLOGIN` | **Deprecated in v22.2: Use the `NOSQLLOGIN` [system privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges).** Allow or disallow a role to log in using the SQL CLI with one of the [client authentication methods]({% link {{ page.version.version }}/authentication.md %}#client-authentication). The role option to `NOSQLLOGIN` prevents the role from logging in using the SQL CLI with any authentication method while retaining the ability to log in to DB Console. It is possible to have both `NOSQLLOGIN` and `LOGIN` set for a role and `NOSQLLOGIN` takes precedence on restrictions.
Without any role options all login behavior is permitted. +`VALID UNTIL` | The date and time (in the [`timestamp`]({% link {{ page.version.version }}/timestamp.md %}) format) after which the [password](#parameters) is not valid. +`VIEWACTIVITY`/`NOVIEWACTIVITY` | **Deprecated in v22.2: Use the `VIEWACTIVITY` [system privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges).** Allow or disallow a role to see other roles' [queries]({% link {{ page.version.version }}/show-statements.md %}) and [sessions]({% link {{ page.version.version }}/show-sessions.md %}) using `SHOW STATEMENTS`, `SHOW SESSIONS`, and the [**Statements**](ui-statements-page.html) and [**Transactions**](ui-transactions-page.html) pages in the DB Console. `VIEWACTIVITY` also permits visibility of node hostnames and IP addresses in the DB Console. With `NOVIEWACTIVITY`, the `SHOW` commands show only the role's own data, and DB Console pages redact node hostnames and IP addresses.
By default, the role option is set to `NOVIEWACTIVITY` for all non-`admin` roles. +`VIEWCLUSTERSETTING` / `NOVIEWCLUSTERSETTING` | **Deprecated in v22.2: Use the `VIEWCLUSTERSETTING` [system privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges).** Allow or disallow a role to view the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}) with `SHOW CLUSTER SETTING` or to access the [**Cluster Settings**]({% link {{ page.version.version }}/ui-debug-pages.md %}) page in the DB Console.
By default, the role option is set to `NOVIEWCLUSTERSETTING` for all non-`admin` roles. +`VIEWACTIVITYREDACTED`/`NOVIEWACTIVITYREDACTED` | **Deprecated in v22.2: Use the `VIEWACTIVITYREDACTED` [system privilege]({% link {{ page.version.version }}/security-reference/authorization.md %}#supported-privileges).** Allow or disallow a role to see other roles' queries and sessions using `SHOW STATEMENTS`, `SHOW SESSIONS`, and the Statements and Transactions pages in the DB Console. With `VIEWACTIVITYREDACTED`, a user will not have access to the usage of statements diagnostics bundle (which can contain PII information) in the DB Console, and will not be able to list queries containing [constants]({% link {{ page.version.version }}/sql-constants.md %}) for other users when using the `listSessions` endpoint through the [Cluster API]({% link {{ page.version.version }}/cluster-api.md %}). It is possible to have both `VIEWACTIVITY` and `VIEWACTIVITYREDACTED`, and `VIEWACTIVITYREDACTED` takes precedence on restrictions. If the user has `VIEWACTIVITY` but doesn't have `VIEWACTIVITYREDACTED`, they will be able to see DB Console pages and have access to the statements diagnostics bundle.
By default, the role option is set to `NOVIEWACTIVITYREDACTED` for all non-`admin` roles. diff --git a/src/current/_includes/v25.1/sql/role-subject-option.md b/src/current/_includes/v25.1/sql/role-subject-option.md new file mode 100644 index 00000000000..c8a71e9af1b --- /dev/null +++ b/src/current/_includes/v25.1/sql/role-subject-option.md @@ -0,0 +1 @@ +You can associate an [X.509](https://en.wikipedia.org/wiki/X.509) certificate's Subject with a [role]({% link {{ page.version.version }}/security-reference/authorization.md %}#roles) as shown below. Note that the Subject fields in the certificate have to be an exact match with what you pass in via the SQL statement. By exact match, we mean that the order of attributes passed in via the SQL statement must match the order of attributes in the certificate. diff --git a/src/current/_includes/v25.1/sql/row-level-ttl-prefer-ttl-expiration-expressions.md b/src/current/_includes/v25.1/sql/row-level-ttl-prefer-ttl-expiration-expressions.md new file mode 100644 index 00000000000..a75ed05e6c6 --- /dev/null +++ b/src/current/_includes/v25.1/sql/row-level-ttl-prefer-ttl-expiration-expressions.md @@ -0,0 +1,5 @@ +Most users should use `ttl_expiration_expression` instead of `ttl_expire_after` for the following reasons: + +- If you add `ttl_expire_after` to an existing table, it **will cause a full table rewrite, which can affect performance**. Specifically, it will result in a [schema change]({% link {{ page.version.version }}/online-schema-changes.md %}) that (1) creates a new [hidden column]({% link {{page.version.version}}/show-create.md%}#show-the-create-table-statement-for-a-table-with-a-hidden-column) `crdb_internal_expiration` for all rows, and (2) backfills the value of that new column to `now()` + `ttl_expire_after`. +- You cannot use `ttl_expire_after` with an existing [`TIMESTAMPTZ`]({% link {{ page.version.version }}/timestamp.md %}) column. +- If you use `ttl_expiration_expression`, you can use an existing [`TIMESTAMPTZ`]({% link {{ page.version.version }}/timestamp.md %}) column called e.g. `updated_at`. diff --git a/src/current/_includes/v25.1/sql/row-level-ttl.md b/src/current/_includes/v25.1/sql/row-level-ttl.md new file mode 100644 index 00000000000..d10ea9b8e87 --- /dev/null +++ b/src/current/_includes/v25.1/sql/row-level-ttl.md @@ -0,0 +1 @@ +CockroachDB has support for Time to Live ("TTL") expiration on table rows, also known as _Row-Level TTL_. Row-Level TTL is a mechanism whereby rows from a table are considered "expired" and can be automatically deleted once those rows have been stored longer than a specified expiration time. diff --git a/src/current/_includes/v25.1/sql/savepoint-ddl-rollbacks.md b/src/current/_includes/v25.1/sql/savepoint-ddl-rollbacks.md new file mode 100644 index 00000000000..57da82ae775 --- /dev/null +++ b/src/current/_includes/v25.1/sql/savepoint-ddl-rollbacks.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_danger}} +Rollbacks to savepoints over [DDL](https://en.wikipedia.org/wiki/Data_definition_language) statements are only supported if you're rolling back to a savepoint created at the beginning of the transaction. +{{site.data.alerts.end}} diff --git a/src/current/_includes/v25.1/sql/savepoints-and-high-priority-transactions.md b/src/current/_includes/v25.1/sql/savepoints-and-high-priority-transactions.md new file mode 100644 index 00000000000..c6de489e641 --- /dev/null +++ b/src/current/_includes/v25.1/sql/savepoints-and-high-priority-transactions.md @@ -0,0 +1 @@ +[`ROLLBACK TO SAVEPOINT`]({% link {{ page.version.version }}/rollback-transaction.md %}#rollback-a-nested-transaction) (for either regular savepoints or "restart savepoints" defined with `cockroach_restart`) causes a "feature not supported" error after a DDL statement in a [`HIGH PRIORITY` transaction]({% link {{ page.version.version }}/transactions.md %}#transaction-priorities), in order to avoid a transaction deadlock. For more information, see GitHub issue [#46414](https://www.github.com/cockroachdb/cockroach/issues/46414). diff --git a/src/current/_includes/v25.1/sql/savepoints-and-row-locks.md b/src/current/_includes/v25.1/sql/savepoints-and-row-locks.md new file mode 100644 index 00000000000..39568092558 --- /dev/null +++ b/src/current/_includes/v25.1/sql/savepoints-and-row-locks.md @@ -0,0 +1,12 @@ +CockroachDB supports exclusive row locks. + +- In PostgreSQL, row locks are released/cancelled upon [`ROLLBACK TO SAVEPOINT`][rts]. +- In CockroachDB, row locks are preserved upon [`ROLLBACK TO SAVEPOINT`][rts]. + +This is an architectural difference that may or may not be lifted in a later CockroachDB version. + +The code of client applications that rely on row locks must be reviewed and possibly modified to account for this difference. In particular, if an application is relying on [`ROLLBACK TO SAVEPOINT`][rts] to release row locks and allow a concurrent transaction touching the same rows to proceed, this behavior will not work with CockroachDB. + +{% comment %} Reference Links {% endcomment %} + +[rts]: rollback-transaction.html diff --git a/src/current/_includes/v25.1/sql/schema-changes.md b/src/current/_includes/v25.1/sql/schema-changes.md new file mode 100644 index 00000000000..c61e9c9a046 --- /dev/null +++ b/src/current/_includes/v25.1/sql/schema-changes.md @@ -0,0 +1 @@ +- Schema changes through [`ALTER TABLE`]({% link {{ page.version.version }}/alter-table.md %}), [`DROP DATABASE`]({% link {{ page.version.version }}/drop-database.md %}), [`DROP TABLE`]({% link {{ page.version.version }}/drop-table.md %}), and [`TRUNCATE`](truncate.html) \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/schema-terms.md b/src/current/_includes/v25.1/sql/schema-terms.md new file mode 100644 index 00000000000..d066d5d979b --- /dev/null +++ b/src/current/_includes/v25.1/sql/schema-terms.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +To avoid confusion with the general term "[schema](https://wiktionary.org/wiki/schema)", in this guide we refer to the logical object as a *user-defined schema*, and to the relationship structure of logical objects in a cluster as a *database schema*. +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/select-for-update-example-partial.md b/src/current/_includes/v25.1/sql/select-for-update-example-partial.md new file mode 100644 index 00000000000..62a2bf9c066 --- /dev/null +++ b/src/current/_includes/v25.1/sql/select-for-update-example-partial.md @@ -0,0 +1,50 @@ +This example assumes you are running a [local unsecured cluster]({% link {{ page.version.version }}/start-a-local-cluster.md %}). + +First, connect to the running cluster (call this Terminal 1): + +{% include_cached copy-clipboard.html %} +~~~ shell +cockroach sql --insecure +~~~ + +Next, create a table and insert some rows: + +{% include_cached copy-clipboard.html %} +~~~ sql +CREATE TABLE kv (k INT PRIMARY KEY, v INT); +INSERT INTO kv (k, v) VALUES (1, 5), (2, 10), (3, 15); +~~~ + +Next, we'll start a [transaction]({% link {{ page.version.version }}/transactions.md %}) and lock the row we want to operate on: + +{% include_cached copy-clipboard.html %} +~~~ sql +BEGIN; +SELECT * FROM kv WHERE k = 1 FOR UPDATE; +~~~ + +Press **Enter** twice in the [SQL client]({% link {{ page.version.version }}/cockroach-sql.md %}) to send the statements to be evaluated. This will result in the following output: + +~~~ + k | v ++---+----+ + 1 | 5 +(1 row) +~~~ + +Now open another terminal and connect to the database from a second client (call this Terminal 2): + +{% include_cached copy-clipboard.html %} +~~~ shell +cockroach sql --insecure +~~~ + +From Terminal 2, start a transaction and try to lock the same row for updates that is already being accessed by the transaction we opened in Terminal 1: + +{% include_cached copy-clipboard.html %} +~~~ sql +BEGIN; +SELECT * FROM kv WHERE k = 1 FOR UPDATE; +~~~ + +Press **Enter** twice to send the statements to be evaluated. Because Terminal 1 has already locked this row, the `SELECT FOR UPDATE` statement from Terminal 2 will appear to "wait". diff --git a/src/current/_includes/v25.1/sql/select-for-update-overview.md b/src/current/_includes/v25.1/sql/select-for-update-overview.md new file mode 100644 index 00000000000..812b9c0eb17 --- /dev/null +++ b/src/current/_includes/v25.1/sql/select-for-update-overview.md @@ -0,0 +1,22 @@ +{% if page.name != "select-for-update.md" %}`SELECT ... FOR UPDATE` exclusively locks the rows returned by a [selection query][selection], such that other transactions trying to access those rows must wait for the transaction that locked the rows to commit or rollback.{% endif %} + +`SELECT ... FOR UPDATE` can be used to: + +- Strengthen the isolation of a [`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %}) transaction. If you need to read and later update a row within a transaction, use `SELECT ... FOR UPDATE` to acquire an exclusive lock on the row. This guarantees data integrity between the transaction's read and write operations. For details, see [Locking reads]({% link {{ page.version.version }}/read-committed.md %}#locking-reads). + +- Order [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) transactions by controlling concurrent access to one or more rows of a table. These other transactions are placed into a queue based on when they tried to read the values of the locked rows. + + Because this queueing happens during the read operation, the [thrashing](https://wikipedia.org/wiki/Thrashing_(computer_science)) that would otherwise occur if multiple concurrently executing transactions attempt to `SELECT` the same data and then `UPDATE` the results of that selection is prevented. By preventing thrashing, `SELECT ... FOR UPDATE` also prevents [transaction retries][retries] that would otherwise occur due to [contention]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#transaction-contention). + + As a result, using `SELECT ... FOR UPDATE` leads to increased throughput and decreased tail latency for contended operations. + +Note that using `SELECT ... FOR UPDATE` does not completely eliminate the chance of [serialization errors]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}). These errors can also arise due to [time uncertainty]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#transaction-conflicts). To eliminate the need for application-level retry logic, in addition to `SELECT FOR UPDATE` your application also needs to use a [driver that implements automatic retry handling]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}#client-side-retry-handling). + +{{site.data.alerts.callout_info}} +By default, CockroachDB uses the `SELECT ... FOR UPDATE` locking mechanism during the initial row scan performed in [`UPDATE`]({% link {{ page.version.version }}/update.md %}) and [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statement execution. To turn off implicit `SELECT ... FOR UPDATE` locking for `UPDATE` and `UPSERT` statements, set the `enable_implicit_select_for_update` [session variable]({% link {{ page.version.version }}/set-vars.md %}) to `false`. +{{site.data.alerts.end}} + +{% comment %} Reference Links {% endcomment %} + +[retries]: transactions.html#transaction-retries +[selection]: selection-queries.html diff --git a/src/current/_includes/v25.1/sql/select-lock-strengths.md b/src/current/_includes/v25.1/sql/select-lock-strengths.md new file mode 100644 index 00000000000..fc0b2cd590e --- /dev/null +++ b/src/current/_includes/v25.1/sql/select-lock-strengths.md @@ -0,0 +1,5 @@ +- `SELECT FOR UPDATE` obtains an *exclusive lock* on each qualifying row, blocking concurrent writes and locking reads on the row. Only one transaction can hold an exclusive lock on a row at a time, and only the transaction holding the exclusive lock can write to the row. {% if page.name == "read-committed.md" %}For an example, see [Reserve rows for updates using exclusive locks](#reserve-rows-for-updates-using-exclusive-locks).{% endif %} + +- `SELECT FOR SHARE` obtains a *shared lock* on each qualifying row, blocking concurrent writes and **exclusive** locking reads on the row. Multiple transactions can hold a shared lock on a row at the same time. When multiple transactions hold a shared lock on a row, none can write to the row. A shared lock grants transactions mutual read-only access to a row, and ensures that they read the latest version of the row. {% if page.name == "read-committed.md" %}For an example, see [Reserve values using shared locks](#reserve-row-values-using-shared-locks).{% endif %} + +When a `SELECT FOR UPDATE` or `SELECT FOR SHARE` read is issued on a row, only the latest version of the row is returned to the client. Under {% if page.name == "read-committed.md" %}`READ COMMITTED`{% else %}[`READ COMMITTED`]({% link {{ page.version.version }}/read-committed.md %}){% endif %} isolation, neither statement will block concurrent, non-locking reads. \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/serializable-tutorial.md b/src/current/_includes/v25.1/sql/serializable-tutorial.md new file mode 100644 index 00000000000..2a03b091b58 --- /dev/null +++ b/src/current/_includes/v25.1/sql/serializable-tutorial.md @@ -0,0 +1,3 @@ +{{site.data.alerts.callout_info}} +This tutorial assumes you are running under [`SERIALIZABLE`]({% link {{ page.version.version }}/demo-serializable.md %}) isolation, which requires client-side retry handling for [serialization errors]({% link {{ page.version.version }}/transaction-retry-error-reference.md %}). +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v25.1/sql/server-side-connection-limit.md b/src/current/_includes/v25.1/sql/server-side-connection-limit.md new file mode 100644 index 00000000000..62300b45619 --- /dev/null +++ b/src/current/_includes/v25.1/sql/server-side-connection-limit.md @@ -0,0 +1 @@ +To control the maximum number of non-superuser ([`root`]({% link {{ page.version.version }}/security-reference/authorization.md %}#root-user) user or other [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role)) connections a [gateway node]({% link {{ page.version.version }}/architecture/sql-layer.md %}#gateway-node) can have open at one time, use the `server.max_connections_per_gateway` [cluster setting](cluster-settings.html). If a new non-superuser connection would exceed this limit, the error message `"sorry, too many clients already"` is returned, along with error code `53300`. diff --git a/src/current/_includes/v25.1/sql/set-transaction-as-of-system-time-example.md b/src/current/_includes/v25.1/sql/set-transaction-as-of-system-time-example.md new file mode 100644 index 00000000000..8e758f1c303 --- /dev/null +++ b/src/current/_includes/v25.1/sql/set-transaction-as-of-system-time-example.md @@ -0,0 +1,24 @@ +{% include_cached copy-clipboard.html %} +~~~ sql +> BEGIN; +~~~ + +{% include_cached copy-clipboard.html %} +~~~ sql +> SET TRANSACTION AS OF SYSTEM TIME '2019-04-09 18:02:52.0+00:00'; +~~~ + +{% include_cached copy-clipboard.html %} +~~~ sql +> SELECT * FROM orders; +~~~ + +{% include_cached copy-clipboard.html %} +~~~ sql +> SELECT * FROM products; +~~~ + +{% include_cached copy-clipboard.html %} +~~~ sql +> COMMIT; +~~~ diff --git a/src/current/_includes/v25.1/sql/shell-commands.md b/src/current/_includes/v25.1/sql/shell-commands.md new file mode 100644 index 00000000000..7586108f3d1 --- /dev/null +++ b/src/current/_includes/v25.1/sql/shell-commands.md @@ -0,0 +1,54 @@ +The following commands can be used within the interactive SQL shell: + +Command | Usage +--------|------------ +`\?`,`help` | View this help within the shell. +`\q`,`quit`,`exit`,`ctrl-d` | Exit the shell.
When no text follows the prompt, `ctrl-c` exits the shell as well; otherwise, `ctrl-c` clears the line. +`\!` | Run an external command and print its results to `stdout`. [See an example]({% link {{ page.version.version }}/cockroach-sql.md %}#run-external-commands-from-the-sql-shell). +
\| | Run the output of an external command as SQL statements. [See an example]({% link {{ page.version.version }}/cockroach-sql.md %}#run-external-commands-from-the-sql-shell).
+`\set