diff --git a/src/current/_includes/v22.2/misc/session-vars.md b/src/current/_includes/v22.2/misc/session-vars.md index c3e9e861d76..bd4fc97d2c9 100644 --- a/src/current/_includes/v22.2/misc/session-vars.md +++ b/src/current/_includes/v22.2/misc/session-vars.md @@ -14,7 +14,7 @@ | `default_transaction_quality_of_service` | The default transaction quality of service for the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level](admission-control.html#set-quality-of-service-level-for-a-session). | `regular` | Yes | Yes | | `default_transaction_read_only` | The default transaction access mode for the current session.
If set to `on`, only read operations are allowed in transactions in the current session; if set to `off`, both read and write operations are allowed. See [`SET TRANSACTION`](set-transaction.html) for more details. | `off` | Yes | Yes | | `default_transaction_use_follower_reads` | If set to on, all read-only transactions use [`AS OF SYSTEM TIME follower_read_timestamp()`](as-of-system-time.html) to allow the transaction to use follower reads.
If set to `off`, read-only transactions will only use follower reads if an `AS OF SYSTEM TIME` clause is specified in the statement, with an interval of at least 4.8 seconds. | `off` | Yes | Yes | -| `disallow_full_table_scans` | If set to `on`, all queries that have planned a full table or full secondary index scan will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | +| `disallow_full_table_scans` | If set to `on`, queries on "large" tables with a row count greater than [`large_full_scan_rows`](#large-full-scan-rows) will not use full table or index scans. If no other query plan is possible, queries will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | | `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes | | `enable_auto_rehoming` | When enabled, the [home regions](alter-table.html#crdb_region) of rows in [`REGIONAL BY ROW`](alter-table.html#set-the-table-locality-to-regional-by-row) tables are automatically set to the region of the [gateway node](ui-sessions-page.html#session-details-gateway-node) from which any [`UPDATE`](update.html) or [`UPSERT`](upsert.html) statements that operate on those rows originate. | `off` | Yes | Yes | | `enable_implicit_select_for_update` | Indicates whether [`UPDATE`](update.html) and [`UPSERT`](upsert.html) statements acquire locks using the `FOR UPDATE` locking mode during their initial row scan, which improves performance for contended workloads.

For more information about how `FOR UPDATE` locking works, see the documentation for [`SELECT FOR UPDATE`](select-for-update.html). | `on` | Yes | Yes | @@ -31,7 +31,7 @@ | `inject_retry_errors_enabled` | If `true`, any statement executed inside of an explicit transaction (with the exception of [`SET`](set-vars.html) statements) will return a transaction retry error. If the client retries the transaction using the special [`cockroach_restart SAVEPOINT` name](savepoint.html#savepoints-for-client-side-transaction-retries), after the 3rd retry error, the transaction will proceed as normal. Otherwise, the errors will continue until `inject_retry_errors_enabled` is set to `false`. For more details, see [Testing transaction retry logic](transactions.html#testing-transaction-retry-logic). | `false` | Yes | Yes | | `intervalstyle` | The input string format for [`INTERVAL`](interval.html) values. Accepted values include `postgres`, `iso_8601`, and `sql_standard`. | The value set by the `sql.defaults.intervalstyle` [cluster setting](cluster-settings.html) (`postgres`, by default). | Yes | Yes | | `is_superuser` | If `on` or `true`, the current user is a member of the [`admin` role](security-reference/authorization.html#admin-role). | User-dependent | No | Yes | -| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` rejects full table or index scans of "large" tables. The default value is `1000`. To reject all full table or index scans, set to `0`. | User-dependent | No | Yes | +| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` prevents queries from performing full table or index scans on those tables. The default value is `1000`. To prevent all full table or index scans, set to `0`. | User-dependent | No | Yes | | `locality` | The location of the node.

For more information, see [Locality](cockroach-start.html#locality). | Node-dependent | No | Yes | | `lock_timeout` | The amount of time a query can spend acquiring or waiting for a single [row-level lock](architecture/transaction-layer.html#concurrency-control).
In CockroachDB, unlike in PostgreSQL, non-locking reads wait for conflicting locks to be released. As a result, the `lock_timeout` configuration applies to writes, and to locking and non-locking reads in read-write and read-only transactions.
If `lock_timeout = 0`, queries do not timeout due to lock acquisitions. | The value set by the `sql.defaults.lock_timeout` [cluster setting](cluster-settings.html) (`0`, by default) | Yes | Yes | | `node_id` | The ID of the node currently connected to.

This variable is particularly useful for verifying load balanced connections. | Node-dependent | No | Yes | diff --git a/src/current/_includes/v23.1/misc/session-vars.md b/src/current/_includes/v23.1/misc/session-vars.md index 554d58e9e17..fc505285062 100644 --- a/src/current/_includes/v23.1/misc/session-vars.md +++ b/src/current/_includes/v23.1/misc/session-vars.md @@ -15,7 +15,7 @@ | `default_transaction_quality_of_service` | The default transaction quality of service for the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#set-quality-of-service-level-for-a-session). | `regular` | Yes | Yes | | `default_transaction_read_only` | The default transaction access mode for the current session.
If set to `on`, only read operations are allowed in transactions in the current session; if set to `off`, both read and write operations are allowed. See [`SET TRANSACTION`]({% link {{ page.version.version }}/set-transaction.md %}) for more details. | `off` | Yes | Yes | | `default_transaction_use_follower_reads` | If set to on, all read-only transactions use [`AS OF SYSTEM TIME follower_read_timestamp()`]({% link {{ page.version.version }}/as-of-system-time.md %}) to allow the transaction to use follower reads.
If set to `off`, read-only transactions will only use follower reads if an `AS OF SYSTEM TIME` clause is specified in the statement, with an interval of at least 4.8 seconds. | `off` | Yes | Yes | -| `disallow_full_table_scans` | If set to `on`, all queries that have planned a full table or full secondary index scan will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | +| `disallow_full_table_scans` | If set to `on`, queries on "large" tables with a row count greater than [`large_full_scan_rows`](#large-full-scan-rows) will not use full table or index scans. If no other query plan is possible, queries will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | | `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes | | `enable_auto_rehoming` | When enabled, the [home regions]({% link {{ page.version.version }}/alter-table.md %}#crdb_region) of rows in [`REGIONAL BY ROW`]({% link {{ page.version.version }}/alter-table.md %}#set-the-table-locality-to-regional-by-row) tables are automatically set to the region of the [gateway node]({% link {{ page.version.version }}/ui-sessions-page.md %}#session-details-gateway-node) from which any [`UPDATE`]({% link {{ page.version.version }}/update.md %}) or [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements that operate on those rows originate. | `off` | Yes | Yes | | `enable_implicit_select_for_update` | Indicates whether [`UPDATE`]({% link {{ page.version.version }}/update.md %}) and [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements acquire locks using the `FOR UPDATE` locking mode during their initial row scan, which improves performance for contended workloads.

For more information about how `FOR UPDATE` locking works, see the documentation for [`SELECT FOR UPDATE`]({% link {{ page.version.version }}/select-for-update.md %}). | `on` | Yes | Yes | @@ -35,7 +35,7 @@ | `inject_retry_errors_enabled` | If `true`, any statement executed inside of an explicit transaction (with the exception of [`SET`]({% link {{ page.version.version }}/set-vars.md %}) statements) will return a transaction retry error. If the client retries the transaction using the special [`cockroach_restart SAVEPOINT` name]({% link {{ page.version.version }}/savepoint.md %}#savepoints-for-client-side-transaction-retries), after the 3rd retry error, the transaction will proceed as normal. Otherwise, the errors will continue until `inject_retry_errors_enabled` is set to `false`. For more details, see [Test transaction retry logic]({% link {{ page.version.version }}/transaction-retry-error-example.md %}#test-transaction-retry-logic). | `false` | Yes | Yes | | `intervalstyle` | The input string format for [`INTERVAL`]({% link {{ page.version.version }}/interval.md %}) values. Accepted values include `postgres`, `iso_8601`, and `sql_standard`. | The value set by the `sql.defaults.intervalstyle` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`postgres`, by default). | Yes | Yes | | `is_superuser` | If `on` or `true`, the current user is a member of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role). | User-dependent | No | Yes | -| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` rejects full table or index scans of "large" tables. The default value is `1000`. To reject all full table or index scans, set to `0`. | User-dependent | No | Yes | +| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` prevents queries from performing full table or index scans on those tables. The default value is `1000`. To prevent all full table or index scans, set to `0`. | User-dependent | No | Yes | | `locality` | The location of the node.

For more information, see [Locality]({% link {{ page.version.version }}/cockroach-start.md %}#locality). | Node-dependent | No | Yes | | `lock_timeout` | The amount of time a query can spend acquiring or waiting for a single [row-level lock]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#concurrency-control).
In CockroachDB, unlike in PostgreSQL, non-locking reads wait for conflicting locks to be released. As a result, the `lock_timeout` configuration applies to writes, and to locking and non-locking reads in read-write and read-only transactions.
If `lock_timeout = 0`, queries do not timeout due to lock acquisitions. | The value set by the `sql.defaults.lock_timeout` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`0`, by default) | Yes | Yes | | `multiple_active_portals_enabled` | Whether to enable the [multiple active portals]({% link {{ page.version.version }}/postgresql-compatibility.md %}#multiple-active-portals) pgwire feature. | `false` | Yes | Yes | diff --git a/src/current/_includes/v23.2/misc/force-index-selection.md b/src/current/_includes/v23.2/misc/force-index-selection.md index 7f9e1292126..ef0d199dd1b 100644 --- a/src/current/_includes/v23.2/misc/force-index-selection.md +++ b/src/current/_includes/v23.2/misc/force-index-selection.md @@ -135,11 +135,8 @@ Time: 1ms total (execution 1ms / network 0ms) ##### Prevent full scan -To prevent the optimizer from planning a full scan for a table, specify the `NO_FULL_SCAN` index hint. For example: +{% include {{ page.version.version }}/sql/no-full-scan.md %} -~~~sql -SELECT * FROM table_name@{NO_FULL_SCAN}; -~~~ - -To prevent a full scan of a [partial index](#force-partial-index-scan), you must specify `NO_FULL_SCAN` _in combination with_ the partial index using `FORCE_INDEX=index_name`. -If you specify only `NO_FULL_SCAN`, a full scan of a partial index may be planned. +{{site.data.alerts.callout_success}} +For other ways to prevent full table scans, refer to [Prevent the optimizer from planning full scans]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#prevent-the-optimizer-from-planning-full-scans). +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v23.2/misc/session-vars.md b/src/current/_includes/v23.2/misc/session-vars.md index ecf6cd815e9..a3914fd2312 100644 --- a/src/current/_includes/v23.2/misc/session-vars.md +++ b/src/current/_includes/v23.2/misc/session-vars.md @@ -15,7 +15,7 @@ | `default_transaction_quality_of_service` | The default transaction quality of service for the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#set-quality-of-service-level-for-a-session). | `regular` | Yes | Yes | | `default_transaction_read_only` | The default transaction access mode for the current session.
If set to `on`, only read operations are allowed in transactions in the current session; if set to `off`, both read and write operations are allowed. See [`SET TRANSACTION`]({% link {{ page.version.version }}/set-transaction.md %}) for more details. | `off` | Yes | Yes | | `default_transaction_use_follower_reads` | If set to on, all read-only transactions use [`AS OF SYSTEM TIME follower_read_timestamp()`]({% link {{ page.version.version }}/as-of-system-time.md %}) to allow the transaction to use follower reads.
If set to `off`, read-only transactions will only use follower reads if an `AS OF SYSTEM TIME` clause is specified in the statement, with an interval of at least 4.8 seconds. | `off` | Yes | Yes | -| `disallow_full_table_scans` | If set to `on`, all queries that have planned a full table or full secondary index scan will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | +| `disallow_full_table_scans` | If set to `on`, queries on "large" tables with a row count greater than [`large_full_scan_rows`](#large-full-scan-rows) will not use full table or index scans. If no other query plan is possible, queries will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | | `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes | | `enable_auto_rehoming` | When enabled, the [home regions]({% link {{ page.version.version }}/alter-table.md %}#crdb_region) of rows in [`REGIONAL BY ROW`]({% link {{ page.version.version }}/alter-table.md %}#set-the-table-locality-to-regional-by-row) tables are automatically set to the region of the [gateway node]({% link {{ page.version.version }}/ui-sessions-page.md %}#session-details-gateway-node) from which any [`UPDATE`]({% link {{ page.version.version }}/update.md %}) or [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements that operate on those rows originate. | `off` | Yes | Yes | | `enable_durable_locking_for_serializable` | Indicates whether CockroachDB replicates [`FOR UPDATE` and `FOR SHARE`]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) locks via [Raft]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft), allowing locks to be preserved when leases are transferred. Note that replicating `FOR UPDATE` and `FOR SHARE` locks will add latency to those statements. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes | @@ -38,7 +38,7 @@ | `inject_retry_errors_enabled` | If `true`, any statement executed inside of an explicit transaction (with the exception of [`SET`]({% link {{ page.version.version }}/set-vars.md %}) statements) will return a transaction retry error. If the client retries the transaction using the special [`cockroach_restart SAVEPOINT` name]({% link {{ page.version.version }}/savepoint.md %}#savepoints-for-client-side-transaction-retries), after the 3rd retry error, the transaction will proceed as normal. Otherwise, the errors will continue until `inject_retry_errors_enabled` is set to `false`. For more details, see [Test transaction retry logic]({% link {{ page.version.version }}/transaction-retry-error-example.md %}#test-transaction-retry-logic). | `false` | Yes | Yes | | `intervalstyle` | The input string format for [`INTERVAL`]({% link {{ page.version.version }}/interval.md %}) values. Accepted values include `postgres`, `iso_8601`, and `sql_standard`. | The value set by the `sql.defaults.intervalstyle` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`postgres`, by default). | Yes | Yes | | `is_superuser` | If `on` or `true`, the current user is a member of the [`admin` role]({% link {{ page.version.version }}/security-reference/authorization.md %}#admin-role). | User-dependent | No | Yes | -| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` rejects full table or index scans of "large" tables. The default value is `1000`. To reject all full table or index scans, set to `0`. | User-dependent | No | Yes | +| `large_full_scan_rows` | Determines which tables are considered "large" such that `disallow_full_table_scans` prevents queries from performing full table or index scans on those tables. The default value is `1000`. To prevent all full table or index scans, set to `0`. | User-dependent | No | Yes | | `locality` | The location of the node.

For more information, see [Locality]({% link {{ page.version.version }}/cockroach-start.md %}#locality). | Node-dependent | No | Yes | | `lock_timeout` | The amount of time a query can spend acquiring or waiting for a single [row-level lock]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#concurrency-control).
In CockroachDB, unlike in PostgreSQL, non-locking reads wait for conflicting locks to be released. As a result, the `lock_timeout` configuration applies to writes, and to locking and non-locking reads in read-write and read-only transactions.
If `lock_timeout = 0`, queries do not timeout due to lock acquisitions. | The value set by the `sql.defaults.lock_timeout` [cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}) (`0`, by default) | Yes | Yes | | `multiple_active_portals_enabled` | Whether to enable the [multiple active portals]({% link {{ page.version.version }}/postgresql-compatibility.md %}#multiple-active-portals) pgwire feature. | `false` | Yes | Yes | diff --git a/src/current/_includes/v23.2/sql/disallow-full-table-scans.md b/src/current/_includes/v23.2/sql/disallow-full-table-scans.md new file mode 100644 index 00000000000..57701770065 --- /dev/null +++ b/src/current/_includes/v23.2/sql/disallow-full-table-scans.md @@ -0,0 +1,12 @@ +- At the cluster level, set `disallow_full_table_scans` for some or all users and roles. For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET disallow_full_table_scans = true; + ~~~ + +- At the application level, add `disallow_full_table_scans` to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters). + +If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error. + +If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint and it is not possible to avoid a full scan, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated. \ No newline at end of file diff --git a/src/current/_includes/v23.2/sql/no-full-scan.md b/src/current/_includes/v23.2/sql/no-full-scan.md new file mode 100644 index 00000000000..304c2ab9697 --- /dev/null +++ b/src/current/_includes/v23.2/sql/no-full-scan.md @@ -0,0 +1,15 @@ +- To prevent the optimizer from planning a full scan for a specific table, specify the `NO_FULL_SCAN` index hint. For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SELECT * FROM table_name@{NO_FULL_SCAN}; + ~~~ + +- To prevent a full scan of a [partial index]({% link {{ page.version.version }}/partial-indexes.md %}) for a specific table, you must specify `NO_FULL_SCAN` in combination with the index name using [`FORCE_INDEX`]({% link {{ page.version.version }}/table-expressions.md %}#force-index-selection). For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SELECT * FROM table_name@{FORCE_INDEX=index_name,NO_FULL_SCAN} WHERE b > 0; + ~~~ + + This forces a constrained scan of the partial index. If a constrained scan of the partial index is not possible, an error is returned. \ No newline at end of file diff --git a/src/current/_includes/v23.2/sql/transactions-limit-rows.md b/src/current/_includes/v23.2/sql/transactions-limit-rows.md new file mode 100644 index 00000000000..0962a211195 --- /dev/null +++ b/src/current/_includes/v23.2/sql/transactions-limit-rows.md @@ -0,0 +1,15 @@ +- When the `transaction_rows_read_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-err) is enabled, transactions that read more than the specified number of rows will fail. In addition, the [optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will not create query plans with scans that exceed the specified row limit. For example, to set a default value for all users at the cluster level: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET transaction_rows_read_err = 1000; + ~~~ + +- When the `transaction_rows_written_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-written-err) is enabled, transactions that write more than the specified number of rows will fail. For example, to set a default value for all users at the cluster level: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET transaction_rows_written_err = 1000; + ~~~ + +To assess the impact of configuring these session settings, use the corresponding session settings [`transaction_rows_read_log`]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-log) and [`transaction_rows_written_log`]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-written-log) to log transactions that read or write the specified number of rows. Transactions are logged to the [`SQL_PERF`]({% link {{ page.version.version }}/logging.md %}#sql_perf) channel. \ No newline at end of file diff --git a/src/current/_includes/v24.1/misc/force-index-selection.md b/src/current/_includes/v24.1/misc/force-index-selection.md index 7f9e1292126..beeb1bcceac 100644 --- a/src/current/_includes/v24.1/misc/force-index-selection.md +++ b/src/current/_includes/v24.1/misc/force-index-selection.md @@ -135,11 +135,8 @@ Time: 1ms total (execution 1ms / network 0ms) ##### Prevent full scan -To prevent the optimizer from planning a full scan for a table, specify the `NO_FULL_SCAN` index hint. For example: +{% include {{ page.version.version }}/sql/no-full-scan.md %} -~~~sql -SELECT * FROM table_name@{NO_FULL_SCAN}; -~~~ - -To prevent a full scan of a [partial index](#force-partial-index-scan), you must specify `NO_FULL_SCAN` _in combination with_ the partial index using `FORCE_INDEX=index_name`. -If you specify only `NO_FULL_SCAN`, a full scan of a partial index may be planned. +{{site.data.alerts.callout_success}} +For other ways to prevent full scans, refer to [Prevent the optimizer from planning full scans]({% link {{ page.version.version }}/performance-best-practices-overview.md %}#prevent-the-optimizer-from-planning-full-scans). +{{site.data.alerts.end}} \ No newline at end of file diff --git a/src/current/_includes/v24.1/misc/session-vars.md b/src/current/_includes/v24.1/misc/session-vars.md index ecf6cd815e9..836a3603dc7 100644 --- a/src/current/_includes/v24.1/misc/session-vars.md +++ b/src/current/_includes/v24.1/misc/session-vars.md @@ -15,8 +15,7 @@ | `default_transaction_quality_of_service` | The default transaction quality of service for the current session. The supported options are `regular`, `critical`, and `background`. See [Set quality of service level]({% link {{ page.version.version }}/admission-control.md %}#set-quality-of-service-level-for-a-session). | `regular` | Yes | Yes | | `default_transaction_read_only` | The default transaction access mode for the current session.
If set to `on`, only read operations are allowed in transactions in the current session; if set to `off`, both read and write operations are allowed. See [`SET TRANSACTION`]({% link {{ page.version.version }}/set-transaction.md %}) for more details. | `off` | Yes | Yes | | `default_transaction_use_follower_reads` | If set to on, all read-only transactions use [`AS OF SYSTEM TIME follower_read_timestamp()`]({% link {{ page.version.version }}/as-of-system-time.md %}) to allow the transaction to use follower reads.
If set to `off`, read-only transactions will only use follower reads if an `AS OF SYSTEM TIME` clause is specified in the statement, with an interval of at least 4.8 seconds. | `off` | Yes | Yes | -| `disallow_full_table_scans` | If set to `on`, all queries that have planned a full table or full secondary index scan will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes | -| `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes | +| `disallow_full_table_scans` | If set to `on`, queries on "large" tables with a row count greater than [`large_full_scan_rows`](#large-full-scan-rows) will not use full table or index scans. If no other query plan is possible, queries will return an error message. This setting does not apply to internal queries, which may plan full table or index scans without checking the session variable. | `off` | Yes | Yes || `distsql` | The query distribution mode for the session. By default, CockroachDB determines which queries are faster to execute if distributed across multiple nodes, and all other queries are run through the gateway node. | `auto` | Yes | Yes | | `enable_auto_rehoming` | When enabled, the [home regions]({% link {{ page.version.version }}/alter-table.md %}#crdb_region) of rows in [`REGIONAL BY ROW`]({% link {{ page.version.version }}/alter-table.md %}#set-the-table-locality-to-regional-by-row) tables are automatically set to the region of the [gateway node]({% link {{ page.version.version }}/ui-sessions-page.md %}#session-details-gateway-node) from which any [`UPDATE`]({% link {{ page.version.version }}/update.md %}) or [`UPSERT`]({% link {{ page.version.version }}/upsert.md %}) statements that operate on those rows originate. | `off` | Yes | Yes | | `enable_durable_locking_for_serializable` | Indicates whether CockroachDB replicates [`FOR UPDATE` and `FOR SHARE`]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) locks via [Raft]({% link {{ page.version.version }}/architecture/replication-layer.md %}#raft), allowing locks to be preserved when leases are transferred. Note that replicating `FOR UPDATE` and `FOR SHARE` locks will add latency to those statements. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes | | `enable_implicit_fk_locking_for_serializable` | Indicates whether CockroachDB uses [shared locks]({% link {{ page.version.version }}/select-for-update.md %}#lock-strengths) to perform [foreign key]({% link {{ page.version.version }}/foreign-key.md %}) checks. To take effect, `enable_shared_locking_for_serializable` must also be enabled. This setting only affects `SERIALIZABLE` transactions and matches the default `READ COMMITTED` behavior when enabled. | `off` | Yes | Yes | diff --git a/src/current/_includes/v24.1/sql/disallow-full-table-scans.md b/src/current/_includes/v24.1/sql/disallow-full-table-scans.md new file mode 100644 index 00000000000..57701770065 --- /dev/null +++ b/src/current/_includes/v24.1/sql/disallow-full-table-scans.md @@ -0,0 +1,12 @@ +- At the cluster level, set `disallow_full_table_scans` for some or all users and roles. For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET disallow_full_table_scans = true; + ~~~ + +- At the application level, add `disallow_full_table_scans` to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters). + +If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error. + +If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint and it is not possible to avoid a full scan, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated. \ No newline at end of file diff --git a/src/current/_includes/v24.1/sql/no-full-scan.md b/src/current/_includes/v24.1/sql/no-full-scan.md new file mode 100644 index 00000000000..304c2ab9697 --- /dev/null +++ b/src/current/_includes/v24.1/sql/no-full-scan.md @@ -0,0 +1,15 @@ +- To prevent the optimizer from planning a full scan for a specific table, specify the `NO_FULL_SCAN` index hint. For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SELECT * FROM table_name@{NO_FULL_SCAN}; + ~~~ + +- To prevent a full scan of a [partial index]({% link {{ page.version.version }}/partial-indexes.md %}) for a specific table, you must specify `NO_FULL_SCAN` in combination with the index name using [`FORCE_INDEX`]({% link {{ page.version.version }}/table-expressions.md %}#force-index-selection). For example: + + {% include_cached copy-clipboard.html %} + ~~~ sql + SELECT * FROM table_name@{FORCE_INDEX=index_name,NO_FULL_SCAN} WHERE b > 0; + ~~~ + + This forces a constrained scan of the partial index. If a constrained scan of the partial index is not possible, an error is returned. \ No newline at end of file diff --git a/src/current/_includes/v24.1/sql/transactions-limit-rows.md b/src/current/_includes/v24.1/sql/transactions-limit-rows.md new file mode 100644 index 00000000000..0962a211195 --- /dev/null +++ b/src/current/_includes/v24.1/sql/transactions-limit-rows.md @@ -0,0 +1,15 @@ +- When the `transaction_rows_read_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-err) is enabled, transactions that read more than the specified number of rows will fail. In addition, the [optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will not create query plans with scans that exceed the specified row limit. For example, to set a default value for all users at the cluster level: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET transaction_rows_read_err = 1000; + ~~~ + +- When the `transaction_rows_written_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-written-err) is enabled, transactions that write more than the specified number of rows will fail. For example, to set a default value for all users at the cluster level: + + {% include_cached copy-clipboard.html %} + ~~~ sql + ALTER ROLE ALL SET transaction_rows_written_err = 1000; + ~~~ + +To assess the impact of configuring these session settings, use the corresponding session settings [`transaction_rows_read_log`]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-log) and [`transaction_rows_written_log`]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-written-log) to log transactions that read or write the specified number of rows. Transactions are logged to the [`SQL_PERF`]({% link {{ page.version.version }}/logging.md %}#sql_perf) channel. \ No newline at end of file diff --git a/src/current/v23.2/cost-based-optimizer.md b/src/current/v23.2/cost-based-optimizer.md index ac0208f043e..cfb59914ce6 100644 --- a/src/current/v23.2/cost-based-optimizer.md +++ b/src/current/v23.2/cost-based-optimizer.md @@ -232,9 +232,7 @@ Even if you have [secondary indexes]({% link {{ page.version.version }}/schema-d You can disable statement plans that perform full table scans with the [`disallow_full_table_scans` session variable]({% link {{ page.version.version }}/set-vars.md %}#disallow-full-table-scans). -If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error. - -If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated. +{% include {{ page.version.version }}/sql/disallow-full-table-scans.md %} ## Control whether the optimizer uses an index diff --git a/src/current/v23.2/performance-best-practices-overview.md b/src/current/v23.2/performance-best-practices-overview.md index 716ed171ebf..adefe963de7 100644 --- a/src/current/v23.2/performance-best-practices-overview.md +++ b/src/current/v23.2/performance-best-practices-overview.md @@ -263,9 +263,7 @@ See [Authorization Best Practices]({% link {{ page.version.version }}/security-r For large tables, avoid table scans (that is, reading the entire table data) whenever possible. Instead, define the required fields in a `SELECT` statement. -#### Example - -Suppose the table schema is as follows: +For example, suppose the table schema is as follows: {% include_cached copy-clipboard.html %} ~~~ sql @@ -298,24 +296,37 @@ This query returns the account balances of the customers. `SELECT DISTINCT` allows you to obtain unique entries from a query by removing duplicate entries. However, `SELECT DISTINCT` is computationally expensive. As a performance best practice, use [`SELECT` with the `WHERE` clause]({% link {{ page.version.version }}/select-clause.md %}#filter-rows) instead. +### Use secondary indexes to optimize queries + +See [Statement Tuning with `EXPLAIN`]({% link {{ page.version.version }}/sql-tuning-with-explain.md %}#issue-full-table-scans). + ### Use `AS OF SYSTEM TIME` to decrease conflicts with long-running queries If you have long-running queries (such as analytics queries that perform full table scans) that can tolerate slightly out-of-date reads, consider using the [`... AS OF SYSTEM TIME` clause]({% link {{ page.version.version }}/select-clause.md %}#select-historical-data-time-travel). Using this, your query returns data as it appeared at a distinct point in the past and will not cause [conflicts]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#transaction-conflicts) with other concurrent transactions, which can increase your application's performance. However, because `AS OF SYSTEM TIME` returns historical data, your reads might be stale. -### Disallow full table scans with the `disallow_full_table_scans` setting +### Prevent the optimizer from planning full scans + +To avoid overloading production clusters, there are several ways to prevent the [cost-based-optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) from generating query plans with [full table and index scans]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#full-table-index-scans). + +#### Use index hints to prevent full scans on tables + +{% include {{ page.version.version }}/sql/no-full-scan.md %} + +#### Disallow query plans that use full scans + +When the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) is enabled, the optimizer will not plan full table or index scans on "large" tables (i.e., those with more rows than [`large_full_scan_rows`]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows)). + +{% include {{ page.version.version }}/sql/disallow-full-table-scans.md %} -To prevent overloading production clusters with [full table scans]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#full-table-index-scans), you have several options: +#### Disallow query plans that scan more than a number of rows -1. At the cluster level, configure the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) for some or all users/roles using the [`ALTER ROLE`]({% link {{ page.version.version }}/alter-role.md %}) statement. +When the `transaction_rows_read_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-err) is enabled, the [optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will not create query plans with scans that exceed the specified row limit. See [Disallow transactions from reading or writing many rows](#disallow-transactions-from-reading-or-writing-many-rows). - {% include_cached copy-clipboard.html %} - ~~~ sql - ALTER ROLE ALL SET disallow_full_table_scans = true; - ~~~ +### Disallow transactions from reading or writing many rows -1. At the application level, add the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters). +{% include {{ page.version.version }}/sql/transactions-limit-rows.md %} diff --git a/src/current/v23.2/transactions.md b/src/current/v23.2/transactions.md index 149ac6118ca..310e3f21d3c 100644 --- a/src/current/v23.2/transactions.md +++ b/src/current/v23.2/transactions.md @@ -228,27 +228,14 @@ For more information about the relationship between these levels, see [A Critiqu You can limit the number of rows written or read in a transaction at the cluster or session level. This allows you configure CockroachDB to log or reject statements that could destabilize a cluster or violate application best practices. -Use the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}) `sql.defaults.transaction_rows_written_log`, -`sql.defaults.transaction_rows_written_err`, `sql.defaults.transaction_rows_read_log`, and -`sql.defaults.transaction_rows_read_err` and [session settings]({% link {{ page.version.version }}/set-vars.md %}) `transaction_rows_written_log`, -`transaction_rows_written_err`, `transaction_rows_read_log`, and -`transaction_rows_read_err` to limit the number of rows written or read in a -transaction. When the `log` limit is reached, the transaction is logged to the `SQL_PERF` channel. -When the `err` limit is reached, the transaction is rejected. The limits are enforced after each -statement of a transaction has been fully executed. - -The "write" limits apply to `INSERT`, `INSERT INTO SELECT FROM`, `INSERT ON CONFLICT`, `UPSERT`, `UPDATE`, -and `DELETE` SQL statements. The "read" limits apply to the `SELECT` -statement in addition to the statements subject to the "write" limits. The limits **do not** -apply to `CREATE TABLE AS`, `SELECT`, `IMPORT`, `TRUNCATE`, `DROP`, `ALTER TABLE`, `BACKUP`, -`RESTORE`, or `CREATE STATISTICS` statements. +{% include {{ page.version.version }}/sql/transactions-limit-rows.md %} + +The limits are enforced after each statement of a transaction has been fully executed. The "write" limits apply to `INSERT`, `INSERT INTO SELECT FROM`, `INSERT ON CONFLICT`, `UPSERT`, `UPDATE`, and `DELETE` SQL statements. The "read" limits apply to the `SELECT` statement in addition to the statements subject to the "write" limits. The limits **do not** apply to `CREATE TABLE AS`, `IMPORT`, `TRUNCATE`, `DROP`, `ALTER TABLE`, `BACKUP`, `RESTORE`, or `CREATE STATISTICS` statements. {{site.data.alerts.callout_info}} Enabling `transaction_rows_read_err` disables a performance optimization for mutation statements in implicit transactions where CockroachDB can auto-commit without additional network round trips. {{site.data.alerts.end}} -{% include {{page.version.version}}/sql/sql-defaults-cluster-settings-deprecation-notice.md %} - ## See also - [`BEGIN`]({% link {{ page.version.version }}/begin-transaction.md %}) diff --git a/src/current/v24.1/cost-based-optimizer.md b/src/current/v24.1/cost-based-optimizer.md index ac0208f043e..cfb59914ce6 100644 --- a/src/current/v24.1/cost-based-optimizer.md +++ b/src/current/v24.1/cost-based-optimizer.md @@ -232,9 +232,7 @@ Even if you have [secondary indexes]({% link {{ page.version.version }}/schema-d You can disable statement plans that perform full table scans with the [`disallow_full_table_scans` session variable]({% link {{ page.version.version }}/set-vars.md %}#disallow-full-table-scans). -If you disable full scans, you can set the [`large_full_scan_rows` session variable]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows) to specify the maximum table size allowed for a full scan. If no alternative plan is possible, the optimizer will return an error. - -If you disable full scans, and you provide an [index hint]({% link {{ page.version.version }}/indexes.md %}#selection), the optimizer will try to avoid a full scan while also respecting the index hint. If this is not possible, the optimizer will return an error. If you do not provide an index hint, the optimizer will return an error, the full scan will be logged, and the `sql.guardrails.full_scan_rejected.count` [metric]({% link {{ page.version.version }}/ui-overview-dashboard.md %}) will be updated. +{% include {{ page.version.version }}/sql/disallow-full-table-scans.md %} ## Control whether the optimizer uses an index diff --git a/src/current/v24.1/performance-best-practices-overview.md b/src/current/v24.1/performance-best-practices-overview.md index 716ed171ebf..adefe963de7 100644 --- a/src/current/v24.1/performance-best-practices-overview.md +++ b/src/current/v24.1/performance-best-practices-overview.md @@ -263,9 +263,7 @@ See [Authorization Best Practices]({% link {{ page.version.version }}/security-r For large tables, avoid table scans (that is, reading the entire table data) whenever possible. Instead, define the required fields in a `SELECT` statement. -#### Example - -Suppose the table schema is as follows: +For example, suppose the table schema is as follows: {% include_cached copy-clipboard.html %} ~~~ sql @@ -298,24 +296,37 @@ This query returns the account balances of the customers. `SELECT DISTINCT` allows you to obtain unique entries from a query by removing duplicate entries. However, `SELECT DISTINCT` is computationally expensive. As a performance best practice, use [`SELECT` with the `WHERE` clause]({% link {{ page.version.version }}/select-clause.md %}#filter-rows) instead. +### Use secondary indexes to optimize queries + +See [Statement Tuning with `EXPLAIN`]({% link {{ page.version.version }}/sql-tuning-with-explain.md %}#issue-full-table-scans). + ### Use `AS OF SYSTEM TIME` to decrease conflicts with long-running queries If you have long-running queries (such as analytics queries that perform full table scans) that can tolerate slightly out-of-date reads, consider using the [`... AS OF SYSTEM TIME` clause]({% link {{ page.version.version }}/select-clause.md %}#select-historical-data-time-travel). Using this, your query returns data as it appeared at a distinct point in the past and will not cause [conflicts]({% link {{ page.version.version }}/architecture/transaction-layer.md %}#transaction-conflicts) with other concurrent transactions, which can increase your application's performance. However, because `AS OF SYSTEM TIME` returns historical data, your reads might be stale. -### Disallow full table scans with the `disallow_full_table_scans` setting +### Prevent the optimizer from planning full scans + +To avoid overloading production clusters, there are several ways to prevent the [cost-based-optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) from generating query plans with [full table and index scans]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#full-table-index-scans). + +#### Use index hints to prevent full scans on tables + +{% include {{ page.version.version }}/sql/no-full-scan.md %} + +#### Disallow query plans that use full scans + +When the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) is enabled, the optimizer will not plan full table or index scans on "large" tables (i.e., those with more rows than [`large_full_scan_rows`]({% link {{ page.version.version }}/set-vars.md %}#large-full-scan-rows)). + +{% include {{ page.version.version }}/sql/disallow-full-table-scans.md %} -To prevent overloading production clusters with [full table scans]({% link {{ page.version.version }}/ui-sql-dashboard.md %}#full-table-index-scans), you have several options: +#### Disallow query plans that scan more than a number of rows -1. At the cluster level, configure the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) for some or all users/roles using the [`ALTER ROLE`]({% link {{ page.version.version }}/alter-role.md %}) statement. +When the `transaction_rows_read_err` [session setting]({% link {{ page.version.version }}/set-vars.md %}#transaction-rows-read-err) is enabled, the [optimizer]({% link {{ page.version.version }}/cost-based-optimizer.md %}) will not create query plans with scans that exceed the specified row limit. See [Disallow transactions from reading or writing many rows](#disallow-transactions-from-reading-or-writing-many-rows). - {% include_cached copy-clipboard.html %} - ~~~ sql - ALTER ROLE ALL SET disallow_full_table_scans = true; - ~~~ +### Disallow transactions from reading or writing many rows -1. At the application level, add the `disallow_full_table_scans` [session setting]({% link {{page.version.version}}/set-vars.md %}#disallow-full-table-scans) to the connection string using the [`options` parameter]({% link {{page.version.version}}/connection-parameters.md %}#additional-connection-parameters). +{% include {{ page.version.version }}/sql/transactions-limit-rows.md %} diff --git a/src/current/v24.1/transactions.md b/src/current/v24.1/transactions.md index 2bd2df11675..5d382b6e705 100644 --- a/src/current/v24.1/transactions.md +++ b/src/current/v24.1/transactions.md @@ -228,27 +228,14 @@ For more information about the relationship between these levels, see [A Critiqu You can limit the number of rows written or read in a transaction at the cluster or session level. This allows you configure CockroachDB to log or reject statements that could destabilize a cluster or violate application best practices. -Use the [cluster settings]({% link {{ page.version.version }}/cluster-settings.md %}) `sql.defaults.transaction_rows_written_log`, -`sql.defaults.transaction_rows_written_err`, `sql.defaults.transaction_rows_read_log`, and -`sql.defaults.transaction_rows_read_err` and [session settings]({% link {{ page.version.version }}/set-vars.md %}) `transaction_rows_written_log`, -`transaction_rows_written_err`, `transaction_rows_read_log`, and -`transaction_rows_read_err` to limit the number of rows written or read in a -transaction. When the `log` limit is reached, the transaction is logged to the `SQL_PERF` channel. -When the `err` limit is reached, the transaction is rejected. The limits are enforced after each -statement of a transaction has been fully executed. - -The "write" limits apply to `INSERT`, `INSERT INTO SELECT FROM`, `INSERT ON CONFLICT`, `UPSERT`, `UPDATE`, -and `DELETE` SQL statements. The "read" limits apply to the `SELECT` -statement in addition to the statements subject to the "write" limits. The limits **do not** -apply to `CREATE TABLE AS`, `SELECT`, `IMPORT`, `TRUNCATE`, `DROP`, `ALTER TABLE`, `BACKUP`, -`RESTORE`, or `CREATE STATISTICS` statements. +{% include {{ page.version.version }}/sql/transactions-limit-rows.md %} + +The limits are enforced after each statement of a transaction has been fully executed. The "write" limits apply to `INSERT`, `INSERT INTO SELECT FROM`, `INSERT ON CONFLICT`, `UPSERT`, `UPDATE`, and `DELETE` SQL statements. The "read" limits apply to the `SELECT` statement in addition to the statements subject to the "write" limits. The limits **do not** apply to `CREATE TABLE AS`, `IMPORT`, `TRUNCATE`, `DROP`, `ALTER TABLE`, `BACKUP`, `RESTORE`, or `CREATE STATISTICS` statements. {{site.data.alerts.callout_info}} Enabling `transaction_rows_read_err` disables a performance optimization for mutation statements in implicit transactions where CockroachDB can auto-commit without additional network round trips. {{site.data.alerts.end}} -{% include {{page.version.version}}/sql/sql-defaults-cluster-settings-deprecation-notice.md %} - ## See also - [`BEGIN`]({% link {{ page.version.version }}/begin-transaction.md %})