Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOCS-758 Adds new commit scope rule degradation functionality for synchronous and group commit. #6044

Merged
merged 43 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
3a5c095
First draft reference changes.
jpe442 Sep 10, 2024
107bd51
Small changes.
jpe442 Sep 10, 2024
2a6f854
Small tweaks.
jpe442 Sep 10, 2024
95b18f8
More small changes in group and synchrnounous commit.
jpe442 Sep 11, 2024
857f090
Small fixes.
jpe442 Sep 17, 2024
30b08b4
More small tweaks.
jpe442 Sep 17, 2024
7d0fcd9
tried to fix link.
jpe442 Sep 17, 2024
d493a71
Changing paths to absolute.
jpe442 Sep 17, 2024
3b842a8
Update product_docs/docs/pgd/5.6/commit-scopes/synchronous_commit.mdx
jpe442 Sep 18, 2024
91e2591
Update product_docs/docs/pgd/5.6/commit-scopes/index.mdx
jpe442 Sep 18, 2024
890e8df
Changed wording in Group commit.
jpe442 Sep 18, 2024
8897f0a
Changed order of commit scopes in Commit scope rules to feature synch…
jpe442 Sep 18, 2024
370a655
changed overview to put synch commit before group commit.
jpe442 Sep 18, 2024
c442329
Removed some white space.
jpe442 Sep 18, 2024
d8faa4c
Degrade options section.
jpe442 Sep 19, 2024
6c823c4
Added Degrade on section in Commit scopes page.
jpe442 Sep 20, 2024
28c2a8e
Fixed some small things.
jpe442 Sep 20, 2024
f98a6ca
Small word change.
jpe442 Sep 20, 2024
c53725b
Move CAMO and adjust accordingly.
jpe442 Sep 24, 2024
63c10a7
Small changes.
jpe442 Sep 24, 2024
b172c32
Moved degrading options to its own page.
jpe442 Sep 24, 2024
b434311
Addressed one of DJ's comments.
jpe442 Sep 24, 2024
1c00222
Update product_docs/docs/pgd/5.6/commit-scopes/degrading.mdx
jpe442 Sep 25, 2024
97291e3
Update product_docs/docs/pgd/5.6/commit-scopes/degrading.mdx
jpe442 Sep 25, 2024
6b26680
Update product_docs/docs/pgd/5.6/commit-scopes/degrading.mdx
jpe442 Sep 25, 2024
f8d86b3
Update product_docs/docs/pgd/5.6/reference/commit-scopes.mdx
jpe442 Sep 25, 2024
6f0e7cf
Update product_docs/docs/pgd/5.6/reference/commit-scopes.mdx
jpe442 Sep 25, 2024
c9a67fc
Moved behavior section up.
jpe442 Sep 25, 2024
bc353f8
Worked on wording given Petr's comments.
jpe442 Sep 26, 2024
bf62aa9
More word changes.
jpe442 Sep 26, 2024
fa22f21
Add degrading bit to Overview.
jpe442 Sep 26, 2024
46dd28a
Added wrap-language support, wrapped blocks, added degrading to index
djw-m Sep 27, 2024
83ff5a8
Remove console.log hangover
djw-m Sep 27, 2024
abf0a6b
Improve feedback
djw-m Sep 27, 2024
d6ec97b
Shorten tooltip
djw-m Sep 27, 2024
48bbdc2
Update product_docs/docs/pgd/5.6/commit-scopes/degrading.mdx
jpe442 Sep 27, 2024
1e440d8
Update product_docs/docs/pgd/5.6/commit-scopes/group-commit.mdx
jpe442 Sep 27, 2024
4b575f0
Update product_docs/docs/pgd/5.6/commit-scopes/index.mdx
jpe442 Sep 27, 2024
277e4b4
Update product_docs/docs/pgd/5.6/commit-scopes/synchronous_commit.mdx
jpe442 Sep 27, 2024
6235d3f
Update product_docs/docs/pgd/5.6/commit-scopes/degrading.mdx
jpe442 Sep 27, 2024
72c286d
Disable startWrap for now
djw-m Sep 27, 2024
3318937
Fix link.
jpe442 Sep 27, 2024
8df3d37
Update product_docs/docs/pgd/5.6/commit-scopes/commit-scope-rules.mdx
jpe442 Oct 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ deepToC: true

## Commit scope

[Commit scopes](/pgd/latest/commit-scopes/) in PGD are a set of rules that describe the behavior of the system as transactions are committed. Because they define how transactions are replicated across a distributed database, they have an effect on consistency, durability, and performance.
[Commit scopes](/pgd/latest/commit-scopes/commit-scopes/) in PGD are a set of rules that describe the behavior of the system as transactions are committed. Because they define how transactions are replicated across a distributed database, they have an effect on consistency, durability, and performance.

The actual behavior depends on the kind of commit scope a commit scope's rule uses: [Group Commit](/pgd/latest/commit-scopes/group-commit/), [Commit At Most Once](/pgd/latest/commit-scopes/camo/), [Lag Control](/pgd/latest/commit-scopes/lag-control/), [Synchronous Commit](/pgd/latest/commit-scopes/synchronous_commit/), or a combination of these.

Expand Down
3 changes: 1 addition & 2 deletions product_docs/docs/pgd/5.6/commit-scopes/camo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ To use CAMO, an application must issue an explicit `COMMIT` message as a separat

## Configuration

Configuration of CAMO happens through [commit scopes](commit-scopes).
See the[`CAMO`](/pgd/latest/reference/commit-scopes/#camo) commit scope reference for configuration parameters.

## Confirmation

Expand All @@ -58,7 +58,6 @@ Configuration of CAMO happens through [commit scopes](commit-scopes).

See the CAMO section of [Limitations](limitations#camo).


## Failure scenarios

Different failure scenarios occur in different configurations.
Expand Down
12 changes: 6 additions & 6 deletions product_docs/docs/pgd/5.6/commit-scopes/commit-scope-rules.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,12 @@ If you're familiar with PostgreSQL's `synchronous_standby_names` feature, be awa

Currently, there are four commit scope kinds. The following is a summary, with links to more details.

### `SYNCHRONOUS_COMMIT`

Synchronous Commit is a commit scope option that's designed to behave like the native Postgres `synchronous_commit` option, but is usable from within the commit scope environment. Unlike `GROUP COMMIT`, it's a synchronous non-two-phase commit operation. Like `GROUP COMMIT`, it supports an optional `DEGRADE ON` clause. The commit scope group that comes before this option controls the groups and confirmation requirements the `SYNCHRONOUS_COMMIT` uses.

For more details, see [`SYNCHRONOUS_COMMIT`](synchronous_commit).

### `GROUP COMMIT`

Group Commit is a synchronous, two-phase commit that's confirmed according to the requirements of the commit scope group. `GROUP COMMIT` has options that control:
Expand All @@ -83,12 +89,6 @@ With Lag Control, when the system's replication performance exceeds specified li

For more details, see [`LAG CONTROL`](lag-control).

### `SYNCHRONOUS_COMMIT`

Synchronous Commit is a commit scope option that's designed to be like the legacy `synchronous_commit` option, but it's accessible in the commit scope environment. Unlike `GROUP COMMIT`, it's a synchronous non-two-phase commit operation, and it has no parameters. The commit scope group that comes before this option controls the groups and confirmation requirements the `SYNCHRONOUS_COMMIT` uses.

For more details, see [`SYNCHRONOUS_COMMIT`](synchronous_commit).

## Combining rules

Commit scope rules are composed of one or more operations that work in combination. Use an AND to form a single rule. For example:
Expand Down
6 changes: 2 additions & 4 deletions product_docs/docs/pgd/5.6/commit-scopes/commit-scopes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Commit scopes

Commit scopes give applications granular control about durability and consistency of EDB Postgres Distributed.

A commit scope is a set of rules that describes the behavior of the system as transactions are committed. The actual behavior depends on which a kind of commit scope a commit scope's rule uses [Group Commit](group-commit), [Commit At Most Once](camo), [Lag Control](lag-control), [Synchronous Commit](synchronous_commit) or combination of these.
A commit scope is a set of rules that describes the behavior of the system as transactions are committed. The actual behavior depends on which a kind of commit scope a commit scope's rule uses [Synchronous Commit](synchronous_commit), [Group Commit](group-commit), [Commit At Most Once](camo), [Lag Control](lag-control), or combination of these.

While most commit scope kinds control the processing of the transaction, Lag Control is the exception as it dynamically regulates the performance of the system in response to replication operations being slow or queued up. It is typically used, though, in combination with other commit scope kinds

Expand Down Expand Up @@ -173,6 +173,4 @@ SELECT bdr.alter_node_group_option(
config_key := 'default_commit_scope',
config_value := 'example_scope'
);
```


```
63 changes: 63 additions & 0 deletions product_docs/docs/pgd/5.6/commit-scopes/degrading.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
title: Degrading commit scope rules
navTitle: Degrading
deepToC: true
---

`SYNCHRONOUS_COMMIT`, `GROUP COMMIT`, and `CAMO` each have the optional capability of degrading the requirements for transactions when particular performance thresholds are crossed.

When a node is applying a transaction and that transaction times out, it can be useful to trigger a process of degrading the requirements of the transaction to be completed, rather than just rolling back.

`DEGRADE ON` offers a route for gracefully degrading the commit scope rule of a transaction. At its simplest, `DEGRADE ON` takes a timeout and a second set of commit scope operations that the commit scope can gracefully degrade to.

For instance, after 20ms or 30ms timeout, the requirements for satisfying a commit scope could degrade from `ALL (node_group_name) GROUP COMMIT` to `MAJORITY (node_group_name) GROUP COMMIT`, making the transactions apply more steadily.

You can also require that the write leader be the originator of a transaction in order for the degrade clause to be triggered. This can be helpful in "split brain scenarios" where you have, say, 2 data nodes and a witness node. Supposing there is a network split between the two data nodes and you have connections to both of the data nodes, only one of them will be allowed to degrade, because only one of them will be elected leader through the raft election with the witness node.

## Behavior

There are two parts to how the generalized `DEGRADE` clause behaves as it is applied to transactions.

Once during the commit, while the commit being processed is waiting for responses that satisfy the commit scope rule, PGD checks for a timeout and, if the timeout has expired, the commit being processed is reconfigured to wait for the commit scope rule in the `DEGRADE` clause. In fact, by this point, the commit scope rule in the `DEGRADE` clause might already be satisfied.

This mechanism alone is insufficient for the intended behavior, as this alone would mean that every transaction—even those that were certain to degrade due to connectivity issues—must wait for the timeout to expire before degraded mode kicks in, which would severely affect performance in such degrading-cluster scenarios.

To avoid this, the PGD manager process also periodically (every 5s) checks the connectivity and apply rate (the one in [bdr.node_replication_rates](/pgd/latest/reference/catalogs-visible/#bdrnode_replication_rates)) and if there are commit scopes that would degrade at that point based on the current state of replication, they will be automatically degraded—such that any transaction using that commit scope when processing after that uses the degraded rule instead of waiting for timeout—until the manager process detects that replication is moving swiftly enough again.

## SYNCHRONOUS_COMMIT and GROUP COMMIT

Both `SYNCHRONOUS_COMMIT` and `GROUP COMMIT` have `timeout` and `require_write_lead` parameters, with defaults of `0` and `false` respectively. You should probably always set the `timeout`, as the default of `0` causes an instant degrade. You can also require that the write leader be the originator of the transaction in order to switch to degraded (again, default is `false`).

Both `SYNCHRONOUS_COMMIT` and `GROUP COMMIT` also have options regarding which rule you can degrade to—which depends on which rule you are degrading from.

First of all, you can degrade to asynchronous operation:

```sql
ALL (left_dc) SYNCHRONOUS_COMMIT DEGRADE ON (timeout=20s) TO ASYNC
```

You can also degrade to a less restrictive commit group with the same commit scope kind (again as long as the kind is either `SYNCHRONOUS_COMMIT` or `GROUP COMMIT`). For instance, you can degrade as follows:

```sql
ALL (left_dc) SYNCHRONOUS_COMMIT DEGRADE ON (timeout=20s) TO MAJORITY (left_dc) SYNCHRONOUS_COMMIT
```

or as follows:

```sql
ANY 3 (left_dc) SYNCHRONOUS_COMMIT DEGRADE ON (timeout=20s) TO ANY 2 (left_dc) SYNCHRONOUS_COMMIT
```

But you cannot degrade from `SYNCHRONOUS_COMMIT` to `GROUP COMMIT` or the other way around.

## CAMO

While [CAMO](camo) supports both the same `timeout` and `require_write_lead` parameters (with the same defaults, `0` and `false` respectively), the options are simpler in that you can only degrade to asynchronous operation.

```sql
ALL (left_dc) CAMO DEGRADE ON (timeout=20ms, require_write_lead=true) TO ASYNC
```

Again, you should set the `timeout` parameter, as the default is `0`.


4 changes: 1 addition & 3 deletions product_docs/docs/pgd/5.6/commit-scopes/group-commit.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,7 @@ See the Group Commit section of [Limitations](limitations#group-commit).

## Configuration

To use Group Commit, first define a [commit scope](commit-scopes). The commit scope
determines the PGD nodes involved in the commit of a transaction.
`GROUP_COMMIT` supports optional `GROUP COMMIT` parameters, as well as `ABORT ON` and `DEGRADE ON` clauses. For a full description of configuration parameters, see the [GROUP_COMMIT](/pgd/latest/reference/commit-scopes/#group-commit) commit scope reference or for more regarding `DEGRADE ON` options in general, see the [Degrade options](degrading) section.

## Confirmation

Expand All @@ -63,7 +62,6 @@ determines the PGD nodes involved in the commit of a transaction.
`durable` | Confirms the transaction after all of its changes are flushed to disk.
`visible` (default) | Confirms the transaction after all of its changes are flushed to disk and it's visible to concurrent transactions.


## Behavior

The behavior of Group Commit depends on the configuration applied by the commit
Expand Down
27 changes: 14 additions & 13 deletions product_docs/docs/pgd/5.6/commit-scopes/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@ navigation:
- commit-scopes
- commit-scope-rules
- comparing
- degrading
- '# Commit Scope kinds'
- synchronous_commit
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why synchronous_commit is being pushed ahead of group commit. Group commit, synchronous commit, camo and then lag control (by use/feature maturity?)

- group-commit
- camo
- lag-control
- synchronous_commit
- '# Working with'
- administering
- legacy-sync
Expand All @@ -24,8 +25,8 @@ redirects:
---

EDB Postgres Distributed (PGD) offers a range of synchronous modes to complement its
default asynchronous replication. These synchronous modes are configured through
commit scopes. Commit scopes are rules that define how operations are handled and when the system
default asynchronous replication. You use commit scopes to configure these synchronous modes.
Commit scopes are rules that define how PGD handles synchronous operations and when the system
considers a transaction committed.

## Introducing
Expand All @@ -34,18 +35,23 @@ considers a transaction committed.
terminology that's used when discussing synchronous commits.

* [Durability terminology](durabilityterminology) lists terms used around PGD's
durability options, including how nodes are referred to in replication.
durability options, including how to refer to nodes in replication.

* [Commit scopes](commit-scopes) is a more in-depth look at the structure
of commit scopes and how to define them for your needs.

* [Commit scope rules](commit-scope-rules) looks at the syntax of and how to formulate
* [Commit scope rules](commit-scope-rules) looks at the syntax of and how to compose
a commit scope rule.

* [Comparing](comparing) compares how each option behaves.
* [Comparing durability options](comparing) compares how commit scope options behave with regard to durability.

* [Degrading commit scope rules](degrading) shows how to set up a commit scope rule that can gracefully degrade to a lower setting in case of timeouts with a stricter setting.

## Commit scope kinds

* [Synchronous Commit](synchronous_commit) is a commit scope mechanism that works
in a similar fashion to legacy synchronous replication, but from within the commit scope framework.

* [Group Commit](group-commit) focuses on the Group Commit option, where you can
define a transaction as done when a group of nodes agrees it's done.

Expand All @@ -57,9 +63,6 @@ retrying. This ensures that their commits only happen at most once.
dynamically throttle nodes according to the slowest node and regulates how far
out of sync nodes may go when a database node goes out of service.

* [Synchronous Commit](synchronous_commit) examines a commit scope mechanism that works
in a similar fashion to legacy synchronous replication, but from within the commit scope framework.

## Working with commit scopes

* [Administering](administering) addresses how to manage a PGD cluster with Group Commit
Expand All @@ -69,8 +72,6 @@ in use.
that aren't currently supported or aren't possible. Refer to this before deciding
on a durability strategy.

* [Legacy synchronous replication](legacy-sync) shows how traditional Postgres synchronous operations
can still be accessed under PGD.
* [Legacy synchronous replication](legacy-sync) shows how you can still access traditional Postgres synchronous operations under PGD.

* [Internal timing of operations](timing) compares legacy replication with PGD's async and synchronous
operations, especially the difference in the order by which transactions are flushed to disk or made visible.
* [Internal timing of operations](timing) compares legacy replication with PGD's async and synchronous operations, especially the difference in the order by which transactions are flushed to disk or made visible.
5 changes: 3 additions & 2 deletions product_docs/docs/pgd/5.6/commit-scopes/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ to applications that are related but can all be implemented individually:

Commit scopes allow four kinds of controlling durability of the transaction:

- [Synchronous Commit](synchronous_commit): This kind of commit scope allows for a behavior where the origin node awaits a majority of nodes to confirm and behaves more like a native Postgres synchronous commit.

- [Group Commit](group-commit): This kind of commit scope controls which and how many nodes
have to reach a consensus before the transaction is considered to be committable
and at what stage of replication it can be considered committed. This option also
Expand All @@ -48,8 +50,7 @@ Commit scopes allow four kinds of controlling durability of the transaction:
- [Lag Control](lag-control): This kind of commit scope controls how far behind nodes can
be in terms of replication before allowing commit to proceed.

- [PGD Synchronous Commit](synchronous_commit): This kind of commit scope allows for a behavior where the origin node awaits a majority of nodes to confirm and behaves more like a native Postgres synchronous commit.

Synchronous commit, group commit, and CAMO each support [degrading commit scope rules](degrading), for even further control of durability.

!!! Note Legacy synchronization availability
For backward compatibility, PGD still supports configuring synchronous
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
---
title: PGD Synchronous Commit
title: Synchronous Commit
jpe442 marked this conversation as resolved.
Show resolved Hide resolved
deepToC: true
---

Commit scope kind: `SYNCHRONOUS_COMMIT`
alias: `SYNCHRONOUS COMMIT`

## Overview

Expand All @@ -26,7 +27,7 @@ SELECT bdr.add_commit_scope(

## Configuration

`SYNCHRONOUS_COMMIT` has no parameters to configure.
`SYNCHRONOUS_COMMIT` supports the optional `DEGRADE ON` clause. See the [`SYNCHRONOUS_COMMIT`](/pgd/latest/reference/commit-scopes/#synchronous_commit) commit scope reference for specific configuration parameters or see [this section](degrading) regarding Degrade on options.

## Confirmation

Expand Down
Loading
Loading