Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update for managed backups cloud 2.0 #18919

Merged
merged 10 commits into from
Sep 24, 2024

Conversation

kathancox
Copy link
Contributor

@kathancox kathancox commented Sep 18, 2024

Fixes DOC-9993

In this PR:

Configurable backup work
Managed-Service Backups --> Managed Backups
Customer-Owned Backups --> Self-Managed Backups

Rendered Previews

Managed Backups page
Self-managed backups

Copy link

netlify bot commented Sep 18, 2024

Netlify Preview

Name Link
🔨 Latest commit f23c498
🔍 Latest deploy log https://app.netlify.com/sites/cockroachdb-docs/deploys/66f30e80fe23e70008f6d4f7
😎 Deploy Preview https://deploy-preview-18919--cockroachdb-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.


For instructions on how to view and configure managed backup settings, use:

- The [Cloud Console](#cloud-console) for {{ site.data.products.standard }} or {{ site.data.products.advanced }}.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to also have links here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added!

You can set backup frequency to one of the following options:

{% include cockroachcloud/backups/frequency-settings.md %}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we will need to add a note here which says that after you configure your backup, we will take another full backup. https://cockroachlabs.slack.com/archives/C07G3TKSKT8/p1726582361854129

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added!

If you have upgraded from a {{ site.data.products.basic }} cluster to a {{ site.data.products.standard }} cluster, the existing backup schedules will still apply, but you can then configure the frequency and retention of future managed backups in the {{ site.data.products.standard }} cluster.

If you have downgraded from a {{ site.data.products.standard }} cluster to a {{ site.data.products.basic }} cluster, existing managed backups will be retained for the configured retention duration. The default managed backups in {{ site.data.products.basic }} clusters will be taken every 24 hours and have a 30-day retention.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have this issue where the retention period really only applies to the last full backup, because once the full backup expires we don't have a way to restore the incrementals. So if a retention period is set to 2d, it's really more like 1d because we take a full back every day. But we're also not exposing the full backup concept to users. I wonder if maybe we should expose the concept that "cockroach will take the correct combination of full/incremental backups on your behalf" otherwise it'll be hard to explain this.

Copy link

@kev-cao kev-cao Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to provide some more info on this, if a full backup is taken once every f, and a retention period is set to r, then an incremental is useable anywhere between r and r-f.

For example, to alleviate this, we have decided to take full backups at least every day. So that means with a 30 day retention, the incrementals could be useable for anywhere between 30 days and 29 days. With a 7 day retention, the incrementals could be useable for anywhere between 7 days and 6 days.

If fulls were taken every 5 minutes, then with a 7 day retention, incrementals would be useable between 7 days and 6 days, 23 hours, and 55 minutes.

We don't want to be so specific in our documentation, but hopefully this guides the wording.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah but fulls are not taken every 5minutes, I feel like we may need to be more explicit about that? what do you think?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh what I mean about "not so specific" is we probably don't need to go into the math so much?

Given current limitations with full/incremental recurrence ratio, I think the highest full backup frequency we can do is 4 hours, and the lowest we are doing is capped at 1 day. So that means for some retention r, the incrementals could be useable for at least r - 4 hours or r - 1 day. But this is highly dependent on the full backup frequency, which customers cannot set separately.

Additionally, these numbers could be changing depending on how we set our full backup frequency (as of right now, the actual highest full backup frequency is once every hour with a 5 minute RPO). So I don't want the documentation to be too specific.

Copy link
Contributor Author

@kathancox kathancox Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alicia-l2 @kev-cao So, are we taking full backups every 24 hours (whatever the frequency is). Or, might it be more often than that?

I'm trying to figure out how customers can identify the usable backups, and how I can explain that. If the full backup frequency is changeable, that's tough to reason about (particularly in Standard where the backup type is not labeled).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the two paragraphs for this that I sent to you both on Slack. Feel free to suggest edits to this still

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me

@kathancox kathancox marked this pull request as ready for review September 20, 2024 13:51
@kathancox
Copy link
Contributor Author

@alicia-l2 + @kev-cao would appreciate your review on where the PR is at now. FYI I have sent this to Melissa as well.

Copy link

@alicia-l2 alicia-l2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some small comments

- The frequency of the backups to meet [recovery point objective (RPO)]({% link {{site.current_cloud_version}}/disaster-recovery-overview.md %}#resilience-strategy) requirements.
- The retention of the backups to set how long Cockroach Labs retains the backups.

In [{{ site.data.products.basic }} clusters](#basic-clusters), you can view the default daily managed backups in the Cloud Console.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know that we need to explicitly say that you can view the daily managed backups in the cloud console, maybe we just say that Basic clusters have a default non configurable schedule? or omit altogether?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, went with "default non-configurable"


Cockroach Labs will take a managed backup every 24 hours. By default, managed backups will be retained for 30 days in {{ site.data.products.basic }} clusters.

Once a cluster or organization is deleted, Cockroach Labs retains the backup for 30 days.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we could change this sentence up to just say "Backups follow the default retention period of 30d even after a cluster or organization deletion." or something like that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

src/current/cockroachcloud/managed-backups.md Show resolved Hide resolved
If you have upgraded from a {{ site.data.products.basic }} cluster to a {{ site.data.products.standard }} cluster, the existing backup schedules will still apply, but you can then configure the frequency and retention of future managed backups in the {{ site.data.products.standard }} cluster.

If you have downgraded from a {{ site.data.products.standard }} cluster to a {{ site.data.products.basic }} cluster, existing managed backups will be retained for the configured retention duration. The default managed backups in {{ site.data.products.basic }} clusters will be taken every 24 hours and have a 30-day retention.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me

@kathancox kathancox force-pushed the configurable-managed-backups-2.0 branch from 3d96c16 to f65d2bf Compare September 20, 2024 18:15
Copy link

@kev-cao kev-cao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@florence-crl florence-crl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm pending suggestions (many repetitions of same comments for different versions or files)

  • change dedicated to advanced
  • fix links with filters
  • liquify links

src/current/cockroachcloud/backup-and-restore-overview.md Outdated Show resolved Hide resolved
src/current/cockroachcloud/managed-backups.md Outdated Show resolved Hide resolved
src/current/cockroachcloud/managed-backups.md Outdated Show resolved Hide resolved
src/current/cockroachcloud/managed-backups.md Outdated Show resolved Hide resolved
src/current/cockroachcloud/managed-backups.md Outdated Show resolved Hide resolved
src/current/v24.1/backup.md Outdated Show resolved Hide resolved
src/current/v24.1/cockroachdb-feature-availability.md Outdated Show resolved Hide resolved
src/current/v24.2/backup.md Outdated Show resolved Hide resolved
src/current/v24.2/cockroachdb-feature-availability.md Outdated Show resolved Hide resolved
Copy link

Files changed:

@kathancox kathancox merged commit dde4862 into cloud-2.0 Sep 24, 2024
4 checks passed
@kathancox kathancox deleted the configurable-managed-backups-2.0 branch September 24, 2024 19:22
mikeCRL added a commit that referenced this pull request Sep 25, 2024
* [DOC-9725] Update Cloud Security pages for new plans

1. Update Security Overview

* Authentication, authorization, client certs, network auth docs

* Network Authorization, Connect to your cluster, AWS Privatelink

* Update CMEK and Vault pages

- Consolidate cmek-ops-*.md into managing-cmek.md
- Use a common include for the Hashicorp tutorial
- Improve the TOC entries

* Update certificate management with Vault

- Also use a common include for the actual content

* Update compliance pages

* Update egress perimeter control and private clusters pages

* Update Cloud Org Audit Log page

* Update Security Hardening page

* Updates to Cluster SSO

* Update TLS page

* Fix links

* [DOC-10386][DOC-11063]  Update Plan a Standard Cluster, Cloud landing page

* Remove nonexistent include

* Add Standard and a couple missing preview features to FA page

* [DOC-9767] Docs for changing between Basic and Standard

* Fix formatting

* Verification fixes

* Update variables in Cloud upgrade policy

* In export-logs.md, removed incorrect mention of Azure monitor. (#18937)

In export-logs-advanced.md, added sentence about Azure monitor log naming format.

* Update for managed backups cloud 2.0 (#18919)

* In essential-metrics.md, replaced crdb_cloud with crdb_dedicated for Datadog prefix. (#18940)

* Update CRDB security reference updates

* Update Choose a deployment option

* Update architecture guide

* Update Learn SQL pages

* Update SQL migration pages

* Variables in remaining CRDB backup pages

More backup

* Variables in remaining 23.1 pages

* Fix link in plan-your-cluster to cloud-api

---------

Co-authored-by: Florence Morris <[email protected]>
Co-authored-by: Kathryn May <[email protected]>
Co-authored-by: Mike Lewis <[email protected]>
Co-authored-by: mikeCRL <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants