Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add documentation changes for Snapshot Status API #8235

Merged
merged 7 commits into from
Sep 16, 2024
Merged

Add documentation changes for Snapshot Status API #8235

merged 7 commits into from
Sep 16, 2024

Conversation

ltaragi
Copy link
Contributor

@ltaragi ltaragi commented Sep 12, 2024

Description

Describe what this change achieves.
This documentation change addresses the necessary changes to Snapshot Status API as part of V2 snapshots:

  • Allow index filter in the status API if the request contains only one snapshot name. Thus, the status API response will only consist of the specified indices in the filter (if any).
  • For non-shallow-V2 snapshots:
    • The status API call works only if the total number of shards across the requested resources (snapshots or the indices of a particular snapshot) is less than a certain amount determined by the cluster setting snapshot.max_shards_allowed_in_status_api
  • For shallow snapshots of type V2, the file count and size stats are 0 at all levels. For these snapshots, snapshot.max_shards_allowed_in_status_api cluster setting is not honoured

Issues Resolved

Closes #8231

Version

This applies to OpenSearch version 2.17 and later

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions GitHub Actions
Copy link

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

@kolchfa-aws kolchfa-aws added release-notes PR: Include this PR in the automated release notes v2.17.0 labels Sep 12, 2024
Naarcha-AWS
Naarcha-AWS reviewed Sep 13, 2024
View reviewed changes
_api-reference/snapshots/get-snapshot-status.md Outdated
Using the API to return state for other than currently running snapshots can be very costly for (1) machine machine resources and (2) processing time if running in the cloud. For each snapshot, each request causes file reads from all a snapshot's shards.
* `GET /_snapshot/<repository>/<snapshot>/<index>/_status` returns detailed status information only for the specified indices in a specific snapshot in the specified repository, for completed snapshots. Note that this variant works only for indices of a specific snapshot.

Using the API to return state for other than currently running snapshots can be very costly for (1) machine machine resources and (2) processing time if running in the cloud. For each snapshot, each request causes file reads from all of the snapshot's shards.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ltaragi: How would a user avoid returning states on snapshots that aren't currently running?

Copy link
Contributor Author

@ltaragi ltaragi Sep 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry did not fully get the question. The user can make use of the snapshot.max_shards_allowed_in_status_api cluster setting or indexes-filter to limit the cost of returning states for completed snapshots (ones that are currently not running).

Naarcha-AWS
Naarcha-AWS reviewed Sep 13, 2024
View reviewed changes
Copy link
Collaborator

@Naarcha-AWS Naarcha-AWS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ltaragi: Need a technical clarification before we can move forward with this one.

@Naarcha-AWS Naarcha-AWS added the 4 - Doc review PR: Doc review in progress label Sep 13, 2024
@Naarcha-AWS Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Sep 16, 2024
natebower
natebower requested changes Sep 16, 2024
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ltaragi @Naarcha-AWS Please see my comments and changes and tag me for approval once addressed. Thanks!

_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Outdated Show resolved Hide resolved
_api-reference/snapshots/get-snapshot-status.md Show resolved Hide resolved
@Naarcha-AWS @natebower
Apply suggestions from code review
042561c 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
natebower
natebower approved these changes Sep 16, 2024
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved with the substitution of my edited text on line 44 and proper formatting of the bulleted list in the table (line 431). Thanks!

@Naarcha-AWS @natebower
Apply suggestions from code review
46c384b 
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
@Naarcha-AWS Naarcha-AWS merged commit d15c7bf into opensearch-project:main Sep 16, 2024
5 checks passed
noahstaveley pushed a commit to noahstaveley/documentation-website that referenced this pull request Sep 23, 2024
@ltaragi @Naarcha-AWS
@natebower @noahstaveley
Add documentation changes for Snapshot Status API (opensearch-project…
640bbb0 
…#8235)

* Snapshot Status API documentation changes for shallow v2 snapshots

Signed-off-by: Lakshya Taragi <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update get-snapshot-status.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Lakshya Taragi <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Noah Staveley <[email protected]>
noahstaveley pushed a commit to noahstaveley/documentation-website that referenced this pull request Sep 23, 2024
@ltaragi @Naarcha-AWS
@natebower @noahstaveley
Add documentation changes for Snapshot Status API (opensearch-project…
01d7dd6 
…#8235)

* Snapshot Status API documentation changes for shallow v2 snapshots

Signed-off-by: Lakshya Taragi <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Update get-snapshot-status.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <[email protected]>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Lakshya Taragi <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
Co-authored-by: Naarcha-AWS <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
Signed-off-by: Noah Staveley <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natebower natebower natebower approved these changes

@Naarcha-AWS Naarcha-AWS Naarcha-AWS approved these changes

@kolchfa-aws kolchfa-aws Awaiting requested review from kolchfa-aws kolchfa-aws is a code owner

@vagimeli vagimeli Awaiting requested review from vagimeli

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@dlvenable dlvenable Awaiting requested review from dlvenable dlvenable is a code owner

@stephen-crawford stephen-crawford Awaiting requested review from stephen-crawford

@epugh epugh Awaiting requested review from epugh epugh is a code owner

Assignees

@Naarcha-AWS Naarcha-AWS

Labels
5 - Editorial review PR: Editorial review in progress release-notes PR: Include this PR in the automated release notes v2.17.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[DOC] Documentation changes for Snapshot Status API for 2.17 and above
4 participants
@ltaragi @Naarcha-AWS @natebower @kolchfa-aws