Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation changes for Snapshot Status API #8235

Merged
merged 7 commits into from
Sep 16, 2024
Merged

Add documentation changes for Snapshot Status API #8235

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b55b4b8
Snapshot Status API documentation changes for shallow v2 snapshots
ltaragi Sep 12, 2024
d563b83
Apply suggestions from code review
Naarcha-AWS Sep 13, 2024
ae56aaa
Apply suggestions from code review
Naarcha-AWS Sep 16, 2024
7079117
Update get-snapshot-status.md
Naarcha-AWS Sep 16, 2024
042561c
Apply suggestions from code review
Naarcha-AWS Sep 16, 2024
f52bbd5
Apply suggestions from code review
Naarcha-AWS Sep 16, 2024
46c384b
Apply suggestions from code review
Naarcha-AWS Sep 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions _api-reference/snapshots/get-snapshot-status.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,24 +23,29 @@
| Parameter | Data type | Description |
:--- | :--- | :---
| repository | String | Repository containing the snapshot. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| snapshot | String | Snapshot to return. |
| snapshot | List | Snapshot(s) to return. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| index | List | Index/Indices to include in the response. |

Check failure on line 27 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'Indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'Indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 27, "column": 24}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

Three request variants provide flexibility:

* `GET _snapshot/_status` returns the status of all currently running snapshots in all repositories.

* `GET _snapshot/<repository>/_status` returns all currently running snapshots in the specified repository. This is the preferred variant.

* `GET _snapshot/<repository>/<snapshot>/_status` returns detailed status information for a specific snapshot in the specified repository, regardless of whether it's currently running or not.
* `GET _snapshot/<repository>/<snapshot>/_status` returns detailed status information for a specific snapshot(s) in the specified repository, regardless of whether it's currently running or not.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

Using the API to return state for other than currently running snapshots can be very costly for (1) machine machine resources and (2) processing time if running in the cloud. For each snapshot, each request causes file reads from all a snapshot's shards.
* `GET /_snapshot/<repository>/<snapshot>/<index>/_status` returns detailed status information only for the specified indices in a specific snapshot in the specified repository, for completed snapshots. Note that this variant works only for indices of a specific snapshot.

Check failure on line 37 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 37, "column": 119}}}, "severity": "ERROR"}

Check failure on line 37 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 37, "column": 242}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

Using the API to return state for other than currently running snapshots can be very costly for (1) machine machine resources and (2) processing time if running in the cloud. For each snapshot, each request causes file reads from all of the snapshot's shards.

Check failure on line 39 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Repetition] 'machine' is repeated.

Raw Output:
{"message": "[OpenSearch.Repetition] 'machine' is repeated.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 39, "column": 101}}}, "severity": "ERROR"}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ltaragi: How would a user avoid returning states on snapshots that aren't currently running?

Copy link
Contributor Author

@ltaragi ltaragi Sep 16, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry did not fully get the question. The user can make use of the snapshot.max_shards_allowed_in_status_api cluster setting or indexes-filter to limit the cost of returning states for completed snapshots (ones that are currently not running).

Accordingly, the API call works only if the total number of shards across the requested resources (snapshot(s)/indices of a snapshot) is less than the limit specified by the following cluster setting

Check failure on line 40 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 40, "column": 112}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
- `snapshot.max_shards_allowed_in_status_api`(Dynamic, integer): The maximum number of shards that can be included in the snapshot status API response. Default value is `200000`. Not applicable for [Shallow V2 Snapshots]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/snapshot-interoperability##shallow-snapshot-v2), where the total number and size of files are returned as 0.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should "Snapshot Status API" be capitalized (unclear if it's the name of the API)? Is "Shallow V2 Snapshots" intentionally capitalized? It should be lowercase if referencing the general noun. "size" => "sizes"? Should 0 be in code font?

Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
{: .warning}

## Request fields

| Field | Data type | Description |
:--- | :--- | :---
| ignore_unavailable | Boolean | How to handles requests for unavailable snapshots. If `false`, the request returns an error for unavailable snapshots. If `true`, the request ignores unavailable snapshots, such as those that are corrupted or temporarily cannot be returned. Defaults to `false`.|
| ignore_unavailable | Boolean | How to handle requests for unavailable snapshots and indices. If `false`, the request returns an error for unavailable snapshots and indices. If `true`, the request ignores unavailable snapshots and indices, such as those that are corrupted or temporarily cannot be returned. Defaults to `false`.|

Check failure on line 48 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 48, "column": 87}}}, "severity": "ERROR"}

Check failure on line 48 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 48, "column": 167}}}, "severity": "ERROR"}

Check failure on line 48 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'indexes' instead of 'indices'.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 48, "column": 233}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

## Example request

Expand Down Expand Up @@ -374,19 +379,19 @@
| Field | Data type | Description |
:--- | :--- | :---
| repository | String | Name of repository that contains the snapshot. |
| snapshot | String | Snapshot name. |
| uuid | String | Snapshot Universally unique identifier (UUID). |
| snapshot | String | Snapshot's name. |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| uuid | String | Snapshot's Universally unique identifier (UUID). |

Check failure on line 383 in _api-reference/snapshots/get-snapshot-status.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Spelling] Error: uuid. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.

Raw Output:
{"message": "[OpenSearch.Spelling] Error: uuid. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_api-reference/snapshots/get-snapshot-status.md", "range": {"start": {"line": 383, "column": 3}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| state | String | Snapshot's current status. See [Snapshot states](#snapshot-states). |
| include_global_state | Boolean | Whether the current cluster state is included in the snapshot. |
| shards_stats | Object | Snapshot's shard counts. See [Shard stats](#shard-stats). |
| stats | Object | Details of files included in the snapshot. `file_count`: number of files. `size_in_bytes`: total of all fie sizes. See [Snapshot file stats](#snapshot-file-stats). |
| stats | Object | Details of files included in the snapshot. `file_count`: number of files. `size_in_bytes`: total of all file sizes. See [Snapshot file stats](#snapshot-file-stats). |
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
| index | list of Objects | List of objects that contain information about the indices in the snapshot. See [Index objects](#index-objects).|

##### Snapshot states

| State | Description |
:--- | :--- |
| FAILED | The snapshot terminated in an error and no data was stored. |
| FAILED | The snapshot terminated in an error and no data was stored. |
| IN_PROGRESS | The snapshot is currently running. |
| PARTIAL | The global cluster state was stored, but data from at least one shard was not stored. The `failures` property of the [Create snapshot]({{site.url}}{{site.baseurl}}/api-reference/snapshots/create-snapshot) response contains additional details. |
| SUCCESS | The snapshot finished and all shards were stored successfully. |
Expand Down Expand Up @@ -420,4 +425,4 @@
:--- | :--- | :--- |
| shards_stats | Object | See [Shard stats](#shard-stats). |
| stats | Object | See [Snapshot file stats](#snapshot-file-stats). |
natebower marked this conversation as resolved.
Show resolved Hide resolved
| shards | list of Objects | List of objects containing information about the shards that include the snapshot. OpenSearch returns the following properties about the shards. <br /><br /> **stage**: Current state of shards in the snapshot. Shard states are: <br /><br /> * DONE: Number of shards in the snapshot that were successfully stored in the repository. <br /><br /> * FAILURE: Number of shards in the snapshot that were not successfully stored in the repository. <br /><br /> * FINALIZE: Number of shards in the snapshot that are in the finalizing stage of being stored in the repository. <br /><br />* INIT: Number of shards in the snapshot that are in the initializing stage of being stored in the repository.<br /><br />* STARTED: Number of shards in the snapshot that are in the started stage of being stored in the repository.<br /><br /> **stats**: See [Snapshot file stats](#snapshot-file-stats). <br /><br /> **total**: Total number and size of files referenced by the snapshot. <br /><br /> **start_time_in_millis**: Time (in milliseconds) when snapshot creation began. <br /><br /> **time_in_millis**: Total time (in milliseconds) that the snapshot took to complete. |
| shards | list of Objects | List of objects containing information about the shards that are included in the snapshot. OpenSearch returns the following properties about the shards. <br /><br /> **stage**: Current state of shards in the snapshot. Shard states are: <br /><br /> * DONE: Number of shards in the snapshot that were successfully stored in the repository. <br /><br /> * FAILURE: Number of shards in the snapshot that were not successfully stored in the repository. <br /><br /> * FINALIZE: Number of shards in the snapshot that are in the finalizing stage of being stored in the repository. <br /><br />* INIT: Number of shards in the snapshot that are in the initializing stage of being stored in the repository.<br /><br />* STARTED: Number of shards in the snapshot that are in the started stage of being stored in the repository.<br /><br /> **stats**: See [Snapshot file stats](#snapshot-file-stats). <br /><br /> **total**: Total number and size of files referenced by the snapshot. <br /><br /> **start_time_in_millis**: Time (in milliseconds) when snapshot creation began. <br /><br /> **time_in_millis**: Total time (in milliseconds) that the snapshot took to complete. |
natebower marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Loading