Skip to content

Commit

Permalink
Added more details regarding shard allocation awareness attributes
Browse files Browse the repository at this point in the history 
Signed-off-by: Smit Patel <[email protected]>
  • Loading branch information
@psmitj
psmitj committed Sep 16, 2024
1 parent 8c74b88 commit 8c4ea94
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion _tuning-your-cluster/index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,11 @@ To better understand and monitor your cluster, use the [CAT API]({{site.url}}{{s
### Shard allocation awareness
If your nodes are spread across several geographical zones, you can configure shard allocation awareness to allocate all replica shards to a zone that’s different from their primary shard.
You can set custom node attributes on Opensearch nodes to be used for shard allocation.

Check failure on line 197 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[Vale.Terms] Use 'OpenSearch' instead of 'Opensearch'.

Raw Output:
{"message": "[Vale.Terms] Use 'OpenSearch' instead of 'Opensearch'.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 197, "column": 39}}}, "severity": "ERROR"}
For example, you can set the `zone` attribute on each node to represent the zone in which the node is located and use this attribute as shard allocation awareness attribute to ensure that primary and its replica shards are allocated in a balanced manner across available distinct zones i.e. maximum shard copies per zone = ceil (number of shard copies / number of distinct zones).

Check failure on line 199 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SpacingSlash] When using '/' between words, do not insert space on either side of it.

Raw Output:
{"message": "[OpenSearch.SpacingSlash] When using '/' between words, do not insert space on either side of it.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 199, "column": 346}}}, "severity": "ERROR"}
If your index has total 5 shard copies (1 primary and 4 replicas) and nodes in only 3 distinct zones, then OpenSearch will not allocate more than 2 shards per zone, so you need atleast 2 nodes in 2 zones each and atleast 1 node in the 3rd zone to allocate all 5 shard copies. If you have 3 nodes in 1st zone and 1 node in remaining zones each, then OpenSearch will only allocate 2,1,1 shard copies leaving the 5th copy unallocated.

Check failure on line 201 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Spelling] Error: atleast. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.

Raw Output:
{"message": "[OpenSearch.Spelling] Error: atleast. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 201, "column": 178}}}, "severity": "ERROR"}

Check failure on line 201 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Spelling] Error: atleast. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.

Raw Output:
{"message": "[OpenSearch.Spelling] Error: atleast. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 201, "column": 214}}}, "severity": "ERROR"}
With shard allocation awareness, if the nodes in one of your zones fail, you can be assured that your replica shards are spread across your other zones. It adds a layer of fault tolerance to ensure your data survives a zone failure beyond just individual node failures.
Expand All @@ -219,6 +223,8 @@ PUT _cluster/settings
}
```

You can also use multiple attributes as the shard allocation awareness attributes, just provide the attributes as a comma separated string in above setting e.g. 'zone,rack'.

Check warning on line 226 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Simple] Don't use 'just' because it's not neutral in tone. If you mean 'only', use 'only' instead.

Raw Output:
{"message": "[OpenSearch.Simple] Don't use 'just' because it's not neutral in tone. If you mean 'only', use 'only' instead.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 226, "column": 84}}}, "severity": "WARNING"}

Check warning on line 226 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.DirectionAboveBelow] Use 'previous, preceding, or earlier' instead of 'above' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions.

Raw Output:
{"message": "[OpenSearch.DirectionAboveBelow] Use 'previous, preceding, or earlier' instead of 'above' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 226, "column": 143}}}, "severity": "WARNING"}

Check failure on line 226 in _tuning-your-cluster/index.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SpacingPunctuation] There should be no space before and one space after the punctuation mark in 'zone,rack'.

Raw Output:
{"message": "[OpenSearch.SpacingPunctuation] There should be no space before and one space after the punctuation mark in 'zone,rack'.", "location": {"path": "_tuning-your-cluster/index.md", "range": {"start": {"line": 226, "column": 163}}}, "severity": "ERROR"}

You can either use `persistent` or `transient` settings. We recommend the `persistent` setting because it persists through a cluster reboot. Transient settings don't persist through a cluster reboot.

Shard allocation awareness attempts to separate primary and replica shards across multiple zones. However, if only one zone is available (such as after a zone failure), OpenSearch allocates replica shards to the only remaining zone.
Expand Down

0 comments on commit 8c4ea94

Please sign in to comment.