Skip to content

Commit

Permalink
(Doc+) Expand watermark resolution
Browse files Browse the repository at this point in the history
Relaunch #116892 since the original one seems to be outdated and hard to update branch.
  • Loading branch information
kunisen committed Dec 20, 2024
1 parent 845021d commit 6209be2
Showing 1 changed file with 25 additions and 10 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ GET _cluster/allocation/explain
[[fix-watermark-errors-temporary]]
==== Temporary Relief

To immediately restore write operations, you can temporarily increase the
To immediately restore write operations, you may consider temporarily increasing the
<<disk-based-shard-allocation,disk watermarks>> and remove the
<<index-block-settings,write block>>.

Expand Down Expand Up @@ -102,19 +102,34 @@ PUT _cluster/settings
[[fix-watermark-errors-resolve]]
==== Resolve

As a long-term solution, we recommend you do one of the following best suited
To resolve, we recommend you do one of the following best suited
to your use case:

* add nodes to the affected <<data-tiers,data tiers>>
+
TIP: You should enable <<xpack-autoscaling,autoscaling>> for clusters deployed using our {ess}, {ece}, and {eck} platforms.
* horizontally scale nodes of the affected <<data-tiers,data tiers>>

* upgrade existing nodes to increase disk space
+
TIP: On {ess}, https://support.elastic.co[Elastic Support] intervention may
become necessary if <<cluster-health,cluster health>> reaches `status:red`.
* vertically scale existing nodes to increase disk space

* delete unneeded indices using the <<indices-delete-index,delete index API>>
* delete indices using the <<indices-delete-index,delete index API>>, either
permanently if unneeded or temporarily to later
<<snapshots-restore-snapshot,snapshot restore>>

* update related <<index-lifecycle-management,ILM policy>> to push indices
through to later <<data-tiers,data tiers>>

TIP: On {ess} and {ece}, indices may need to be temporarily deleted via
its {cloud}/ec-api-console.html[Elasticsearch API Console] to later
<<snapshots-restore-snapshot,snapshot restore>> in order to resolve
<<cluster-health,cluster health>> `status:red` which will block
{cloud}/ec-activity-page.html[attempted changes]. If you experience issues
with this resolution flow on {ess}, kindly reach out to
https://support.elastic.co[Elastic Support] for assistance.

To avoid this situation, we recommend you do one of the following best
suited to your use case:

* enable <<xpack-autoscaling,autoscaling>> for clusters deployed using our
{ess}, {ece}, and {eck} platforms

* setup {kibana-ref}/kibana-alerts.html[Stack Monitoring Alerts] on top of
<<monitor-elasticsearch-cluster,{es} Monitoring>> to be notified before
flood-stage watermark is reached

0 comments on commit 6209be2

Please sign in to comment.