Skip to content

Commit

Permalink
Small improvement to UTF-8 docs (#9081)
Browse files Browse the repository at this point in the history
  • Loading branch information
grobinson-grafana authored Sep 2, 2024
1 parent 59092d3 commit 673cbe2
Showing 1 changed file with 9 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -184,15 +184,16 @@ Grafana Mimir provides a number of tools to help you identify whether any existi
To identify affected tenant configurations, take the following steps:

1. Make sure Mimir is running version 2.12 or later with debug-level logging enabled.
2. To identify any tenant configurations that are incompatible with UTF-8 (meaning the tenant configuration fails to load and the [fallback configuration](#fallback-configuration) is used instead), search Mimir server logs for lines containing `Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible`. Each log line includes the invalid matcher from the tenant configuration and the ID of the affected tenant. For example:

1. To identify any tenant configurations that are incompatible with UTF-8 (meaning the tenant configuration fails to load and the [fallback configuration](#fallback-configuration) is used instead), search Mimir server logs for lines containing `Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible`. Each log line includes the invalid matcher from the tenant configuration and the ID of the affected tenant. For example:

```
msg="Alertmanager is moving to a new parser for labels and matchers, and this input is incompatible. Alertmanager has instead parsed the input using the classic matchers parser as a fallback. To make this input compatible with the UTF-8 matchers parser please make sure all regular expressions and values are double-quoted. If you are still seeing this message please open an issue." input="foo=" err="end of input: expected label value" suggestion="foo=\"\"" user="1"
```

In this example, the tenant with User ID `1` has an incompatible matcher in their tenant configuration `foo=` and should to be changed to the suggestion `foo=""`.

3. To identify any tenant configurations that are compatible with UTF-8 but contain matchers that might change in behavior when its enabled, search Mimir server logs for lines containing `Matchers input has disagreement`. Disagreement occurs when a matcher is valid, but due to adding support for UTF-8, it can behave differently when UTF-8 is enabled.
1. To identify any tenant configurations that are compatible with UTF-8 but contain matchers that might change in behavior when its enabled, search Mimir server logs for lines containing `Matchers input has disagreement`. Disagreement occurs when a matcher is valid, but due to adding support for UTF-8, it can behave differently when UTF-8 is enabled.

```
msg="Matchers input has disagreement" input="foo=\"\\xf0\\x9f\\x99\\x82\"" user="1"
Expand All @@ -208,7 +209,7 @@ To fix any identified tenant configurations, take the following steps:

1. Use the `migrate-utf8` [command]({{< relref "./../../../manage/tools/mimirtool#migrate-alertmanager-configuration-for-utf-8-in-mimir-212-and-later" >}}) in mimirtool to fix any tenant configurations that are incompatible with UTF-8. This command can migrate existing tenant configurations in a way that is backwards-compatible, doesn't change the behavior of existing matchers, and works even in Mimir installations that don't have UTF-8 enabled. If you cannot use mimirtool, you can edit tenant configurations by hand through applying each suggestion from the Mimir server logs.

2. You must look at tenant configurations that have disagreement on a case-by-case basis. Depending on the nature of the disagreement, you might not need to fix a matcher with disagreement. For example `\xf0\x9f\x99\x82` is the byte sequence for the 🙂 emoji. If the intention is to match a literal 🙂 emoji then no change is required. However, if the intention is to match the literal `\xf0\x9f\x99\x82` then you need to change the matcher to use `\\xf0\\x9f\\x99\\x82` instead.
1. You must look at tenant configurations that have disagreement on a case-by-case basis. Depending on the nature of the disagreement, you might not need to fix a matcher with disagreement. For example `\xf0\x9f\x99\x82` is the byte sequence for the 🙂 emoji. If the intention is to match a literal 🙂 emoji then no change is required. However, if the intention is to match the literal `\xf0\x9f\x99\x82` then you need to change the matcher to use `\\xf0\\x9f\\x99\\x82` instead.

{{< admonition type="note" >}}
It's rare to find cases of disagreement in a tenant configuration, as most tenants do not need to match alerts that contain literal UTF-8 byte sequences in their labels.
Expand All @@ -218,8 +219,10 @@ It's rare to find cases of disagreement in a tenant configuration, as most tenan

1. After identifying and fixing all affected tenant configurations, check the Mimir server logs again to make sure you haven't missed any tenant configurations.

2. You can now enable UTF-8 by setting [`utf8-strict-mode-enabled`]({{< relref "./../../../configure/configuration-parameters#alertmanager" >}}) to `true`.
1. To enable UTF-8, set [`utf8-strict-mode-enabled`]({{< relref "./../../../configure/configuration-parameters#alertmanager" >}}) to `true`. You must restart Mimir for the changes to take effect.

1. To confirm UTF-8 is enabled, search for `Starting Alertmanager in UTF-8 strict mode` in the Mimir server logs. If you find `Starting Alertmanager in classic mode` instead then UTF-8 is not enabled.

3. Any incompatible tenant configurations will fail to load. You can identify if any tenant configurations are failing to load by searching the Mimir server logs for lines containing `error applying config`, or querying the `alertmanager_config_last_reload_successful` gauage for `0`.
1. Any incompatible tenant configurations will fail to load. To identify if any tenant configurations are failing to load, search the Mimir server logs for lines containing `error applying config`, or query the `cortex_alertmanager_config_last_reload_successful` gauge for `0`.

4. You can disable debug-level logging.
1. You can disable debug-level logging.

0 comments on commit 673cbe2

Please sign in to comment.