Alertmanager templating

[tagathi]

Hi,
I am using mimirtool to configure alertmanager.
I have loaded the configuration below:

route:
  receiver: slack
  group_interval: 5m
  group_wait: 60s
  repeat_interval: 24h
receivers:
- name: slack
  slack_configs:
  - api_url: "https://hooks.slack.com/services/xxxxxx"
    channel: "XXXXXXX"
    title: '{{ template "slack.title" . }}'
    text: '{{ template "slack.message" . }}'
    send_resolved: true


Templates:
slack.template:
{{ define "slack.title" }}
{{ len .Alerts.Firing }} firing alert(s), {{ len .Alerts.Resolved }} resolved alert(s)
{{ end }}
{{ define "slack.print_alert" -}}
[{{.Status}}] {{ .Labels.alertname }}
Labels:
{{ range .Labels.SortedPairs -}}
- {{ .Name }}: {{ .Value }}
{{ end -}}
{{ if .Annotations -}}
Annotations:
{{ range .Annotations.SortedPairs -}}
- {{ .Name }}: {{ .Value }}
{{ end -}}
{{ end -}}
{{ if .SilenceURL -}}
  Silence: {{ .SilenceURL }}
{{ end -}}
{{ if .DashboardURL -}}
  Go to dashboard: {{ .DashboardURL }}
{{- end }}
{{- end }}
{{ define "slack.message" -}}
{{ if .Alerts.Firing -}}
{{ len .Alerts.Firing }} firing alert(s):
{{ range .Alerts.Firing }}
{{ template "slack.print_alert" . }}
{{ end -}}
{{ end }}
{{ if .Alerts.Resolved -}}
{{ len .Alerts.Resolved }} resolved alert(s):
{{ range .Alerts.Resolved }}
{{ template "slack.print_alert" .}}
{{ end -}}
{{ end }}
{{- end }}

Whenever an alert is firing, alertmanager reports the following error:
ts=2023-08-29T07:34:43.673294812Z caller=dispatch.go:352 level=error component=MultiTenantAlertmanager user=Clearview component=dispatcher insight=true component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="slack/slack[0]: notify retry canceled due to unrecoverable error after 1 attempts: template: :1:12: executing \"\" at <{{template \"slack.title\" .}}>: template \"slack.title\" not defined"
Any ideas what might be wrong?

[pschiffe]

Hi @tagathi, were you able to solve this? I'm seeing the same issue with Mimir 2.12.0:

...
receivers:
  - name: email-all
    email_configs:
      - to: some@mail
        send_resolved: true
        html: '{{ template "ws.email.default.html" . }}'

Templates:
email.tpl:
{{ define "ws.email.default.html" }}
...

email-all/email[0]: notify retry canceled due to unrecoverable error after 1 attempts: execute html template: html/template::1:12: no such template \"ws.email.default.html\"

[tagathi]

Hi @pschiffe,
Yes, you need to refer the template in your Alert Manager config.
My alertmanager.yaml file looks like this, note the last two lines:

route:
  receiver: slack
  group_interval: 5m
  group_wait: 60s
  repeat_interval: 24h
receivers:
- name: slack
  slack_configs:
  - api_url: "https://hooks.slack.com/services/xxxxxx"
    channel: "XXXXXXX"
    title: '{{ template "slack.title" . }}'
    text: '{{ template "slack.message" . }}'
    send_resolved: true
templates:
- slack.tmpl