Netdata allows you to send custom notifications to any endpoint you choose.
To configure custom notifications, you will need to customize health_alarm_notify.conf
. Open the file for editing
using edit-config
from the Netdata config
directory, which is typically at /etc/netdata
.
You can look at the other senders in /usr/libexec/netdata/plugins.d/alarm-notify.sh
for examples of how to modify the custom_sender()
function in health_alarm_notify.conf
.
As with other notifications, you will also need to define the recipient list in DEFAULT_RECIPIENT_CUSTOM
and/or the role_recipients_custom
array.
The following is a sample custom_sender
function in health_alarm_notify.conf
, to send an SMS via an imaginary HTTPS endpoint to the SMS gateway:
custom_sender() {
# example human readable SMS
local msg="${host} ${status_message}: ${alarm} ${raised_for}"
# limit it to 160 characters and encode it for use in a URL
urlencode "${msg:0:160}" >/dev/null; msg="${REPLY}"
# a space separated list of the recipients to send alarms to
to="${1}"
for phone in ${to}; do
httpcode=$(docurl -X POST \
--data-urlencode "From=XXX" \
--data-urlencode "To=${phone}" \
--data-urlencode "Body=${msg}" \
-u "${accountsid}:${accounttoken}" \
https://domain.website.com/)
if [ "${httpcode}" = "200" ]; then
info "sent custom notification ${msg} to ${phone}"
sent=$((sent + 1))
else
error "failed to send custom notification ${msg} to ${phone} with HTTP error code ${httpcode}."
fi
done
}
Variables available to the custom_sender:
${to_custom}
the list of recipients for the alarm${host}
the host generated this event${url_host}
same as${host}
but URL encoded${unique_id}
the unique id of this event${alarm_id}
the unique id of the alarm that generated this event${event_id}
the incremental id of the event, for this alarm id${when}
the timestamp this event occurred${name}
the name of the alarm, as given in Netdata health.d entries${url_name}
same as${name}
but URL encoded${chart}
the name of the chart (type.id)${url_chart}
same as${chart}
but URL encoded${family}
the family of the chart${url_family}
same as${family}
but URL encoded${status}
the current status : REMOVED, UNINITIALIZED, UNDEFINED, CLEAR, WARNING, CRITICAL${old_status}
the previous status: REMOVED, UNINITIALIZED, UNDEFINED, CLEAR, WARNING, CRITICAL${value}
the current value of the alarm${old_value}
the previous value of the alarm${src}
the line number and file the alarm has been configured${duration}
the duration in seconds of the previous alarm state${duration_txt}
same as${duration}
for humans${non_clear_duration}
the total duration in seconds this is/was non-clear${non_clear_duration_txt}
same as${non_clear_duration}
for humans${units}
the units of the value${info}
a short description of the alarm${value_string}
friendly value (with units)${old_value_string}
friendly old value (with units)${image}
the URL of an image to represent the status of the alarm${color}
a color in #AABBCC format for the alarm${goto_url}
the URL the user can click to see the Netdata dashboard${calc_expression}
the expression evaluated to provide the value for the alarm${calc_param_values}
the value of the variables in the evaluated expression${total_warnings}
the total number of alarms in WARNING state on the host${total_critical}
the total number of alarms in CRITICAL state on the host
The following are more human friendly:
${alarm}
like "name = value units"${status_message}
like "needs attention", "recovered", "is critical"${severity}
like "Escalated to CRITICAL", "Recovered from WARNING"${raised_for}
like "(alarm was raised for 10 minutes)"