Skip to content

Commit

Permalink
[DPE-3735] Integration of custom alerting rules and dashboards (#180)
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] authored Apr 10, 2024
1 parent ce5ec60 commit 002289f
Show file tree
Hide file tree
Showing 2 changed files with 72 additions and 1 deletion.
71 changes: 71 additions & 0 deletions docs/how-to/h-integrate-alerts-dashboards.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# Integrate custom alerting rules and dashboards

This guide shows you how to integrate an existing set of rules and/or dashboards to your Charmed Kafka and Charmed Zookeeper deployment to be consumed with the [Canonical Observability Stack (COS)](https://charmhub.io/topics/canonical-observability-stack).
To do so, we will sync resources stored in a git repo to COS Lite.

## Prerequisites

Deploy the cos-lite bundle in a Kubernetes environment and integrate Charmed Kafka and Charmed ZooKeeper to the COS offers, as shown in the [How to Enable Monitoring](/t/charmed-kafka-documentation-how-to-enable-monitoring/10283) guide.
This guide will refer to the models that charms are deployed into as:

* `<cos-model>` for the model containing observabilities charms (and deployed on k8s)

* `<apps-model>` for the model containing Charmed Kafka and Charmed ZooKeeper

* `<apps-model>` for other optional charms (e.g. tls-certificates operators, `grafana-agent`, `data-integrator`, etc.).

## Create a repository with a custom monitoring setup


Create an empty git repository, or in an existing one, save your alert rules and dashboard models under the `<path_to_prom_rules>`, `<path_to_loki_rules>` and `<path_to_models>` folders.

If you want a primer to rule writing, refer to the [Prometheus documentation](https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/).
You may also find an example in the [kafka-test-app repository](https://github.com/canonical/kafka-test-app).

Then, push your changes to the remote repository.


## Deploy the COS configuration charm

Deploy the [COS configuration](https://charmhub.io/cos-configuration-k8s) charm in the `<cos-model>` model:

```shell
juju deploy cos-configuration-k8s cos-config \
--config git_repo=<repository_url> \
--config git_branch=<branch> \
```

The COS configuration charm keeps the monitoring stack in sync with our repository, by forwarding resources to Prometheus, Loki and Grafana.
Refer to the [documentation](https://charmhub.io/cos-configuration-k8s/configure) for all configuration options, including how to access a private repository.
Adding, updating or deleting an alert rule or a dashboard in the repository will be reflected in the monitoring stack.

[Note]
You need to manually refresh `cos-config`'s local repository with the *sync-now* action if you do not want to wait for the next [update-status event](/t/event-update-status/6484) to pull the latest changes.
[/Note]


## Forward the rules and dashboards

The path to the resource folders can be set after deployment:

```shell
juju config cos-config \
--config prometheus_alert_rules_path=<path_to_prom_rules>
--config loki_alert_rules_path=<path_to_loki_rules>
--config grafana_dashboards_path=<path_to_models>
```

Then, integrate the charm to the COS operator to forward the rules and dashboards:

```shell
juju integrate cos-config prometheus
juju integrate cos-config grafana
juju integrate cos-config loki
```

After this is complete, the monitoring COS stack should be up, and ready to fire alerts based on our rules.
As for the dashboards, they should be available in the Grafana interface.

## Conclusion

In this guide, we enabled monitoring on a Kafka deployment and integrated alert rules and dashboards by syncing a git repository to the COS stack.
2 changes: 1 addition & 1 deletion docs/tutorial/t-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
The Charmed Kafka Operator delivers automated operations management from [day 0 to day 2](https://codilime.com/blog/day-0-day-1-day-2-the-software-lifecycle-in-the-cloud-age/) on the [Apache Kafka](https://kafka.apache.org/) event streaming platform.
It is an open source, end-to-end, production-ready data platform [on top of Juju](https://juju.is/). As a first step this tutorial shows you how to get Charmed Kafka up and running, but the tutorial does not stop there.
As currently Kafka requires a paired [ZooKeeper](https://zookeeper.apache.org/) deployment in production, this operator makes use of the [ZooKeeper Operator](https://github.com/canonical/zookeeper-operator) for various essential functions.
Through this tutorial you will learn a variety of operations, everything from adding replicas to advanced operations such as enabling Transcript Layer Security (TLS).
Through this tutorial you will learn a variety of operations, everything from adding replicas to advanced operations such as enabling Transport Layer Security (TLS).

In this tutorial we will walk through how to:
- Set up your environment using LXD and Juju.
Expand Down

0 comments on commit 002289f

Please sign in to comment.