Skip to content

Commit

Permalink
[OMON-753] Update docs for reading metrics from Kafka
Browse files Browse the repository at this point in the history
  • Loading branch information
awegrzyn committed Feb 7, 2024
1 parent 9c4de85 commit b56beca
Showing 1 changed file with 13 additions and 4 deletions.
17 changes: 13 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,13 +206,15 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr
## Receiving metrics from Monitoring system (development instructions)

### Requirements
- CS8 or CC7 with `devtoolset-9`
- Boost >= 1.70
- RHEL8, RHE9, CS8 or CC7 with `devtoolset-9`
- Boost >= 1.70, CMake

### Compile Monitoring library with Kafka backend

#### Manually
- Compile `librdkafka`
```bash
git clone -b v1.8.2 https://github.com/edenhill/librdkafka && cd librdkafka
git clone -b v2.3.0 https://github.com/edenhill/librdkafka && cd librdkafka
cmake -H. -B./_cmake_build -DENABLE_LZ4_EXT=OFF -DCMAKE_INSTALL_LIBDIR=lib -DRDKAFKA_BUILD_TESTS=OFF -DRDKAFKA_BUILD_EXAMPLES=OFF -DCMAKE_INSTALL_PREFIX=~/librdkafka_install
cmake --build ./_cmake_build --target install -j
```
Expand All @@ -223,6 +225,11 @@ This guide explains manual installation. For `ansible` deployment see [AliceO2Gr
cmake --build ./_cmake_build --target install -j
```

#### aliBuild
- Modify `monitoring.sh`: add `- librdkafka` to "requires"
- Compile Monitoring: `aliBuild build Monitoring --defaults o2-dataflow --always-prefer-system`
- Add `Monitoring` as dependency of your project

### Look for Monitoring library in your CMake
As `librdkafka` is optional dependency of Monitoring it is not handled by CMakeConfig, therefore you need:
```cmake
Expand All @@ -238,7 +245,7 @@ And then, link against `AliceO2::Monitoring` target.
...

std::vector<std::string> topics = {"topic-to-subscribe"};
auto client = MonitoringFactory::GetPullClient("kafka-server:9092", topics);
auto client = MonitoringFactory::GetPullClient("kafka-server:9092", topics, "<client-id>");
for (;;) {
auto metrics = client->pull();
if (!metrics.empty()) {
Expand All @@ -248,5 +255,7 @@ for (;;) {
}
```

`<client_id>` should be unique, self-explainable string describing the client, eg. `dcs-link-status` or `its-link-status`.

### Data format
Native data format is [Influx Line Protocol](https://docs.influxdata.com/influxdb/latest/reference/syntax/line-protocol/) but metrics can be converted into any format listed in here: https://docs.influxdata.com/telegraf/latest/data_formats/output/

0 comments on commit b56beca

Please sign in to comment.