Skip to content

Latest commit

 

History

History
124 lines (84 loc) · 4.21 KB

README.md

File metadata and controls

124 lines (84 loc) · 4.21 KB
Raw

Prometheus Prefect Exporter

A Prometheus exporter for Prefect metrics, written in Python. Originally contributed by @ialejandro!

Usage

By default prometheus-prefect-exporter will listen on port 8000.

Installing released versions

Docker

docker run -d \
  -p 8000:8000 \
  -e PREFECT_API_URL=<PREFECT_ENDPOINT> \
  prefecthq/prometheus-prefect-exporter:latest

Helm

helm repo add prefect https://prefecthq.github.io/prefect-helm
helm search repo prefect
helm install prometheus-prefect-exporter prefect/prometheus-prefect-exporter

Installing development versions

You will need to have a Prefect Server up and running for the exporter to communicate with. Follow these steps to set one up, or skip to step 3.

  1. Install Prefect Server
helm repo add prefect https://prefecthq.github.io/prefect-helm
helm search repo prefect
helm install prefect-server prefect/prefect-server
  1. Port forward the api to your localhost:
kubectl port-forward svc/prefect-server 4200:4200
  1. Clone repository
git clone https://github.com/PrefectHQ/prometheus-prefect-exporter.git
  1. Change to this directory
cd prometheus-prefect-exporter`
  1. Setup a virtual environment of your choice & install dependencies
virtualenv prom-exporter
source prom-exporter/bin/activate
pip install -r requirements.txt
python main.py

Using Docker Compose

Alternatively, you can use the Docker Compose configuration to stand up the exporter alongside Prefect and Prometheus.

To get started, run:

docker compose up -d

Confirm the services are running and healthy:

docker compose ps

You can now reach each service locally:

Configuration

You can modify environment variables to change the behavior of the exporter.

  • An API Key is only required for auth-enabled, on-prem, self-managed solutions.
  • An API key is not required for open-source or Prefect Server.
Environment Variable Description Default
LOG_LEVEL Logging level INFO
MAX_RETRIES Number of retries to attempt when fetching metrics from Prefect API 3
METRICS_PORT Port to expose metrics on 8000
OFFSET_MINUTES Number of minutes to offset the start time when fetching metrics from Prefect API 5
PREFECT_API_URL Prefect API URL https://localhost:4200/api
PREFECT_API_KEY Prefect API KEY (Optional) ""
PREFECT_CSRF_ENABLED Enable compatibilty with Prefect Servers using CSRF protection False
PAGINATION_ENABLED Enable pagination usage. (Uses more resources) True
PAGINATION_LIMIT Pagination limit 200
SCRAPE_INTERVAL_SECONDS Interval in seconds to scrape metrics from Prefect API 30

Contributing

Contributions to the Prometheus Prefect Exporter is always welcome! We welcome your help - whether it's adding new functionality, tweaking documentation, or anything in between. In order to successfully contribute, you'll need to fork this repository and commit changes to your local prometheus-prefect-exporter repo. You can then open a PR against this upstream repo that the team will review!

Documentation

Please make sure that your changes have been linted & the documentation has been updated. The easiest way to accomplish this is by installing pre-commit.

Testing & validation

Make sure that any new functionality is well tested! You can do this by installing the exporter locally, see above for how to do this.

Opening a PR

A helpful PR explains WHAT changed and WHY the change is important. Please take time to make your PR descriptions as helpful as possible. If you are opening a PR from a forked repository - please follow these docs to allow prometheus-prefect-exporter maintainers to push commits to your local branch.