Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus Remote Write v2 Implementation #6324

Open
SungJin1212 opened this issue Nov 8, 2024 · 1 comment
Open

Prometheus Remote Write v2 Implementation #6324

SungJin1212 opened this issue Nov 8, 2024 · 1 comment

Comments

@SungJin1212
Copy link
Contributor

SungJin1212 commented Nov 8, 2024

I'm implementing Prometheus remote write v2 (PRW2.0) based on PRW2.0 docs (https://prometheus.io/docs/specs/remote_write_spec_2_0).

There are two milestones:

  • Milestone1.
    The distributor can accept PRW2.0 requests. It requires small changes between the Distributor and Ingester, including adding response headers. The ingestion is done by converting the PRW2.0 request to PRW1.0, then using existing Push methods in the Distributor and Ingester.

push response headers to be added:

X-Prometheus-Remote-Write-Samples-Written <count of all successfully written Samples>
X-Prometheus-Remote-Write-Histograms-Written <count of all successfully written Histogram samples>
X-Prometheus-Remote-Write-Exemplars-Written <count of all successfully written Exemplars>

[] Change the push handler so that the Distributor can accept PRW2.0 #6330
[] Add response headers

  • Milestone2.
    Add PRW2.0 proto and update the protocol used between the Distributor and Ingester.

[] Add RPW2.0 proto
[] Implement Distributor & Ingester Push functions
[] Add fallback logic for rolling updates: when the Distributor can accept PRW2.0 but not Ingester, we have to ingest metrics.
[] Add some performance tests between PRW2.0 and 1.0.

@yeya24
Copy link
Contributor

yeya24 commented Nov 12, 2024

Thanks!
I think we should start with milestone 1 and having milestone 1 is good enough to support RW 2.0

We need to think more about milestone 2 before actually changing the internal write protocol between distributor and ingester. They are not user facing and we can do whatever we want so doesn't need to be same as RW 2.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants