Cannot view 1 minute rates on a 15 second scrape interval #9289
-
|
Hello! I think I might be confused on how a rate query is supposed to work and I haven't found any information online that quite explains it. From what I understand, a rate query requires two samples within the time window you're searching. My setup is Alloy scraping a target -> remote write to Mimir. Take the following as an example: What does my 1 minute rate window not work in this case? I don't remember having this issue in Prometheus before I switched to Mimir with similar rates. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Hello. To understand why You can see raw samples in Grafana by using 1) Instant query type, 2) using query: 
Given that
Range query will produce result at every "step" of the requested interval. This just means that your step is 15 seconds. But the fact that rate of 2m windows shows results proves that in every 2m window there are at least two samples it can work with. |
Beta Was this translation helpful? Give feedback.
-
|
This was a wonderfully detailed answer. Thank you.
This was exactly my misconception, seeing 15 second results made me think I was scraping at 15 seconds but that wasn't the case. The tip for running an instant query worked perfectly for seeing what the raw data was! |
Beta Was this translation helpful? Give feedback.
Hello. To understand why
rate(cortex_request_duration_seconds_count[1m])returns no results, we would need to see raw samples used for the computation.You can see raw samples in Grafana by using 1) Instant query type, 2) using query:
cortex_request_duration_seconds_count[$__range]. (In my example, I also used{namespace="mimir-dev-12"}to reduce number of returned series).Given that
rate(cortex_request_duration_seconds_count[2m])returns data, most likely explanation is that there are not 2 samples in 1m window. (that is, your scrape interval is likely >= 1 minute).