-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Job state metrics: split into gauge and counter #14390
base: devel
Are you sure you want to change the base?
Conversation
Thank you for opening this PR. Our team will review it shortly and let you know if there are any changes needed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there anything in prometheus_client that allows us to add aliases for the old names?
These numbers don't look right, or is there something I'm not understanding? |
This is a Unix time stamp showing when the metric was created— checking this value myself, this comes out to a date of this past Monday which makes sense. |
Oh, I'm showing my lack of understanding of this system then. so the metrics This is certainly cool. But this segways into the obvious question here - when you run cleanup_jobs system management job, it will reduce the total counts. After that happens, I could see the data collector freaking out. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No blockers from my side
Good question @gundalow ! Not that I am aware of, and none that I see in their documentation. Since the underlying data driving these metrics hasn't been changed, it would still be possible to keep the original gauges if that's deemed to be needed. However that seems extraneous since it would be the same data.
This is a good point @AlanCoding , while writing this I couldn't conceive of a situation where the counter |
Also, what exactly is the timestamp for the |
Hello,
If you could go ahead and get that test updated we would be happy to further investigate this. Thank you again for your time! |
Signed-off-by: onefourfive <>
these shouldn't be needed for local testing, but can be changed
a7f8a00
to
6b54071
Compare
Signed-off-by: onefourfive <>
SUMMARY
Closes #14369
Prometheus
counter
metric types should be used for metrics that can only increase, eg terminal jobs states likefailed
,canceled
,error
, andsuccessful
.ISSUE TYPE
COMPONENT NAME
Metrics
AWX VERSION
ADDITIONAL INFORMATION
Metrics Before
Metrics After
Note the
_created
metrics are automatically exported by Prometheus for counters. This can be disabled with an environment variablePROMETHEUS_DISABLE_CREATED_SERIES=True
(see docs.