Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add otel collector service telemetry metrics #364

Merged
merged 5 commits into from
Aug 18, 2023

Conversation

mkocher
Copy link
Member

@mkocher mkocher commented Aug 16, 2023

Configure a prometheus endpoint which serves metrics about the otel-collector itself. Also adds a prom_scraper_config so promscraper scrapes the metrics.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Testing performed?

  • Unit tests
  • Integration tests
  • Acceptance tests

Checklist:

  • This PR is being made against the main branch, or relevant version branch
  • I have made corresponding changes to the documentation
  • I have added testing for my changes

@mkocher mkocher requested a review from a team as a code owner August 16, 2023 22:29
@mkocher
Copy link
Member Author

mkocher commented Aug 16, 2023

we aren't sure why prom scraper configs have both origin and source_id. But cargo culting what the other agents do just in case until we can say for sure.

Turns out source_id and origin are different for app logs/metrics, but synced for component logs/metrics which this is.

Configure a prometheus endpoint which serves metrics about the
otel-collector itself. Also adds a prom_scraper_config so promscraper
scrapes the metrics.

Signed-off-by: Rebecca Roberts <[email protected]>
@mkocher mkocher force-pushed the add-otel-service-telemetry-metrics branch from 13d0d01 to 0815d9d Compare August 17, 2023 21:36
ctlong
ctlong previously approved these changes Aug 18, 2023
ctlong and others added 3 commits August 18, 2023 21:52
When OTel Collector jobs are disabled, don't output any prom scraper
configuration so that the prom scraper skips over these jobs. This
avoids continual error messages logged by the prom scraper when it can't
scrape a destination.
Since we already dropped exact parity with OTel Collector settings with
`ingress.grpc.tls.*`, we think it's preferable to remove the `service.`
prefix from telemetry properties in the OTel Collector jobs.

Signed-off-by: Carson Long <[email protected]>
@ctlong ctlong merged commit 8021d40 into main Aug 18, 2023
5 checks passed
@ctlong ctlong deleted the add-otel-service-telemetry-metrics branch August 18, 2023 23:16
acrmp pushed a commit to cloudfoundry/otel-collector-release that referenced this pull request Dec 6, 2023
…r-agent-release#364)

* add otel collector service telemetry metrics

Configure a prometheus endpoint which serves metrics about the
otel-collector itself. Also adds a prom_scraper_config so promscraper
scrapes the metrics.

* add otel service telemetry metrics to windows jobs

* fix: selectively enable prom scraping of otel collectors

When OTel Collector jobs are disabled, don't output any prom scraper
configuration so that the prom scraper skips over these jobs. This
avoids continual error messages logged by the prom scraper when it can't
scrape a destination.

* fix(otel-collector-windows): align default port with otel-collector

* refactor: remove service prefix from telemetry properties

Since we already dropped exact parity with OTel Collector settings with
`ingress.grpc.tls.*`, we think it's preferable to remove the `service.`
prefix from telemetry properties in the OTel Collector jobs.

---------

Signed-off-by: Rebecca Roberts <[email protected]>
Signed-off-by: Andrew Crump <[email protected]>
Signed-off-by: Carson Long <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants