Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FDS-2525] Authenticated export of telemetry data #1527

Merged
merged 53 commits into from
Nov 13, 2024

Conversation

BryanFauble
Copy link
Collaborator

@BryanFauble BryanFauble commented Oct 29, 2024

Problem:

  1. When creating telemetry data where an open-telemetry collector is not available to perform an oauth2 token exchange the only way to attach an auth token on outbound requests was to define a static header an environment variables.

Solution:

  1. Adding the requests-oauth2client library that will handle the oauth2 token exchange with Auth0 for us if the environment variables are set.
  2. Attach the auth mechanism on the requests.Session() object that the opentelemetry exporter uses to send data out.
  3. This is the first portion of https://sagebionetworks.jira.com/browse/FDS-2525 - Which allows developers to export telemetry data locally. It also verifies that all we need to do for FDS-2525 is set a few environment variables and allow schematic running within AWS to export it's data to the OTEL collector, which will then forward the data to the DPE K8s cluster.

Testing:

  1. I ran this code locally with the environment variables set and confirmed that both traces AND log were exported and viewable within SigNoz on the sandbox kubernetes cluster.
  2. I also verified I could point this at the https://dev.sagedpe.org URL too

TODO:

  • Remove hacky code around verify=False for the HTTP POST logic when non-self signed certs can be used.

Base automatically changed from attach-additional-telemetry-data to develop November 1, 2024 19:01
@BryanFauble BryanFauble marked this pull request as ready for review November 4, 2024 23:21
@BryanFauble BryanFauble requested a review from a team as a code owner November 4, 2024 23:21
Copy link
Member

@thomasyu888 thomasyu888 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔥 LGTM! Thanks for doing this. Make sense to me, I'll leave it to at least one other developer to review this.

@GiaJordan
Copy link
Contributor

@BryanFauble The changes look good to me!
I just see that a couple tests are failing and there are a couple merge conflicts.

Copy link

sonarcloud bot commented Nov 13, 2024

@BryanFauble BryanFauble merged commit 778bf54 into develop Nov 13, 2024
7 checks passed
@BryanFauble BryanFauble deleted the fds-2525-authenticated-exports branch November 13, 2024 16:35
GiaJordan pushed a commit that referenced this pull request Nov 13, 2024
* Authenticated export of telemetry data, updating to HTTP otel library
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants