Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and Feature: standardize config behavior across tracing libs #2873

Merged
merged 13 commits into from
Oct 3, 2024

Conversation

mtoffl01
Copy link
Contributor

@mtoffl01 mtoffl01 commented Sep 17, 2024

Description

This change makes Go "consistent" with other tracing libraries for a given list of configurations; for some configs, the behavior has been changed; for others, support is introduced for the first time. RFC is linked under "Motivation." This is not an exhaustive list.
Also, adds telemetry report for trace_rate_limit.

Each change is in its own commit for ease of review, except for the very last commit, which just does some cleanup.

Telemetry changes: Commit

Configurations made consistent:
(This means they already existed in dd-trace-go, but their behavior was modified):

  1. HTTP Client errors: Default behavior was changed from reporting error on 5xx codes to reporting error on 4xx codes. Commit
  2. Client IP headers: When DD_TRACE_CLIENT_IP_ENABLED is set to true, we report a http.client_ip tag whose value is populated from a list of "default headers." This PR fixes this list to include cf-connecting-ipv6 instead of cf-connecting-ip6 (The former is correct according to the spec). Commit

Configurations introduced:

  1. DD_TRACE_HTTP_CLIENT_TAG_QUERY_STRING: Boolean that determines whether or not to include query params in http.url tag for http client spans. Commit
  2. DD_TRACE_HTTP_CLIENT_ERROR_STATUSES: Customize errors on http client spans. Commit
  3. DD_TRACE_HTTP_CLIENT_ERROR_STATUSES: Customize errors on http server spans. Commit

Note: support for DD_TRACE_LOG_DIRECTORY is also added as part of this effort, but for ease of review, the changes are in a separate PR since the implementation was slightly more complex.

Motivation

RFC
Ensure behavior is consistent across tracing libs for a given list of configurations.

Reviewer's Checklist

  • Changed code has unit tests for its functionality at or near 100% coverage.
  • System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
  • There is a benchmark for any new code, or changes to existing code.
  • If this interacts with the agent in a new way, a system test has been added.
  • Add an appropriate team label so this PR gets put in the right place for the release notes.
  • Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.

Unsure? Have a question? Request a review!

@pr-commenter
Copy link

pr-commenter bot commented Sep 17, 2024

Benchmarks

Benchmark execution time: 2024-10-03 19:34:39

Comparing candidate commit 09b6d5d in PR branch mtoff/config-consistency with baseline commit ef90025 in branch main.

Found 1 performance improvements and 1 performance regressions! Performance is the same for 56 metrics, 1 unstable metrics.

scenario:BenchmarkInjectW3C-24

  • 🟥 execution_time [+124.107ns; +143.293ns] or [+3.053%; +3.525%]

scenario:BenchmarkSetTagStringer-24

  • 🟩 execution_time [-7.668ns; -4.172ns] or [-5.331%; -2.900%]

@mtoffl01 mtoffl01 force-pushed the mtoff/config-consistency branch 2 times, most recently from 8912f64 to 72c22f6 Compare September 23, 2024 16:39
@mtoffl01 mtoffl01 force-pushed the mtoff/config-consistency branch 3 times, most recently from 0989e3d to c7e46fe Compare September 25, 2024 18:18
Small note: Only overwrite 0 -> 200 code if 0 was not specified in DD_TRACE_HTTP_SERVER_ERROR_STATUSES
@mtoffl01 mtoffl01 force-pushed the mtoff/config-consistency branch 2 times, most recently from 57de51d to bb3adb9 Compare October 1, 2024 17:42
@github-actions github-actions bot added the apm:ecosystem contrib/* related feature requests or bugs label Oct 1, 2024
@mtoffl01 mtoffl01 marked this pull request as ready for review October 1, 2024 18:00
@mtoffl01 mtoffl01 requested review from a team as code owners October 1, 2024 18:00
@mtoffl01 mtoffl01 changed the title [WIP] Config Consistency Config Consistency Oct 1, 2024
@mtoffl01 mtoffl01 changed the title Config Consistency Fix and Feature: standardize config behavior across tracing libs Oct 1, 2024
Copy link
Contributor

@eliottness eliottness left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving for ˋinternal/appsec` files (Thanks for the fix 😄)

// treat 5XX as errors
if res.StatusCode/100 == 5 {
// Client spans consider 4xx as errors
if res.StatusCode/100 == 4 {
Copy link
Member

@darccio darccio Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this a breaking behavioural change? Should we introduce an environment variable to allow keeping the old behaviour or would it be enough to set DD_TRACE_HTTP_CLIENT_ERROR_STATUSES to the right values?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@darccio , yes it is a breaking change, although in my opinion it's a "fix" as the http client spans should always have had 4xx as errors, not 5xx.
We can discuss further.

@@ -516,11 +516,11 @@ func router(muxOpts ...Option) http.Handler {
return mux
}

func handler200(w http.ResponseWriter, _ *http.Request) {
func handler200(w http.ResponseWriter, r *http.Request) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the body of the function this variable isn't used. Is it needed due to another reason?

@mtoffl01 mtoffl01 enabled auto-merge (squash) October 3, 2024 13:37
@mtoffl01 mtoffl01 merged commit e90c07d into main Oct 3, 2024
165 of 166 checks passed
@mtoffl01 mtoffl01 deleted the mtoff/config-consistency branch October 3, 2024 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:ecosystem contrib/* related feature requests or bugs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants