fix(llmobs): safely handle non-json serializable arguments #10694

Yun-Kim · 2024-09-17T21:33:50Z

Safely handle non-JSON serializable tag arguments in LLMObs.annotate() and the OpenAI/LangChain/Bedrock/Anthropic integrations.

LLMObs.annotate(): we previously just discarded the entire argument to annotate if the argument was non-JSON serializable. Now, we safely convert non-JSON serializable fields/objects to a default placeholder text, meaning users can still send some data even if some of it may be invalid.
Same idea with each integration, we ensure we safely handle non-JSON serializable args and default to placeholder texts if necessary.
We've moved all json.dumps() call into a private helper safe_json() which does the above for us.

Note: This PR removes some tests in test_llmobs_service.py regarding truly unserializable objects as this is highly unlikely, someone would have to go out of their way to make a truly unserializable object (i.e. override __repr__() with a non-json serializable value). We still catch these so any resulting crashes should not be from our code.

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2024-09-17T21:34:19Z

CODEOWNERS have been resolved as:

releasenotes/notes/fix-llmobs-safe-json-4aee3aacf05fd69f.yaml           @DataDog/apm-python
tests/snapshots/tests.contrib.langchain.test_langchain_community.test_base_tool_invoke_non_json_serializable_config.json  @DataDog/apm-python
ddtrace/contrib/internal/anthropic/utils.py                             @DataDog/ml-observability
ddtrace/contrib/internal/langchain/patch.py                             @DataDog/ml-observability
ddtrace/llmobs/_integrations/anthropic.py                               @DataDog/ml-observability
ddtrace/llmobs/_integrations/bedrock.py                                 @DataDog/ml-observability
ddtrace/llmobs/_integrations/gemini.py                                  @DataDog/ml-observability
ddtrace/llmobs/_integrations/langchain.py                               @DataDog/ml-observability
ddtrace/llmobs/_integrations/openai.py                                  @DataDog/ml-observability
ddtrace/llmobs/_llmobs.py                                               @DataDog/ml-observability
ddtrace/llmobs/_utils.py                                                @DataDog/ml-observability
tests/contrib/anthropic/test_anthropic.py                               @DataDog/ml-observability
tests/contrib/anthropic/test_anthropic_llmobs.py                        @DataDog/ml-observability
tests/contrib/langchain/test_langchain_community.py                     @DataDog/ml-observability
tests/contrib/openai/test_openai_llmobs.py                              @DataDog/ml-observability
tests/llmobs/test_llmobs_service.py                                     @DataDog/ml-observability
tests/llmobs/test_utils.py                                              @DataDog/ml-observability
tests/snapshots/tests.contrib.anthropic.test_anthropic.test_anthropic_llm_unserializable_arg.json  @DataDog/apm-python

datadog-dd-trace-py-rkomorn · 2024-09-17T21:48:19Z

Datadog Report

Branch report: yunkim/llmobs-safe-json
Commit report: 8c88074
Test service: dd-trace-py

✅ 0 Failed, 2762 Passed, 2317 Skipped, 31m 8.47s Total duration (38m 39.73s time saved)

pr-commenter · 2024-09-17T22:18:34Z

Benchmarks

Benchmark execution time: 2024-09-19 19:14:36

Comparing candidate commit 8c88074 in PR branch yunkim/llmobs-safe-json with baseline commit cc824a2 in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 353 metrics, 47 unstable metrics.

tests/llmobs/test_llmobs_service.py

sabrenner

lgtm! just left a small question, probably more for my own edification.

ddtrace/llmobs/_integrations/anthropic.py

releasenotes/notes/fix-llmobs-safe-json-4aee3aacf05fd69f.yaml

ddtrace/llmobs/_integrations/langchain.py

ddtrace/llmobs/_llmobs.py

github-actions · 2024-09-19T19:58:17Z

The backport to 2.12 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.12 2.12
# Navigate to the new working tree
cd .worktrees/backport-2.12
# Create a new branch
git switch --create backport-10694-to-2.12
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9922df5c7ab752cd6b0a7517ee96fa3814e51a00
# Push it to GitHub
git push --set-upstream origin backport-10694-to-2.12
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.12

Then, create a pull request where the base branch is 2.12 and the compare/head branch is backport-10694-to-2.12.

github-actions · 2024-09-19T19:58:18Z

The backport to 2.13 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.13 2.13
# Navigate to the new working tree
cd .worktrees/backport-2.13
# Create a new branch
git switch --create backport-10694-to-2.13
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9922df5c7ab752cd6b0a7517ee96fa3814e51a00
# Push it to GitHub
git push --set-upstream origin backport-10694-to-2.13
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.13

Then, create a pull request where the base branch is 2.13 and the compare/head branch is backport-10694-to-2.13.

Safely handle non-JSON serializable tag arguments in LLMObs.annotate() and the OpenAI/LangChain/Bedrock/Anthropic integrations. - LLMObs.annotate(): we previously just discarded the entire argument to annotate if the argument was non-JSON serializable. Now, we safely convert non-JSON serializable fields/objects to a default placeholder text, meaning users can still send *some* data even if some of it may be invalid. - Same idea with each integration, we ensure we safely handle non-JSON serializable args and default to placeholder texts if necessary. - We've moved all json.dumps() call into a private helper `safe_json()` which does the above for us. Note: This PR removes some tests in `test_llmobs_service.py` regarding truly unserializable objects as this is highly unlikely, someone would have to go out of their way to make a truly unserializable object (i.e. override `__repr__()` with a non-json serializable value). We still catch these so any resulting crashes should not be from our code. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

…10694 to 2.13] (#10727) Backports #10694 to 2.13 Safely handle non-JSON serializable tag arguments in LLMObs.annotate() and the OpenAI/LangChain/Bedrock/Anthropic integrations. - LLMObs.annotate(): we previously just discarded the entire argument to annotate if the argument was non-JSON serializable. Now, we safely convert non-JSON serializable fields/objects to a default placeholder text, meaning users can still send *some* data even if some of it may be invalid. - Same idea with each integration, we ensure we safely handle non-JSON serializable args and default to placeholder texts if necessary. - We've moved all json.dumps() call into a private helper `safe_json()` which does the above for us. Note: This PR removes some tests in `test_llmobs_service.py` regarding truly unserializable objects as this is highly unlikely, someone would have to go out of their way to make a truly unserializable object (i.e. override `__repr__()` with a non-json serializable value). We still catch these so any resulting crashes should not be from our code. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

wip wip Update scripts/freshvenvs.py Co-authored-by: datadog-datadog-prod-us1[bot] <88084959+datadog-datadog-prod-us1[bot]@users.noreply.github.com> wip wip wip wip wip wip wip wip wip move to scripts move to scripts wip wip wip wip wip wip wip wip wip wip wip chore: [SVLS-5262] Span Pointer standard hashing (#10687) Adding a standard hashing function for Span Pointers. This continues to be a private API for now. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) fix(llmobs): safely handle non-json serializable arguments (#10694) Safely handle non-JSON serializable tag arguments in LLMObs.annotate() and the OpenAI/LangChain/Bedrock/Anthropic integrations. - LLMObs.annotate(): we previously just discarded the entire argument to annotate if the argument was non-JSON serializable. Now, we safely convert non-JSON serializable fields/objects to a default placeholder text, meaning users can still send *some* data even if some of it may be invalid. - Same idea with each integration, we ensure we safely handle non-JSON serializable args and default to placeholder texts if necessary. - We've moved all json.dumps() call into a private helper `safe_json()` which does the above for us. Note: This PR removes some tests in `test_llmobs_service.py` regarding truly unserializable objects as this is highly unlikely, someone would have to go out of their way to make a truly unserializable object (i.e. override `__repr__()` with a non-json serializable value). We still catch these so any resulting crashes should not be from our code. - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) fix(llmobs): ensure integration metrics are disabled in agentless mode (#10712) Follows up on #10084, which missed a condition where agentless mode should always disable integration metrics. Adds standard testing cases for the `BaseLLMIntegration` class, which was missing prior to this (and thus the reason for missing this edge case prior) - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) chore(crashtracking): add python version to crashtracker tags (#10720) During escalations it's currently difficult to find the exact Python version relevant for a defect. This is a simple fix for that. I'm recommending a backport because 1. The fix is simple 2. 2.11 and onward introduce crashtracker, and we're still dealing with crashes from those versions - [X] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) - [X] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting) minor changes minor changes

…10694 to 2.12] (#10726) Backports #10694 to 2.12. Safely handle non-JSON serializable tag arguments in LLMObs.annotate() and the OpenAI/LangChain/Bedrock/Anthropic integrations. - LLMObs.annotate(): we previously just discarded the entire argument to annotate if the argument was non-JSON serializable. Now, we safely convert non-JSON serializable fields/objects to a default placeholder text, meaning users can still send *some* data even if some of it may be invalid. - Same idea with each integration, we ensure we safely handle non-JSON serializable args and default to placeholder texts if necessary. - We've moved all json.dumps() call into a private helper `safe_json()` which does the above for us. Note: This PR removes some tests in `test_llmobs_service.py` regarding truly unserializable objects as this is highly unlikely, someone would have to go out of their way to make a truly unserializable object (i.e. override `__repr__()` with a non-json serializable value). We still catch these so any resulting crashes should not be from our code. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

#10780) This PR follows up on #10727, which accidentally omitted a `integration="openai"` line in a backported test case. Due to all the other changes to the OpenAI integration which were backported at the same time, this line got omitted during the merge conflict resolution. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Safely handle non-JSON serializable tag arguments in LLMObs.annotate() and the OpenAI/LangChain/Bedrock/Anthropic integrations. - LLMObs.annotate(): we previously just discarded the entire argument to annotate if the argument was non-JSON serializable. Now, we safely convert non-JSON serializable fields/objects to a default placeholder text, meaning users can still send *some* data even if some of it may be invalid. - Same idea with each integration, we ensure we safely handle non-JSON serializable args and default to placeholder texts if necessary. - We've moved all json.dumps() call into a private helper `safe_json()` which does the above for us. Note: This PR removes some tests in `test_llmobs_service.py` regarding truly unserializable objects as this is highly unlikely, someone would have to go out of their way to make a truly unserializable object (i.e. override `__repr__()` with a non-json serializable value). We still catch these so any resulting crashes should not be from our code. ## Checklist - [x] PR author has checked that all the criteria below are met - The PR description includes an overview of the change - The PR description articulates the motivation for the change - The change includes tests OR the PR description describes a testing strategy - The PR description notes risks associated with the change, if any - Newly-added code is easy to change - The change follows the [library release note guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html) - The change includes or references documentation updates if necessary - Backport labels are set (if [applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)) ## Reviewer Checklist - [x] Reviewer has checked that all the criteria below are met - Title is accurate - All changes are related to the pull request's stated goal - Avoids breaking [API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces) changes - Testing strategy adequately addresses listed risks - Newly-added code is easy to change - Release note makes sense to a user of the library - If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment - Backport labels are set in a manner that is consistent with the [release branch maintenance policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Safely handle json serializable issues

da8fdbc

Yun-Kim added backport 2.12 backport 2.13 labels Sep 17, 2024

Yun-Kim added 2 commits September 17, 2024 18:33

Add tests

2fed7ad

Fix tests

3dd9170

Yun-Kim marked this pull request as ready for review September 17, 2024 22:51

Yun-Kim requested review from a team as code owners September 17, 2024 22:51

Yun-Kim requested review from taegyunkim and vitor-de-araujo September 17, 2024 22:51

datadog-datadog-prod-us1 bot reviewed Sep 17, 2024

View reviewed changes

tests/llmobs/test_llmobs_service.py Show resolved Hide resolved

sabrenner reviewed Sep 18, 2024

View reviewed changes

ddtrace/llmobs/_integrations/anthropic.py Outdated Show resolved Hide resolved

releasenotes/notes/fix-llmobs-safe-json-4aee3aacf05fd69f.yaml Outdated Show resolved Hide resolved

Reintroduce try/catch

1b2dd8d

vitor-de-araujo approved these changes Sep 19, 2024

View reviewed changes

ddtrace/llmobs/_integrations/langchain.py Outdated Show resolved Hide resolved

Yun-Kim force-pushed the yunkim/llmobs-safe-json branch from 33386c9 to 78ab838 Compare September 19, 2024 16:59

Extract safe json into helper, fix tests, move try/catch to helper

776e86b

Yun-Kim force-pushed the yunkim/llmobs-safe-json branch from 78ab838 to 776e86b Compare September 19, 2024 17:31

Yun-Kim mentioned this pull request Sep 19, 2024

feat(llmobs): support annotating primitive prompt templates #10677

Merged

2 tasks

Typing fix

8c88074

Yun-Kim enabled auto-merge (squash) September 19, 2024 19:26

lievan approved these changes Sep 19, 2024

View reviewed changes

ddtrace/llmobs/_llmobs.py Show resolved Hide resolved

Yun-Kim merged commit 9922df5 into main Sep 19, 2024
571 checks passed

Yun-Kim deleted the yunkim/llmobs-safe-json branch September 19, 2024 19:57

Yun-Kim mentioned this pull request Sep 19, 2024

fix(llmobs): safely handle non-json serializable arguments [backport #10694 to 2.12] #10726

Merged

2 tasks

Yun-Kim mentioned this pull request Sep 19, 2024

fix(llmobs): safely handle non-json serializable arguments [backport #10694 to 2.13] #10727

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(llmobs): safely handle non-json serializable arguments #10694

fix(llmobs): safely handle non-json serializable arguments #10694

Yun-Kim commented Sep 17, 2024 •

edited

Loading

github-actions bot commented Sep 17, 2024 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Sep 17, 2024 •

edited

Loading

pr-commenter bot commented Sep 17, 2024 •

edited

Loading

sabrenner left a comment

github-actions bot commented Sep 19, 2024

github-actions bot commented Sep 19, 2024

fix(llmobs): safely handle non-json serializable arguments #10694

fix(llmobs): safely handle non-json serializable arguments #10694

Conversation

Yun-Kim commented Sep 17, 2024 • edited Loading

Checklist

Reviewer Checklist

github-actions bot commented Sep 17, 2024 • edited Loading

datadog-dd-trace-py-rkomorn bot commented Sep 17, 2024 • edited Loading

Datadog Report

pr-commenter bot commented Sep 17, 2024 • edited Loading

Benchmarks

sabrenner left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 19, 2024

github-actions bot commented Sep 19, 2024

Yun-Kim commented Sep 17, 2024 •

edited

Loading

github-actions bot commented Sep 17, 2024 •

edited

Loading

datadog-dd-trace-py-rkomorn bot commented Sep 17, 2024 •

edited

Loading

pr-commenter bot commented Sep 17, 2024 •

edited

Loading