Skip to content

Commit

Permalink
feat(llmobs): add Anthropic LLMObs Integration (#9462)
Browse files Browse the repository at this point in the history
This PR adds LLM Observability for Anthropic messages. 

## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

---------

Co-authored-by: Yun Kim <[email protected]>
Co-authored-by: Yun Kim <[email protected]>
  • Loading branch information
3 people authored Jun 5, 2024
1 parent 52e83bf commit 3bc3051
Show file tree
Hide file tree
Showing 23 changed files with 650 additions and 148 deletions.
32 changes: 18 additions & 14 deletions .riot/requirements/76db01b.txt → .riot/requirements/121ef70.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,41 +2,45 @@
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/76db01b.in
# pip-compile --no-annotate .riot/requirements/121ef70.in
#
ai21==2.4.0
ai21==2.4.1
ai21-tokenizer==0.9.1
aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anthropic==0.28.0
anyio==4.4.0
attrs==23.2.0
boto3==1.34.114
botocore==1.34.114
certifi==2024.2.2
boto3==1.34.120
botocore==1.34.120
certifi==2024.6.2
charset-normalizer==3.3.2
cohere==5.5.3
cohere==5.5.4
coverage[toml]==7.5.3
dataclasses-json==0.6.6
defusedxml==0.7.1
distro==1.9.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.5.0
fsspec==2024.6.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.23.2
huggingface-hub==0.23.3
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
jiter==0.4.1
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
langchain==0.1.20
langchain-anthropic==0.1.11
langchain-aws==0.1.6
langchain-community==0.0.38
langchain-core==0.1.52
Expand All @@ -57,9 +61,9 @@ packaging==23.2
pinecone-client==3.2.2
pluggy==1.5.0
psutil==5.9.8
pydantic==2.7.2
pydantic-core==2.18.3
pytest==8.2.1
pydantic==2.7.3
pydantic-core==2.18.4
pytest==8.2.2
pytest-asyncio==0.21.1
pytest-cov==5.0.0
pytest-mock==3.14.0
Expand All @@ -76,10 +80,10 @@ sortedcontainers==2.4.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tokenizers==0.19.1
tokenizers==0.15.2
tqdm==4.66.4
types-requests==2.32.0.20240523
typing-extensions==4.12.0
types-requests==2.32.0.20240602
typing-extensions==4.12.1
typing-inspect==0.9.0
urllib3==2.2.1
vcrpy==6.0.1
Expand Down
46 changes: 25 additions & 21 deletions .riot/requirements/e17f33e.txt → .riot/requirements/144795f.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,65 +2,69 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/e17f33e.in
# pip-compile --no-annotate .riot/requirements/144795f.in
#
ai21==2.4.0
ai21==2.4.1
ai21-tokenizer==0.9.1
aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anthropic==0.28.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
boto3==1.34.114
botocore==1.34.114
certifi==2024.2.2
boto3==1.34.120
botocore==1.34.120
certifi==2024.6.2
charset-normalizer==3.3.2
cohere==5.5.3
cohere==5.5.4
coverage[toml]==7.5.3
dataclasses-json==0.6.6
defusedxml==0.7.1
distro==1.9.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.5.0
fsspec==2024.6.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.23.2
huggingface-hub==0.23.3
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
jiter==0.4.1
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
langchain==0.2.1
langchain==0.2.2
langchain-anthropic==0.1.15
langchain-aws==0.1.6
langchain-community==0.2.1
langchain-core==0.2.1
langchain-openai==0.1.7
langchain-community==0.2.3
langchain-core==0.2.4
langchain-openai==0.1.8
langchain-pinecone==0.1.1
langchain-text-splitters==0.2.0
langsmith==0.1.63
langchain-text-splitters==0.2.1
langsmith==0.1.73
marshmallow==3.21.2
mock==5.1.0
multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.10.0
numpy==1.26.4
openai==1.30.4
openai==1.31.1
opentracing==2.4.0
orjson==3.10.3
packaging==23.2
pinecone-client==3.2.2
pluggy==1.5.0
psutil==5.9.8
pydantic==2.7.2
pydantic-core==2.18.3
pytest==8.2.1
pydantic==2.7.3
pydantic-core==2.18.4
pytest==8.2.2
pytest-asyncio==0.21.1
pytest-cov==5.0.0
pytest-mock==3.14.0
Expand All @@ -77,11 +81,11 @@ sortedcontainers==2.4.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tokenizers==0.19.1
tokenizers==0.15.2
tomli==2.0.1
tqdm==4.66.4
types-requests==2.32.0.20240523
typing-extensions==4.12.0
types-requests==2.32.0.20240602
typing-extensions==4.12.1
typing-inspect==0.9.0
urllib3==2.2.1
vcrpy==6.0.1
Expand Down
32 changes: 18 additions & 14 deletions .riot/requirements/8297334.txt → .riot/requirements/16311ec.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,47 @@
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/8297334.in
# pip-compile --no-annotate .riot/requirements/16311ec.in
#
ai21==2.4.0
ai21==2.4.1
ai21-tokenizer==0.9.1
aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anthropic==0.28.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
boto3==1.34.114
botocore==1.34.114
certifi==2024.2.2
boto3==1.34.120
botocore==1.34.120
certifi==2024.6.2
charset-normalizer==3.3.2
cohere==5.5.3
cohere==5.5.4
coverage[toml]==7.5.3
dataclasses-json==0.6.6
defusedxml==0.7.1
distro==1.9.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.5.0
fsspec==2024.6.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.23.2
huggingface-hub==0.23.3
hypothesis==6.45.0
idna==3.7
importlib-metadata==7.1.0
iniconfig==2.0.0
jiter==0.4.1
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
langchain==0.1.20
langchain-anthropic==0.1.11
langchain-aws==0.1.6
langchain-community==0.0.38
langchain-core==0.1.52
Expand All @@ -59,9 +63,9 @@ packaging==23.2
pinecone-client==3.2.2
pluggy==1.5.0
psutil==5.9.8
pydantic==2.7.2
pydantic-core==2.18.3
pytest==8.2.1
pydantic==2.7.3
pydantic-core==2.18.4
pytest==8.2.2
pytest-asyncio==0.21.1
pytest-cov==5.0.0
pytest-mock==3.14.0
Expand All @@ -78,15 +82,15 @@ sortedcontainers==2.4.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tokenizers==0.19.1
tokenizers==0.15.2
tomli==2.0.1
tqdm==4.66.4
types-requests==2.31.0.6
types-urllib3==1.26.25.14
typing-extensions==4.12.0
typing-extensions==4.12.1
typing-inspect==0.9.0
urllib3==1.26.18
vcrpy==6.0.1
wrapt==1.16.0
yarl==1.9.4
zipp==3.19.0
zipp==3.19.2
46 changes: 25 additions & 21 deletions .riot/requirements/17e8568.txt → .riot/requirements/1bd8488.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,66 +2,70 @@
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/17e8568.in
# pip-compile --no-annotate .riot/requirements/1bd8488.in
#
ai21==2.4.0
ai21==2.4.1
ai21-tokenizer==0.9.1
aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anthropic==0.28.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
boto3==1.34.114
botocore==1.34.114
certifi==2024.2.2
boto3==1.34.120
botocore==1.34.120
certifi==2024.6.2
charset-normalizer==3.3.2
cohere==5.5.3
cohere==5.5.4
coverage[toml]==7.5.3
dataclasses-json==0.6.6
defusedxml==0.7.1
distro==1.9.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.5.0
fsspec==2024.6.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.23.2
huggingface-hub==0.23.3
hypothesis==6.45.0
idna==3.7
importlib-metadata==7.1.0
iniconfig==2.0.0
jiter==0.4.1
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
langchain==0.2.1
langchain==0.2.2
langchain-anthropic==0.1.15
langchain-aws==0.1.6
langchain-community==0.2.1
langchain-core==0.2.1
langchain-openai==0.1.7
langchain-community==0.2.3
langchain-core==0.2.4
langchain-openai==0.1.8
langchain-pinecone==0.1.1
langchain-text-splitters==0.2.0
langsmith==0.1.63
langchain-text-splitters==0.2.1
langsmith==0.1.73
marshmallow==3.21.2
mock==5.1.0
multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.10.0
numpy==1.26.4
openai==1.30.4
openai==1.31.1
opentracing==2.4.0
orjson==3.10.3
packaging==23.2
pinecone-client==3.2.2
pluggy==1.5.0
psutil==5.9.8
pydantic==2.7.2
pydantic-core==2.18.3
pytest==8.2.1
pydantic==2.7.3
pydantic-core==2.18.4
pytest==8.2.2
pytest-asyncio==0.21.1
pytest-cov==5.0.0
pytest-mock==3.14.0
Expand All @@ -78,15 +82,15 @@ sortedcontainers==2.4.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tokenizers==0.19.1
tokenizers==0.15.2
tomli==2.0.1
tqdm==4.66.4
types-requests==2.31.0.6
types-urllib3==1.26.25.14
typing-extensions==4.12.0
typing-extensions==4.12.1
typing-inspect==0.9.0
urllib3==1.26.18
vcrpy==6.0.1
wrapt==1.16.0
yarl==1.9.4
zipp==3.19.0
zipp==3.19.2
Loading

0 comments on commit 3bc3051

Please sign in to comment.