Skip to content

Commit

Permalink
chore(llmobs): extract and refactor langchain llmobs tests (#9407)
Browse files Browse the repository at this point in the history
This PR does a couple things:
- Extract all langchain llmobs tests from `test_langchain.py`,
`test_langchain_community.py` to `test_langchain_llmobs.py` to ease
maintainability for LLMObs testing in the future
- Refactors LLMObs langchain tests to be more readable and simplify
tests.
- Update tested versions of langchain from `0.0.192, 0.1.9` to `0.0.192
(start of ddtrace support), 0.1.20 (latest 0.1.x release), latest` and
regenerates cassette/snapshot files accordingly.


## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
  • Loading branch information
Yun-Kim authored May 29, 2024
1 parent 110f4e4 commit ecc56cf
Show file tree
Hide file tree
Showing 26 changed files with 1,136 additions and 1,165 deletions.
50 changes: 25 additions & 25 deletions .riot/requirements/a186e90.txt → .riot/requirements/1084a71.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,30 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/a186e90.in
# pip-compile --no-annotate .riot/requirements/1084a71.in
#
ai21==1.3.4
aiohttp==3.9.3
aiohttp==3.9.5
aiosignal==1.3.1
anyio==4.3.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
backoff==2.2.1
certifi==2024.2.2
charset-normalizer==3.3.2
cohere==4.53
coverage[toml]==7.4.3
cohere==4.57
coverage[toml]==7.5.3
dataclasses-json==0.5.14
dnspython==2.6.1
exceptiongroup==1.2.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.13.1
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.2.0
fsspec==2024.5.0
greenlet==3.0.3
huggingface-hub==0.21.4
huggingface-hub==0.23.2
hypothesis==6.45.0
idna==3.6
idna==3.7
importlib-metadata==6.11.0
iniconfig==2.0.0
jsonpatch==1.33
Expand All @@ -36,41 +36,41 @@ langchain-core==0.1.23
langchainplus-sdk==0.0.4
langsmith==0.0.87
loguru==0.7.2
marshmallow==3.21.1
marshmallow==3.21.2
mock==5.1.0
multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.9.0
numexpr==2.10.0
numpy==1.26.4
openai==0.27.8
openapi-schema-pydantic==1.2.4
opentracing==2.4.0
packaging==23.2
pinecone-client==2.2.4
pluggy==1.4.0
pluggy==1.5.0
psutil==5.9.8
pydantic==1.10.14
pytest==8.1.1
pydantic==1.10.15
pytest==8.2.1
pytest-asyncio==0.21.1
pytest-cov==4.1.0
pytest-mock==3.12.0
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.15.0
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2023.12.25
requests==2.31.0
regex==2024.5.15
requests==2.32.3
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
sqlalchemy==2.0.28
tenacity==8.2.3
tiktoken==0.6.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tomli==2.0.1
tqdm==4.66.2
typing-extensions==4.10.0
tqdm==4.66.4
typing-extensions==4.12.0
typing-inspect==0.9.0
urllib3==2.2.1
vcrpy==6.0.1
wrapt==1.16.0
yarl==1.9.4
zipp==3.17.0
zipp==3.19.0
92 changes: 92 additions & 0 deletions .riot/requirements/17e8568.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
#
# This file is autogenerated by pip-compile with Python 3.9
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/17e8568.in
#
ai21==2.4.0
ai21-tokenizer==0.9.1
aiohttp==3.9.5
aiosignal==1.3.1
annotated-types==0.7.0
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
boto3==1.34.114
botocore==1.34.114
certifi==2024.2.2
charset-normalizer==3.3.2
cohere==5.5.3
coverage[toml]==7.5.3
dataclasses-json==0.6.6
distro==1.9.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.5.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.23.2
hypothesis==6.45.0
idna==3.7
importlib-metadata==7.1.0
iniconfig==2.0.0
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==2.4
langchain==0.2.1
langchain-aws==0.1.6
langchain-community==0.2.1
langchain-core==0.2.1
langchain-openai==0.1.7
langchain-pinecone==0.1.1
langchain-text-splitters==0.2.0
langsmith==0.1.63
marshmallow==3.21.2
mock==5.1.0
multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.10.0
numpy==1.26.4
openai==1.30.4
opentracing==2.4.0
orjson==3.10.3
packaging==23.2
pinecone-client==3.2.2
pluggy==1.5.0
psutil==5.9.8
pydantic==2.7.2
pydantic-core==2.18.3
pytest==8.2.1
pytest-asyncio==0.21.1
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.15.0
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2024.5.15
requests==2.32.3
s3transfer==0.10.1
sentencepiece==0.2.0
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tokenizers==0.19.1
tomli==2.0.1
tqdm==4.66.4
types-requests==2.31.0.6
types-urllib3==1.26.25.14
typing-extensions==4.12.0
typing-inspect==0.9.0
urllib3==1.26.18
vcrpy==6.0.1
wrapt==1.16.0
yarl==1.9.4
zipp==3.19.0
50 changes: 25 additions & 25 deletions .riot/requirements/8ccebca.txt → .riot/requirements/1ec5924.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,29 @@
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/8ccebca.in
# pip-compile --no-annotate .riot/requirements/1ec5924.in
#
ai21==1.3.4
aiohttp==3.9.3
aiohttp==3.9.5
aiosignal==1.3.1
anyio==4.3.0
anyio==4.4.0
attrs==23.2.0
backoff==2.2.1
certifi==2024.2.2
charset-normalizer==3.3.2
cohere==4.53
coverage[toml]==7.4.3
cohere==4.57
coverage[toml]==7.5.3
dataclasses-json==0.5.14
dnspython==2.6.1
exceptiongroup==1.2.0
exceptiongroup==1.2.1
fastavro==1.9.4
filelock==3.13.1
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.2.0
fsspec==2024.5.0
greenlet==3.0.3
huggingface-hub==0.21.4
huggingface-hub==0.23.2
hypothesis==6.45.0
idna==3.6
idna==3.7
importlib-metadata==6.11.0
iniconfig==2.0.0
jsonpatch==1.33
Expand All @@ -35,40 +35,40 @@ langchain-core==0.1.23
langchainplus-sdk==0.0.4
langsmith==0.0.87
loguru==0.7.2
marshmallow==3.21.1
marshmallow==3.21.2
mock==5.1.0
multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.9.0
numexpr==2.10.0
numpy==1.26.4
openai==0.27.8
openapi-schema-pydantic==1.2.4
opentracing==2.4.0
packaging==23.2
pinecone-client==2.2.4
pluggy==1.4.0
pluggy==1.5.0
psutil==5.9.8
pydantic==1.10.14
pytest==8.1.1
pydantic==1.10.15
pytest==8.2.1
pytest-asyncio==0.21.1
pytest-cov==4.1.0
pytest-mock==3.12.0
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.15.0
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2023.12.25
requests==2.31.0
regex==2024.5.15
requests==2.32.3
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
sqlalchemy==2.0.28
tenacity==8.2.3
tiktoken==0.6.0
tqdm==4.66.2
typing-extensions==4.10.0
sqlalchemy==2.0.30
tenacity==8.3.0
tiktoken==0.7.0
tqdm==4.66.4
typing-extensions==4.12.0
typing-inspect==0.9.0
urllib3==2.2.1
vcrpy==6.0.1
wrapt==1.16.0
yarl==1.9.4
zipp==3.17.0
zipp==3.19.0
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/da7b0f6.in
# pip-compile --no-annotate .riot/requirements/1f6f978.in
#
ai21==2.4.0
ai21-tokenizer==0.9.1
Expand All @@ -25,6 +25,7 @@ fastavro==1.9.4
filelock==3.14.0
frozenlist==1.4.1
fsspec==2024.5.0
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
Expand All @@ -40,17 +41,17 @@ langchain==0.1.20
langchain-aws==0.1.6
langchain-community==0.0.38
langchain-core==0.1.52
langchain-openai==0.1.5
langchain-pinecone==0.1.1
langchain-openai==0.1.6
langchain-pinecone==0.1.0
langchain-text-splitters==0.0.2
langsmith==0.1.63
langsmith==0.1.58
marshmallow==3.21.2
mock==5.1.0
multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.10.0
numpy==1.26.4
openai==1.12.0
openai==1.30.3
opentracing==2.4.0
orjson==3.10.3
packaging==23.2
Expand All @@ -67,7 +68,7 @@ pytest-randomly==3.15.0
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2024.5.15
requests==2.32.2
requests==2.32.3
s3transfer==0.10.1
sentencepiece==0.2.0
six==1.16.0
Expand Down
Loading

0 comments on commit ecc56cf

Please sign in to comment.