Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: avoid JSONDecodeError for empty line stream response #393

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

johnny0120
Copy link

@johnny0120 johnny0120 commented Dec 25, 2024

r.iter_lines() may yield empty string, which results in JSONDecodeError and terminates the stream response.

the stackstrace is as follows:

File "/root/.pyenv/versions/3.10.16/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/root/Developer/github/LightRAG/.venv/lib/python3.10/site-packages/tenacity/asyncio/__init__.py", line 114, in __call__
    result = await fn(*args, **kwargs)
  File "/root/Developer/github/LightRAG/lightrag/llm.py", line 335, in ollama_model_if_cache
    for chunk in response:
  File "/root/Developer/github/LightRAG/.venv/lib/python3.10/site-packages/ollama/_client.py", line 171, in inner
    part = json.loads(line)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/root/.pyenv/versions/3.10.16/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

the fix: check line before json.loads, continue to next line if is empty

Client

for line in r.iter_lines():
    if not line: continue
    part = json.loads(line)

AsyncClient

async for line in r.aiter_lines():
    if not line: continue
    part = json.loads(line)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant