Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unhandled IndexError or JSONDecodeError when exceeding max_execution_time. #93

Open
tomasfarias opened this issue Jul 10, 2023 · 0 comments

Comments

@tomasfarias
Copy link

tomasfarias commented Jul 10, 2023

Steps to reproduce:

  1. Setup a ClickHouse db with a lot of fake data (I used about 10 million key-value pairs).
  2. Set max_execution_time to something relatively low, I used 120 secs.
  3. Send a request using ChClient (I used aiohttp, maybe the same happens with httpx, although I haven't tested):
from aiochclient import ChClient
from aiohttp import ClientSession, ClientTimeout, TCPConnector
connector = TCPConnector(ssl=False)
session = ClientSession(connector=connector)
client = ChClient(session, url=f"http://localhost:8123", user="default", password="", database="default")
iterator = client.iterate("select key, value from test")
rows = [row async for row in iterator]
  1. Assuming we hit the max_execution_time, two things can happen depending on whether we are calling iterate with json=True or json=False:
    a. json=True

An exception will be raised as we iterate:

File "/aiochclient/client.py", line 396, in iterate
    async for row in self._execute(
  File "/aiochclient/client.py", line 179, in _execute
    yield rf.new(line)
  File "/aiochclient/records.py", line 101, in new
    return self.loads(row)
  File "/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This happens as the line returned from the resp.content.iter_any() is not a valid JSON, but a TOO_SLOW error message from ClickHouse:

b'Code: 160. DB::Exception: Estimated query execution time (180.2056076714325 seconds) is too long. Maximum: 20. Estimated rows to process: 114844900: While executing MergeTreeThread. (TOO_SLOW) (version 23.4.2.11 (official build))\n'

b. json=False

We don't get an exception when iterating over the results. However, if we check the list of rows, the last record will actually be the error message from ClickHouse:

>>> [val for val in rows[-1].values()]
Traceback (most recent call last):
  File "/_collections_abc.py", line 930, in __iter__
    yield self._mapping[key]
  File "/aiochclient/records.py", line 47, in __getitem__
    return self._getitem(key)
  File "/aiochclient/records.py", line 52, in _getitem
    return self._row[self._names[key]]
IndexError: tuple index out of range
>>> rows[-1]._row
(b'Code: 160. DB::Exception: Estimated query execution time (134.46100121598155 seconds) is too long. Maximum: 20. Estimated rows to process: 114844900: While executing MergeTreeThread. (TOO_SLOW) (version 23.4.2.11 (official build))',)

Expected behavior:

Instead of hiding the underlying error, aiochclient should properly inform the user that their queries are slow for ClickHouse (or at least that ClickHouse returned an error string). Unfortunately the response is successful as this error happens after we have received the response, so there is no easy way to check this. One way to go about it is by catching the JSONDecodeError and IndexError and raising a more informative exception, for example for the JSONDecodeError case:

if is_json:
    rf = FromJsonFabric(loads=self._json.loads)
        async for line in response:
            try:
                yield rf.new(line)
            except JSONDecodeError:
                raise ChClientError(f"Data received from ClickHouse could not be decoded as JSON, potentially because of an error message: {line}")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant