[Clickhouse] Ibis significantly slower than clickhouse native client for python

[rongxin-yu]

Hi,

I am testing using ibis against clickhouse and found that ibis seems to run significantly slower than native clickhouse client. I wonder if someone could help explain the reason and advise on how to solve it.

My experiment is as follows:

1. I write IBIS expression and run .execute() method (still running after 5 minutes)
2. I convert the ibis expression to sql and run the sql with native clickhouse client (10 seconds)

I currently plan to go with the second way, only using ibis as a SQL generator, but it's not as convenient as calling the .execute() method directly.

Any advice is greatly appreciated. Thanks!

[gforsyth]

Hey @rongxin-yu -- do you have an example to reproduce this behavior? That would help us track down what sounds like a bug.

We expect Ibis to be a touch slower than using any backend directly, but on the order of seconds or fractions of seconds.

[rongxin-yu]

@cpcloud

I tried to run the following command but still ran into this issue. Is there any way I can check whether I have actually checked out the master branch?

pip install 'git+https://github.com/ibis-project/ibis.git#egg=ibis-framework[clickhouse]'

DEPRECATION: git+https://github.com/ibis-project/ibis.git#egg=ibis-framework[clickhouse] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617

Installing collected packages: zstandard, clickhouse-connect Attempting uninstall: clickhouse-connect Found existing installation: clickhouse-connect 0.3.6 Uninstalling clickhouse-connect-0.3.6: Successfully uninstalled clickhouse-connect-0.3.6 Successfully installed clickhouse-connect-0.5.25 zstandard-0.21.0

[cpcloud]

@rongxin-yu Thanks for giving that a try.

Were you able to execute that install command in a fresh environment to rule out any pip installation issues?

[rongxin-yu]

Hi @cpcloud,

I just tried to start refresh.
With version 5.1.0-post.411+53ed88e92, the issue is gone.

Thanks for the help.

[cpcloud]

Awesome!

For future people who may observe this issue, the solution is: upgrade ibis :)

This change will be in the next release (6.0.0) which is slated for the end of June.