query clickhouse table with uint256 type columns

[xiaom]

Hey,

I tried to use ibis to query a ClickHouse table with a uint256 column type but got the following exceptions:

t = ch.table(name=identifier, database="community")
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/ibis/backends/sql/__init__.py", line 137, in table
    table_schema = self.get_schema(name, catalog=catalog, database=database)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/ibis/backends/clickhouse/__init__.py", line 507, in get_schema
    dict(zip(names, map(self.compiler.type_mapper.from_string, types)))
  File ".venv/lib/python3.11/site-packages/ibis/backends/sql/datatypes.py", line 200, in from_string
    return cls.to_ibis(sgtype, nullable=nullable)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/ibis/backends/sql/datatypes.py", line 173, in to_ibis
    dtype = _from_sqlglot_types[typecode](nullable=cls.default_nullable)

I noticed that the type is commented out in the source code as an unsupported datatype. Is there any workaround to query such a table in ibis?

Thanks!

[gforsyth]

What version of Ibis are you using? We added a fallback in 9.2.0 to treat unsupported datatypes as unknown -- so while you can't operate on the unsupported dtypes, you can still view the table and interact with the other columns.

[xiaom]

9.3.0. I also tried 9.2.0 but it has the same issue.

It fails when I create the table object, so I cannot access other columns as well.

ch.table(name=identifier, database="community")

[gforsyth]

Hmm, yep, this should work but doesn't. Can you file a bug report?

[cpcloud]

Moving this to an issue.

[cpcloud]

Fixed in #9868.

[xiaom]

Thanks for fixing this so quickly!

A follow-up question: assuming we have the table object, is there an easy way to cast the uint256 column to a string and access it via the ibis API?

[gforsyth]

Sure, if you have some table ch_table with a column called my_big_uint you can do:

ch_table = ch_table.mutate(my_big_uint=ch_table.my_big_uint.cast("str"))

[xiaom]

nice! Thanks for the tip!