Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Interval precision is None when ibis handle interval type of BigQuery #10357

Open
1 task done
grieve54706 opened this issue Oct 23, 2024 · 0 comments
Open
1 task done
Labels
bug Incorrect behavior inside of ibis

Comments

@grieve54706
Copy link
Contributor

grieve54706 commented Oct 23, 2024

What happened?

When selecting interval or getting the return type is interval, ibis can't handle the type in the schema to ibis type

ibis.bigquery.connect(...).sql('SELECT INTERVAL '1' DAY AS col')

I found the type that bigquery python client returned is not with precision in the schema.

def _get_schema_using_query(self, query: str) -> sch.Schema:
job = self.client.query(
query,
job_config=bq.QueryJobConfig(dry_run=True, use_query_cache=False),
project=self.billing_project,
)
return BigQuerySchema.to_ibis(job.schema)

And the _from_sqlglot_INTERVAL of SqlglotType uses a class field default_interval_precision.

def _from_sqlglot_INTERVAL(
cls,
precision_or_span: sge.IntervalSpan | None = None,
nullable: bool | None = None,
) -> dt.Interval:
if precision_or_span is None:
precision_or_span = cls.default_interval_precision
if isinstance(precision_or_span, str):
return dt.Interval(precision_or_span, nullable=nullable)
elif isinstance(precision_or_span, sge.IntervalSpan):
if (expression := precision_or_span.expression) is not None:
unit = expression.this
else:
unit = precision_or_span.this.this
return dt.Interval(unit=unit, nullable=nullable)
elif isinstance(precision_or_span, sge.Var):
return dt.Interval(unit=precision_or_span.this, nullable=nullable)
elif precision_or_span is None:
raise com.IbisTypeError("Interval precision is None")
else:
raise com.IbisTypeError(precision_or_span)

The class BigQueryType does not implement it. I am not sure this issue should just make the default_interval_precision with a unit will be okay.

What version of ibis are you using?

9.5.0

What backend(s) are you using, if any?

bigquery

Relevant log output

../app/model/connector.py:45: in query
    return self.connection.sql(sql).limit(limit).to_pandas()
../.venv/lib/python3.11/site-packages/ibis/backends/sql/__init__.py:176: in sql
    schema = self._get_schema_using_query(query)
../.venv/lib/python3.11/site-packages/ibis/backends/bigquery/__init__.py:659: in _get_schema_using_query
    return BigQuerySchema.to_ibis(job.schema)
../.venv/lib/python3.11/site-packages/ibis/backends/bigquery/datatypes.py:67: in to_ibis
    return sch.Schema({f.name: cls._dtype_from_bigquery_field(f) for f in fields})
../.venv/lib/python3.11/site-packages/ibis/backends/bigquery/datatypes.py:67: in <dictcomp>
    return sch.Schema({f.name: cls._dtype_from_bigquery_field(f) for f in fields})
../.venv/lib/python3.11/site-packages/ibis/backends/bigquery/datatypes.py:52: in _dtype_from_bigquery_field
    dtype = BigQueryType.from_string(typ)
../.venv/lib/python3.11/site-packages/ibis/backends/sql/datatypes.py:204: in from_string
    return cls.to_ibis(sgtype, nullable=nullable)
../.venv/lib/python3.11/site-packages/ibis/backends/sql/datatypes.py:173: in to_ibis
    dtype = method(*typ.expressions)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'app.custom_ibis.backends.sql.datatypes.BigQueryType'>
precision_or_span = None

    @classmethod
    def _from_sqlglot_INTERVAL(
        cls, precision_or_span: sge.IntervalSpan | None = None
    ) -> dt.Interval:
        nullable = cls.default_nullable
        if precision_or_span is None:
            precision_or_span = cls.default_interval_precision
    
        if isinstance(precision_or_span, str):
            return dt.Interval(precision_or_span, nullable=nullable)
        elif isinstance(precision_or_span, sge.IntervalSpan):
            if (expression := precision_or_span.expression) is not None:
                unit = expression.this
            else:
                unit = precision_or_span.this.this
            return dt.Interval(unit=unit, nullable=nullable)
        elif isinstance(precision_or_span, sge.Var):
            return dt.Interval(unit=precision_or_span.this, nullable=nullable)
        elif precision_or_span is None:
>           raise com.IbisTypeError("Interval precision is None")
E           ibis.common.exceptions.IbisTypeError: Interval precision is None

Code of Conduct

  • I agree to follow this project's Code of Conduct
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior inside of ibis
Projects
Status: backlog
Development

No branches or pull requests

1 participant