You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When doing a very large query using lazy mode, if there's any error in the query, you will get it when you call collect which is normal, but we are receiving errors like this in production:
polars.exceptions.ComputeError: cannot compare string with numeric type (i32)
Which column name, compared to which other column produced this error? Impossible to know. My query expression tree is constructed from several methods adding filters dynamically with a lot of columns, it is very difficult to investigate which column specifically has the wrong type in production when this doesn't happen locally.
But given that we are in a lazy query, we have all the query expression tree, so we could potentially at least return in the error message which column name are having a problem.
The only solution right now is to transform this big query in several chunk in eager mode and see where it will crash in production which is not ideal.
The text was updated successfully, but these errors were encountered:
Description
When doing a very large query using lazy mode, if there's any error in the query, you will get it when you call collect which is normal, but we are receiving errors like this in production:
polars.exceptions.ComputeError: cannot compare string with numeric type (i32)
Which column name, compared to which other column produced this error? Impossible to know. My query expression tree is constructed from several methods adding filters dynamically with a lot of columns, it is very difficult to investigate which column specifically has the wrong type in production when this doesn't happen locally.
But given that we are in a lazy query, we have all the query expression tree, so we could potentially at least return in the error message which column name are having a problem.
The only solution right now is to transform this big query in several chunk in eager mode and see where it will crash in production which is not ideal.
The text was updated successfully, but these errors were encountered: