feat(python): Various Schema improvements (equality/init dtype checks)

[alexander-beedie]

A few improvements (arguably fixes) for equality/inequality checks, and dtype init:

• We require that a Schema is initialised with instantiated dtypes (and make a point of it in the docstring), but don't actually check that we are given such; this PR validates that nested dtypes are given in instantiated form (eg: pl.List(Float64), not just pl.List), as without the inner dtype we don't actually have a valid Schema. I haven't done a full recursive check (eg: walking into Struct fields), but can look at it in a second pass.

• Uninstantiated non-nested types (such as Int64 that do not take modifying parameters) are allowed, and are automatically instantiated on the way in to guarantee that a Schema instance is always composed of instantiated dtypes. Those that take units/specifiers (such as Datetime) require specific user-side instantiation, as above.

• Fixed equality check between Schema and dict objects; similar to the above, we could compare a Schema with a {name: dtype, …} dict and get potentially invalid results if the dictionary had uninstantiated dtypes.

• Added a convenience base_types method, returning a {name: dtype_class, …} dict (as returned by the individual dtype equivalent base_type method). (update: removed the new method)

• Optimised internal Schema init where the dtypes are already known to be valid (eg: we are initialising from internal state, so we know that we don't have to check them).

• Drive-by; The "schema_overrides" param in read_excel should handle python dtypes (as this param does elsewhere), eg: schema_overrides={"col": float}.

[codecov]

Codecov Report

Attention: Patch coverage is 86.66667% with 6 lines in your changes missing coverage. Please review.

Project coverage is 80.19%. Comparing base (27289b2) to head (db59be1).
Report is 16 commits behind head on main.

Files with missing lines	Patch %	Lines
py-polars/polars/schema.py	86.84%	3 Missing and 2 partials ⚠️
py-polars/polars/series/struct.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #19379      +/-   ##
==========================================
+ Coverage   80.18%   80.19%   +0.01%     
==========================================
  Files        1523     1527       +4     
  Lines      209897   210268     +371     
  Branches     2434     2440       +6     
==========================================
+ Hits       168314   168633     +319     
- Misses      41028    41079      +51     
- Partials      555      556       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ritchie46]

I do agree this is valuable, but I am afraid we will add features we cannot support when we move to Rust.

I want our schema to be a python wrapper around PolarsSchema, just like we do with DataFrame and Series. I think we should do that refactor first and then add functionality there.

[alexander-beedie]

I do agree this is valuable, but I am afraid we will add features we cannot support when we move to Rust.

Yup, can remove the base_types method, no problem - was merely a minor convenience for lazy people (like myself) 😆

However, I think we still need to do the dtype-checks here, as we will always want to pass correct (and instantiated) dtypes into the PolarsSchema object, when exposed (which, you are quite right!, is where any significant new Schema functionality should live) 👍