feat(python): improve large Enum reprs

[alexander-beedie]

Closes #13337.

Split-out from #13345 following some insight from @stinodego, and contains only the improved __repr__.

Examples

Improved repr for Enum with a large number of categories:

import polars as pl

pl.Enum(f"c{i}" for i in range(5))
# Enum(categories=['c0','c1','c2','c3','c4'])

pl.Enum(f"c{i}" for i in range(500))
# Enum(categories=['c0','c1','c2' … 'c497','c498','c499'])

---

ℹ️ RFC: DataType serialisation

Parked this PR in Draft as there is apparently an implicit expectation that the DataType repr should be guaranteed to eval back to the equivalent instantiated object, acting as a form of serialisation. I think we need a real API point for this instead, as the repr is a fragile/non-standard way to handle that, and not obvious.

Given the desired use-case¹ I think we might want to add write_json and from_json methods for DataType, which would decouple serialisation from the repr and offer a solid/consistent API for this use-case. We already have such methods available for Expr, for example.

Footnotes

1. See Implement DataType.serialize/deserialize for serializing to/from JSON #13152 ↩

[ritchie46]

apparently an implicit expectation that the DataType repr should be guaranteed to eval back to the equivalent instantiated object, acting as a form of serialisation

Are we dependent on that? I don't think we should. Repr and serde a different things.

[alexander-beedie]

Are we dependent on that? I don't think we should. Repr and serde a different things.

I entirely agree ;)
However, it looks like we suggested this for patito here, so we need to untangle ourselves from that a little.

[ritchie46]

Ai.. Doesn't pickle work out of the box?

[alexander-beedie]

Ai.. Doesn't pickle work out of the box?

It does, but you don't want pickle if your interop is via JSON (like patito) or you are serialising more generally. I figure we probably need a write_json / from_json pair, as we have for other parts of the API 🤔 (@stinodego?)

[stinodego]

Random idea: can we have a __str__ implementation that is short and readable (max 6 categories), and a __repr__ implementation that is a full representation (like currently)? I think generally the repr is the 'official' full spec of an object, while str is the user-readable version.

I think that could address the situation adequately (repr can be eval'd, printing the object gets a compact result).

EDIT: I worked out the idea a bit further: #13439

[stinodego]

I think we should go ahead with this, but first implement serialization as requested in #13152

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.02%. Comparing base (4656342) to head (be0033a).

❗ Current head be0033a differs from pull request most recent head fc48302. Consider uploading reports for the commit fc48302 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #13357      +/-   ##
==========================================
- Coverage   81.22%   81.02%   -0.21%     
==========================================
  Files        1348     1332      -16     
  Lines      175333   172883    -2450     
  Branches     2508     2461      -47     
==========================================
- Hits       142420   140075    -2345     
+ Misses      32433    32340      -93     
+ Partials      480      468      -12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.