Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Parquet Files with Delta Encoding and other Parquet V2 Features #11371

Open
3 tasks
RussellSpitzer opened this issue Oct 21, 2024 · 0 comments
Open
3 tasks

Support Parquet Files with Delta Encoding and other Parquet V2 Features #11371

RussellSpitzer opened this issue Oct 21, 2024 · 0 comments
Labels
improvement PR that improves existing functionality

Comments

@RussellSpitzer
Copy link
Member

Feature Request / Improvement

We currently do not support Parquet files generated by several engines/fabrics that use

Encodings - DELTA_BINARY_PACKED, DELTA_LENGTH_BYTE_ARRAY, DELTA_BYTE_ARRAY
or V2 Pages

Support was added to Spark 3.3.0 but we still don't support this in our vectorized readers.

Query engine

None

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
improvement PR that improves existing functionality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@RussellSpitzer