Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BorrowDatum, FlatArray, and Text #1856

Open
wants to merge 67 commits into
base: develop
Choose a base branch
from

Conversation

workingjubilee
Copy link
Member

@workingjubilee workingjubilee commented Sep 11, 2024

Introduce a new Rust type that serves as a definition of what Postgres ArrayTypes "really are", called FlatArray. This is meant to be manipulated via &FlatArray<'_, T>... it is an "unsized type". To support this, we also introduce BorrowDatum, which allows borrowing elements directly and is what powers FlatArray's simplicity in iteration, compared to the incredibly jank implementation in pgrx/src/datum/array.rs!

The cost is that it is much more nitpicky about what it can work with, as the type must implement BorrowDatum, which means it must be directly borrowable! This means FlatArray can't be, say, FlatArray<'mcx, String>, and implicitly unbox the elements into an allocated type. You can still map the outputs, of course. The good news is that this means that all FlatArrays must truly be "zero-copy", as the type definition doesn't allow anything else. As part of this compromise, we introduce a new Text type so that it is still possible to work with common array needs like strings.

These are effectively the first varlena types that allow directly interacting with their header instead of "forgetting" it.

@workingjubilee
Copy link
Member Author

workingjubilee commented Sep 28, 2024

This needs more tests, and I'll probably split it into components so that it can be reviewed in smaller chunks, but I'm satisfied with the fundamentals of the implementation here.

@workingjubilee workingjubilee marked this pull request as ready for review September 28, 2024 21:06
@workingjubilee workingjubilee changed the title BorrowDatum, FlatArray, and Text? BorrowDatum, FlatArray, and Text Sep 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant