Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ListingTableUrl should allow direct construction #12581

Open
rtyler opened this issue Sep 22, 2024 · 2 comments
Open

ListingTableUrl should allow direct construction #12581

rtyler opened this issue Sep 22, 2024 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@rtyler
Copy link
Contributor

rtyler commented Sep 22, 2024

Is your feature request related to a problem or challenge?

When investigating delta-io/delta-rs#2834 I discovered that the crux of the problem was that our (delta-rs) code has Urls coming from object_store, which are then passed back into Datafusion via read_parquet which does a ListingTableUrl::parse on those strings, thereby turning URLs into strings and "corrupting" them in the process.

Describe the solution you'd like

I think a ListingTableUrl constructor or means of construction without a parse would suffice for our use-case. On the delta-rs side we have effectively all the necessary information to construct a ListingTableUrl (url, scheme, object store, etc) but we are not able to build that ourselves and therefore have to jump through a couple hoops to get ListingTableUrl::parse to produce the right thing.

Describe alternatives you've considered

Right now I'm passing str::replcae(meta.location.as_ref(), "%", "%25") to read_parquet and it feels yucky 😆 :naus

Additional context

No response

@rtyler rtyler added the enhancement New feature or request label Sep 22, 2024
@alamb
Copy link
Contributor

alamb commented Sep 23, 2024

I agree -- being able to take a direct ListingTableUrl seems like a good idea to me

@OussamaSaoudi
Copy link
Contributor

take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants