Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Creating external the table with SQL with unbounded and ordered options #5126

Closed
metesynnada opened this issue Jan 31, 2023 · 3 comments
Closed
Labels
enhancement New feature or request

Comments

@metesynnada
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

CREATE EXTERNAL TABLE t(c1 int) STORED AS CSV PARTITIONED BY (p1 int) LOCATION 'foo.csv'

is supported, however, we cannot inject unbounded and ordered information like other flags. A basic usage can be

CREATE EXTERNAL TABLE t(c1 int) STORED AS CSV ORDERED BY (p1 int) LOCATION 'foo.csv' UNBOUNDED

but, of course, I am open to better word selection. The problem is not limited to CSV, we need to support JSON and AVRO types also.

Describe the solution you'd like
The problem requires the sqlparser crate to be changed first. Then, we apply necessary flag declarations and logical plan changes here.

Describe alternatives you've considered
NA

Additional context
Na

@metesynnada metesynnada added the enhancement New feature or request label Jan 31, 2023
@alamb
Copy link
Contributor

alamb commented Feb 2, 2023

This might be a good feature to add as a custom DataFUsion parser extension -- i.e. you can follow the model of CREATE EXTERNAL TABLE which is not supported by sqlparser but rather in DataFusion

@metesynnada
Copy link
Contributor Author

Sure, thanks for the feedback. If no one owns the issue in a couple of weeks, we will start it.

@alamb
Copy link
Contributor

alamb commented Sep 29, 2024

I believe WITH ORDER and UNBOUNDED are now supported so we can close this ticket

Doc reference: https://datafusion.apache.org/user-guide/sql/ddl.html#create-external-table

@alamb alamb closed this as completed Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants