Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make supported types in dialect yaml files as a map #81

Merged
merged 5 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bft/dialects/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ class DialectFile(NamedTuple):
scalar_functions: List[DialectFunction]
aggregate_functions: List[DialectFunction]
uri_to_func_prefix: Dict[str, str]
supported_types: List[str]
supported_types: Dict[str, Dict]


class SqlMapping(NamedTuple):
Expand Down
30 changes: 20 additions & 10 deletions dialects/cudf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,26 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- i8
- i16
- i32
- i64
- fp32
- fp64
- bool
- str
- ts
- date
i8:
df_type_name: int8
i16:
df_type_name: int16
i32:
df_type_name: int32
i64:
df_type_name: int64
fp32:
df_type_name: float32
fp64:
df_type_name: float64
bool:
df_type_name: bool
str:
df_type_name: string
ts:
df_type_name: datetime64[s]
date:
df_type_name: datetime64[s]
scalar_functions:
- name: arithmetic.add
local_name: add
Expand Down
37 changes: 24 additions & 13 deletions dialects/datafusion.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,19 +26,30 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- i8
- i16
- i32
- i64
- fp32
- fp64
- bool
- str
- date
- time
- ts
- tstz
- interval
i8:
sql_type_name: int8
i16:
sql_type_name: int16
i32:
sql_type_name: int32
i64:
sql_type_name: int64
fp32:
sql_type_name: float32
fp64:
sql_type_name: float64
bool:
sql_type_name: bool_
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the underscore in SQL?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this from runner.py

str:
sql_type_name: string
date:
sql_type_name: timestamp
time:
sql_type_name: timestamp
ts:
sql_type_name: timestamp
tstz:
sql_type_name: timestamp
scalar_functions:
- name: arithmetic.add
local_name: +
Expand Down
42 changes: 28 additions & 14 deletions dialects/duckdb.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,34 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- i8
- i16
- i32
- i64
- fp32
- fp64
- bool
- str
- date
- time
- ts
- tstz
- interval
- dec
i8:
sql_type_name: tinyint
i16:
sql_type_name: smallint
i32:
sql_type_name: integer
i64:
sql_type_name: bigint
fp32:
sql_type_name: real
fp64:
sql_type_name: double
bool:
sql_type_name: boolean
str:
sql_type_name: varchar
date:
sql_type_name: date
time:
sql_type_name: time
ts:
sql_type_name: timestamp
tstz:
sql_type_name: timestamptz
interval:
sql_type_name: interval
decimal:
sql_type_name: decimal
scalar_functions:
- name: arithmetic.add
local_name: +
Expand Down
36 changes: 24 additions & 12 deletions dialects/postgres.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,18 +26,30 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- i16
- i32
- i64
- fp32
- fp64
- bool
- str
- date
- time
- ts
- tstz
- interval
i16:
sql_type_name: smallint
i32:
sql_type_name: integer
i64:
sql_type_name: bigint
fp32:
sql_type_name: float4
fp64:
sql_type_name: float8
bool:
sql_type_name: boolean
str:
sql_type_name: text
date:
sql_type_name: date
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these type names lowercase while most of the other dialects are capitalized?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed everything to lower case.

time:
sql_type_name: time
ts:
sql_type_name: timestamp
tstz:
sql_type_name: timestamptz
interval:
sql_type_name: interval
scalar_functions:
- name: arithmetic.add
local_name: +
Expand Down
28 changes: 19 additions & 9 deletions dialects/snowflake.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,25 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- fp64
- bool
- str
- date
- time
- ts
- tsz
- interval
- dec
fp64:
sql_type_name: float
bool:
sql_type_name: boolean
str:
sql_type_name: varchar
date:
sql_type_name: date
time:
sql_type_name: time
ts:
sql_type_name: timestamp
tstz:
sql_type_name: timestamptz
interval:
sql_type_name: interval
supported_as_column: false
dec:
sql_type_name: numeric
scalar_functions:
- name: arithmetic.add
local_name: +
Expand Down
24 changes: 16 additions & 8 deletions dialects/sqlite.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,22 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- i8
- i16
- i32
- i64
- fp32
- fp64
- bool
- str
i8:
sql_type_name: tinyint
i16:
sql_type_name: smallint
i32:
sql_type_name: int
i64:
sql_type_name: hugeint
fp32:
sql_type_name: real
fp64:
sql_type_name: real
bool:
sql_type_name: boolean
str:
sql_type_name: text
scalar_functions:
- name: arithmetic.add
local_name: +
Expand Down
12 changes: 8 additions & 4 deletions dialects/velox_presto.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,14 @@ dependencies:
string:
https://github.com/substrait-io/substrait/blob/main/extensions/substrait/extensions/functions_string.yaml
supported_types:
- i64
- fp64
- bool
- str
i64:
sql_type_name: i64
fp64:
sql_type_name: fp64
bool:
sql_type_name: boolean
str:
sql_type_name: string
scalar_functions:
- name: arithmetic.add
local_name: +
Expand Down
Loading