Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Easier scalar func support #23642

Open
wants to merge 19 commits into
base: master
Choose a base branch
from

Conversation

auden-woolfson
Copy link
Contributor

@auden-woolfson auden-woolfson commented Sep 12, 2024

Description

Fix for issue #23605

Currently in order to add new operator/function mappings a new function needs to be defined for each possible mapping. By introducing some new annotations for functions and handling the case we can re use function definitions with different possible type signatures.

== RELEASE NOTES ==

General Changes
* Add SupportedSignatures and SqlSignature annotations :pr:`23642`
* Update JDBC OperatorTranslators to use new annotations :pr:`23642`

@auden-woolfson auden-woolfson self-assigned this Sep 12, 2024
@auden-woolfson auden-woolfson requested a review from a team as a code owner September 12, 2024 21:59
@auden-woolfson auden-woolfson marked this pull request as draft September 12, 2024 21:59
@aaneja
Copy link
Contributor

aaneja commented Sep 13, 2024

Can you add tests ? Take a look at TestRowExpressionTranslator - this uses translations defined in TestRowExpressionTranslator.TestFunctions

@aaneja aaneja marked this pull request as ready for review September 13, 2024 13:51
@@ -33,8 +35,8 @@ private OperatorTranslators()
}

@ScalarOperator(ADD)
@SqlType(StandardTypes.BIGINT)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the file we want to change is in presto-base-jdbc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I can update all the cases in the JDBC file, should we ultimately change all operator translator classes?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should support all translators eventually. I am not familiar with the SQL dialect that Clickhouse supports, but I think the basic operators + types should translate and work with it. Feel free to take it up in another PR if this reduces the testing overhead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, will do once this one gets merged!

@@ -33,8 +35,8 @@ private OperatorTranslators()
}

@ScalarOperator(ADD)
@SqlType(StandardTypes.BIGINT)
public static ClickHouseExpression add(@SqlType(StandardTypes.BIGINT) ClickHouseExpression left, @SqlType(StandardTypes.BIGINT) ClickHouseExpression right)
@SupportedSignatures(@SqlSignature({StandardTypes.BIGINT, StandardTypes.BIGINT}))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should both of these be BIGINT ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, one of these values represents the argument types, and one represents the return type


@Retention(RUNTIME)
@Target(METHOD)
public @interface SqlSignature {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call this SqlTypes instead ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made some updates that I think clarify the purpose of the annotations, I think SqlSignature might be a bit more descriptive

@aaneja aaneja marked this pull request as draft September 13, 2024 13:52
Copy link
Contributor

@aaneja aaneja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tdcmeehan Can you validate the approach being proposed here to support multiple type translations. Is there a better approach you can think of ?

{
return new JdbcExpression(infixOperation("-", left, right), forwardBindVariables(left, right));
}

@ScalarOperator(EQUAL)
@SqlType(StandardTypes.BOOLEAN)
public static JdbcExpression equal(@SqlType(StandardTypes.BIGINT) JdbcExpression left, @SqlType(StandardTypes.BIGINT) JdbcExpression right)
@SupportedSignatures(@SqlSignature(argumentType = StandardTypes.BIGINT, returnType = StandardTypes.BOOLEAN))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since most JDBC support EQUALS (=), NOT_EQUALS (<>) , comparison functions (greater than, less than) out-of-the-box, lets add support for these standard type variants - bigint, integer, smallint, tinyint, boolean, date, decimal, real, double, varchar, timestamp

TypeSignature typeSignature = parseTypeSignature(type.value(), ImmutableSet.of());
argumentTypes.add(typeSignature);
}
SupportedSignatures supportedSignatures = method.getAnnotation(SupportedSignatures.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just saw this if block, I think migrating the Clickhouse OperatorTranslators right away makes sense to avoid the need for this if block

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🆕 Unprioritized
Development

Successfully merging this pull request may close these issues.

2 participants