Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support trailing comments in passed SQL queries #580

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

sfc-gh-hachouraria
Copy link

@sfc-gh-hachouraria sfc-gh-hachouraria commented Aug 19, 2024

To support trailing comments, particularly the inline types with -- or //, the query wrapping performed
in the connector needs to ensure the query is placed into a new line of its own

Otherwise, passing such an inline comment will break the Snowflake SQL syntax.

Attempting any of the following, for example, fails today:

spark.sql("select * from table -- trailing comment").collect()
spark.read.[…].option("query", "select * from table -- trailing comment").load()

The failure occurs due to Spark wrapping around the query and executing a resulting invalid syntax:

select col from (select * from table -- trailing comment)

The change made here works by adjusting such wrapping to always, safely place provided statements into their own line:

select col from (
select * from table -- trailing comment
)

This allows for a valid syntax to be used despite users accidentally placing trailing comments at the end.

The tests added fail without the fixes added (tested via sbt test it:test).

To support trailing comments, particularly the inline
types with -- or //, the query wrapping performed
in the connector needs to ensure the query is placed
into a new line of its own

Otherwise, passing such an inline comment will break
the Snowflake SQL syntax:

  select * from table -- trailing comment
@sfc-gh-hachouraria sfc-gh-hachouraria changed the title Wrap passed queries with newlines Support trailing comments in passed SQL queries Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant