-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Recursive CTEs: Stage 2 - add support for sql -> logical plan generation #8839
Recursive CTEs: Stage 2 - add support for sql -> logical plan generation #8839
Conversation
98f77ff
to
531d541
Compare
531d541
to
12d401a
Compare
a3b6463
to
22661af
Compare
update docs from script update slt test for doc change
* impl cte as work table * move SharedState to continuance * impl WorkTableState wip: readying pr to implement only logical plan fix sql integration test wip: add sql test for logical plan wip: format test assertion
some docs more docs
22661af
to
8d38c92
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @matthewgapp -- I think this PR is basically ready to go. I took the liberty of merging up from main to resolve a conflict and some more comments in 675a6df
I also have a few suggestions about error messages and naming but I don't think any of them are required on this PR (we could potentially do them (or not) as a follow on PR)
format!("{plan:?}"), | ||
"Projection: numbers.n\ | ||
\n SubqueryAlias: numbers\ | ||
\n RecursiveQuery: is_distinct=false\ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
pub recursive_term: Arc<LogicalPlan>, | ||
/// Should the output of the recursive term be deduplicated (`UNION`) or | ||
/// not (`UNION ALL`). | ||
pub is_distinct: bool, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder if using the term all
would be easier here as it more closely matches the CTE Syntax (UNON vs UNION ALL)
Something like
pub is_distinct: bool, | |
pub all: bool, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about dedupe
? Regardless, I would prefer if we made this naming change in a follow-on PR once we merge this and the execution PR.
Thanks for the review and for layering in those tweaks. I'll layer in changes based on your comments real quick and then should be good 2 go. 🚢 |
FYI @jonahgao |
Onwards! |
This PR implements sql -> logical plan generation support for Recursive CTEs. Builds on top of #8828.
The logical plan assumes that execution is performed iteratively, using a worktable to write to and read from the intermediate results during the recursive CTE's execution.
Todos
Which issue does this PR close?
Part of closing #462.
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?