Query planner passes #588
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| package org.partiql.lang.planner | ||
| import org.partiql.lang.errors.Problem | ||
|
|
||
| sealed class PassResult<TResult> { | ||
| /** | ||
| * Indicates query planning was successful and includes a list of any warnings that were encountered along the way. | ||
| */ | ||
| data class Success<TResult>(val result: TResult, val warnings: List<Problem>) : PassResult<TResult>() | ||
|
|
||
| /** | ||
| * Indicates query planning was not successful and includes a list of errors and warnings that were encountered | ||
|
There was a problem hiding this comment.
There was a problem hiding this comment. This is designed to allow us to continue at least the current pass in the event of a semantic error instead of throwing an exception. That means its entirely possible to encounter both warnings and errors. I'll add a note. |
||
| * along the way. Encountering both errors and warnings, as well as multiple errors is possible since we are not | ||
| * required to stop when encountering the first error. | ||
| */ | ||
| data class Error<TResult>(val errors: List<Problem>) : PassResult<TResult>() | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| package org.partiql.lang.planner | ||
|
|
||
| import org.partiql.lang.eval.BindingCase | ||
| import org.partiql.lang.eval.BindingName | ||
|
|
||
| /** Indicates the result of an attempt to resolve a global variable to its customer supplied unique identifier. */ | ||
| sealed class ResolutionResult { | ||
| /** | ||
| * A success case, indicates the [uniqueId] of the match to the [BindingName] in the global scope. | ||
| * Typically, this is defined by the storage layer. | ||
| */ | ||
| data class GlobalVariable(val uniqueId: String) : ResolutionResult() | ||
|
|
||
| /** | ||
| * A success case, indicates the [index] of the only possible match to the [BindingName] in a local lexical scope. | ||
| * This is `internal` because [index] is an implementation detail that shouldn't be accessible outside of this | ||
| * library. | ||
| */ | ||
| internal data class LocalVariable(val index: Int) : ResolutionResult() | ||
|
|
||
| /** A failure case, indicates that resolution did not match any variable. */ | ||
| object Undefined : ResolutionResult() | ||
| } | ||
|
|
||
| fun interface UniqueIdResolver { | ||
| /** | ||
| * Implementations try to resolve a global variable which is typically a database table to a unique identifier | ||
| * using [bindingName]. [bindingName] includes both the name as specified by the query author and a [BindingCase] | ||
| * which indicates if query author included double quotes (") which mean the lookup should be case-sensitive. | ||
| * | ||
| * Implementations of this function must return: | ||
| * | ||
| * - [ResolutionResult.GlobalVariable] if [bindingName] matches a global variable (typically a database table). | ||
| * - [ResolutionResult.Undefined] if no identifier matches [bindingName]. | ||
| * | ||
| * When determining if a variable name matches a global variable, it is important to consider if the comparison | ||
| * should be case-sensitive or case-insensitive. @see [BindingName.bindingCase]. In the event that more than one | ||
| * variable matches a case-insensitive [BindingName], the implementation must still select one of them | ||
| * without providing an error. (This is consistent with Postres's behavior in this scenario.) | ||
| * | ||
| * Note that while [ResolutionResult.LocalVariable] exists, it is intentionally marked `internal` and cannot | ||
| * be used by outside this project. | ||
| */ | ||
| fun resolve(bindingName: BindingName): ResolutionResult | ||
| } | ||
|
|
||
| private val EMPTY = UniqueIdResolver { ResolutionResult.Undefined } | ||
|
|
||
| /** Convenience function for obtaining an instance of [UniqueIdResolver] with no defined variables. */ | ||
| fun emptyUniqueIdResolver(): UniqueIdResolver = EMPTY |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| package org.partiql.lang.planner.transforms | ||
|
|
||
| import org.partiql.lang.domains.PartiqlAst | ||
| import org.partiql.lang.eval.visitors.FromSourceAliasVisitorTransform | ||
| import org.partiql.lang.eval.visitors.PipelinedVisitorTransform | ||
| import org.partiql.lang.eval.visitors.SelectListItemAliasVisitorTransform | ||
| import org.partiql.lang.eval.visitors.SelectStarVisitorTransform | ||
|
|
||
| /** | ||
| * Executes the [SelectListItemAliasVisitorTransform], [FromSourceAliasVisitorTransform] and | ||
| * [SelectStarVisitorTransform] passes on the receiver. | ||
| */ | ||
| fun PartiqlAst.Statement.normalize(): PartiqlAst.Statement { | ||
| // Since these passes all work on PartiqlAst, we can use a PipelinedVisitorTransform which executes each | ||
| // specified VisitorTransform in sequence. | ||
| val transforms = PipelinedVisitorTransform( | ||
| // Synthesizes unspecified `SELECT <expr> AS ...` aliases | ||
| SelectListItemAliasVisitorTransform(), | ||
| // Synthesizes unspecified `FROM <expr> AS ...` aliases | ||
| FromSourceAliasVisitorTransform(), | ||
| // Changes `SELECT * FROM a, b` to SELECT a.*, b.* FROM a, b` | ||
| SelectStarVisitorTransform() | ||
| ) | ||
| return transforms.transformStatement(this) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| package org.partiql.lang.planner.transforms | ||
|
|
||
| import org.partiql.lang.domains.PartiqlAst | ||
| import org.partiql.lang.domains.PartiqlAstToPartiqlLogicalVisitorTransform | ||
| import org.partiql.lang.domains.PartiqlLogical | ||
|
|
||
| /** | ||
| * Transforms an instance of [PartiqlAst.Statement] to [PartiqlLogical.Statement]. | ||
| * | ||
| * Performs no semantic checks. | ||
| * | ||
| * This conversion (and the logical algebra) are early in their lifecycle and so only a very limited subset of | ||
| * SFW queries are transformable. See tests for this class to see which queries are transformable. | ||
| */ | ||
| internal fun PartiqlAst.Statement.toLogicalPlan(): PartiqlLogical.Plan = | ||
| PartiqlLogical.build { | ||
| plan( | ||
| AstToLogicalVisitorTransform.transformStatement(this@toLogicalPlan), | ||
| version = PLAN_VERSION_NUMBER | ||
| ) | ||
| } | ||
|
|
||
| private object AstToLogicalVisitorTransform : PartiqlAstToPartiqlLogicalVisitorTransform() { | ||
|
|
||
| override fun transformExprSelect(node: PartiqlAst.Expr.Select): PartiqlLogical.Expr { | ||
| checkForUnsupportedSelectClauses(node) | ||
|
|
||
| var algebra: PartiqlLogical.Bexpr = FromSourceToBexpr.convert(node.from) | ||
|
|
||
| algebra = node.fromLet?.let { fromLet -> | ||
| PartiqlLogical.build { | ||
| let(algebra, fromLet.letBindings.map { transformLetBinding(it) }, node.fromLet.metas) | ||
| } | ||
| } ?: algebra | ||
|
|
||
| algebra = node.where?.let { | ||
| PartiqlLogical.build { filter(transformExpr(it), algebra, it.metas) } | ||
| } ?: algebra | ||
|
|
||
| algebra = node.offset?.let { | ||
| PartiqlLogical.build { offset(transformExpr(it), algebra, node.offset.metas) } | ||
| } ?: algebra | ||
|
|
||
| algebra = node.limit?.let { | ||
| PartiqlLogical.build { limit(transformExpr(it), algebra, node.limit.metas) } | ||
| } ?: algebra | ||
|
|
||
| return convertProjectionToBindingsToValues(node, algebra) | ||
| } | ||
|
|
||
| private fun convertProjectionToBindingsToValues(node: PartiqlAst.Expr.Select, algebra: PartiqlLogical.Bexpr) = | ||
| PartiqlLogical.build { | ||
| bindingsToValues( | ||
| when (val project = node.project) { | ||
| is PartiqlAst.Projection.ProjectValue -> transformExpr(project.value) | ||
| is PartiqlAst.Projection.ProjectList -> { | ||
| struct( | ||
| List(project.projectItems.size) { idx -> | ||
| when (val projectItem = project.projectItems[idx]) { | ||
| is PartiqlAst.ProjectItem.ProjectExpr -> | ||
| structField( | ||
| lit( | ||
| projectItem.asAlias?.toIonElement() | ||
| ?: errAstNotNormalized("SELECT-list item alias not specified") | ||
| ), | ||
| transformExpr(projectItem.expr), | ||
| ) | ||
| is PartiqlAst.ProjectItem.ProjectAll -> { | ||
| structFields(transformExpr(projectItem.expr), projectItem.metas) | ||
| } | ||
| } | ||
| } | ||
| ) | ||
| } | ||
| is PartiqlAst.Projection.ProjectStar -> | ||
| // `SELECT * FROM bar AS b` is rewritten to `SELECT b.* FROM bar as b` by | ||
| // [SelectStarVisitorTransform]. Therefore, there is no need to support `SELECT *` here. | ||
| errAstNotNormalized("Expected SELECT * to be removed") | ||
|
|
||
| is PartiqlAst.Projection.ProjectPivot -> TODO("PIVOT ...") | ||
| }, | ||
| algebra, | ||
| node.project.metas | ||
| ) | ||
| }.let { q -> | ||
| // in case of SELECT DISTINCT, wrap bindingsToValues in call to filter_distinct | ||
| when (node.setq) { | ||
| null, is PartiqlAst.SetQuantifier.All -> q | ||
| is PartiqlAst.SetQuantifier.Distinct -> PartiqlLogical.build { call("filter_distinct", q) } | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Throws [NotImplementedError] if any `SELECT` clauses were used that are not mappable to [PartiqlLogical]. | ||
| * | ||
| * This function is temporary and will be removed when all the clauses of the `SELECT` expression are mappable | ||
| * to [PartiqlLogical]. | ||
| */ | ||
| private fun checkForUnsupportedSelectClauses(node: PartiqlAst.Expr.Select) { | ||
| when { | ||
| node.group != null -> TODO("Support for GROUP BY") | ||
| node.order != null -> TODO("Support for ORDER BY") | ||
| node.having != null -> TODO("Support for HAVING") | ||
| } | ||
| } | ||
|
|
||
| override fun transformLetBinding(node: PartiqlAst.LetBinding): PartiqlLogical.LetBinding = | ||
| PartiqlLogical.build { | ||
| letBinding( | ||
| transformExpr(node.expr), | ||
| varDecl_(node.name, node.name.metas), | ||
| node.metas | ||
| ) | ||
| } | ||
|
|
||
| override fun transformStatementDml(node: PartiqlAst.Statement.Dml): PartiqlLogical.Statement { | ||
| TODO("Support for DML") | ||
| } | ||
|
|
||
| override fun transformStatementDdl(node: PartiqlAst.Statement.Ddl): PartiqlLogical.Statement { | ||
| TODO("Support for DDL") | ||
| } | ||
lziq marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| override fun transformExprStruct(node: PartiqlAst.Expr.Struct): PartiqlLogical.Expr = | ||
| PartiqlLogical.build { | ||
| struct( | ||
| node.fields.map { | ||
| structField( | ||
| transformExpr(it.first), | ||
| transformExpr(it.second) | ||
| ) | ||
| }, | ||
| metas = node.metas | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| private object FromSourceToBexpr : PartiqlAst.FromSource.Converter<PartiqlLogical.Bexpr> { | ||
|
|
||
| override fun convertScan(node: PartiqlAst.FromSource.Scan): PartiqlLogical.Bexpr { | ||
| val asAlias = node.asAlias ?: errAstNotNormalized("Expected as alias to be non-null") | ||
| return PartiqlLogical.build { | ||
| scan( | ||
| AstToLogicalVisitorTransform.transformExpr(node.expr), | ||
| varDecl_(asAlias, asAlias.metas), | ||
| node.atAlias?.let { varDecl_(it, it.metas) }, | ||
| node.byAlias?.let { varDecl_(it, it.metas) }, | ||
| node.metas | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| override fun convertUnpivot(node: PartiqlAst.FromSource.Unpivot): PartiqlLogical.Bexpr { | ||
| TODO("Support for UNPIVOT") | ||
| } | ||
|
|
||
| override fun convertJoin(node: PartiqlAst.FromSource.Join): PartiqlLogical.Bexpr = | ||
| PartiqlLogical.build { | ||
| join( | ||
| joinType = AstToLogicalVisitorTransform.transformJoinType(node.type), | ||
| left = convert(node.left), | ||
| right = convert(node.right), | ||
| predicate = node.predicate?.let { AstToLogicalVisitorTransform.transformExpr(it) }, | ||
| node.metas | ||
| ) | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While at it, shouldn't we just release the PIG and remove this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but partiql/partiql-ir-generator#41 hasn't been fixed yet.