v0.8.2-alpha

Description

This is a minor version release, v0.8.2-alpha.

Added

• Adds simple auto-completion to the CLI.

Changed

• Increases the performance of the PartiQLParser by changing the parsing strategy
  • The PartiQLParser now attempts to parse queries using the SLL Prediction Mode set by ANTLR
  • If unable to parse via SLL Prediction Mode, it attempts to parse using the slower LL Prediction Mode
  • Modifications have also been made to the ANTLR grammar to increase the speed of parsing joined table references
  • Updates how the PartiQLParser handles parameter indexes to remove the double-pass while lexing

List of commits: v0.8.1-alpha...v0.8.2-alpha.

v0.8.1-alpha

Description

This is a minor version release, v0.8.1-alpha.

Added

• Extends statement redaction to support INSERT/REPLACE/UPSERT INTO.

List of commits: v0.8.0-alpha...v0.8.1-alpha

v0.8.0-alpha

Description

This is a major version release for v0.8.0-alpha.

Breaking Changes

• Replaced union, intersect, except IR nodes with common bag_op IR node
• Renamed PassResult to PlannerPassResult for clarity. (This is part of the experimental query planner API.)
• Removed [ExprValueType.typeNames] as needed by the future work of legacy parser removal and OTS

Added

• CHANGELOG.md with back-filling of the previous releases to the change log to provide more visibility on unreleased
  changes and make the release process easier by using the unreleased section of change log. The CONTRIBUTING.md
  has also been updated to ensure this is part of the process.
• backward-incompatiblity and dependency questions are added to the project's PR process to provide more context
  on the changes that include these and the alternatives that have been considered.
• README.md badges for GitHub Actions status, codecov, and license
• An experimental (pending #15) embedding of a subset of
  the GPML (Graph Pattern Matching Language) graph query, as a new expression
  form <expr> MATCH <gpml_pattern>, which can be used as a bag-of-structs data source in the FROM clause.
  The use within the grammar is based on the assumption of a new graph data type being added to the
  specification of data types within PartiQL, and should be considered experimental until the semantics of the graph
  data type are specified.
  • basic and abbreviated node and edge patterns (section 4.1 of the GPML paper)
  • concatenated path patterns (section 4.2 of the GPML paper)
  • path variables (section 4.2 of the GPML paper)
  • graph patterns (i.e., comma separated path patterns) (section 4.3 of the GPML paper)
  • parenthesized patterns (section 4.4 of the GPML paper)
  • path quantifiers (section 4.4 of the GPML paper)
  • restrictors and selector (section 5.1 of the GPML paper)
  • pre-filters and post-filters (section 5.2 of the GPML paper)
• Added EvaluatonSession.context: A string-keyed map of arbitrary values which provides a way to make
  session state such as current user and transaction details available to custom [ExprFunction] implementations
  and custom physical operator implementations.
• Replaces union, intersect, except IR nodes with common bag_op IR node
• Add support for CallAgg in Type Inferencer.
• A GitHub Workflow to automatically sync the docs directory with the GitHub Wiki
• Introduces the PartiQLParser, an implementation of Parser using ANTLR
  • Matches the functionality of the existing SqlParser
  • Now catches a StackOverflowError and throws a ParserException
  • Support for DQL, DDL, DML, GPML, and EXEC
  • Handles consistency and precedence issues seen in SqlParser
    • See GitHub Issues #709, #708,
      #707, #683,
      and #730
• Parsing of INSERT DML with ON CONFLICT DO REPLACE EXCLUDED based on RFC-0011
• Adds a subset of REPLACE INTO and UPSERT INTO parsing based on RFC-0030
  • Parsing of target attributes is not supported yet and is pending #841
• Logical plan representation and evaluation support for INSERT DML with ON CONFLICT DO REPLACE EXCLUDED and REPLACE INTO based on RFC-0011
• Logical plan representation of INSERT DML with ON CONFLICT DO UPDATE EXCLUDED and UPSERT INTO based on RFC-0011
• Enabled projection alias support for ORDER BY clause

Experimental Planner Additions

• Renamed PassResult to PlannerPassResult for clarity. (This is part of the experimental query planner API.)
• The PlannerPipeline API now has experimental and partial support for INSERT and DELETE DML statements—
  tracking PartiQL specification issues are partiql-docs/#4 (only
  a subset has been implemented--see examples below) and
  partiql-docs/#19.
  • Examples of supported statements include:
    • INSERT INTO foo << { 'id': 1, 'name': 'bob' }, { 'id': 2, 'name' : 'sue' } >> (multi record insert)
    • INSERT INTO foo SELECT c.id, c.name FROM customer AS c (insert the results of a query into another table)
    • DELETE FROM foo (delete all records in a table)
    • DELETE FROM foo AS f WHERE f.zipCode = '90210' (delete all records matching a predicate)
• Introduced planner event callbacks as a means to provide a facility that allows the query to be visualized at every
  stage in the PlannerPipeline and to generate performance metrics for the individual phases of query planning. See
  PlannerPipe.Builder.plannerEventCallback for details.
• Adds the following optimization passes, none of which are enabled by default:
  • FilterScanToKeyLookupPass which performs a simple optimization common to most databases: it converts a filter
    predicate covering a table's complete primary key into a single get-by-key operation, thereby avoiding a full table
    scan. This may pass leave behind some useless and expressions if more and operands exist in the filter predicate
    other than primary key field equality expressions.
  • RemoveUselessAndsPass, which removes any useless and expressions introduced by the previous pass or by the
    query author, e.g. true and x.id = 42 -> x.id = 42), true and true -> true, etc.
  • RemoveUselessFiltersPass, which removes useless filters introduced by the previous pass or by the query author
    (e.g. (filter (lit true) <bexpr>)) -> <bexpr>.
• Add support for UNPIVOT, the behavior is expected to be compatible with the evaluating compiler.
• Adds support for GROUP BY (aggregations, group keys, etc)
• Adds support for ORDER BY in Planner
• Adds support for PIVOT in the planner consistent with EvaluatingCompiler

Changed

• The default parser for all components of PartiQL is now the PartiQLParser -- see the deprecation of SqlParser
• Parsing of ORDER BY clauses will no longer populate the AST with defaults for the 'sort specification'
  (i.e., ASC or DESC) or 'nulls specification' (i.e., NULLS FIRST or NULLS LAST) when the are not provided in
  the query text. Defaulting of sort order is moved to the evaluator.

Deprecated

• Deprecates SqlLexer and SqlParser to be replaced with the PartiQLParserBuilder.
• Deprecates helper method, blacklist, within org.partiql.lang.eval and introduced a functionally equivalent
  org.partiql.lang.eval.denyList method.
• Deprecates TypedOpParameter.LEGACY to be replaced with TypedOpParameter.HONOR_PARAMETERS

Fixed

• Codecov report uploads in GitHub Actions workflow
• GitHub Actions capability to run on forks
• Negation overflow caused by minimum INT8
• Type mismatch error caused by evaluator's integer overflow check
• Cast function's behavior on positive_infinity, negative_infinity, and NaN explicitly defined and handled.
• Changed Trim Function Specification handling(fixed error message, and now can take case-insensitive trim spec)

Removed

• README.md badge for travisci
• Breaking Change: removed [ExprValueType.typeNames] as needed by the future work of legacy parser removal and OTS

List of commits: v0.7.0-alpha...v0.8.0-alpha

v.0.7.0-alpha

This is a major version release for v0.7.0-alpha; it is highlighted by addition of an experimental query planner API, new built-in functions, 2.3x times more performant LIKE operator, and various CLI and REPL improvements (see features more details).

Breaking changes are:

• Removal of Field EVALUATOR_SQL_EXCEPTION from ErrorCode class:
  A client program may be interrupted by NoSuchFieldError exception.
• Removal of NodeMetadata from org.partiql.lang.eval:
  A client program may be interrupted by NoClassDefFoundError exception.
• Removal of the following classes from org.partiql.lang.eval.like:
  • CodepointCheckpointIterator
  • PatternPart
  • PatternPart.AnyOneChar
  • PatternPart.ExactChars
  • PatternPartKt
    A client program may be interrupted by NoClassDefFoundError exception.

See the attached compat_report.html file for full report on compatibility between this and previous version.

• Removal of the following interface from org.partiql.lang.eval.like:
  A client program may be interrupted by NoClassDefFoundError exception.

Features

• An experimental query planner API along with logical and physical plans structures with the support of non-default physical operator implementations.
• An optional flag, --wrap-ion, to give users the old functionality of reading multiple Ion values (previous behavior).
• Benchmark framework and benchmark implementation for LIKE performance
• Convenience StaticType for TEXT and NUMERIC
• Enable MIN and MAX to work with all the data-types.
• Introduction of extensions and addition of the query_ddb function to allow querying AWS DynamodB from the CLI.
• Replacement of REPL with JLine shell
• Syntax highlighting for CLI
• Three additional CLI flags:
  • -r --projection-iter-behavior: Controls the behavior of ExprValue.iterator in the projection result: (default: FILTER_MISSING) [FILTER_MISSING, UNFILTERED]
  • -t --typed-op-behavior: indicates how CAST should behave: (default: HONOR_PARAMETERS) [LEGACY, HONOR_PARAMETERS]
  • -v --undefined-variable-behavior: Defines the behavior when a non-existent variable is referenced: (default: ERROR) [ERROR, MISSING]
• --input-format flag to the CLI
• CEIL and FLOOR functions
• DATE/TIME formatting and the support for DATE/TIME in Ion data format

Fixes

• Fix write_file CLI function; the old function required the input to be a string, but it must be a generic type.
• Add ktlint task dependency to enable execution optimizations and reducing he build time by ~ 30%.
• Adjust handling of Ion input (requiring single value)
• Adjust handling of Ion output (outputting the real value)
• Adds missing metas to ORDER BY ExprNode and PartiqlAst (E.g. source location), which limits error message reporting.

Misc

• LIKE matching via compilation to java.util.regex.Pattern
• Run ktlint before tests.

List of commits: v0.6.0-alpha...v0.7.0-alpha

v0.6.0-alpha

This is a major version release for v0.6.0-alpha. It is highlighted by the deprecation of ExprNode, permissive mode option in the CLI/REPL, ORDER BY implementation in the evaluator, and PIG upgrade to v0.5.0.

A breaking change was changing ExprFunction's usage of Environment to EvaluationSession along with some other classes containing implementation details made internal as part of #559.

Features

• Deprecate ExprNode (#535)
• [cli] Add permissive mode evaluation option to CLI/REPL (#545)
• For ExprFunction, replace Environment with EvaluationSession (#559)
• ORDER BY implementation in evaluator (#554)
• Migrate to PIG v0.5.0 (#563)

Fixes

• Changed Path AST node to use its root node source location (#527)

Misc

• CAST: Improve the CAST assertion assertEval (#523)
• Clean up ExprFunction test (#529)
• [build] Increase build performance w/ Gradle upgrade to 7.4 (#539)
• [build] Adds ktlint to gradle build (#542)
• [build] Bump Kotlin version to 1.4.32 from 1.4.0 (#548)
• Fix all compiler warnings (#562)
• [build] Upgrade dokka to 1.6.10, set org.gradle.jvmargs (#568)

List of commits: v0.5.0-alpha...v0.6.0-alpha

v0.5.0-alpha

This is a major version release for v0.5.0-alpha. It is highlighted by the addition of permissive evaluation mode, static type inferencer, dataguide, CLI improvements, a new ExprFunction API, and deprecating ExprNode in the parser.

Features

• Adds a static type inferencer for static query checks and query type inference
• Adds multiple exception logging and severity level API
• Adds the dataguide API which can be used to infer Ion schema from Ion data
  • Also adds mappers to and from PartiQL’s static type and ISL
• Refactor of PartiQL’s StaticType
• Refactors ExprFunction interface
• Adds evaluator option for PERMISSIVE mode
• Adds support for CAN_CAST and CAN_LOSSLESS_CAST
• Adds evalution-time function call (ExprFunction) argument type checks
• Adds integer8, int8, bigint, int2, and integer2 as type names
• Adds support for OFFSET (#451)
• [cli] Uses Apache's CSVParser for file reading (#474) and ability to read custom CSV configurations (#480)

Fixes

• Fixes evaluator behavior to error for structs with non-text keys
• Corrects the parser error for unexpected reserved keywords in a select list
• Fixes static initializing cycle with lazy initialization of SqlDataType
• Fixes unknown propagation for IN operator
• Fixes bug in precision check for NUMERIC
• Makes unknown simple CASE WHEN predicate the same as false
• Make unknown branch predicates the same as false for searched CASE WHEN
• Disallows duplicate projected fields in select list query
• Fixes EXTRACT ExprFunction to return a decimal instead of float
• Fixes EXISTS and DATE_DIFF function signatures
• Fixes GROUP BY for more than 2 nested path expressions (#461)
• [cli] Fixes CLI command bug when input data is empty (#478)
• [cli] Fixes CLI bug when outputting IONTEXT to file (#479)

Misc

• Upgrades Kotlin version to 1.4
• Modeled NULLIF and COALESCE as PartiqlAst nodes rather than ExprFunctions
• Started parameterization of evaluation tests
• Deprecate ExprNode in parser (#464)
• Removes wildcard imports in cli (#483) and lang (#488)
• Removes DateTimeType sealed class (#489)
• Renames DateTimePart type to DatePart (#506)

Changes not corresponding to a PR were part of #517.

List of commits: v0.4.0-alpha...v0.5.0-alpha

v0.4.0-alpha

Release v0.4.0-alpha. New major version to use new PIG major version v0.4.0, which may introduce some breaking changes related to imported builders.

Features

• Sets up JMH for PartiQL (#427)
• Allows for default timezone configuration (#449)

Fixes

• Fixes struct handling of non-text struct field keys (#450)

Misc

• Uses new PIG major version v0.4.0 (#454)
• Moves usage of default timezone from parser to evaluator (#448)

List of commits: v0.3.4-alpha...v0.4.0-alpha

v0.2.7-alpha

Release v0.2.7-alpha

Fixes

• Cherry picks "Fix bug causing multiple nested nots to parse very slowly (#436)" for v0.2.7 release #439
• Cherry picks "Use LazyThreadSafteyMode.PUBLICATION instead of NONE (#433)" for v0.2.7 release #440

List of commits: v0.2.6-alpha...v0.2.7-alpha

v0.1.7-alpha

Release v0.1.7-alpha

Fixes

• Cherry picks "Fix bug causing multiple nested nots to parse very slowly (#436)" for v0.1.7 release #441
• Cherry picks "Use LazyThreadSafteyMode.PUBLICATION instead of NONE (#433)" for v0.1.7 release #442

List of commits: v0.1.6-alpha...v0.1.7-alpha

v0.3.4-alpha

Release v0.3.4-alpha

Fixes

• Bug causing multiple nested nots to parse very slowly (#436)