Drill 2.0 Proposal

Drill 2.0 Proposal

This page serves to document a proposal for Drill 2.0. At the time of writing, we are currently gearing up to release Drill 1.20.0 which means that we have released 19 versions of Drill since it was deemed stable enough to warrant a 1.0 label. Since the second phase of this project's life began there have been some things which have been discussed which are breaking changes. This page serves to document proposed breaking changes that could be included in a Drill 2.0. Please feel free to add your ideas in the knowledge that a subset will be ultimately be selected by the dev team as the basis for a 2.0 release.

APIs and connectors

Config system

• Add a shared component that applies configuration priorities (... session opt > storage/format config opt > system opt ...) and make all plugins use this component for reading options.

Query planner

• Rebase on current Calcite and review our customisations.

Packaging and distribution

• Split Drill installation packages into "core" and "extra".
• Install plugins and UDFs from an online marketplace, a la the Eclipse marketplace.

Storage and format plugins

• Scrap the columns[] array wherever it occurs (only TextReader?) in favour of distinct, numbered fields column1, column2, column3, ...
• Increase consistency of available options across plugins, e.g. column name and type information only allowed in provided schema not format config.

Vector engine

• Do something about ObjectHolder.

Web UI