Druid 31.0.0 release notes

docs/release-info/upgrade-notes.md

@@ -38,6 +38,66 @@ For more information, see [Migration guide: front-coded dictionaries](./migr-fro
 
 If you're already using this feature, you don't need to take any action. 
 
+## 31.0.0
+
+### Upgrade notes
+
+#### Array ingest mode now defaults to array
+
+MSQ ingestion context flag `arrayIngestMode` now defaults to `array` instead of `mvd`. This means that SQL `VARCHAR ARRAY` types is no longer implicitly translated and stored in `VARCHAR` columns, but is instead stored as `VARCHAR ARRAY`. This change permits other array types such as `BIGINT ARRAY` and `DOUBLE ARRAY` to be inserted with MSQ into their respective array column types instead of failing as they do in `mvd` mode.
+
+To continue to store multi-value strings, modify any insert/replace queries to wrap the array types with the `ARRAY_TO_MV` operator.
+
+Validation is in place to prevent mixing `VARCHAR` and `VARCHAR ARRAY` columns in the same table, so any ingestions affected by this change will fail and provide a descriptive error message instead of exhibiting unexpected behavior.
+
+The `arrayIngestMode` option of `none` has been removed. It was introduced prior to the table validation logic as a means for cluster operators to force query writers to explicitly set `array` or `mvd` on their query contexts, but provides little utility in Druid 31.
+
+See the following topics for more information:
+* [Ingest multi-value dimensions](../querying/multi-value-dimensions.md#sql-based-ingestion) for how to ingest multi-value strings.
+* [Ingest arrays](../querying/arrays.md#sql-based-ingestion) for ingesting arrays.
+
+[#16789](https://github.com/apache/druid/pull/16789)
+
+#### Removed task action audit logging
+
+The deprecated task action audit logging has been removed. This change includes the following updates:
+
+- The endpoint `/indexer/v1/task/{taskId}/segments` is no longer supported.
+- Druid doesn't write to or read from the metadata table `druid_taskLog`.
+- Druid ignores the property `druid.indexer.auditlog.enabled`.
+- Druid doesn't emit the metric `task/action/log/time`.
+
+These changes are backward compatible with all existing metadata storage extensions.
+
+[#16309](https://github.com/apache/druid/pull/16309)
+
+#### Removed Firehose and FirehoseFactory
+
+Removed Firehose and FirehoseFactory and remaining implementations.
+Apache deprecated support for Druid firehoses in version 0.17. Support for firehose ingestion was removed in version 26.0.
+
+[#16758](https://github.com/apache/druid/pull/16758)
+
+### Incompatible changes
+
+#### Removed the scan query legacy mode
+
+The native scan query legacy mode has been removed. It was introduced in Druid 0.11 to maintain compatibility during an upgrade from older versions of Druid where the scan query was part of a `contrib` extension.
+
+[#16659](https://github.com/apache/druid/pull/16659)
+
+Hard-coded `"legacy":false` following removal of the legacy mode to prevent error during rolling upgrades or downgrades.
+
+[#16793](https://github.com/apache/druid/pull/16793)
+
+#### ZK-based segment loading
+
+ZK-based segment loading is now disabled. ZK `servedSegmentsPath` was deprecated in Druid 0.7.1. This legacy path has been replaced by `liveSegmentsPath`.
+
+Segment-serving processes such as Peons, Historicals and Indexers no longer create ZK `loadQueuePath` entries. The `druid.zk.paths.loadQueuePath` and `druid.zk.paths.servedSegmentsPath` properties are no longer used.
+
+Move to HTTP-based segment loading first and then perform the version upgrade.
+
 ## 30.0.0
 
 ### Upgrade notes

website/.spelling

@@ -233,6 +233,7 @@ Splunk
 SqlParameter
 SslContextFactory
 StatsD
+stateful
 SYSTEM_TABLE
 TaskRunner
 TabItem
@@ -254,6 +255,7 @@ UTF8
 XMLs
 ZK
 ZSTD
+Axios
 accessor
 ad-hoc
 aggregator
@@ -323,6 +325,7 @@ deserialization
 deserialize
 deserialized
 deserializes
+deserializing
 downtimes
 druid
 druid–kubernetes-extensions
@@ -345,6 +348,7 @@ filterValue
 firefox
 firehose
 firehoses
+FirehoseFactory
 fromPigAvroStorage
 frontends
 granularities
@@ -535,8 +539,10 @@ stdout
 storages
 stringDictionaryEncoding
 stringified
+structs
 sub-conditions
 subarray
+submenu
 subnet
 subqueries
 subquery
@@ -636,6 +642,7 @@ IS_NULLABLE
 JDBC_TYPE
 MIDDLE_MANAGER
 MILLIS_TO_TIMESTAMP
+NPE
 NULLable
 NUMERIC_PRECISION
 NUMERIC_PRECISION_RADIX
@@ -711,6 +718,7 @@ total_size
 useApproximateCountDistinct
 useGroupingSetForExactDistinct
 useApproximateTopN
+UX
 wikipedia
 your-table
 enableTimeBoundaryPlanning
@@ -1090,6 +1098,7 @@ reverseLoadingCacheSpec
 OAuth
 Okta
 OpenID
+OpenRewrite
 pac4j
 Env
 POD_NAME
@@ -1222,6 +1231,8 @@ chunk2
 stddev
 t1
 t2
+v4
+v5
 variance1
 variance2
 varianceFold
@@ -1631,6 +1642,7 @@ un
 5MB
 64KiB
 8GiB
+GBs
 G1GC
 GroupBys
 QoS-type
@@ -2203,6 +2215,7 @@ PiB
 protobuf
 Golang
 multiValueHandling
+multivalue
 _n_
 KLL
 KllFloatsSketch