Release v0.13.0-rc.2

Fixes

• Add clear error messages for pipelines for console in, pipeline out, pipeline in
• Added a new port in ConnectInput and weaved them together
• Added a new port in ConnectOutput and weaved them together
• Fixed linking pipelines to pipelines
• Fixed error reporting in ConnectOutput and ConnectInput functions
• Fixed kv test cases

Release v0.13.0-rc.1

Breaking Changes

• cb connector config path changed to paths.
• Introduce breaking change in kafka_consumer configuration, by adding mode.
• Update grok to 0.2, optional patterns are now omitted no longer ""
• Unify to url instead of endpoint for google and s3 connectors so they're in line with every other connector
• Unify to urls for elastic connector
• Unify to path for bench, kv and wal connector so it's in line with file
• Rename s3_writer to s3_streamer

New features

• Add gcl_writer Google Cloud Platform Cloud Logging connector for writing log entries
• Add reverse functions and tests for arrays and strings, add sort test for arrays
• Add a ClickHouse connector
• Add a GCS streamer connector

Fixes

• Fix bug in the runtime, swallowing some event acknowledgements and fails for batched events from multiple streams
• Fix custom metrics for kafka_consumer, kafka_producer not being reported in the requested interval
• Fix metrics connector not emitting all metrics it did receive.
• Fix invalid scope handling for flow definition default arguments
• Fix elastic sink not acking events handled successfully if no source is connected.
• Fix kafka_consumer possibly committing earlier offsets, thus replaying events that have already been handled.
• Fix off-by-one error in kafka_consumer committing offsets, thus replaying the last committed event.
• Allow kafka_consumer connector to reconnect upon more error conditions and avoid stalls.
• Include flow alias in pipeline and connector aliases reported via metrics events and logging in order to deduplicate entries

Release v0.12.4

Fixes

• Fix startup startup script for systemd installations
• Avoid hangs in elastic connector when not connected as source via its out or err port.
• Default udp client bind ip to 0.0.0.0:0 instead of 127.0.0.1:0

New features

• Add compression level for lz4, xz2 and zstd

Release v0.12.3

New features

• Added the gpubsub_publisher connector
• Added the gpubsub_consumer connector
• Added new metadata options to elastic connector: version, version_type, retry_on_conflict, if_primary_term, if_seq_no

Fixes

• Don't fail unrecoverably when the connection to the server is lost in Google PubSub consumer.
• Update bert operators to work with tremor 0.12
• Fix race condition leading to quiescence timeout when shutting down Tremor
• Fix timeout config unit mismatch in qos::backpressure and qos::percentile operators. Changed to nanoseconds precision.
• Update WAL operator to fix issue with overwriting first entries.

Release v0.12.1

Fixes

• Fix misbehaving stdio connector when stdio is a pipe
• Fix Docker entrypoint script argument ordering
• Fix id encoding for discord connector

Release v0.12.0

New features

• Add full TLS support for elastic connector (including client certificate auth)
• Add several auth methods for elastic and HTTP connector (Basic Auth, Api-Key, Bearer Token, ...)
• Add support for specifying client certificates with tls config for http_client and tcp_client connectors.
• Add tremor new to create new template projects.
• Support elastic.raw_payload for update
• Add support for chunked requests and responses for the HTTP connectors
• Add the gbq connector for Google BigQuery
• Add support for tuple patterns inside of record patterns.
• Refactor visitors to seperate walker and visitor and visit all nodes.
• Add support for modular subqueries in Trickle
• Add check_topic_metadata configuration flag to kafka source to bypass topic metadata fetch
• port json! improvements to literal!
• print names of failing tests
• Add check_topic_metadata configuration flag to kafka source to bypass topic metadata fetch
• port json! improvements to literal!
• Add support for troy deployment language. The language adds the pipeline, connector, flow, links, connect, to, deploy and config reserved keywords. These are now reserved in the scripting and query languages and must now be escaped when they appear in event data. This is a breaking change in the query and script language dialects.
• Add support for FLOAT4 and FLOAT8 serialization/deserialization to postgres connectors
• Add std::path::try_default fn
• Experimental ARM support
• Add default => {...} and default "key" => "value" to patch
• Add zstd pre- and post-processors #1100
• Remove rental from Event #1031 #1037
• Put event raw payload into Arc to improve cloning perf
• Remove rental from the entire runtime #1037
• Restructure operators and select to avoid transmutation #1024
• Fix tremor-cli to use it's own binary when possible #1096
• Restructure functions to avoid transmutation #1028
• Update blackhole to print events / s #1129
• Improve soundness and documentation of SRS code.
• Add support for concatenating arrays #1113
• Allow gcp headers to be included in rest offramp with auth: gcp
• Add env source #1136
• Remove need for emit_emtpy_window by automatic eviction for groups where no window holds data
• Ensure all windows of a group are alligned and can not go out of sync
• Box apropriately in the rest sink
• replace macros with functions in gcp code
• Add -q flag and clarify -v flag for unit tests
• Add win::cardinality function
• Backpressure now allows lossless (no discard) circuit breaker behaviour #1119
• Allow functions, constants and expressions as roots for path lookups
• Add the unix-socket source
• Enabale automatic benchmakrs
• Add the csv codec
• Add the permissions option for setting the file mode for the unix-socket source
• Tests can be run without their suite. #1238
• Add the std::size module to convert sizes
• Add custom function calls to constant folding
• Integration test names are added as a tag so they can be run by name
• integration tests low log stdout/stderr for before and after

Breaking Changes

• the tremor api sub command in the cli has been removed.
• the - is no longer a valid part of identifiers.
• binaries now use _ to sepoerate type names as - is no longer a identifier.
• changed naming for record object to avoid keywords like select and merge. New names are record.extract and record.combine.
• command separators are now unified, both patch, match and for now use ; the same way the rest of the language does
• in all definitional statements args now specifies interface arguments that are overwritable in the correspanding create statement, while with specifies non-overwritable configuration in both define and create statements - this unifies the use of with and args between trickle and troy
• file connector no longer splits by lines - it now requires a preconnector
• define for both troy and trickle now follow the same principle of define <type> <alias> from <source>
• wal is no longer an operator but a connector
• for the elastic connector indexes have not to be set on the batch not the individual event so one batch can only be to a single index.
• metronome interval is now in nanoseconds (as all other timings)
• Most connectors require a specified codec now instead of using JSON as a default
• merge no longer treats null in the spec as a delete option but rather as a normal value
• Combine all compression and decompression pre/postprocessors.

Fixes

• Fix detection of *.troy files in entrypoint.sh causing duplicate configs to be loaded when using Kubernetes.
• Fix a one-off error in the bench connector leading to it producing one event too much
• Avoid acking events that failed while preprocessing or decoding using a codec.
• Remove acknowledging events when dropped or sent to a dead end (e.g. unconnected port) as this was causing confusing and unwanted behaviour with error events
• move EXIT connector to debug connectors to avoid shutting down tremor
• Fix bug in grok extractor that would never return good matches
• Properly terminate after tremor run ...
• Fix the bench connector to actually stop after the given amount of events when iters is configured.
• http headers to allow strings and arrays
• fix the use of args in the with part of a create insode of a flow.
• fix silent swallowing of unknown fields in connector definition.
• Make otel severity_number optional: #1248
• Don't allow duplicate stream names: #1212
• Fix memory safety issue when using merge or patch with state as target and reassigning the resulting value to state #1217
• Fix delayed event re-execution in case of errors in a branched pipeline
• Skip instead of fail EQC on out of repo PRs
• Ensure patch keys are strings to move runtime errors into the compiletime
• Fix issue with the token " being presented as a tick in errors
• Fix heredoc_start and heredoc_end showing up in error messages instead of """
• Fix some errors in otel pb <-> json translation
• Fix windowed queries emitting events with null metadata on tick
• Fix sorting for artefacts
• Fix issue where the test framework would generate reports without being asked for it #1072
• Remove the need for eviction_period for time based windows
• Remove dead code and unneeded allows in otel and gcp code
• Fix test and suite names not being printed
• Fix badly nested structure in unit tests
• Fix bug in the unit testing framework that would ignore all tags
• Fix the illogical structure of suites that required a doubly nested record
• Fix -v flag
• Fix issue with double counting of unit test stats
• Fix issue with wrong script snippets being shown for unit tests
• Fix argument order in test cases
• Fix GCS go-auth token refresh
• Fix create script syntax for aliased scripts with overridden params
• Add benchmark names to benchmark tags
• Kafka onramp: Remove failing metadata fetch in order to verify topic existance. Instead detect subscription errors and stop the onramp in that case.
• Unix offramp: Add the missing StartStream message
• tremor-script: Add more details about Unicode in the documentation of the string module
• Fix hdr and dds aggregation function losing events when aggregating > 8192 events
• Ensure merge can only happen on objects

Release v0.12.0-rc.9

Fixes

• Fix detection of *.troy files in entrypoint.sh causing duplicate configs to be loaded when using Kubernetes.
• Fix a one-off error in the bench connector leading to it producing one event too much
• Avoid acking events that failed while preprocessing or decoding using a codec.
• Remove acknowledging events when dropped or sent to a dead end (e.g. unconnected port) as this was causing confusing and unwanted behaviour with error events
• move EXIT connector to debug connectors to avoid shutting down tremor
• Fix bug in grok extractor that would never return good matches

New features

• Add full TLS support for elastic connector (including client certificate auth)
• Add several auth methods for elastic and HTTP connector (Basic Auth, Api-Key, Bearer Token, ...)
• Add support for specifying client certificates with tls config for http_client and tcp_client connectors.
• Add tremor new to create new template projects.

Release v0.12.0-rc.8

Fixes

• Properly terminate after tremor run ...
• Fix the bench connector to actually stop after the given amount of events when iters is configured.

New features

• Support elastic.raw_payload for update

Release v0.12.0-rc.7

Fixes

• Properly terminate after tremor run ...
• Fix the bench connector to actually stop after the given amount of events when iters is configured.

New features

• Support elastic.raw_payload for update

Release v0.12.0-rc.6

Fixes

• http headers to allow strings and arrays