Skip to content

4.0.0
Compare
Choose a tag to compare
@github-actions github-actions released this 26 May 12:32
· 229 commits to master since this release
4.0.0
f880946

In this release, we are introducing our new Databricks Loader. Databricks Loader will be able to load Parquet transformed data therefore we've added wide row Parquet support to both Batch Transformer and Stream Transformer.

Also, we've included various improvements and bug fixes for Stream Transformer in order to get one step closer to making it production-ready.

Common

  • Change http4s client backend to async-http-client (#903)
  • Bump http4s to 0.21.33 (#902)

Loader

  • Add Databricks as a destination (#860)
  • Check if target is ready before submitting the statement (#846)
  • Emit latency statistics on constant intervals (#795)
  • Add load_tstamp (#815, #571)

Batch Transformer

  • Support Parquet output option (#896)

Transformer Kinesis

  • Support Parquet output option (#900)
  • Report metrics (#862)
  • Add telemetry (#863)
  • Write shredding_complete.json to S3 (#867)
  • Use output of transformation in updating global state (#824)
  • Fix updating total and bad number of events counter in global state (#823)
  • Add tests for whole processing pipeline (#835)
  • Fix passing checkpoint action during creation of windowed records (#762)
Assets 7
Loading