0.5.0

• Disable asynchronous deletes for Hudi (#94)
• Upgrade dependencies (#92)
• DynamoDB runtime dependencies for Delta S3 multi-writer
• PDP-1323 Handle initialization errors for GCP
• PDP-1324 - Handle initialization errors for lake loader Delta Azure
• Upgrade common-streams to 0.8.0-M5 (#83)
• Delete files asynchronously (#82)
• Upgrade common-streams 0.8.0-M4 (#81)
• Avoid error on duplicate view name (#80)
• Add option to exit on missing Iglu schemas (#79)
• common-streams 0.8.x with refactored health monitoring (#78)
• Create table concurrently with subscribing to stream of events (#77)
• Iceberg fail fast if missing permissions on the catalog (#76)
• Make alert messages more human-readable (#75)
• Hudi loader should fail early if missing permissions on Glue catalog (#72)
• Add alert & retry for delta/s3 initialization (#74)
• Implement alerting and retrying mechanisms
• Bump aws-hudi to 1.0.0-beta2 (#71)
• Bump hudi to 0.15.0 (#70)
• Allow disregarding Iglu field's nullability when creating output columns (#66)
• Extend health probe to report unhealthy on more error scenarios (#69)
• Fix bad rows resizing (#68)

Full Changelog: 0.4.1...0.5.0

Version 0.4.1

• Fixes missing s3 dependency for Delta (#64)

Version 0.4.0

• Improve default Iceberg table properties (#62)
• Bad rows must not exceed the maximum size allowed by the sink (#62)
• Trap SIGTERM and start graceful shutdown (#62)
• Improve default Hudi configuration settings (#62)
• Enable syncing Hudi metadata to the Glue Catalog (#62)
• Turn on metrics logging for Iceberg (#62)
• Improve default Delta table properties (#62)
• Bump common-streams and iceberg to latest versions (#62)
• Fix AWS dependencies (#61)
• Optimizations for larger compute instances (#60)
• Bump spark and lake libraries to latest versions (#59)
• Bump common-streams to 0.6.0 (#58)

Version 0.3.0

• Add AWS Glue as an Iceberg catalog (#55)
• Bump common-streams to 0.5.0 (#56)
• Fix crashing on invalid view name (#54)
• Remove Snowflake as an Iceberg catalog (#53)
• Fix and simplify BigLake as Iceberg catalog (#52)
• Change default log level of Delta libraries (#50)
• Adjust default spark memory configuration (#51)
• Azure: authenticate with Event Hubs using OAuth2 (#49)

Version 0.2.0

• Fix regression: it is taking longer to commit a batch to the lake (#46)
• Use SLF4J for Cats Effect starvation warning message (#45)
• Custom AWS credentials provider should fall back to default credentials chain (#42)
• Bump dependencies to latest versions for the 0.2.0 release (#41)
• Seperate docker builds for minority lake formats (#40)
• Integration tests for Hudi and Iceberg output formats (#38)
• Parse events from bytes instead of string (#44)
• Custom AWS credentials provider with support for external ID (#35)
• Health probe should not be healthy until after table is created (#34)
• Set user-agent headers when using GCP SDKs (#25)
• Add Hudi as a table format (#32)
• Refactor table writers so we can extend to more table formats (#31)
• AWS variant of the Lake Loader (#20)
• Bump common-streams to 0.2.0 (#30)
• Add latency statsd metrics (#23)
• Import common loaders lib from maven (#18)
• Downgrade to scala 2.12 (#21)
• Update the license to Snowplow Limited Use License Agreement 1.0 (#43)

Version 0.1.3

Changelog

• Improvements to example config files (#14)
• Use TIMESTAMP_MICROS as parquet timestamp type (#13)
• Bump scala analytics sdk to 3.1.0 (#12)
• Remove delayThreshold as a config option of pubsub sink (#11)
• Fix GCP app hanging when event size exceeds buffer size (#10)

Version 0.1.2

Changelog

• Delta: Support data skipping on tstamp columns (#9)
• Retry writing delta after concurrent modification exceptions (#7)

Version 0.1.1

Changelog

• Add license file to packaged jar (#5)
• Fix loading when a schema needs recovery columns (#4)

Version 0.1.0

Initial 0.1.0 release