Skip to content

v0.1.1
Compare
Choose a tag to compare
@thtmnisamnstr thtmnisamnstr released this 07 Mar 05:00
· 66 commits to main since this release
v0.1.1
1d5cf01
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

V0.1.1 - Current status -- what works and what doesn't

What works? 🚀

rtdl's initial feature set is built and working. You can use the API on port 80 to
configure streams that ingest json from an rtdl endpoint on port 8080, process them into Parquet,
and save the files to a destination configured in your stream. rtdl can write files locally, to
AWS S3, GCP Cloud Storage, and Azure Blob Storage and you can query your data via Dremio's web UI
at http://localhost:9047 (login with Username: rtdl and Password rtdl1234).

What's new? 💥

  • Replaced Kafka & Zookeeper with Redpanda.
  • Added support for HDFS.
  • Fixed issue with handling booleans when writing Parquet.
  • Added several logo variants and a banner to the public directory.

What doesn't work/what's next on the roadmap? 🚴🏼

  • Dremio Cloud support.
  • Apache Hudi support.
  • Start using GitHub Projects for work tracking.
  • Research and implementation for Apache Iceberg, Delta Lake, and Project Nessie.
  • Community contribution: Stateful Function for PII detection and masking.
  • Graphical user interface.
Assets 2
Loading
21 Join discussion