-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* mv md to adoc * adoc format * fix typo in fn doc
- Loading branch information
Showing
3 changed files
with
74 additions
and
69 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
= Waterfall | ||
|
||
image:https://github.com/robertluo/waterfall/actions/workflows/main.yml/badge.svg[CI,link=https://github.com/robertluo/waterfall/actions/workflows/main.yml] | ||
image:https://img.shields.io/clojars/v/io.github.robertluo/waterfall.svg[Clojars project,link=https://clojars.org/io.github.robertluo/waterfall] | ||
image:https://cljdoc.org/badge/io.github.robertluo/waterfall[cljdoc,link=https://cljdoc.org/d/io.github.robertluo/waterfall] | ||
|
||
== Rational | ||
|
||
Try to use Apache Kafka clients in idiomatic Clojure. | ||
|
||
Kafka has Java/Scala clients API, and programmers have no problem calling them directly in Clojure, so we do not need to "wrap" Java API. | ||
|
||
However, from a Clojure programmer's perspective, direct call Kafka Java API does not feel very good: | ||
|
||
- It is complex, with many interfaces, classes, and methods, which is not a good experience for a dynamic language. | ||
- It looks more like a framework or even frameworks. Users have to have a lot of background knowledge and implementation details. | ||
- Clojure programmers could use transducers to deal with data and do not need to use Kafka streams. | ||
|
||
Hence, Waterfall is an attempt at a minimalist library with additional optional batteries. | ||
|
||
- Only two core functions: `producer` and `consumer`. | ||
- Relying on the excellent https://github.com/clj-commons/manifold[manifold library], `producer`, and `consumer` is just the implementation of the Manifold stream. | ||
- Minimum necessary dependencies. | ||
- Clojure version SERDES, in `robertluo.waterfall.shape` namespace. Underlying, give Kafka API byte array SERDES. | ||
- Optional features require optional dependencies. For example, if you want to use nippy as SERDES, you put it in your classpath. | ||
|
||
== Development experience | ||
|
||
Because all functions have their schemas incorporated, you can get the best development experience if dependency `metosin/malli` is in your classpath. | ||
|
||
Put `(malli.dev/start!)` in your `user.clj` will https://github.com/metosin/malli/blob/master/docs/function-schemas.md#tldr[enable clj-kondo] to use the schemas when editing. | ||
|
||
## Single API | ||
|
||
Powered by https://github.com/robertluo/fun-map[fun-map], you can use one single API for accessing a Kafka cluster without any further knowledge: | ||
|
||
- https://cljdoc.org/d/io.github.robertluo/waterfall/CURRENT/api/robertluo.waterfall#kafka-cluster[`kafka-cluster`] | ||
|
||
You can see an example in link:notebook/easy.clj[this easy example notebook]. | ||
|
||
## API namespaces | ||
|
||
[cols="1,1"] | ||
|=== | ||
| namespace | Description | ||
|
||
| robertluo.waterfall | main API | ||
| robertluo.waterfall.shape | optional Clojure data shape transformation | ||
|=== | ||
|
||
## Files Description | ||
|
||
[cols="1,1"] | ||
|=== | ||
| Filename | Description | ||
|
||
| deps.edn | Clojure tools.deps configuration | ||
| tests.edn | kaocha test runner configuration | ||
| build.clj | Clojure tools.build building script | ||
| notebook | https://github.com/nextjournal/clerk[Clerk] notebooks to demostrate usage, use `clojure -M:dev:notebook` to use them | ||
|=== | ||
|
||
== https://unlicense.org[Unlicense] | ||
|
||
2022, Robertluo |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters