Skip to content

Commit

Permalink
Initial commit of plugin with local Kafka details
Browse files Browse the repository at this point in the history
  • Loading branch information
jasononeil committed Aug 14, 2024
1 parent 4338302 commit ec796c4
Show file tree
Hide file tree
Showing 4 changed files with 87 additions and 0 deletions.
39 changes: 39 additions & 0 deletions plugins/ca-kafka-local/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# CA Kafka Local Devbox Plugin

At Culture Amp we use Kafka for sharing data between services, and the private [kafka-local](https://github.com/cultureamp/kafka-local) repo is how we run that locally.

This plugin is added to other services to allow you to interact with `kafka-local` for your service. You still need to launch Kafka on your own, but this plugin provides environment variables and tools to make it easy for your project to connect to `kafka-local`.

What it provides:

- Environment variables
- `KAFKA_BROKERS` set to `"localhost:14231"`
- `SCHEMA_REGISTRY_URL` set to `"localhost:14228"`
- Process Compose job
- Will print information to the terminal about Kafka, once `kafka-local` is up and running.
- Allows you to add a `depends_on` clause so other services wait for `kafka-local`.

## Usage

To start Kafka

- Use [Hotel CLI](https://github.com/cultureamp/hotel)
- Run `hotel services up kafka`
- In your repo run `devbox services up`

Include the plugin in your `devbox.json`:

{
"$schema": "https://raw.githubusercontent.com/jetify-com/devbox/main/.schema/devbox.schema.json",
"include": [
"github:cultureamp/devbox-extras?dir=plugins/ca-kafka-local"
]
}

You can have another service depend on Kafka in your `process-compose.yaml`:

projectors_process:
command: "./gradlew run-projectors"
depends_on:
kafka_local:
condition: process_healthy
15 changes: 15 additions & 0 deletions plugins/ca-kafka-local/bin/kafka-local-readme
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/sh

set -e

echo
echo "== LOCAL KAFKA =="
echo "To start Kafka Local (https://github.com/cultureamp/kafka-local):"
echo " hotel services up kafka"
echo
echo "Once all Kafka services are up and running, this step will be considered healthy"
echo
echo "== Environment variables =="
echo "KAFKA_BROKERS: ${KAFKA_BROKERS}"
echo "SCHEMA_REGISTRY_URL: ${SCHEMA_REGISTRY_URL}"
echo
18 changes: 18 additions & 0 deletions plugins/ca-kafka-local/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"name": "ca-kafka-local",
"version": "0.0.1",
"readme": "README.md",
"env": {
"KAFKA_BROKERS": "localhost:14231",
"SCHEMA_REGISTRY_URL": "localhost:14228"
},
"create_files": {
"{{.Virtenv}}/bin/kafka-local-readme": "bin/kafka-local-readme",
"{{ .Virtenv }}/process-compose.yaml": "./process-compose.yaml"
},
"shell": {
"scripts": {
"kafka-local-readme": "{{.Virtenv}}/bin/kafka-local-readme"
}
}
}
15 changes: 15 additions & 0 deletions plugins/ca-kafka-local/process-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
version: "0.5"

processes:
kafka_local:
# This doesn't actually start Local Kafka, but it prints information about Kafka running, and allows you to do `depends_on: kafka_local`
command: "devbox run kafka-local-readme; tail -f /dev/null"
readiness_probe:
# This ensures kafka-ui is up and running, which means Kafka and Schema Registry are up and healthy.
# The ports are static and defined in our `local-ops` repo, and it's safe to assume it won't change.
http_get:
host: localhost
port: 12288
failure_threshold: 999
availability:
restart: always

0 comments on commit ec796c4

Please sign in to comment.