Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

Latest commit

 

History

History
157 lines (117 loc) · 3.62 KB

quickstart.md

File metadata and controls

157 lines (117 loc) · 3.62 KB

Quickstart Guide

Local Install

1. Run minikube.sh

deployment/quickstart_install/sh-scripts/minikube.sh

or

deployment/quickstart_install/ps-scripts/minikube.ps1

2. Install dependencies

1. Run dependencies.sh

deployment/quickstart_install/sh-scripts/dependencies.sh

or

deployment/quickstart_install/ps-scripts/dependencies.ps1

3. Prepare GitHub

1. Prepare Siembol Config Repository

  1. Go to https://github.com/G-Research/siembol-config
  2. Fork into your own organization or personal account

2. Create GitHub token

  1. Go to https://github.com/settings/tokens
  2. Click Generate new token
  3. Select "repo - Full control of private repositories" scope
  4. Hit "Generate token"
  5. Keep this token value as you will need it for the next step.

3. Run demoInstall.sh

  1. This will ask for your github details related to the Siembol Config repository and the token created in previous step.
deployment/quickstart_install/sh-scripts/demoInstall.sh

or

deployment/quickstart_install/ps-scripts/demoInstall.ps1

4. Siembol install

To install Siembol in the cluster

helm install siembol deployment/helm-k8s/ -f deployment/helm-k8s/values.yaml -n=siembol

This step might take a few minutes depending on the specs of your development machine.

Check it out!

In a browser, go to:

You should now see the Siembol UI homepage. You can also try Storm UI to see running topologies:

Enrichment tables

To add enrichment tables.

deployment/quickstart_install/sh-scripts/enrichmentStore.sh

or

deployment/quickstart_install/ps-scripts/enrichmentStore.ps1

Monitoring

We are scraping Siembol metrics using Prometheus and displaying the metrics in a Grafana dashboard.

To install these components:

  1. Download charts:
helm dependency update deployment/helm-k8s/monitoring-tools/
  1. Install the charts:
helm install monitoring deployment/helm-k8s/monitoring-tools/ -n=siembol

To see the dashboard:

You can also see the raw Prometheus metrics from:

Kafka UI

We are using Kafdrop UI to view topics and messages. You can send messages to parsing topics and to test siembol configs.

  1. To install this and create a kafka client pod:
deployment/quickstart_install/sh-scripts/kafkaExtra.sh

or

deployment/quickstart_install/ps-scripts/kafkaExtra.ps1
  1. Exec into the kafka client pod:
kubectl exec --tty -i kafka-client --namespace siembol -- bash
  1. Connect to the broker:
kafka-console-producer.sh \
--broker-list kafka-0.kafka-headless.siembol.svc.cluster.local:9092 \
--topic <your-topic>
  1. Produce your message in terminal window

Traffic Generator

We have a simple deployment which generates messages which Siembol processes. The Kafka topics, frequency and messages can be modified here

  1. To run this:
helm install traffic-generator deployment/helm-k8s/kafka-traffic-generator/ -n=siembol

Cleaning up

If you're done poking about on a local instance, you can clean up with:

  1. For cleaning up siembol resources and dependencies:
deployment/quickstart_install/sh-scripts/cleanUp.sh

or

deployment/quickstart_install/ps-scripts/cleanUp.ps1
  1. For deleting everything; delete the siembol minikube profile:
minikube delete -p siembol
sudo rm /etc/resolver/minikube-*