Skip to content

Latest commit

 

History

History
114 lines (73 loc) · 2.73 KB

README.md

File metadata and controls

114 lines (73 loc) · 2.73 KB
Raw

siddhi-kafka-cep

  • This is a simple CEP Engine leveraging the Kafka Streams platform.

Key Features

  • This project integrates Kafka Streams with WSO2 Siddhi CEP Engine and allows the users to leverage the best features of both platforms.

  • The complex CEP patterns supported by Siddhi (link) can be used for processing events.

Getting started

  • Compile the cep engine with maven.
mvn clean install

  • Start the cep engine with the scripts in the scripts directory.

  • Create a siddhi rule using the command

java -cp example-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.kafka.SiddhiRuleGenerator
  • Start the event receiver using the command
java -cp example-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.kafka.SiddhiStreamsDataReceiver
  • Publish data to the Engine using the command
java -cp example-1.0-SNAPSHOT-jar-with-dependencies.jar org.apache.kafka.SiddhiStreamDataGenerator

Architecture

Alt text

Api

  • The Java api documentation to create a new Siddhi Rule can be found here.

  • The Java api documentation to publish & receive data from the CEP Engine can be found here & here.

Examples

  • Examples for using the Java api s can be found the example directory.

Using Kafka Producers & Consumers

  • The standard Kafka Producers & Consumers can also be used to publish & retrieve data out of this engine.

Rest Api

  • A simple REST server is included with this project.

Getting started with the Rest server

  • Run the maven jetty plugin to start the server.
mvn jetty:run

Payloads

  • To create a Siddhi Rule:

    POST http://localhost:8080/cep/{streamId}/rule

    Body:

     {
 	"topic": "t1",
 	"bootstrapServers": "localhost:9092",
 	"definitions": ["define stream siddhiStream1 (symbol string, price double, volume long);"],
 	"rule": "@info(name = 'query1') from siddhiStream1[price < 20.0] select symbol, price, volume insert into outputStream"
 }

  • To post streams of data:

    POST http://localhost:8085/cep/{streamId}/stream/data

    Body:

      {
  	"topic": "t1",
  	"bootstrapServers": "localhost:9092",
  	"data": [["Rectangle", 19.0, 19], ["Square", 21.0, 21]]
  }