NATS Streaming/Jetstream Replicator [SJR]

Introduction

This project replicates messages from streaming channels to jetstream. but why? At Snapp when we started using the nats-streaming there was no jetstream for nats, so streaming was our only solution to have reliable delivery for important events, but now streaming is going to be deprecated as mentioned here and jetstream has new features which we will use.

Right now all of our projects use streaming and we want to have an smooth migration so with this project we can replicate events on the jetstream side then remove them from streaming side.

How it works?

It uses the configuration as below to first creates the stream on jetstream and then binds the given topics to it as subjects. then it starts subscribing on given topics from nats streaming and publishes them into jetstream. if you want to bind topics into different streams you need to run different instances of sjr.

---
monitoring:
  enabled: true
  address: ":8080"
output:
  url: "nats://127.0.0.1:4222"
logger: {}
telemetry: {}
input:
  type: "streaming"
  group: "sjr"
  streaming:
    url: "nats://127.0.0.1:4222"
    clientid: "snapp"
channel: "koochooloo"
topics:
  - k.1
  - k.2
stream:
  maxage: "1h"
  storagetype: 1
  replicas: 1

Pyroscope

sjr has buitin support for pyroscope and you can use by enabling it throuth the following configuration.

telemetry:
  profiler:
    enabled: true
    address: http://127.0.0.1:4040

Tracing with OpneTelemetry

sjr inject its context into the jetstream messages. right now we don't support context extraction from streaming messages because there is no standard way to do that.