Apache Storm is a distributed realtime computation system. The DC/OS Storm service is a Mesos framework that allows you to manage and use Storm in a flexible and scalable way.
- Estimated time for completion: 5 minutes
- Target audience: Data engineers and data scientists a stream processing engine.
- Scope: Install and use Apache Storm.
Table of Contents:
- A running DC/OS 1.8 cluster with 2 agents with each 1 CPU and 1 GB of RAM available.
- DC/OS CLI installed.
To install Storm, do:
$ dcos package install storm
Note that the Apache Storm DCOS Service is beta and there may be bugs, incomplete features, incorrect documentation or other discrepancies.
Continue installing? [yes/no] yes
Installing Marathon app for package [storm] version [0.1.0]
Package is already installedAfter this, you should see the Storm service running via the Services tab of the DC/OS UI:
To use Storm, we will use a DC/OS CLI extension called tunnel, effectively allowing us to create a VPN to access the Storm UI running inside the DC/OS cluster.
Now, once you've dcos tunnel set up and created a VPN tunnel you should see something like the following (in the example here we're using OpenVPN):
$ sudo dcos tunnel vpn --client=/Applications/Tunnelblick.app/Contents/Resources/openvpn/openvpn-2.3.12/openvpn
Password:
*** Unknown ssh-rsa host key for 52.50.225.76: 1565ae7d574857729b625205416eae1e
ATTENTION: IF DNS DOESN'T WORK, add these DNS servers!
198.51.100.1
198.51.100.2
198.51.100.3
Waiting for VPN server in container 'openvpn-28frfq1q' to come up...
VPN server output at /tmp/tmp_62teeyl
VPN client output at /tmp/tmpbm3_w8wpWhen the VPN is up you can access cluster services directly from you local environment (try for example curl leader.mesos) you can access the Storm UI via the following URL:
http://storm.mesos:$UI_PORT/index.html
With $UI_PORT being the first port assigned to the Storm service by the System Marathon. Note that you can either look up that port via the Services tab or directly using a DC/OS CLI command like the following:
$ dcos marathon app show /storm | grep -A 30 tasks | grep -A 2 ports
"ports": [
11585,
11586So in our example case the value for $UI_PORT (ui.port in the Storm config) would be 11585. Note also that the second value you see here, 11586, is nimbus.thrift.port.
To uninstall Storm:
$ dcos package uninstall storm