-
Notifications
You must be signed in to change notification settings - Fork 218
Luckily, the Clojure DSL and Java APIs for Storm haven't changed that much from the 0.5.0 release version (though of course, this could change). This allows our Clojure wrapper to define, run and submit topologies between different releases. If you change the version of the Storm JAR via :dependencies
in your project.clj file, you'll get support for a different Storm version for free. Here's a sample project.clj file supporting Storm 0.8.2:
(defproject wordcount "0.0.1-SNAPSHOT"
:source-paths ["topologies"]
:resource-paths ["_resources"]
:target-path "_build"
:min-lein-version "2.0.0"
:jvm-opts ["-client"]
:dependencies [ [org.clojure/data.json "0.2.4"]
[org.clojure/tools.cli "0.3.1"]
[storm "0.8.2"] ; <-------------------------------------
[com.parsely/streamparse "0.0.1-SNAPSHOT"]]
:jar-exclusions [#"log4j\.properties" #"backtype" #"trident" #"META-INF" #"meta-inf" #"\.yaml"]
:uberjar-exclusions [#"log4j\.properties" #"backtype" #"trident" #"META-INF" #"meta-inf" #"\.yaml"]
)
Of course upgrading or downgrading topologies may still require manual steps if you wish to use/stop using newer features that were not available between Storm versions.
Storm bundles a file, storm.py, which serves a prototype implementation of Storm multi-lang and ShellSpout
/ ShellBolts
for Python. But it is not very robust (just a prototype) and isn't made available as an actual Python library that can be upgraded over time.
We have also made some important changes:
- You call
self.emit
,self.ack
, andself.fail
rather thanstorm.emit
,storm.ack
, orstorm.fail
. -
Spout.nextTuple
was renamed toSpout.next_tuple
(hooray, Python names!). - The code is now neatly reorganized into modules.
- Errors handled a little more gracefully; tracebacks are properly propagated.
- sys.stdout/stdin are redirected to a log handler so that you can use the
print
statement.
Over time, we plan to improve this library even further; e.g. supporting custom serialization formats or adding functional / generator / co-routine / decorator variants.