Skip to content
Andrew Montalenti edited this page May 13, 2014 · 5 revisions

How do I use a different version of Storm?

Luckily, the Clojure DSL and Java APIs for Storm haven't changed that much from the 0.5.0 release version (though of course, this could change). This allows our Clojure wrapper to define, run and submit topologies between different releases. If you change the version of the Storm JAR via :dependencies in your project.clj file, you'll get support for a different Storm version for free. Here's a sample project.clj file supporting Storm 0.8.2:

(defproject wordcount "0.0.1-SNAPSHOT"
  :source-paths ["topologies"]
  :resource-paths ["_resources"]
  :target-path "_build"
  :min-lein-version "2.0.0"
  :jvm-opts ["-client"]
  :dependencies [     [org.clojure/data.json "0.2.4"]
                      [org.clojure/tools.cli "0.3.1"]
                      [storm "0.8.2"] ; <-------------------------------------
                      [com.parsely/streamparse "0.0.1-SNAPSHOT"]]
  :jar-exclusions     [#"log4j\.properties" #"backtype" #"trident" #"META-INF" #"meta-inf" #"\.yaml"]
  :uberjar-exclusions [#"log4j\.properties" #"backtype" #"trident" #"META-INF" #"meta-inf" #"\.yaml"]
  )

Of course upgrading or downgrading topologies may still require manual steps if you wish to use/stop using newer features that were not available between Storm versions.

How is this different from Storm's storm.py?

Storm bundles a file, storm.py, which serves a prototype implementation of Storm multi-lang and ShellSpout / ShellBolts for Python. But it is not very robust (just a prototype) and isn't made available as an actual Python library that can be upgraded over time.

We have also made some important changes:

  • You call self.emit, self.ack, and self.fail rather than storm.emit, storm.ack, or storm.fail.
  • Spout.nextTuple was renamed to Spout.next_tuple (hooray, Python names!).
  • The code is now neatly reorganized into modules.
  • Errors handled a little more gracefully; tracebacks are properly propagated.
  • sys.stdout/stdin are redirected to a log handler so that you can use the print statement.

Over time, we plan to improve this library even further; e.g. supporting custom serialization formats or adding functional / generator / co-routine / decorator variants.