Release 0.15.0

Highlights:

• Complete support for running Heron in Kubernetes
• Heron API server for easy deployment and topology management
• Support for Storm like python API with PEX support
• Python DSL with PEX support
• Exactly once and stateful processing in Java and Python
• Initial version of Dhalion for policy based self regulation
• Pulsar Spout
• Google cloud storage uploader
• Several bug fixes

Release Notes:

• Add base framework for api server. (#2094)
• Heron Python Code Cleanup (#2173)
• Update pulsar to apache client. (#2171)
• Added docstrings to python dsl (#2166)
• Add kubernetes stateful config. (#2164)
• added update certificates for java (#2163)
• Fix NPE when logging exceptions that have no message using SLF4J (#2161)
• Dhalion: Add implementation for resolvers and health policy (#2162)
• Added Pulsar Spout and Pulsar Streamlet (#2160)
• Added exactly once semantics for python dsl (#2159)
• Don't print topology context since it just prints a class pointer info (#2157)
• Misc Python fixes (#2158)
• Sort the task ids to ensure that identical fields grouping tuples (#2155)
• Make sure that metric registration and adding tasks are done once. (#2152)
• GetInstances will return only active instances. Instead we want to (#2148)
• Refactored common code by moving it to base instance (#2151)
• Correct the way we refer to the class member (#2150)
• Add MetricsCacheMetricsProvider (#2113)
• Add Dhalion detectors and diagnoser implementation for heron (#2132)
• Use a default value for checkpoint interval if user does not specify it (#2147)
• Fix the paths for local stateful storage (#2145)
• For debugging purporses count all words as 1 (#2144)
• Corrected stateful checkpoint location (#2143)
• Add a string method to print out the state (#2142)
• Use appropriate method while manipulating state (#2141)
• Spout should read input tuples even in non acking mode (#2140)
• Initialize instance state if it's null (#2138)
• Register for stateful messages and ensure that state is not empty before parsing it (#2139)
• Initialize the instance state before open/prepare instance (#2134)
• Add java doc gen for heron api, spi and storm (#2123)
• Added exactly once semantics for python (#2135)
• Update sendOutState (#2131)
• While unzipping pex files, don't overwrite existing files. (#2129)
• Use Java ver 8 as source and target version (#2130)
• Add Dhalion-sensor implementation for heron (#2101)
• Limit stmgr client reconnect attempts (#2125)
• Make instance send out state (#2120)
• Register stateful processing messages with StreamManagerClient (#2119)
• Added a new socket option to take the maximum packet size accepted (#2118)
• Add stateful support into Slave (#2106)
• Introduce StatefulController in TMaster and handle all stateful messages (#2105)
• Added ability in tmaster to accept stateful processing messages (#2102)
• For stateful topologies, TMaster should connect with checkpoint manager (#2100)
• Dhalion health manager (#1980)
• Add stateful controller into tmaster dir (#1973)
• Add clean stateful checkpoint handler into tcontroller (#2096)
• Introduce stateful restorer in stmgr. (#2091)
• New API for topology reliability modes (#2089)
• Correct heap memory usage of stream manager (#2086)
• Limit number of messages in mempool, instead of total byte sizes (#2073)
• Add Pulsar spout. (#2069)
• Instantiate ClientMgr in constructor (#2078)
• Instantiate checkpoint manager client in stmgr (#2076)
• Added Stateful processing related message handling in TMasterClient (#2075)
• Fix kafka spout HeapByteBuffer not serializable problem (#2063)
• Added stateful processing restore state machine to stmgr. (#2070)
• packaging python with all the modules (#2072)
• Fix the comparison to be string rather than numerical (#2071)
• Change ci-docker.sh to use sh instead of bash (#2068)
• Added stateful processing message handlers in stmgr-server (#2067)
• Remove API dep on c.t.h.common.basics.TypeUtils (#2061)
• Add hooks in stmgr client/clientmgr to send checkpoint markers. (#2060)
• Added a script to build/publish docker images (#2065)
• Mempool acquire should always return a Clean protobuf (#2057)
• Adding debugging to integration test spout, increasing timeouts for get condition (#2055)
• Added Checkpoint Gateway Layer while sending data to instances (#2047)
• Increase timeout for integration test result checking (#2053)
• Add countdown latch to runnable in testRegisterTimerEventInNanoSeconds (#2050)
• Added Spout for Apache Pulsar (#2048)
• Created heron/spouts to house all well-known spouts. (#2046)
• Added applatix scripts and tools (#2045)
• Package python dsl package (#2043)
• Python DSL (#2033)
• Extract HeronTuple from current_tuples while acking it (#2032)
• Applatix workflow with build followed by test (#2015)
• Made cloudpickle the default serializer (#2030)
• Window tuples should take care of handling acks (#2028)
• Changed the custom grouping to take in an actual instance of (#2027)
• Added WindowBolt interface in python (#2021)
• Added Checkpoint Gateway (#2024)
• Add security check for shell-kill-executor (#2013)
• flatten the directory structure for integration tests (#2018)
• Updated Python documentation (#2017)
• Added Neighbour Calculator, a utility class to calculate (#2016)
• Expose TopologyContext in heronpy (#2014)
• Moved Herons pypi to heronpy. Updated requirements as well (#2012)
• Remove pyheron since now python is first class api in Heron (#2006)
• Rename tools/travis-ci to tools/travis and update the references to it (#2007)
• Restart subprocess with shell_env. (#2004)
• Discover the platform in travis scripts (#2001)
• change the default gcc 4.8 to just gcc (#1999)
• remove the space in applatix array parameters (#1998)
• Made heron instance and examples depend on api and not on pyheron (#1992)
• Applatix CI integration (#1986)
• Add support for src task id in the storm api (#1938)
• Kubernetes Manual Scaling (#1970)
• Use TopologyContext as the base for TopologyContextImpl (#1989)
• Refactored constants (#1988)
• Add GCSUploader for uploading topologies to Google Cloud Storage. (#1976)
• Heron client unit test (#1983)
• Rephrase references to Storm (#1979)
• Native Python API for Heron. (#1982)
• Add checking server starting return (#1984)
• Change TMaster to watch the packing plan for changes (#1955)
• Added get topology name to context (#1978)
• Pex files need to be unzipped so that depdencies can be set/found (#1977)
• Import metrics from pyheron module (#1975)
• Added the python requirements to be >= (#1974)
• Allow the tracker to take command line args for a state manager. (#1967)
• Fix inaccurate execute-latency metric (#1969)
• Changed the python requirements for pyheron so that it doesn't force to downgrade (#1966)
• Topic Real Time Streaming does not exist as list of allowed Topic Names (#1965)
• Streamline preparation of docker images for distribution (#1961)
• Remove extra initialize method from ContainerFileDownloadHandler. (#1962)
• Allow any type of file to be downloaded from a container through the shell. (#1959)
• Increase delta in Python loopers test (#1958)
• Add support for watching packing plan to c++ state managers (#1953)
• Scripts to build docker artifacts (#1948)
• Add note on hostname in /etc/hosts (#1945)