Skip to content

Releases: yahoo/TensorFlowOnSpark

v2.0.0

02 Oct 16:31
3052fd3
Compare
Choose a tag to compare
  • initial release compatible with TensorFlow 2.x.
  • API changes:
    • removed TFNode.start_cluster_server, which is not required for tf.keras and tf.estimator.
    • removed TFNode.export_saved_model, which can be replaced by TF native APIs now.
    • added TFNodeContext.num_workers to count master, chief, and worker nodes.
  • Spark ML Pipeline API changes:
    • Scala API has been removed for now, since the Java library for TensorFlow 2.0 is not available yet.
    • removed InputMode.TENSORFLOW support for ML Pipelines, since the input data is always a Spark DataFrame for this API.
    • added HasMasterNode and HasGraceSecs params.
    • removed optional export_fn argument for Spark ML TFEstimator (use TF export APIs instead).
    • added standard default values for signature_def_key and tag_set for Spark ML TFModel.
    • modified inferencing code in TFModel for TF2.x APIs.
  • older TF 1.x examples have been replaced with TF 2.x compatible examples.

v1.4.4

30 Sep 22:50
29e9f07
Compare
Choose a tag to compare
  • last expected release compatible with TensorFlow 1.x (aside from any critical fixes), since the master branch will be moving to TF 2.0 compatibility.
  • handle multiple outputs with signaturedef (thanks to @markromedia).
  • handle exceptions after data feeding.
  • moved API docs to sphinx_rtd_theme.
  • updated to Spark 2.4.4.

v1.4.3

06 Apr 01:37
26c08a0
Compare
Choose a tag to compare
  • removed tensorflow as a dependency, in order to support other variants like tensorflow-gpu or tf-nightly.
  • allow use of evaluator node type in cluster (thanks to @bbshetty)
  • refactored cluster template generation.
  • updated wide-deep example to use models/official code.
  • restore termination of feed in mnist/spark example.
  • updated sample notebook instructions.
  • updated to use Spark 2.3.3.

v1.4.2

22 Jan 18:46
02af44e
Compare
Choose a tag to compare
  • Set TF_CONFIG for "chief" clusters (required by DistributionStrategy APIs)
  • Fix GPU allocation for multi-gpu nodes
  • Updated examples for MNIST
  • Updated Hadoop and Spark dependency versions

v1.4.1

03 Dec 21:58
e392a02
Compare
Choose a tag to compare
  • Added util.single_node_env(), which can be used to initialize the environment (HDFS compatibility + GPU allocation) for running a single-node instance of TensorFlow on the Spark driver.
  • Added an example of parallelized inferencing from a pre-trained SavedModel.

v1.4.0

16 Nov 18:43
7b36521
Compare
Choose a tag to compare
  • More deterministic GPU allocation for multi-GPU nodes.
  • Added timeout argument to TFCluster.shutdown() (default is 3 days). This is intended to shutdown the Spark application in the event that any of the TF nodes hang for any reason. Set to -1 to disable timeout.
  • Added ability to start reservation server on a specific port (contributed by @AvihayTsayeg).
  • Updated pipeline API for latest TF APIs (contributed by @AvihayTsayeg)
  • Added unit test for tf.SparseTensor support.
  • Updated examples to latest TF APIs (including workaround for tensorflow/tensorflow#21745).
  • Updated Spark version dependency for Scala Inferencing API.
  • Added __version__ to module.

v1.3.4

27 Sep 22:02
60a07be
Compare
Choose a tag to compare
  • Travis CI integration for Python documentation and Scala Inferencing API builds.
  • Added sys.path to tensorboard search path.

v1.3.3

06 Sep 20:44
Compare
Choose a tag to compare
  • Only set TF_CONFIG environment variable if cluster_spec has a "master", i.e. when using tf.estimator.
  • Updated mnist/keras/mnist_mlp_estimator.py with example of distributed/parallel inferencing via estimator.predict.
  • Added optional feed_timeout argument to TFCluster.train() for InputMode.SPARK.
  • Added optional grace_secs argument to TFCluster.shutdown().
  • Workaround for firewall proxy issue with get_ip_address (contributed by @viplav).
  • Add support for all Hadoop-compatible File System schemes (contributed by @vishnu2kmohan).
  • Added error messages to assert statements.
  • Initial Travis CI integration.

v1.3.2

13 Jul 21:13
ca1d6df
Compare
Choose a tag to compare
  • add grace period to TFCluster.shutdown()
  • add wide & deep example (contributed by @crafet)
  • update mnist/pipeline examples to tf.data, add instructions, and misc code cleanup (from @yileic)
  • parameterize versions in pom.xml and code cleanup (from @tmielika)
  • update Scala Inferencing pom.xml to latest tensorflow-hadoop artifact (contributed by @psuszyns)

v1.3.1

13 Jul 17:52
df03f49
Compare
Choose a tag to compare
  • Add keras/estimator example
  • Update original keras example to latest tf.keras apis
  • Update Scala Inferencing pom.xml to latest TF java version
  • Allow PS to use CPU on TF-GPU builds (contributed by @dratini6)
  • More pep8
  • More py2/py3 compat