Skip to content

Latest commit

 

History

History
155 lines (104 loc) · 8.68 KB

README.md

File metadata and controls

155 lines (104 loc) · 8.68 KB
Raw

Chainer: A deep learning framework

pypi GitHub license travis coveralls Read the Docs

Website | Docs | Install Guide | Tutorials (ja) | Examples (Official, External) | Concepts | ChainerX

Forum (en, ja) | Slack invitation (en, ja) | Twitter (en, ja)

My Contributions (Aniket Didolkar)

Before GSoC selection

PR Description Merged?
#6472 Implementation of sigmoid for ChainerX yes
#6476 Dot Product for higher dimensions for ChainerX yes
#6496 Elementwise power operator for ChainerX yes
#6715 Implementation of absolute for ChainerX yes
#6731 Implementation of Relu for ChainerX yes

After GSoC selection

Main Feature-Based PRs

These PRs include the main features that were to be implemented during GSoC.

PR Description Docs Merged?
#7764 This PR includes the CUDNN and CPU implementation of LSTM/BiLSTM, GRU/BiGRU and RNN/BiRNN link yes
#7783 This includes the implementation of S-LSTM routine as an activation function link yes
#7720 This includes the implementation of TreeLstm as an activation function link yes
#7784 This includes the implementation of word embeddings no

Supplementary PRs

These are the supplementary PRs that were implemented that were necessary for merging the feature-based PRs mentioned above. These mainly include

  • Chainer test fixes so that ChainerX routines could be tested from Chainer.
  • Documentation fixes for the features above
PR Description Merged?
#7804 Simplify n_step_rnn/birnn test in Chainer yes
#7806 Simplify n_step_gru/bigru test in Chainer yes
#7807 Simplify n_step_lstm/bilstm test in Chainer yes
#7805 Simplify slstm test in Chainer yes
#7808 Simplify lstm (as an activation function) test in Chainer yes
#7881 Simplify TreeLSTM test in Chainer yes
#7903 Simplify word embedding test in Chainer yes
#7985 Fix RNN docs for ChainerX yes

Short Description of GSoC project

The main aim of this project was to implement all the recurrent neural network based routines available in Chainer (under chainer.function.rnn) in ChainerX. All the feature based PRs that have been merged can be called using the C++ and the Python API of ChainerX. For each of the chainer rnn functions, I have implemented the forward_chainerx function, which means the ChainerX rnn functions can be used by calling the Chainer rnn functions and specifying the appropriate device as mentioned here.

For each feature the basic procedure was as follows :

  • Implement the feature using the other routines already available if possible.
  • If CUDNN supports the given feature, then implement the forward and backward CUDNN kernels for the given feature.
  • Write tests for the given feature.
  • Write documentation for the given feature.

Most of the Supplementary PRs involve test fixes. When implementing the forward_chainerx, there had to be a way to test the ChainerX code from Chainer. The original tests were not able to do that. Hence, I had to modify the tests such that the ChainerX code could be tested seemlessly from Chainer.

Future Work

  • Support Dropout in the n_step_lstm/bilstm, n_step_gru/bigru and n_step_rnn/birnn routines.
  • Demonstrate through an example, the speed-up that ChainerX offers over Chainer and PyTorch in the case of RNNs.

Chainer is a Python-based deep learning framework aiming at flexibility. It provides automatic differentiation APIs based on the define-by-run approach (a.k.a. dynamic computational graphs) as well as object-oriented high-level APIs to build and train neural networks. It also supports CUDA/cuDNN using CuPy for high performance training and inference. For more details about Chainer, see the documents and resources listed above and join the community in Forum, Slack, and Twitter.

Stable version

The stable version of current Chainer is separated in here: v5.

Installation

To install Chainer, use pip.

$ pip install chainer

To enable CUDA support, set up CUDA and install CuPy.

$ pip install cupy

See the installation guide for more details.

Docker image

We are providing the official Docker image. This image supports nvidia-docker. Login to the environment with the following command, and run the Python interpreter to use Chainer with CUDA and cuDNN support.

$ nvidia-docker run -it chainer/chainer /bin/bash

Contribution

Any contributions to Chainer are welcome! If you want to file an issue or send a pull request, please follow the contribution guide.

ChainerX

See the ChainerX documentation.

License

MIT License (see LICENSE file).

More information

Reference

Tokui, S., Oono, K., Hido, S. and Clayton, J., Chainer: a Next-Generation Open Source Framework for Deep Learning, Proceedings of Workshop on Machine Learning Systems(LearningSys) in The Twenty-ninth Annual Conference on Neural Information Processing Systems (NIPS), (2015) URL, BibTex

Akiba, T., Fukuda, K. and Suzuki, S., ChainerMN: Scalable Distributed Deep Learning Framework, Proceedings of Workshop on ML Systems in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS), (2017) URL, BibTex