A MATLAB / MEX / CUDA-MEX implementation of Convolutional Restricted Boltzmann Machines.
Note: This repository is no longer actively maintained, supported, or updated. Use at your own risk.
- Norouzi, M. Convolutional Restricted Boltzmann Machines for Feature Learning. 2009.
- Lee, H., Grosse R., Ranganath, R., and Ng A. Y. Convolutioanl Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations. ICML 2009.
- Norouzi, M., Ranjbar, M., and Mori, G. Stacks of Convolutional Restricted Boltzmann Machines for Shift-Invariant Feature Learning. IEEE Computer Vision and Pattern Recognition (CVPR), 2009.
(From 2014)
When I was doing research on deep learning structures, I was amazed by convolutional RBMs when I read about them because they possess the amazing capability of learning object parts and "recovering" missing parts (see Reference). Although Alex Krizhevsky has an implementation of such network in pure CUDA, I think some (like myself) might want to use a more friendly Matlab version to process smaller amount of data with a bit more time-consumption --- so here we are.
| Current Version | 0.3 | ||||||||
| Note (Feb. 7, 2014) | New CPU code updated for new data structure. GPU code might not work. Some updates on help text, code comments, as well as mex option files. New nvmex option file for MacOS added. | ||||||||
| Important Note | This code is not an official release by the authors and may contain defect resulting in erroneous outcome. Please use with caution and all possible losses is at the user’s own risk. For research / personal purposes only and should be kept public with this note. | ||||||||
| Update Log |
|
- The CUDA code might crash or produce rubbish results. Although I make the best effort to keep the CPU code of this project clean and usable, I sometimes struggle to do the same for the CUDA code in the same timely manner. If that happens, please accept my deepest apologies and hang tight for my next release.
- The CUDA code is currently not running properly (at all) on Mac OS 10.9.1. This seems to be a general CUDA issue since the Mavericks update. Please sit tight before nVidia releases new drivers to save the world.
- The CUDA code might not compile properly on Linux systems (Ubuntu, etc). I don’t have a Linux system at hand set up with CUDA and everything for the moment to test the building scripts. But anyone’s help on this is very much welcomed.
- I can’t run the code. Functions not exist?
To run this code properly, run make, and follow the instructions to setup your compiler for MEX to compile necessary MEX files. - How to use your CUDA-MEX?
See A Guide to Using NVMEX Tool. I’ve modified the nvmexopts.bat file a bit to link the CUDA libs. make will ask you whether you want to compile CUDA MEX files. This feature is limited by the nvmexopts.bat file and currently only work under Win32. Some configuration of the bat file is needed, which is covered in the software documentation (to come). - How fast can this code run?
I can’t say it runs fast: after all the current (0.1 alpha) backbone code is still implemented in Matlab. But with mex implementation, it’s about 5x faster than a pure Matlab implememtation. However since CUDA-MEX is not quite taken advantage of, the current version (0.1 alpha) using CUDA-MEX runs about 3x slower on my laptop, than the MEX version. - I found a bug. Where do I report to?
Both commenting below this post and adding an issue at the project homepage will do. - Future development plans?
This code is still under heavy development before it’s usable in real-world applications instead of toy-sized problems. A full MEX / CUDA-MEX implementation is planned. Besides, a gadget script for compiling MEX / CUDA-MEX files, as well as sample data and usage, are considered.
The current version contains a tested CPU version only. The GPU code in this version is incompatible with the CPU implementation, and may result in memory leakage or other issues. The GPU version is still in development. Any updates will be posted here.
CUDA compilation support for multiple platforms has not been thoroughly tested.
For general information on functions in this project, try help function_name, as most .m files in this project are self-documented.
Please leave questions in Github issues in this repository. Even if I am not around to actively maintain the repository, hopefully the Github community can help you answer your questions more quickly.
That being said, here are some questions from recent comments
- How do I get the data and parameters for
trainCRBM?
You'll have to download a dataset on your own, and transform it to a format that is compatible with trainCRBM. For format of the input, try using the MATLAB help command on the function.
As for the param parameter in that function, try the getparams function for an example.
- What is the
oldModelparameter intrainCRBM?
Firstly, this parameter is optional. That is, you can simply ignore it when you first train your CRBM.
This parameter sort of a fail-safe. As is known to researchers, CDBN's can take a relatively long time to train, and there's usually no guarantee for the system state of your machine running the traning process. Therefore, in my implementation, the model parameters are saved to a .mat file periodically during the training process.
In case your training was interrupted accidentally, you can load the intermediate model from the fail-safe .mat file, and use the model there as the oldModel parameter in this function. The training process will instantly continue based on the saved progress.