A Deep Learning Audio Plugin that Classifies Beatboxes and converts to midi in Real Time.
Deepbox is an audio plugin that uses a convolutional neural network to classify and predict beatboxes in realtime. You can then create a midi file from the plugin with the corresponding beats. It works at any tempo and uses the DAW as it's master tempo. You can view a demo of Deepbox here.
AU/VST3
- macOS: OSX 10.10 and above
All the code for creating a CNN model is in ModelCreation
- To Build Your Own Keras Model:
You can run this python file and change the global test and training data variables.
python CNNAudioClassifer.py
- Convert Keras Model into an fdeep json model:
python ConvertModel.py <input .h5 filename here> <output .json filename here>
The model used for this plugin was trained with audio sample sizes of 512 samples.
- Frugally Deep: Used for inferencing Keras models in C++
- Essentia: Used to extract audio features for training and inference.
- Juce: Used as the core audio plugin framework