Skip to content

Software Tools for DL

Jump to bottom
Thomas Lidy edited this page May 17, 2019 · 3 revisions

Very good list of Software toolboxes for Deep Learning: http://deeplearning.net/software_links/

Most Popular Software Frameworks

Keras easy to use library to create deep neural networks in few lines of code (using Tensorflow underneath, previously also Theano) http://keras.io (Github: https://github.com/fchollet/keras)

Tensorflow https://www.tensorflow.org

PyTorch "open source deep learning platform that provides a seamless path from research prototyping to production deployment" https://pytorch.org

Other DL Software Frameworks

MXnet (https://github.com/dmlc/mxnet) deep learning framework designed for both efficiency and flexibility, written in C++, supports many languages (Python, R, C++ and Julia), multi-GPU and cloud setups Documentation (http://mxnet.readthedocs.org/en/latest/index.html)

Caffe(http://caffe.berkeleyvision.org/) Deep learning framework mainly for image retrieval.

cxxnet (https://github.com/dmlc/cxxnet/) a fast, concise, distributed deep learning framework. - outdated in favor of mxnet

DeepLearning4j (https://deeplearning4j.org) For the ones in favor of typesafe, fluent APIs in Java/Scala

Apache SINGA (https://singa.incubator.apache.org/en/index.html) Common c++ NN framework can be used in python and c++ or c applications Distributed Neural Networks!!

nolearn (https://pythonhosted.org/nolearn/) seems outdated - recommends to use Lasagne (see above)

Deep Learning on Amazon EC2 GPU with Python and no learn (http://www.pyimagesearch.com/2014/10/13/deep-learning-amazon-ec2-gpu-python-nolearn/)

LeNet library for Convolutional Neural Networks (http://yann.lecun.com/exdb/lenet/) (deeplearning.net Tutorial) (http://deeplearning.net/tutorial/lenet.html) (Tutorial PDF) (http://www.cs.cmu.edu/~aarti/Class/10701_Spring14/slides/DeepLearning.pdf) (more simple for beginners than Theano)

cuda-convnet2 (https://github.com/akrizhevsky/cuda-convnet2) (old Google Code repo) (https://code.google.com/p/cuda-convnet2/) a fast C++/CUDA implementation of convolutional (or more generally, feed-forward) neural networks. Fermi-generation GPU (GTX 4xx, GTX 5xx, or Tesla equivalent) required.

pendant (https://github.com/coreyker/pendant) Python ENvironment for Deep learning with Artificial Neural neTworks - simple python package for training and inference using (deep) artificial neural networks (with music in mind)

dnn-mgr (https://github.com/coreyker/dnn-mgr) deep neural networks (DNNs) for music genre recognition (MGR)

scikit-neuralnetwork (https://pypi.python.org/pypi/scikit-neuralnetwork/0.3) Deep neural network implementation without the learning cliff! This library implements multi-layer perceptrons as a wrapper for the powerful pylearn2 library that’s compatible with scikit-learn for a more user-friendly and Pythonic interface.<br supports:

  • Activation Functions: Nonlinear: Sigmoid, Tanh, Rectifier, Maxout. Linear: Linear, Gaussian, Softmax.
  • Layer Types: Convolution (greyscale and color, 2D), Dense (standard, 1D).
  • Learning Rules: sgd, momentum, nesterov, adadelta, adagrad, rmsprop.
  • Regularization: L1, L2 and dropout.

Chainer (http://chainer.org/) Relatively new compared to some other libraries on this list but buzzing with development activity. This library also supports CUDA for running your models on the GPU, but does not rely on Theano to do so. It has instead its own implementation. Chainer supports various network architectures including feed-forward nets, convnets, recurrent nets and recursive nets. It also supports per-batch architectures. Chainer looks extremely promising due to the way it builds its computational graph, which differs from the way one declares a model in e.g. Theano or Torch7. This strategy also makes it easy to write multi-GPU parallelization, since logic comes closer to network manipulation. (from http://creative-punch.net/2015/07/frameworks-and-libraries-for-deep-learning/)

more deep learning libraries: https://news.ycombinator.com/item?id=9283105

Data Augmentation

Data Augmentation for Audio

muda - Python library for augmenting annotated Audio data https://github.com/bmcfee/muda

SpecAugment (Google):

Parameter Optimization

hyperopt - Distributed Asynchronous Hyperparameter Optimization in Python (https://github.com/jaberg/hyperopt)

Spearmint (https://github.com/HIPS/Spearmint) Practical Bayesian Optimization of Machine Learning Algorithms. support for Spearmint for hyper parameter search planned in keras (https://github.com/fchollet/keras/issues/100)

DataValoris http://www.datavaloris.com/en/ Commercial cloud platform to optimize your neural networks.

Python toolboxes

Anaconda (https://store.continuum.io/cshop/anaconda/) Scientific Python distribution for large-scale data processing, predictive analytics, and scientific computing (can install parallel to an existing Python installation and has a package updater)

MKL - Math Kernel Library(https://store.continuum.io/cshop/mkl-optimizations/) Boost the speed of popular numerical Python libraries

lrn2(http://lrn2cre8.eu/?q=workpackage1/deliverable1.1) A python framework for representation learning from data

Python IDEs

PyCharm(https://www.jetbrains.com/pycharm/) A really good IDE for Python with code assistance/completition, refactoring etc. It is free for students and for research projects (besides a free community edition).

Theano-based Libraries

Note: Theano is no longer developed since v1.0 in 2017.

Theano - deep learning library(http://deeplearning.net/software/theano/) Evaluate mathematical expressions involving multi-dimensional arrays efficiently. It incorporates automatic optimization in order to reduce calculating overhead an to increase mathematical stability.

Theano tutorial code (https://github.com/lisa-lab/tutorial_theano)

Speeding up your Neural Network with Theano and the GPU (http://www.wildml.com/2015/09/speeding-up-your-neural-network-with-theano-and-the-gpu/)

Fast LeNet5 CNN in Theano for GPU (https://www.kaggle.com/lukasz8000/digit-recognizer/test1) (by Kaggle user)

The fastest convolutions in Theano with meta-optimization (http://benanne.github.io/2014/12/09/theano-metaopt.html) by Jan Schlüter on Sander Dielemans blog

Good overview and short-summaries of Theano, Pylearn2, Blocks, Keras, Lasagne and Chainer (http://creative-punch.net/2015/07/frameworks-and-libraries-for-deep-learning/)

TheaNet (https://github.com/lmjohns3/theanets) (github) Feedforward and recurrent neural networks for Python (good and easy to use but prepare your data very well)

Pylearn2(http://deeplearning.net/software/pylearn2/): Pylearn2 is a machine learning library. Most of its functionality is built on top of Theano. This means you can write Pylearn2 plugins (new models, algorithms, etc) using mathematical expressions, and Theano will optimize and stabilize those expressions for you, and compile them to a backend of your choice (CPU or GPU).

Pylearn2 Tutorials on GitHub (https://github.com/lisa-lab/pylearn2/tree/master/pylearn2/scripts/tutorials) (+ Intro slides) (https://bfeba431-a-62cb3a1a-s-sites.googlegroups.com/site/deeplearningcvpr2014/pylearn2_theano.pdf?attachauth=ANoY7crakAy3wF-_Cagw6ohTK7zLWJM-F_zjAKJvZ0vfu5K7sTmWUXwA7VQNT3C-ees9lIgNp1XyNGQZ6R1Cm4csC5CP-THFJKGgOiiE_c1n_faaRfJyX4EAUYN26Nc1ka-rszKZ8BbBSUZ068K9BB1AHqc7n4xiTvNbiQ3MRbnc5cR2FjnlWQVzVwj81goEorNliSiTjCmfAX7BXqYJfaT0aMQKDzETsP47K1nf6C1ABP-OW7VKr00%3D&attredirects=0)

Lasagne (https://github.com/Lasagne/Lasagne) a lightweight library to build and train neural networks in Theano by Sander Dieleman (Music IR researcher) User Guide and API Reference (http://lasagne.readthedocs.org/en/latest/)

Good Lasagne CNN Tutorial (http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/)

Difference between Keras and Lasagne (https://news.ycombinator.com/item?id=9283105)

Blocks (https://github.com/mila-udem/blocks) a Theano framework for building and training neural networks

theano-lstm pip install theano-lstm Theano-based LSTM network

Clone this wiki locally