What is nnForge?
nnForge is a library for training convolutional and fully-connected neural networks. It includes CPU and GPU (CUDA) backends.
Layers
The library has a number of layers defined:
- Convolutional (1D, 2D, 3D, and 4D)
- Sparse (in feature map dimension) convolutional (1D, 2D, 3D, and 4D, Kepler+ only support in GPU backend)
- Fully connected
- Local contrast subtractive (CPU backend - no limitations, GPU backend - 2D only)
- RGB->YUV conversion (CPU backend: forward propagation only, GPU backend - forward + backward prop)
- Max spatial subsampling (1D, 2D, 3D, and 4D)
- Average spatial subsampling (CPU backend - no limitations, GPU backend - 2D only)
- Maxout - arXiv:1302.4389
- Rectification - |x|
- Hyperbolic tangent - 1.7159*tanh(0.666666*x)
- Sigmoid
- Rectified linear unit - max(0,x)
- Softmax
Training
The library implements Stochastic Gradient Descent training algorithm.
Mini-batch training supported.
Regularization
- Dropout, see Improving neural networks by preventing co-adaptation of feature detectors, Geoffery Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, arXiv:1207.0580
- Weight decay
Error functions
Error functions available:
- MSE - Mean Squared Error
- SHL - Squared Hinge Loss
- NLL - Negative Log-Likelihood
- CE - Cross-Entropy
License
nnForge is an open-source software distributed under the Apache License v2.0.
Download
Download the latest version. Access all the releases along with release notes on GitHub.
The package contains nnForge library as well as examples - applications using the library.
Prerequisites
The library depends on Boost and OpenCV.
If you want to use CUDA backend you will also need CUDA Toolkit installed.
Authors
nnForge is designed and implemented by Maxim Milakov.