Provides and wraps the mathematical functions from the Cephes mathematical library, developed by Stephen L. Moshier. This C library provides a lot of mathematical functions. It is used, among many other places, at the heart of SciPy.

grad = require 'autograd'
grad.optimize(true) -- global
local df = grad(f, { optimize = true }) -- for this function only
local grads = df(params)

From Torch.ch

Neural Network Package

This package provides an easy and modular way to build and train simple or complex neural networks using Torch:

Modules are the bricks used to build neural networks. Each are themselves neural networks, but can be combined with other networks using containers to create complex neural networks:
- [Module]
  (https://github.com/torch/nn/blob/master/doc/module.md#nn.Module): abstract class inherited by all modules;
- Containers: container classes like Sequential, Parallel and Concat;
- Transfer functions: non-linear functions like Tanh and Sigmoid;
- Simple layers: like Linear, Mean, Max and Reshape;
- Table layers: layers for manipulating tables like SplitTable, ConcatTable and JoinTable;
- Convolution layers: Temporal, Spatial and Volumetric convolutions;
Criterions compute a gradient according to a given loss function given an input and a target:
- Criterions: a list of all criterions, including Criterion, the abstract class;
- MSECriterion: the Mean Squared Error criterion used for regression;
- ClassNLLCriterion: the Negative Log Likelihood criterion used for classification;
Additional documentation:
- Overview of the package essentials including modules, containers and training;
- Training: how to train a neural network using StochasticGradient;
- Testing: how to test your modules.
- Experimental Modules: a package containing experimental modules and criteria.