What is Differentiable Programming?
DRANK

The term “differentiable programming” was coined very recently by Yann LeCunas a rebranding of deep learning:His reasoning is that the commonality between all successful contemporary machine learning techniques is not hidden layers in neural networks, but rather training through some form of differential calculus—most often stochastic gradient descent.Despite the neologism, differentiable programming actually refers to a particular approach to a relatively old technique: automatic differentiation (or AD for short). AD is nearly as old as the perceptron. In fact, it was the subject of one of several papers often claimed to be the first dissertation in the field of computer science: A Simple Automatic Derivative Evaluation Program, written by Robert Edwin Wengert in 1964.Okay, But What is Automatic Differentiation?I’ve found AD is best explained by comparison with the two more common methods of computing derivatives: numerical approximation and symbolic differentiation. The differen…

quora.com
Related Topics: Deep Learning Lisp Machine Learning