Differentiable Programming - A Primer

Why does it exist?

A program is simply a sequence of instructions that a computer can execute to obtain a desired output. Classical programming is all about a human (or a bunch of them) writing down these instructions that the computer will follow.

A neural network is also a program, but it is a program that is learned from data. The programer just specifies the architecture of the network and the learning algorithm, and the network learns to perfom the desired task from data.

What is Differentiable Programming?

How does it actually work?

Example: Modeling a single neuron

Open questions

Final thoughts