[fathead_glacier]

You can refer to a talk about Differentiable Programming here: https://www.youtube.com/watch?v=Sv3d0k7wWHk the talk is for Julia, albeit the principles are general. If you skip to https://youtu.be/Sv3d0k7wWHk?t=2877 you can see a question asked: why would you need this.

In essence there are cases outside the well developed uses (CNN, LSTM etc.) such as Neural ODEs where you need to mix different tools (ODE solvers and neural networks) and the ability to do Differentiable Programming is helpful otherwise it is harder the get gradients.

The way I can see it being useful is that it helps speed up development work so we can explore more architectures, again Neural ODEs being a great example.

[pabl0rg]

Is it very different from probabilistic programming (a term that is both older and easier to understand)?

Erik Meijer gave two great talks on the concept.

https://m.youtube.com/watch?v=NKeHrApPWlo

https://m.youtube.com/watch?v=13eYMhuvmXE

[zeroxfe]

Yes, these are two very different things.

Differential programming is about building software that is differentiable end-to-end, so that optimal solutions can be calculated with gradient descent.

Probabilistic programming (which is a bit more vague) is about specifying probabilistic models in an elegant and consistent way (which than then be used for training and inference.)

So, you can build some kinds of probabilistic programs with differential programming languages, but not vice versa.