[sfletcher]

Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.

[KenoFischer]

The differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.

[pixelpoet]

Here's a stunning recent example from computer graphics: http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba...

[bionhoward]

Attention is cool but not to the level of higher-order cognitive architecture. Further, there are many uses of differentiable programming besides AI