[freeone3000]

The downside of differential programming is the absolutely massive amounts of training data and time required. Several orders of magnitude over boosted decision trees or even SVMs. If your function’s domain is fairly well understood, save yourself a few weeks and a few thousand dollars.

[time_to_smile]

You can implement SVM, gradient boosted decisions trees, and almost all classical models using the techniques of differentiable programming and it will have 0 impact on the amount of data required.

Massive Neural Nets do require a lot of data and are often not the best solution, but differentiable programming in general does not have higher data requirements than manually computing your derivatives or using OLS. You can still approach classical ML from the perspective of differentiable programming (and likely end up with a better sense of our how your models work in the end).