[ftxbro]

> Advanced math is mostly useless because of the dimensionality of neural nets.

It depends what you mean by advanced math. There is a lot of math that only really comes into play because of the high dimensionality! For example math related to tensor wrangling, low rank approximations, spectral theory, harmonic theory, matrix calculus derivatives, universality principles, and other concepts that could be interesting or bewildering or horrifying depending how you react to it. Of course some of it is only linear algebra of the 'just high school math' kind but that's not how I would normally describe it. If you look at the math in the proofs in the appendices of the more technical AI papers on arxiv there is often some weird stuff in there, not just matrix multiply and softmax.

[amelius]

Yes but do you have examples of "higher" math not being just a curiosity and actually making it into real world models and training algorithms?

[ftxbro]

Well I suppose that in some sense you are right. You can do deep learning without even knowing any math at all, by plugging together libraries and frameworks that other people wrote.

Also maybe you will say that "higher" math is by definition a curiosity and if it's practical then it's not "higher".

But if those aren't your arguments, then you can consider one example that the tensor 'differentiable programming' libraries used in deep learning use automatic differentiation and matrix calculus. Matrices are taught in high school, and calculus is taught in high school, but matrix calculus generally isn't as far as I know. Or at least not at my high school. https://en.wikipedia.org/wiki/Matrix_calculus