[SpaceManNabs]

My bullet list, which might be too ambitious and theory-focused, but this is what I used from my physics background.

Learn some:

Calc up to 3 (you can skip some of the divergence and curl stuff)

Linear algebra (no need for Jordan change of basis)

Real analysis

Intermediate probability theory (MAE, MAP, conjugate priors minus the measure theory stuff)

A little bit of differential geometry (at least geodesics. This is for dimension reduction)

Discrete math (know counting and sums really well)

Learn a little bit of Physics (at least know Lagrangians and Hamiltonians)

A little bit of complex analysis (to know contour integration and fourier/laplace transforms)

Some differential equations (up to Frobenius and wave equations)

Some graph theory (my weak spot, but I have used the matrix representations a few times)

After all that, read some Kevin Murphy and Peter Norvig.

Congrats, now you can read most machine learning papers. The above will also give you the toolkit to learn things as they come up like Robbins-Monro.

OP's article is much better if you are trying to be a ML developer/practitioner. Like I said, this list might be too theory focused, but it lets me read lots of applied math papers that aren't ML focused.

[blt]

I'm interested to know where you encountered contour integrals in machine learning?

[mlevental]

ya lol and Hamiltonians. sometimes people just reel off all the math they've heard of to sound impressive. next we'll have people talking about de rham cohomology because of TDA (or something like that)

[cfcf14]

Hamiltonian mechanics, along with many other seemingly out of place 'advanced' maths, show up in modern Bayesian statistics pretty frequently. Hamiltonian Monte Carlo/Riemannian Manifold Monte Carlo are pretty cutting edge (although are implemented in popular libraries like MC-Stan and Pymc3) and both require fairly advanced physics to really understand.

Additionally, we're seeing the introduction of even more sophisticated stochastic samplers (stochastic gradient hamiltonian monte-carlo, etc) that require even more esoteric branches of math and physics to really grok. I have a strong math background but frequently find myself struggling with a lack of knowledge in statistical mechanics when trying to read papers in these areas.

So yeah - there's plenty of bullshit and exaggeration. But there's also some wicked cool stuff happening which requires very sophisticated (and specialized) knowledge to understand.

[blt]

I agree. My original question came from curiosity, not incredulity :)

[cfcf14]

:) Personally I've never used any serious complex analysis in my job (I'm very grateful too, because I always struggled a bit with it). The closest thing I've seen, which I did run into recently, is the use of complex numbers to compute very accurate finite differences. It's one of those delightful tricks that is both elegant and useful: https://blogs.mathworks.com/cleve/2013/10/14/complex-step-di...

I've been working in golang, which fortunately has built-in complex128 types, so it's proved very helpful in a project!

[blt]

oh cool, I really like Cleve Moler's Matlab posts.

I took a complex analysis class and did OK, but I get the feeling that EEs are the ones who really benefit from it (at least in the applied world). They seem to have some very rich analyses of linear dynamical systems using frequency domain methods.

[SpaceManNabs]

https://arxiv.org/abs/1111.4246

I believe the above was used in sk-learn or PyMC3 at some point.

Like I said, the list was a bit too theory focused and not just for ML. Hope that clears things up.

Also, maybe a bit out of place, but it makes my day happier when I assume good intentions out of random tidbits posted online.

[SpaceManNabs]

I have only seen it used to evaluate some integrals in a few papers via Residue Theorem, so maybe I should have said Residue theorem instead of contour integration. I am sure there were other methods that the authors could have used, but I was sure glad to know of contour integration then. I'd say some complex analysis still deserves to be on the list to have a base understanding of Fourier transforms. Of course, you can arrive there without complex analysis.

[imh]

Can you give any recommendations for a little bit of differential geometry?

[tMcGrath]

I think the standard reference is probably Spivak's 'Calculus on Manifolds' but this never really did it for me.

If you have a background in physics then some combination of Nakahara's 'Geometry, Topology and Physics' and Baez and Muniain's 'Gauge Fields, Knots and Gravity' might be good (I haven't included relativity textbooks as I assume it you have a background in GR then you have enough differential geometry).

An unusual recommendation that I think is really nice is 'Stochastic Models, Information Theory and Lie Groups' by Chirikjian. It covers a few other topics mentioned in this thread and is really nice. It's _extremely_ concrete and spells out a lot of calculations in great detail. Plus, the connection to engineering applications is much more obvious.

[imh]

Chirikjian's book looks really cool! Its website says that in volume 1 "The author reviews stochastic processes and basic differential geometry in an accessible way for applied mathematicians, scientists, and engineers." And I can't tell if that means 'brief review because this is a prereq to the book' or if this is a good first take on it. Do you know which it is?

[SpaceManNabs]

This is a little difficult for me since I learned it mostly in the sense of general relativity (which is why I said some differential geometry). For that course, I mostly used the instructor's lecture notes. However, the books for the course were:

Hartle, James B., Gravity: An Introduction to Einstein's General

Schutz, Bernard, A First Course in General Relativity

The first few chapters would be all you need, but they don't include the nice things I learned from the lecture notes like how to derive the gradient, divergence, and curl in any curvilinear coordinate system by using the Christoffel symbols.

Sorry that I can't be of much more help.

[tntn]

Sounds like EE.