[godelski]

Is it fair to call something an introduction if it uses math from an upper division undergrad math criteria? Such as metric theory. My opinion is that it is context driven. E.g. Introduction to Differential Geometry or Introduction to Homotopy Theory. But I think you can't look at the title and infer prerequisites that are within the ballpark. I'd wager most people outside math and some physics students are familiar with Galerkin methods (maybe a handful of engineers) at the undergraduate level. I don't think most outside math and physics even learn PDEs (my engineering friends mostly didn't and my uni's CS program doesn't even require DE).

[WhitneyLand]

What percent of LLM knowledge requires proficiency in anything you mentioned?

From what I’ve seen it’s a small percentage, and there’s no reason for most people to be put off by it.

Everyone come on in the water is fine.

[godelski]

Between 0% and idk 70%? depending on what you're doing.

[WhitneyLand]

Looking at the theory as a whole it’s a very small minority.

I’m trying to think if it’s 0 percent outside of backprop…

Arguably high school math gets you quite a bit of understanding. After that in descending order I’d guess Linear Algebra, Statistics/Probability, Basic Calculus, Partial Derivatives…

In other words it’s not all or nothing. The easiest stuff gets you a lot of bang for your buck.

[godelski]

Are you a researcher in ML? What is your focus? I'm in image synthesis/explicit density modeling.