[6gvONxR4sf7o]

I disagree strongly. In your analogy, if the compiler broke down all the time, you would probably need to understand assembly to do programming. ML is amazing today, but still kinda sucks. In general you’ll have a bunch of failures on the way to a successful novel application, so it’s more critical to understand what’s going on under the hood in ML than in your programming analogy.

If you just want to apply well known things to well known things, sure you’re right. But as soon as things go wrong, I couldn’t imagine how much more inefficient my iteration cycles would be trying to do novel work without understanding linear algebra (for some kinds of novel work) or calc (for other kinds of novel work). I think you kinda get at this when you say it’s not necessary but it helps. It’s not necessary, but it helps a lot with anything off the beaten track.

[sillysaurusx]

We agree, I think!

And certainly, if you're one of those people who can pull it off, studying ML from first principles is probably an advantage. I just wince every time since I wouldn't have gotten into ML in the first place if I had to start with a big Calculus tome. There are probably a lot of people like me out there.

[PartiallyTyped]

OP asked for foundational, and I provided _foundational_. In my opinion, everyone should start from some sound foundations in LinAlg and Calculus.

Here are a couple of errors that stem from a single foundational problem:

- a linear regressor can not be more than the number of datapoints

- dimensionality reduction when you have NxM with M > N is bogus and you need a bigger dataset to do anything meaningful other than clustering

- input dimension of output layer is larger than the number of samples

The underlying issue in all of these is the rank nullity theorem which is pretty foundational for ML, and yet many practitioners don't know about it or haven't made the connection.

I am not expressing that you should have gone through Spivak or build bottom up. There are books like mathematics of ML that condense everything you need, giving you a decent enough foundation for what you will need.

[PartiallyTyped]

Correction:

A linear regressor can not have more parameters than the number of data points.