[PartiallyTyped]

OP asked for foundational, and I provided _foundational_. In my opinion, everyone should start from some sound foundations in LinAlg and Calculus.

Here are a couple of errors that stem from a single foundational problem:

- a linear regressor can not be more than the number of datapoints

- dimensionality reduction when you have NxM with M > N is bogus and you need a bigger dataset to do anything meaningful other than clustering

- input dimension of output layer is larger than the number of samples

The underlying issue in all of these is the rank nullity theorem which is pretty foundational for ML, and yet many practitioners don't know about it or haven't made the connection.

I am not expressing that you should have gone through Spivak or build bottom up. There are books like mathematics of ML that condense everything you need, giving you a decent enough foundation for what you will need.

[PartiallyTyped]

Correction:

A linear regressor can not have more parameters than the number of data points.