[thaumasiotes]

> if I am not mistaken then least squares was invented largely due to the central limit theorem by Gauss

I'm not really speaking from expertise here, but I thought least-squares error measurement was based on the fact that the metric is easy to minimize, because taking the derivative of x^2 is easy, whereas taking the derivative of |x| is complicated.

Least cubes doesn't really work conceptually, as it would imply that if an outlier above the fitted curve is bad, then an outlier below the fitted curve is good. That's not what you want.

[mtrimpe]

https://www.jessicayung.com/mse-as-maximum-likelihood/

[mikorym]

As someone else mentioned, the BLUE is what we are usually looking for the Gauss-Markov theorem linked tells us why it is least squares.

EDIT: I am a bit suspicious though, since the definition of variance implicitly uses least squares. Maybe someone else can explain this better, but my expectation is that variance and the reason why it also uses squares (and not something else) should follow from some set of first principles.

[whatshisface]

Least squares is also popular because it is equivalent to projection in a vector space.