[mikorym]

If you vary the size of the opening, am I correct that that just changes the parameters on the curve?

I've known about the central limit theorem for a long time and was probably taught about it in first year, but I have never managed to sit down and understand how to prove it properly. One side effect of the theorem should be to explain least squares—if I am not mistaken then least squares was invented largely due to the central limit theorem by Gauss.

We can always do least cubes, but that does not provide us (usually) with better results.

[thaumasiotes]

> if I am not mistaken then least squares was invented largely due to the central limit theorem by Gauss

I'm not really speaking from expertise here, but I thought least-squares error measurement was based on the fact that the metric is easy to minimize, because taking the derivative of x^2 is easy, whereas taking the derivative of |x| is complicated.

Least cubes doesn't really work conceptually, as it would imply that if an outlier above the fitted curve is bad, then an outlier below the fitted curve is good. That's not what you want.

[mtrimpe]

https://www.jessicayung.com/mse-as-maximum-likelihood/

[mikorym]

As someone else mentioned, the BLUE is what we are usually looking for the Gauss-Markov theorem linked tells us why it is least squares.

EDIT: I am a bit suspicious though, since the definition of variance implicitly uses least squares. Maybe someone else can explain this better, but my expectation is that variance and the reason why it also uses squares (and not something else) should follow from some set of first principles.

[whatshisface]

Least squares is also popular because it is equivalent to projection in a vector space.

[lordnacho]

This might be what you're looking for:

https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem

[mikorym]

Yes, I think so. Intuitively, how I would like to think of this is that we are getting to this from the Euclidean distance metric. But skimming the proof seems like we use variance (which is defined using squares) as a presupposition.

[flopska]

Ah, the good old BLUE (Best linear unbiased estimator). Had this in statistics and really really liked the proof.