[lottin]

> Because you can solve the equation to minimize squared error using linear algebra in closed form.

Exactly right. It has nothing to do with probability distributions.