[Rainymood]

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial?

There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

[triplesec]

My suspicion, for the Eeyores round here, is that this has upvotes because people like to bookmark a reminder of references to remind you of the maths behind what most of us do automagically nowadays, for when we might need it. And often it's the comments that provide even better and advance sources, as is the case here. And indeed there are some programmers new to this kind of thing.

[VHRanger]

I think a lot of people here didn't take econometrics courses.

If you see all regression problems under the foil of maximum likelihood estimation, you might not know that ordinary least squares regression has a closed form solution

[theophrastus]

One issue that I nearly always find missing in intro discussions about linear regression is the near universal assumption of no error in the abcissal/"x" values. And while this is true-ish for time series data, (we know for certain which day we collected the data on - yet the same hour every day?), I'd be rich if I had a nickel for every time I saw standard linear regression done when the "x" had significant (and known) error. In which case you're biasing yourself unless you use some sort of 2d regression, like Deming.[1]

[1] https://en.wikipedia.org/wiki/Deming_regression

[VHRanger]

Regression with measurement error is usually treated in much higher level statistics/econometrics classes.

If you're interested in this you can read more in Mostly Harmless Econometrics [1] about adressing this with IV methods

[1] http://www.development.wne.uw.edu.pl/uploads/Main/recrut_eco...

[pacbard]

To build on this a little bit more, there are also generalized linear models that allow to specify the reliability (i.e., error level) of a variable.

Regarding, 2SLS models, I find them more useful to account for endogeneity in the model rather than measurement error. After all, measurement error is usually unobserved (otherwise you would just take it out). 2SLS “just” reweigh the point estimates by identifying the good variation in the instrumented variable using the instrument (for example, using draft lottery results to account for the endogenous choice to attend college).

[thess24]

I agree, it seems like no help just to list a formula to memorize. If someone knows enough linear algebra to understand what the formula represents, they can do the derivation. This link [1] is a good one if anyone is interested.

[1] https://eli.thegreenplace.net/2014/derivation-of-the-normal-...

[abecedarius]

What I expected "by hand" to mean was something like a handmade analog computer. E.g. print your scatterplot, tape a penny over each data point, push a tack through the origin and let the printout swing around it under gravity until it comes to rest (while keeping it flat -- I guess I should've first stuck the printout onto some cardboard). Is there some generalization of this idea that lets the intercept vary too?

[luckyt]

Well, unless you're doing original research, everything you write will be trivial to somebody.

[mr_toad]

There aren’t that many good articles on basic statistics that are freely available on the net. Textbooks are expensive, and most of the free material is of dubious quality. Wikipedia is particularly terrible on Statistics.

[bluusteel]

It seems like blog posts on simple statistical methods like this one land on the front page of HNews a lot more than one would expect.

[thaumaturgy]

https://www.xkcd.com/1053/

I almost certainly know something I'd consider "trivial" that you haven't encountered yet. I try to be really excited when that happens.

[baking]

I think that perhaps the issue is that machine learning courses skip over the fact that there is a trivial closed form solution to 99.9% of all real-world machine learning problems.