Hacker News new | ask | show | jobs
by fanzhang 1708 days ago
First, a nitpick about a theme prevalent in psychological research: drawing sweeping conclusions from a single specific experiment.

Here the "broad conclusion drawn" is that depressed people see the world more realistically -- which if true could have sweeping implications in many areas of life (e.g. they would make better stock pickers, they should be put in command of armies).

The single specific experiment is a 1979 study on correlation between a light bulb and a button. Has this study been replicated? Is the result robust to trivial variations, like predicting a biased random walk, or a continuous correlation (like how hard someone hits a pad versus how many lights light up)? Where's the literature review article for the broad conclusion? (Oh right, this is Vice!)

Now that the nitpick is out of the way, suppose the conclusion is true. A thought experiment would be would a person who estimates light-button correlation accurately be the best at a particular job?

If you wanted to get the best statistical estimate of a correlation, then the more realistic estimator is best for the job. But imagine a situation where you press a button and a green light would light up 10% more often if you press the button versus not. If your goal instead is to get the maximum amount of green light, you'd actually want the person to be pressing the button as often as possible. You'd actually want the person who overestimates his own effectiveness because he'll put in more effort and yield better results!

Generally speaking, it's possible that someone who moderately overestimates her own effectiveness in the world will be better at getting things done in the world, versus the realistic estimate.

Partially related, but I'm reminded of Elon's quote: > I’d Rather Be Optimistic and Wrong Than Pessimistic and Right

2 comments

> drawing sweeping conclusions from a single specific experiment

It seems that you are bringing up the main issue with modern scientific production. Research studies that do not have sweeping/popular/weird conclusions do not get press and the ones that do get coverage. That creates a perverse incentive for researchers to either do research in "popular" topics and to have new insights/conclusions even when these are not really supported by the results.

It is not an accident that you are reading about this specific study in vice rather than the 100s of other incrementalist studies about depression that got published over the past year.

Without having dug out the study, I'm not sure that applies here. "Here's an experiment that suggests X could be a thing", later studies find "yes, might be a thing, but doesn't translate very well to other setups and cases" is completely fine process. What's not is someone coming 40 years later and throwing a breathless generalized headline on it, despite actually mentioning the negative results later.
I don't think it's fair to pick on psychological research, this is a phenomenon present in all reporting on science... and I think it's more of a reporting issue (they're incentivized on collecting eyeballs, not accuracy). I've often read the studies behind the articles to find that the study is much more realistic and less apt to draw conclusions beyond "one possibility is X, but more research is needed."