[rstevenson542]

It is ridiculous to suppose that one's score on a multiple choice test is an accurate measure of innate ability or real-world intelligence.

The SAT, ACT, IQ tests, and all standardized test like them are socially constructed concepts that ATTEMPT a method of measuring intelligence. Intelligence (in the real world) reaches far beyond one's abilities to answer multiple choice reading comprehension, basic math and writing. Not to mention that problem solving in the real world has no time constraints.

Beethoven would not have gotten a perfect score on his SAT's. However, we all can attest to his innovation, creativity and musical genius. How can a multiple choice test measure the creative abilities of people like Sir Richard Branson, Steve Jobs, or Pablo Picasso?

The idea that "smarter people... were slightly more vulnerable to common mental mistakes" is a nonsensical conclusion. These findings are completely worthless.

[ajross]

How on earth would any test be anything more than a "socially constructed concept that ATTEMPTs [sic] [to be] a method of measuring intelligence". That just sounds like a definition to me, not an indictment. Do you have a better test?

No one claims the SAT, or any other test, is the final word. But at the same time pretty much everyone accepts that that "general intelligence" (or something like it) exists, and that tests are a reasonably good proxy for detecting it. To first approximation, students who do well on the SAT are successful in other ways associated with "intelligence".

And that -- the fact that the SAT correlates with something under study -- is all that is needed for good science. Even poor correlations can be enlightening if the data (and scientist) is good enough.

[rstevenson542]

The conclusion that "Smart People are Stupid" is wholly inaccurate. My point was that there is no test that can accurately measure intelligence. As we know, intelligence is often intangible and abstract.

Wikipedia's Definition of intelligence:

Intelligence has been defined in many different ways, including the abilities, but not limited to, abstract thought, understanding, self-awareness, communication, reasoning, learning, having emotional knowledge, retaining, planning, and problem solving.

Sure, many people that scored well on the SAT are intelligent, but that doesn't mean that those that didn't score well are not just as intelligent or capable.

The findings more accurately reflect the conclusion that: "Those With High Scores on the SAT Are Stupid"

[ajross]

The title and your point are both just playing on the semantics of the words. The study is measuring "general intelligence", which is a better defined (if still poorly understood and somewhat controversial) subject amenable to scientific study. Basically everyone in the field accepts that it's real. Even informally, think back to your school peers: I'm willing to bet good money that, on balance, the ones that everyone called "smart" got the best grades, got the best test scores and ultimately got the best jobs. All those things are correlations, and they can be measured scientifically. And they're real.

Basically, if you want to quibble with the headline of the post, then I'll grant that it's a little confusing (intentionally so, as are most good headlines), even if IMHO that point is a little specious. If you really want to claim "there is no test that can accurately measure intelligence" as a matter of scientific fact, you're just plain wrong, sorry.

[cheatercheater]

I wanted to upvote, but at this point you're just indulging a troll.

[jwoah12]

First off, I agree that the SAT is not a good indicator of intelligence. That being said, this argument is missing the point:

> Beethoven would not have gotten a perfect score on his SAT's. However, we all can attest to his innovation, creativity and musical genius. How can a multiple choice test measure the creative abilities of people like Sir Richard Branson, Steve Jobs, or Pablo Picasso?

These tests are not meant to measure creative abilities. They are meant to measure the ability to solve math, reading comprehension, and writing problems. The SAT is used as one of several criteria in the college admissions process, NOT to try to predict who will be on a short list of history's most innovative people.

[rstevenson542]

My point was that intelligence isn't easily defined. And that using the SATs as a measure of intelligence is absolutely ridiculous. The title "Why Smart People are Stupid" is misleading. The title "Why Good Standardized Test Takers are Stupid" more accurately reflects the findings of the research.

[nandemo]

But how come you accept the claim they're "stupid"? Are you saying that SAT is not a good measure of intelligence but Kahneman's tests are?

[hef19898]

I completely agree to the point that any IQ-test like test is just a proxy to meassure what is commonly called intelligence. And these tests are just proxies, they bear in themselves the risk that any studiy based on them is more analysing the proxy (in this case the IQ-test, SAT or whatever) than the real thing (in this case intelligence).

On the orther hand I completly agree that is damn hard to meassure intelligence correctly. But as as long as you don't have proxy that works as well for "educated" westerners as it does for "uneducated" bush or jungle tribesmen you still have quite a high risk of error in your studies. Just my 5 cents.

P.S.: Upvoted rstevensons posts, don't see any reason to down vote him for being critical about multiple choice tests as a basis for such studies.

[RobAtticus]

The questions, judging from the ones in the article, seem to be about testing one's logical reasoning skills. The SATs, at least the qualitative portion, attempt to measure one's math and logical reasoning skills. It seems entirely reasonable to me to assume that someone who scored highly on the SATs should not be tripped up by these kind of questions, and the finding that they are actually more inclined to be tripped up surprises me.

It might be better just to look at a student's qualitative portion of the exam, since one could score highly on the SATs while still getting a (relatively) poor score on the section most similar to these kinds of questions.

[achompas]

Don't worry, you're both right. No self-respecting researcher uses SAT scores as proxies for intelligence in the social sciences.

[ajross]

The study detailed in the linked article does exactly that...

[joshmaker]

Which is why the paper used "various cognitive measurements" which together could be taken as a proxy for intelligence instead of relying on only on S.A.T. scores.

[cheatercheater]

warning: reading this post is a loss of time unless you're rstevenson542.

> all standardized test are socially constructed concepts that ATTEMPT a method of measuring intelligence

as opposed to ones that don't attempt and have not been constructed by a society? What are we hoping for here, exactly: some ray of light shone upon us by god almighty which will let us know that, without doubt, those men are smart and those other ones are stupid?

> The idea that "smarter people... were slightly more vulnerable to common mental mistakes" is a nonsensical conclusion. These findings are completely worthless.

> Intelligence (in the real world) reaches far beyond one's abilities to answer multiple choice reading comprehension, basic math and writing.

Good thing I'm in La-La-Land, then, so I can answer multiple-choice reading comprehension all day long.

> Beethoven would not have gotten a perfect score on his SAT's

And you know this because you dug up his skeleton and it wouldn't mark answers? I'm not sure what to make of your assertion.

You hadn't quoted any passages from the original study that would display inadequate methodology or statistical error. Your findings are completely worthless.