[ebrenes]

The SATs have a top score, if it were easy to game you'd find many people would clump at the top. That would make it impossible to normalize that distribution into a bell-shape because they're all clustered into the same bucket with no way of spreading them.

OP's point was that the perfect scores remain consistently outside of people's grasp despite the variety of resources available to prep for the exam. I only once managed to hit perfect score and my other best scores were one or two questions off. I had been taking the test since I was 11 (for various extracurricular camps/activities) and prepped multiple times for them. The biggest scores jumps were more closely related with my age and academic achievements than anything else.

[disgruntledphd2]

That is correct.

So the SATs are (almost certainly) 3-pl (actually 2) IRT models. Essentially, it's a multivariate generalised linear mixed model to estimate both question difficulty and participant ability.

Normally, they'll estimate the abilities on the logistic scale, and use the percentile to back transform to a standard normal.

Most people don't cluster at the top because they are a good proxy for g, which is an imaginary statistical construct that we use to explain differences in school outcomes.

So I had a long digression here about the usefulness of penalties for guessing, but it turns out the SATs don't do that anymore, so wth?

(ETS invented IRT, that's why I'm pretty sure).

[chongli]

A lot of the reason the SATs are able to maintain a spread that lets them normalize the distribution is that they fill the tests with stupid tricks that fool people into wasting time. If you do a ton of prep then you learn to spot these tricks and then a lot of the questions become really easy and you can knock them off quickly.

A poor smart student may have mastery of the material but their score will suffer if they don't know the tricks.

[barry-cotter]

And you can find out about all these tricks for free on Khan Academy or for $20 in a SAT prep book.

[saagarjha]

Not everyone can afford to sink time into learning SAT tricks.

[dumb_kid]

No, but I think the point is a combination of Chesterton fence and Goodhart's law. The personal statments and the like, by becoming the primary targets, will cease to measure achivement, and instead measure wealth. So instead of changing without a plan, make a plan to replace SAT2 with something less subject to wealth

MIT is removing a test which rewards the wealthy and punishes the poor. That's good. But the other metrics that MIT currently use are more punishing to the poor, and more rewarding to the wealthy.

With the removal of this element, MIT is left relying on personal statements, recommendations, and the like. With those metrics as the targets, the kids of the wealthy will fund their child's various activities for the sake of giving them a better application.

[cycomanic]

I find this truly baffling. How can it be considered good use of time for someone to sink endless hours of time into learning the ins and outs of some stupid test that is not useful for anything? Just think about the man years of enthusiasm and creativity wasted on learning to score on a test, instead of something truly useful.

I understand why the students do it, I don't understand why anyone would consider it a good system.

[chongli]

There are plenty of kids in the US who can't afford to bring lunch to school, let alone pay $20 for a SAT prep book. Many of these kids don't have a computer or Internet access at home for that matter, so Khan Academy isn't much good to them either. When we talk about rich vs poor kids, there's a HUGE gap that many people don't realize.

[banachtarski]

Anecdote: When I was younger, I used to take a public bus to a local bookstore and work through the SAT practice books (on separate scrap paper of course).

[chongli]

It sounds like you didn't have to pick up younger siblings at school when you were younger. That's the reality for a lot of poor kids. Raised by a single parent, the older siblings are in charge of looking after the younger ones. The parent often does not get home from work until late (and may have multiple jobs).

[banachtarski]

I did actually. I was the oldest of three.

I'm not saying the poor aren't disadvantaged. It's just that the disadvantages are more along the lines of attitudes towards education in the first place (along with everything else). There are also fewer people nearby that serve as an "example" on how to learn or get ahead.

[convivialdingo]

I don’t think that gap is huge in terms of internet access - there’s certainly a gap but it’s mostly a function of parents limiting access in my experience.

84% of teens have smartphones in the US. A large percentage of that remaining 16% have access to internet at school, as 98% of schools have broadband internet.

[pishpash]

They have public libraries with both computers and books. Access has never been the issue if there is the will.

[austincheney]

> The SATs have a top score, if it were easy to game you'd find many people would clump at the top.

I understand and appreciate my school was statistically abnormal but that is exactly what happened. I graduated in a class 334 students and several achieved perfect scores. Perhaps more than a third of the students achieved greater than 1300 and greater than 20% achieved greater than 1400. This is when 1600 was the perfect score.

All these scores indicate is the degree of conditioning imposed upon a student. I know people want these tests to mean something more for personal reasons, but according to all available data this is biased wishful thinking. The research on standardized convergent testing indicates it is not a measure of academic success or potential, but rather an indicator/discriminator of class distinctions due to availability of preparation.

Because I did not come from well groomed pedigree, did not value the subculture of excess vanity, and came from a family that was lower positioned financially I deliberately inverted the goals for a personal social experiment. I wanted to see how close to the bottom I could get and still graduate on time. This was exciting because the risks were greater. If you fail to estimate the conditions correctly you don’t graduate whereas other people get a slightly lower test score or grade point average. Because the goals were different you had plan and weigh the conditions in unexpected ways. Even with all the effort I put in there were still 5 people who graduated with a lower class rank than me.

What impact did that have in later life? I did not get a free ride to an Ivy League school like many of my classmates, enter corporate management immediately out of college, or become a corporate executive within 10 years. I did enter and graduate college. I became a self taught software developer without much challenge and have found very low resistance attaining employment as a senior developer in my full time job. I also became a managing principal in my part time job without as much effort. The greatest tragedy in all this isn’t lost status or earning potential but how boring life has turned out.