[aianus]

One thing I never understood about standardized testing is why do they create them in a predictable way that makes them easy to prepare for (and thus game)?

Why not create totally different tests each year with all kinds of difficult problems and then grade the whole thing on a curve? The only way to succeed is to actually understand what you're doing better than your peers.

[VLM]

"We are so cool here, that we exclude kids from our campus who achieved less than X score"

That PR message becomes a puzzle when X becomes a randomly distributed number from year to year. So is 1580 a great number this year while 1400 was a great number last year?

The greatest danger would be an "emperor has not clothes" moment when random test results randomly categorize random students into random schools. That might make the test irrelevant in the future, thus not taken. And the test providers income stream disappears.

[aianus]

The scores would be more like 99th percentile or 80th percentile etc. So the message would be "We are so cool, we only accept the top 1%" which works fine and is more meaningful than 1580 or w/e.

[VLM]

That would assume the cream rises to the top, rather than random testing smoothly distributing them into some kind of spread spectrum signal across the test results.

Or it creates a lotto effect. In some ways this is honest and respectable; if we accept that most people are where they are solely because of whom their parents are, providing a lotto to decide who gets the best deal is in some ways fair and honest. "I was lucky to be born to certain parents" rhymes well with "I was lucky to attend certain schools"

[dTal]

Because it's "standardized" testing, that's the whole point. If the nature of the test changes drastically yearly, how can you compare results from year to year?

[aianus]

It's "standardized" as in everyone gets the same test in a given year. Absolute results on something like a college admissions exam are pretty meaningless since getting into college is a zero-sum game for all but the worst schools. All that matters is doing better than X% of the population.

Seems to me like incentivizing deep learning over memorization outweighs losing the ability to compare absolute results from one year to the next.