[elgenie]

> If you take the students who performed at the 10th-25th percentile in any school in one year, on average they would do better the next year because of reversion to the mean.

This is shoddy reasoning that assumes that each school year is an independent trial. In reality, school years build on each other and usually success in the next year requires familiarity with and competence in the previous year's material, so a more reasonable assumption IMO would be that those kids' next year would if fact be closer to a normal distribution with a mean at the 17.5th percentile.

It's certainly arguable that some of the reason for the school's success is that they're selecting students into classrooms in which they're all 10th-25th percentile which lets the teachers zero out the effect of them having fallen behind without the stigma of being "the dumb class" if they were tracked that way inside another school. But if that's the case, isn't that a rather valuable effect?

[collective-intl]

That's a valid point that someone who fell behind their usual performance may also do just as bad the next year since school material is cumulative. I strongly suspect that this effect is smaller than the reversion to the mean, but you are right we would need to look at data to know for sure.