[collective-intl]

I think there is a major flaw which you are getting at.

If you take the students who performed at the 10th-25th percentile in any school in one year, on average they would do better the next year because of reversion to the mean.

The way to understand it is that the population they chose did so poorly on their test last year, it is likely that they did worse than they usually do. They are more likely to have had an off year.

For example, the NYT article mentions the girl who missed 50 days of school the previous year. It's more likely she won't miss so many days this year. That's reversion to the mean.

IMO, that throws all the results into question, as you would expect them to do better already.

In general, there are no panaceas in education. Any school which is claiming really great results pretty much never holds up. We've had decades of these articles with experts trying to figure out how to achieve better educational outcomes, and very few can be isolated. Even Bill Gates tried for a while.

Anyone who studies this stuff seriously will tell you educational outcomes are mostly based on innate talent.

[elgenie]

> If you take the students who performed at the 10th-25th percentile in any school in one year, on average they would do better the next year because of reversion to the mean.

This is shoddy reasoning that assumes that each school year is an independent trial. In reality, school years build on each other and usually success in the next year requires familiarity with and competence in the previous year's material, so a more reasonable assumption IMO would be that those kids' next year would if fact be closer to a normal distribution with a mean at the 17.5th percentile.

It's certainly arguable that some of the reason for the school's success is that they're selecting students into classrooms in which they're all 10th-25th percentile which lets the teachers zero out the effect of them having fallen behind without the stigma of being "the dumb class" if they were tracked that way inside another school. But if that's the case, isn't that a rather valuable effect?

[collective-intl]

That's a valid point that someone who fell behind their usual performance may also do just as bad the next year since school material is cumulative. I strongly suspect that this effect is smaller than the reversion to the mean, but you are right we would need to look at data to know for sure.

[bsder]

> Any school which is claiming really great results pretty much never holds up.

That actually isn't true.

We know (we have had multiple state programs demonstrate in real programs) what works in the elementary to early middle crowd--roughly $15,000-$16,000 per pupil spent on the worst performers sustained every single year will bring the worst performers closer to mean ever year. Even the Gates foundation documented this.

We simply do not have the political will to put it into broad action.

[mac01021]

$15,000 spent how?

[bsder]

Quoting myself from a much earlier comment:

It's been a while since I chewed through the Gates foundation stuff but this one seems up-to-date about early programs:

https://docs.gatesfoundation.org/documents/Lessons%20from%20....

Page 16-17 talks about known effective programs and their costs. Note that the more expensive programs (almost all exceeding $15K per student--sometimes dramatically) are almost always more effective. Under $10K is almost uniformly not helpful and the further you get from 10K the less helpful they get. You can have effective programs for $10K, but it's really hard. Money really does make things easier.

From Page 21: "At the highest level, this “doing many things well” requirement results in a high degree of difficulty and is a key reason why high-quality early learning that sticks is so infrequently seen."

From Page 22: "ESSENTIAL ELEMENTS OF HIGH-QUALITY PRE-K THAT STICKS"

"3. Teachers delivering high-quality instruction is a key differentiator between early learning that sticks and early learning that, more than likely, will not stick. ...

4. All exemplar programs have two adults in the classroom—one lead teacher and one paraprofessional/aide— at all times. ...

5. All exemplar programs have maximum class size of 22 children or fewer and adult-to-child ratios ranging from 2:15 to 2:22. Adult-to-child ratios at the lower end of the range are particularly advantageous for classrooms where a significant number of English language learners (ELLs) are present and/or where a significant number of children with special needs are present.

6. Lead teachers with a B.A. plus suitable early learning credential, paid at same level as K-3 teachers. ...

7. Dosage. Three of the four exemplars offer pre-K that runs 6-6.5 hours/day, for 180-205 days/year. The other (Maryland) offers full-day (6.5 hours/day, 180 days/year) and part-day (3 hours/day, 180 days/year) options. It is clear from the exemplars and consistent with research findings that within high-quality pre-K programs the dosage required is related to the size of the achievement gap that must be closed for each low-income child. For low-income children who enter pre-K already on a trajectory to be kindergarten-ready, a high-quality part- day option may be sufficient. For most low-income children, at least one year in full-day, high-quality pre-K is needed to be kindergarten-ready. For low-income children for whom English is not spoken at home, children with special needs, and children who are significantly below age-level competency in one or more domains, it is likely that two years of high- quality, full-day pre-K is ideal and, in fact, may be necessary for most of these children to be kindergarten- ready on time. "

It goes on to other things as well.

And these exemplars are at the $10K-$12K per student mark, roughly. And even successful ones still can't get funding--"New Jersey was poised to expand the Abbott Pre-K Program in 2013, but budget pressures have delayed that expansion.".

And the Gates foundation is VERY gently suggesting that all the mediocre, non-useful programs should be shut down in preference to spending ALL that money on the most underperforming students. While this is likely the best use of resource, it is going to be a politically unviable one.

The upshot is that teaching properly is expensive, and money really DOES have an impact. And the effectiveness "breakpoint" is somewhere around $12K with some adjustmemts for cost of living. And your primary expense is the teacher vs class size--see page 17. The cost per student with a teacher at BA I qualification ranges from $10K with a 15 student class size to $8K with a 20 student class size. Of course, teaching effectiveness is inversely related to class size--pick your optimization point.

I don't always like the Gates foundation because I think they sometimes helicopter in, muck things up, leave, and then other people have to clean up the mess. However, they have been quite forthright with publishing their information and do acknowledge when they have NOT succeeded even when it goes against their agenda. That I applaud.

[jameskraus]

Link is broken

[bsder]

Sorry.

https://docs.gatesfoundation.org/documents/Lessons%20from%20...

[socrates1998]

>average they would do better the next year because of reversion to the mean.

I don't think it works like that. Students who do poorly often do WORSE the next year. They tend to get more and more behind and eventually just drop out.

The only way to reverse this trend is to identify them for more academic instruction, like an extra math or reading class.

Even then, it often doesn't work. It's really really hard to turn a low performing student into even an average one.

"Reversion to the mean" has no meaning when you are talking about one student from one year to the next.

[collective-intl]

It's definitely not the pattern that doing worse once snowballs inevitably into future years, at least not on average, because students' scores still always assemble in a nice bell curve.

I think a better model is that you are not likely to stray too far from your "true" ability, since even if you forgot (or missed) everything from previous years, you could still learn a certain amount in just one year (and probably you remember some things).

[socrates1998]

I never said it was doing worse once. And it absolutely does snowball. A student who is a poor reader in grade school often falls further and further behind.

And grades are not "bell curved" shape. That's ridiculous. Do you know the average grade at Harvard for every class, it's an A.

That's definitely not Guassian.

Man made things like income, grades, stock market prices and home prices are not "bell shaped".

They are heavily skewed one way or another.

I would recommend you do a little research before making such patently wrong statements.