Hacker News new | ask | show | jobs
by perlgeek 5210 days ago
The increased test scores of 6.4% in contrast to 5.2% in the control group sounds like it could be purely noise, especially since I couldn't find the value for the standard deviations of these numbers.

The difference is so small that even if it's not noise, it could be very well caused by other factors (teachers more motivated in the test group, for example).
3 comments
dicroce 5210 days ago
Deep down in the report it is revealed that that 5.2% consists of a third of the class improving by 10% or more... and the rest remaining essentially flat. So it appears to me that it's particularly effective for a subset of learners.

Also note: the teacher doesn't have to really lead the class. The report says that the teacher spent most of their time 1 on 1 helping students who were stuck. Teachers may see value in simply having to do less work to achieve the same (or slightly better) result.

Also, this class was only a couple months. There is a question whether the small improvement would have continued and would have become wider when applied over a whole school year.

link
ippisl 5210 days ago
The class only took 5 weeks (24X2 hours sessions).
link
ippisl 5210 days ago
6.4/5.2 = 1.23 .

Is 23% improvement considered that small(assuming it's statistically significant) ?

link
pessimizer 5210 days ago
I think you're looking at the wrong derivative. This is 23% improvement of improvement. It's still just 6.4% improvement.

Might be easier to see if you think of it as 106.6/105.2, in order to include the baseline.

link
ippisl 5210 days ago
I'm not sure 106.6/105.2 is the correct way to look at it either.

Intuitively, every point you got in a test corresponds to a micro-subject.

So before the class , you knew 100 micro-subjects(which probably took a few years to teach). And you gained 6.4 or 5.2 micro-subjects during the 5 week class.

Since one class taught 6.4 micro-subjects and the other only 5.2 micro-subjects ,you can say the difference is 32%.

link
kiba 5210 days ago
The Khan Academy group spent 50% of their time on basic mathematical skills while the control group spend all their time on algebra. That might be a reason as to why we don't see better improvement in their score.
link