[siboehm]

Here's a good review of the research around this problem: https://nintil.com/bloom-sigma/

The author concludes (IMO, but you should read it yourself) that while tutoring does have positive effects, the 2 Sigma effect size measured by Bloom was probably an outlier.

[dr_dshiv]

Keep in mind that Benjamin Bloom attributes much of the success of tutoring to the affective learning components. That is not addressed in the meta analysis.

Bloom described how the tutor and the student achieved an emotional connection that is often difficult to achieve with a class of students.

This is so critical, because the barrier to learning for many students is emotional. It’s not that they are really trying and just don’t get it; it’s that they don’t have the capacity to care enough to engage over time. Emotional barriers to learning are super widespread — being “bored” for instance is an emotional response. Transformational learning takes place when there is an authentic emotional motivation to succeed. Human contact can support that. It’s also why stuff like ALECKS only goes so far. There is not emotional resonance, like with a human.

[mleonhard]

I think you could combine both models: Students use tutoring software and their teacher spends time with each student celebrating their progress and encouraging and teaching them when they struggle. The teacher's primary job would be to teach the children how to learn and succeed in academics. Specifically, teachers would teach students how to make plans, follow plans, focus, how to think about success, how to think about failure, determine the cause of failure, update their plans, develop determination, evaluate their own mood, and recognize their mental habits. Teachers would also assign, grade, and give feedback on student projects. Projects would have multiple iterations before a final grade.

[barry-cotter]

> being “bored” for instance is an emotional response

Don't be so patronising. People are bored (not "bored") because they're being asked to do something they don't care about. That's an extremely common experience. About a third of high school students are bored every day in every class, and another third report being bored in at least one class every day. Most people have no interest in intellectual pursuits. Their preferences are completely valid.

[dr_dshiv]

Patronizing?

I’m sure we can sit here all day and debate what material kids should learn. My point is that 1. people do well in any subject when they care about it and 2. tutors often help kids care. I’m trying to distinguish this emotional effect of tutoring from the cognitive effect —- otherwise it is difficult to explain the 2 sigma findings.

[todd8]

A relevant quote from the work you cite:

> The history of the educational research literature is one plagued with low quality small sample size studies that were done decades ago, with less work being done now. It can be that now researchers are focusing on studying other instructional methods. Still, the fact that most large RCTs tend to find little effects should make us have a sceptical prior when presented with a new educational method.

[mleonhard]

Thanks. My takeaway from the Nintil article is that nobody has performed a good study on the effectiveness of mastery learning vs traditional teaching. All of the studies have some fatal flaw: not randomized, small sample size, study duration too short, interval between exams too long, not providing specialized remedial content to students, or not actually requiring mastery.

I think the massive effects shown by software tutoring in the DARPA studies point to the mechanism: frequent exams and specialized remedial content. Good tutoring software continually tests students for mastery, identifies specific misconceptions, and provides specialized remedial content for each misconception. The automated software can perform this iteration for each core concept, multiple times per hour. Students frequently get feedback on problems so they waste little time trying to learn material when they don't have the pre-requisite concepts. Students also frequently pass section mini-exams and enjoy feelings of accomplishment. These positive feelings help with learning.

Compare that to the mastery learning studies performed. The studies gave exams once a week or once every 4 weeks. A student with a crucial misconception will struggle for weeks before the they finally understand the content. During that time, they feel frustrated and unmotivated.

We need a good study of mastery learning.

We also need researchers to design their studies better.

IDEA: A new kind of journal with an open study design process. Researchers submit their study proposal, experimental procedures, example raw data, code for cleaning and filtering the raw data, code for statistical analyses, code for generating tables and graphs from data, and a paper template that includes different conclusions based on the values produced by the code. The paper template pulls in the tables and graphs generated by the checked-in code. All of this content is public. Anyone may register an account and provide feedback. Vetted researchers volunteer to review the proposal and code. They receive credit in the resulting paper. When reviewers give LGTM, then the journal and researchers commit to publishing the paper, regardless of the results, and before they have done any experiments. A separate LGTM is required from an experienced statistician. The code includes assertions for sample sizes and valid data ranges.

The researchers must record video of themselves as they perform the experiments. They must also record raw data from their instruments. They must upload these recordings and raw data. The reviewers must LGTM the recordings and any PII redactions. The researchers must get LGTM for all changes to the code and paper template. The journal's servers execute the template and generate the final paper. When someone later discovers an error in the analysis or code, they can file a ticket or send a pull-request with a proposed change. The researchers commit to reviewing every issue and PR within a time limit. If they fail to do that, then the reviewers must handle it. If the reviewers also fail to do it, then the journal assigns another qualified volunteer as a new reviewer to handle it. After making a change, the system generates a new version of the paper.

Anyone may "star" the paper and receive notifications whenever it changes or there is a change to any of the papers it references. If a paper is withdrawn, the system automatically adds warnings to all papers that reference it.

[jimhefferon]

Thank you for the link. There is a lot to take in there. (Perhaps the author could have provided a computer-based tutor ;-) )