[doctorpangloss]

The increase in scores is really small. It’s 1.1 percentage points.

My interpretation is, the pandemic is a root cause of lower test scores for many reasons, one reason is that kids started using cell phones way more during the pandemic, and that new stuff on the phone (TikTok, let’s be real) causes lower test scores. Reducing usage during school is addressing a real problem, but it’s one of many real problems, and some are way bigger.

[eitally]

I think you're correct, but another piece of this is that students (especially in HS, but probably also in MS) have realized they can accomplish most assigned tasks in a fraction of the time using online resources (whether copies of old tests, agentic AI, or other sites), so they lean on their phone for this. A big piece of the missing equation here is the fact that home PC/laptop use has also been consistently decreasing, in favor of phones & tablets, causing phones themselves to be even more indispensable for youth.

I read a position paper last week suggesting the solution to this is to take a zero tolerance policy in the classroom and move all course testing back to pencil & paper / bluebooks. I would support that (as a parent of two current high schoolers).

[phantasmish]

Funnily enough, as a parent, I'd have loved for my (young-ish at the time) kids to have been on old-school pencil and paper stuff during the pandemic.

Tear-out worksheet books or a weekly trip to the schools to grab a packet of physical papers with the week's lessons and work (or, hell, send the buses around to drop them off) would have been SO MUCH easier to manage and help with than all the online horse-shit.

Like, I truly think my 80s and 90s classrooms would have been better prepared to deal with the pandemic than the modern computerized ones. You'd think it'd be the other way around, but from what I saw, no. It's just so much harder to keep track of what's going on in several different computer programs, than a stack of paper and a couple books for each kid.

[pavon]

I think there is a typo in the paper, that was carried over to the article. These two sentences appear to contradict one another as written:

> Interestingly, we observe significantly improved student test scores in the second year of the ban (about 2-3 percentiles higher than the year before the ban) when suspensions revert to pre-ban levels.

> Overall, we show that student test scores improved by 0.6 percentiles, with the ban increasing spring test scores 1.1 percentiles in the second year relative to the spring test right before the ban took effect.

Instead, I think the 1.1 percentile gain should be about the first year, and a 2-3 percentile gain by the second year. That is consistent with the graph.

But yes, a fairly small gain. I agree that much of the gain could be recovering from losses during the pandemic. Also the FAST is a new test that started in the 2022-2023 school year, so some of this could also be due to students and teachers adjusting to the new test and improving over time.

[tylermw]

When you're dealing with large populations (here, the study include 230,065 students--a very large number), even small shifts due to some treatment can be significant. It is very hard to generate top-down policy interventions that shift the mean of a population in significant ways: if this treatment effect (banning phones) is real, 1.1 points represents a very big policy win that can easily be applied elsewhere. The devil is in the details, however: they exclude some recent data based on the pandemic, but baseline off of 2022-2023, which was still in the throes of the pandemic. The data they show looks to have around a 0.5-1 sigma variation in percentile from 2022-2024, so the shift from the baseline of around 1 to 4 definitely looks significant, but it will be interesting to see if sticks over time.

[malshe]

Where did you get 1.1 percentage point? The paper uses percentiles.

"Overall, we show that student test scores improved by 0.6 percentiles, with the ban increasing spring test scores 1.1 percentiles in the second year relative to the spring test right before the ban took effect. These positive test score effects are larger for male students (an effect of 1.4 percentiles on the spring test in the second year) and for students in middle and high schools (1.3 percentiles)."

[johnnyanmac]

There are of course bigger problems, but it seems like phones are one of those "attainable" problems for a school to fix.

Schools aren't exactly much better equipped to make sure parents don't both need to work 50 hours to survive, nor bring housing prices down. They can barely pay their teachers to begin with.