| HN Mirror

5. David Zweig makes the argument that some schools are open for six weeks instead of the three. The article has some ambiguity here due to the writing style. Zweig states, "After reviewing school calendars and speaking with several school administrators in Maricopa and Pima Counties, I found that only a small proportion of the schools in the study were open at any point during July. Some didn’t begin class until August 10; others were open from July 19 or July 21. That means students in the latter group of schools had twice as much time—six weeks instead of three weeks—in which to develop a COVID outbreak." It is unclear whether the "latter group of schools" here refers to those open on July 19/20 or to the group of schools with mask mandates. I think it's referencing the dates. Megan Jehn, one of the study's authors responded to this by saying that the median start date for the non-mandate schools was August 3rd and those with mandates started on average on August 5th. She also responded with, "It is highly improbable that this difference alone could explain the strong association observed between mask policies and school outbreaks." I would like to see more disaggregated data in the study but at some point you've gotta believe an epidemiologist when they tell you that your counter point isn't so good. However, Ketcham is quoted again here saying, "If schools with mask mandates had fewer school days during the study, that alone could explain the difference in outbreaks." I agree with Ketcham, that could explain the difference in outbreaks. Does it, though? The authors of the study, who have done the work in this case say its improbable, the economist Jonathan Ketcham says it might explain the diffence. I'm not sure this qualifies as a severe issue in the study, but it's something that could likely be put to rest with a little more public data.

6. Louise-Anne McNutt and Ketcham point to a possible detection error in the study. Namely, "...according to Maricopa County guidelines, students are considered 'close contacts' of an infected student—and thus subject to potential testing and quarantine—only if they (or that infected student) were unmasked. As a result, students in Maricopa schools with mask mandates may have been less likely than students in schools without mandates to get tested following an initial exposure." To this the study authors responded that it is, "highly speculative to make the assumption that identified close contacts are more likely to be tested than other students." This is a little harder to muddle through as we are getting a core argument on designing a reliable study. So, we are looking at the definition of an outbreak. The study states its definition thusly, "A school-associated outbreak was defined as the occurrence of two or more laboratory-confirmed COVID-19 cases§ among students or staff members at the school within a 14-day period and at least 7 calendar days after school started, and that was otherwise consistent with the Council for State and Territorial Epidemiologists 2020 outbreak definition¶ and Arizona’s school-associated outbreak definition." The CSTE defines an outbreak thusly, "Two or more laboratory-confirmed COVID-19 cases among students or staff with onsets within a 14-day period, who are epidemiologically linked, do not share a household, and were not identified as close contacts of each other in another setting during standard case investigation or contact tracing." The confounding factor that McNutt and Ketcham are referring to, which is that if two people are considered an outbreak they could have caught covid from outside of the school instead of each other, is sort of moot here as the paper states they conducted adjusted logical regression analysis against many factors including the, "7-day COVID-19 case rate in the school’s zip code during the week school commenced". That means that detection bias should not be present and we can be fairly certain that the definition of outbreak in this study is both consistent and likely means that the two or more cases are in fact school related.

7. Jason Abaluck makes the point that vaccination status could be a confounding variable. It can be! This is a flaw in the study that the study itself acknowledges. I'm not sure how this is an argument for the study being junk science. It's a little like one of the first probability and statistics examples I was taught. In this example a study was done on Coney Island where the scientists looked at ice cream sales and the number of drownings in the area. They found that there is a clear correlation between them as when ice cream sales went up so did drownings. In the study they unfortunately could not get access to temperature data or visitor counts. They concluded that ice cream sales are a good indicator of drownings. Of course, the reason ice cream sales and drownings are correlated is because people go to Coney Island when it is hot out. They also eat ice cream when it is hot out. With more people going to Coney Island there are more people that are likely to drown. The study did not make the claim that ice cream sales cause drownings, just like this study does not claim that mask mandates reduce covid outbreaks. However, in both cases the indicator is useful. In the case of mask mandates we also have data that shows masks reduce covid transmission in many other scenarios. So, yes, vaccination status is a known confounding variable that I'm sure the authors of the study would like to control for. Since they can't they did the next best thing and established that mask mandates are a good indicator for reduced covid outbreaks.

8. David Zweig attempts to reproduce some of the data himself. He attempts to build part of the data set used in the study for Maricopa county. In the end he gets the list of schools from the study authors. He writes, "Yet it still included at least three schools in Pima County, along with at least one virtual academy, one preschool, and more than 80 entries for vocational programs that are not actual schools. In response to a follow-up inquiry, they acknowledged having included the online school by mistake, while attributing any other potential misclassifications to the Arizona Department of Education." This is interesting, but ultimately the misclassification of some schools does not ultimately change the statistical methods used.

This is it. Eight arguments meant to show, "...the study’s methodology and data set appear to have significant flaws." I don't see the points made as exposing significant flaws. However, I am a simple programmer who studied mathematics and reads technical specifications in my spare time.

At the end of the day please remember these truths. Covid kills a lot of people, mainly the older ones. Covid transmits via aerosolized bodily fluids (coughs, sneezes, breathing), and things like wiping your nose then touching doorknobs where someone does the same in reverse order. Masks, preferably well fitting n95 and kn95 ones, irrefutably reduce the spread of covid. Washing your hands well and often irrefutably reduces the spread of covid. The science here is icing on the cake and it just tells us, sometimes roughly, how much these things help. Beyond those two things the vaccines are an entirely different story (because thanks Trump), but the science is more clear there, they also help.