A re-examination of an influential and widely cited study which claimed that school mask mandates led to significantly reduced COVID-19 cases among students and staff in Massachusetts suggests that the conclusion may be flawed due to challenges inherent in the study methodology. This analysis is important because it shows that results of observational studies can be misleading and may lead to inappropriate health policy recommendations. The paper is published in Annals of Internal Medicine.

A study of Boston-area school districts by Cowger and colleagues found that lifting mask mandates led to an additional 44.9 COVID-19 cases per 1000 students and staff over a 15-week period. However, sampling bias and confounding may have muddied the findings. Researchers from the Massachusetts Institute of Technology, the University of Toronto, Stanford, University of California-San Francisco and the United Kingdom’s Health Security Agency studied publicly available data of district case rates among students and staff in 72 Boston-area school districts originally studied by Cowger and colleagues. They then added three alternative control groups in Massachusetts to assess changes in COVID-19 case rates in greater Boston area school districts that did and did not lift mask mandates during the 2021-to-2022 academic year. The goal was to identify shortcomings of the difference-in-difference (DiD) methodology the original study used to estimate the effects of the mask mandate.

Using both alternative statistical methods and comparison groups, the authors failed to find consistent evidence in support of the Cowger and colleagues paper conclusions. The researchers found that the data showed an inconclusive association of school mask mandates with case rates. These findings demonstrate how the reported effectiveness of an intervention based on ecological data can be highly dependent on choice of comparison group, time period, or statistical method.

Source: American College of Physicians (ACP)