Comprehensive Review of Face Mask Studies Finds No Evidence of Benefit

The Cato Institute has published its latest working paper, a critical review of the evidence for face masks to prevent the spread of Covid. Entitled “Evidence for Community Cloth Face Masking to Limit the Spread of SARS‐​CoV‑2: A Critical Review” and written by Ian Liu, Vinay Prasad and Jonathan Darrow, the paper is an admirably thorough and balanced overview of the published evidence on the efficacy of face masks. While even-handedly acknowledging and summarising the studies that show benefit, the authors’ overall conclusion is that: “More than a century after the 1918 influenza pandemic, examination of the efficacy of masks has produced a large volume of mostly low- to moderate-quality evidence that has largely failed to demonstrate their value in most settings.”

At 61 pages in length, however, not everyone will make it through to the end, so here’s a TL;DR, with some key quotes to serve as a handy overview. The paper is, of course, worth reading in full, though.

Here’s the authors’ own summary from the abstract:

The use of cloth facemasks in community settings has become an accepted public policy response to decrease disease transmission during the COVID-19 pandemic. Yet evidence of facemask efficacy is based primarily on observational studies that are subject to confounding and on mechanistic studies that rely on surrogate endpoints (such as droplet dispersion) as proxies for disease transmission. The available clinical evidence of facemask efficacy is of low quality and the best available clinical evidence has mostly failed to show efficacy, with fourteen of sixteen identified randomised controlled trials comparing face masks to no mask controls failing to find statistically significant benefit in the intent-to-treat populations. Of sixteen quantitative meta-analyses, eight were equivocal or critical as to whether evidence supports a public recommendation of masks, and the remaining eight supported a public mask intervention on limited evidence primarily on the basis of the precautionary principle. Although weak evidence should not preclude precautionary actions in the face of unprecedented events such as the COVID-19 pandemic, ethical principles require that the strength of the evidence and best estimates of amount of benefit be truthfully communicated to the public.

The authors open by recalling the initial advice on masks from the WHO and others and the pre-Covid evidence it was based on.

Until April 2020, World Health Organization COVID-19 guidelines stated that “[c]loth (e.g. cotton or gauze) masks are not recommended under any circumstance”, which were updated in June 2020 to state that “the widespread use of masks by healthy people in the community setting is not yet supported by high quality or direct scientific evidence”. In the surgical theatre context, a Cochrane review found “no statistically significant difference in infection rates between the masked and unmasked group in any of the trials”. Another Cochrane review, of influenza-like-illness, found “low certainty evidence from nine trials (3,507 participants) that wearing a mask may make little or no difference to the outcome of influenza-like illness (ILI) compared to not wearing a mask (risk ratio 0.99, CI 0.82 to 1.18).”

Considering mechanisms for transmission and thus modes of operation for masks, the authors review the evidence for aerosol transmission and find it very likely. They argue that the ability of masks to inhibit the passage of sufficient aerosols to protect the wearer or others, whether through tiny holes in the material or leaking round the sides, is limited. They write:

[F]iltering capability is unlikely to be reliable surrogate for infection control, since exhaled air necessarily either leaks around a mask’s edges or passes through it. Such leakage has been shown to account for the vast majority (~5:1 ratio) of particle penetration of standardised surgical masks, and exhaled air easily passes around the edges of most cloth masks. One study of cloth masks simulated leakage and found that a hole equal to ~1% of the mask area decreased mask efficiency by over 60%… In a study of N95 respirators, 25% (158 of 643) professional healthcare workers failed to properly fit their mask, despite knowing they were being studied and receiving instructions on how to achieve a proper respirator fit.

Cloth masks, they note, had particularly poor filtration properties in simulated real-world settings: “Cloth masks sewn to CDC specifications offered ~18% inward and 0% outward filtration efficacy at the 0.5 micron size… One mannequin study found that between 5%-20% of respiratory secretions were captured by standard surgical masks during simulated tidal breathing due to face mask leakage.”

Turning to clinical evidence, they summarise the findings of two gold-standard randomised controlled trials (RCTs), which indicate little to no benefit from mask-wearing.

There have been two large-scale RCTs evaluating the use of facemasks at limiting the spread of SARS-CoV-2. One [DANMASK] failed to show a statistically significant benefit to those randomised to wear high-quality surgical masks in both the intention-to-treat and per protocol (i.e., excluding those who reported not wearing masks as specified in the protocol) analyses. The other [in Bangladesh] failed to find a statistically significant benefit to cloth masks, but found an 11% relative reduction in COVID-19 prevalence for surgical masks that was marginally statistically significant, with the confidence interval spanning 0% to 22%. In the latter trial, absolute reductions in COVID-19-like illness associated with mask-wearing were only 1% (reduced from 8.6% in control villages to 7.6% in intervention villages), while absolute reductions in symptomatic seroprevalence were less than 0.1% (from 0.76% in control villages to 0.68% in intervention village), raising questions about whether resources devoted to mask production, awareness, utilization, and enforcement could be deployed to greater public health benefit if directed at alternate interventions, such as vaccination, contact-tracing, or isolation.

They add that “a large RCT (around 40,000 participants) in Guinea-Bissau on community cloth face mask use against COVID-19 is ongoing”, which is worth looking out for.

They critically review the evidence from observational studies, and also from other influenza-like illness. For the latter, they summarise:

In non-healthcare settings, of the 14 RCTs identified by the authors that evaluated face mask efficacy compared to no-mask controls in protecting against respiratory infections other than COVID-19, 13 failed to find statically significant benefits from facemask use under intention-to-treat analyses. In communal living settings, four of five RCTs failed to show statistically significant benefits to masking, and the promising results of the fifth study were not confirmed when its authors sought to replicate the results in a much larger follow-up trial. Of eight RCTs that evaluated face mask efficacy against respiratory illness transmission in nonhealthcare household settings, all eight failed to find a statistically significant benefit for the use of face masks alone compared to controls in their intention-to-treat analyses, and only three found statistically significant benefit in highly selective sub-group analyses.

The follow-up trial mentioned actually found masks were counterproductive.

[A] much larger (7,687 participants) randomised controlled follow-up study by the same research group not only failed to show a statistically significant benefit for mask wearing, but the per-protocol analysis showed higher point estimates for mask wearers compared to non-mask wearers for both clinical respiratory infections (12% (97/828) vs. 9% (141/1497); odds ratio [OR]: 1.3) and laboratory-confirmed respiratory infections (50% (46/93) vs. 41% (50/122); OR: 1.2).

More broadly, they consider the issue of mask harms, focusing in particular on elevated infection risk:

Although some evidence suggests masks may cause non-infection-related harms, such as breathing difficulties, psychological burdens, impaired communication, skin irritation or breakdown, and headaches, the most concerning potential harm to health is an increased rate of disease spread.

While evidence of this isn’t necessarily high quality, it is there.

Multiple household studies have found higher instances of respiratory sickness in masked intervention groups than in unmasked controls. In one household source-control medical mask trial, point estimates of the primary outcome measure of influenza-like illness in the intention-to-treat analysis were higher in the surgical mask group than in the no mask group (22.3% (21/94) vs. 16.0% (16/100)), but the results were not statistically significant and adherence was poor… In a cluster-randomised trial of cloth masks compared with medical masks in healthcare workers, rates of influenza-like illness in the cloth mask intervention arm, where 56.8% of workers wore a mask more than 70% of the time, were more than three times higher compared to the “standard practice” control arm, where 23.6% did so (2.3% (13/569) vs. 0.7% (3/458)).

As a possible cause of this, they suggest mask-wearing might encourage people to interact more, or to lean-in when talking. Masks might also split larger droplets into smaller droplets, they suggest, or become contaminated through touching or re-use.

On the cost of masks, they note: “Although masks are individually inexpensive, the collective costs of producing and distributing an adequate and continuous supply of masks to a global community of 7.8 billion people is not trivial, nor are the environmental harms that result when they are discarded.”

They caution against unthinking application of the precautionary principle, pointing out: “Recommendations to impose mask mandates based on the precautionary principle fail to account for the possibility that masks cause harm, or that masks may have varying benefits and risks in different settings.”

They are concerned by the lack of interest in getting to the truth on masks, putting it down to the widespread public commitment to masking policies making the scientific endeavour politically fraught.

Once officials or others became publicly committed to a position on masks, it became difficult to advocate for high-quality evidence generation, leading to a situation in which, despite the prevalence of masking policies, only two randomised trials have been performed to address the question of face mask efficacy for SARS-CoV-2.

They close with a warning, that a hypothetical future single high-quality study finding benefit must not be taken to override all the evidence to date of a lack of benefit:

When repeated attempts are undertaken to demonstrate an expected or desired outcome, there is a risk of declaring the effort resolved once results consistent with preconceived notions are generated, regardless of the number or extent of previous failures. Scientists and public health officials should exercise caution to ensure that this potential bias does not lead to a cessation of research once the first high-quality study demonstrating mask efficacy is reported.

As I say, worth reading in full.