The Bangladesh Mask Study Is a Missed Opportunity

On August 31st, a new randomised controlled trial on mask efficacy was published online. This study is the first of its kind. While the well-known Danish mask study looked at whether wearing a mask reduces one’s own risk of infection, the new study looked at whether community masking reduces the general level of infection.

The main argument for wearing a mask has always been that it makes infected people less likely to transmit the virus, rather than it provides any protection to the wearer. Hence the new study is a far more valid test of the claim that community masking ‘works’.

As I’ll explain, however, it’s a missed opportunity. And this is a shame because some aspects of the experimental design are quite powerful, and – given the number of people involved – it must have been very expensive (meaning there’s now less money available for the next big study).

The basic set-up was as follows. The authors randomly assigned 600 villages in rural Bangladesh – comprising more than 300,000 people – to one of two groups: an intervention group and a control group.

Villages in the intervention group received an eight-week mask promotion campaign, which involved distribution of masks, promotion of masks in public spaces, and role-modelling by community leaders (including imams at local mosques). Villages in the control group did not receive any interventions.

The main outcome variables were measures of seroprevalence. Using phone and in-person surveys, the researchers obtained data on the number of people who experienced COVID-19 symptoms after the intervention. They then collected blood samples from some of those individuals to gauge the number who were seropositive.

Overall, mask-wearing reached 42.3% in the intervention villages, compared to 13.3% in the control villages. What about the outcome variables? 7.6% of participants in the intervention villages reported COVID-19 symptoms, compared to 8.6% in the control villages.

And when the researchers zoomed in on those who reported COVID-19 symptoms and tested positive for antibodies, the seroprevalence was 0.68% in the intervention villages, versus 0.76% in the control villages. (Note: the figure in the chart below is ‘0.69’ because it’s an adjusted value from a model.)

Among the intervention villages, some had been randomly assigned to get cloth masks, whereas others had been randomly assigned to get surgical masks. The researchers compared these two groups, and found that the reduction in seroprevalence was larger for the surgical mask group. They also found that the reduction was concentrated among individuals aged 50+ (see p.28).

Given that the intervention increased mask-wearing by 29 percentage points, the results suggest that going from 0% to 100% mask-wearing could reduce the seroprevalence of COVID-19 by as much as 36%. (That is, 100/29 x [0.76 – 0.68]/0.76.) And the reduction could be even greater for individuals aged 50 and over.

However, there are several weaknesses of the study, which mean that this interpretation is not necessarily correct, and – in fact – may very well be wrong.

One criticism is that the ‘p-values’ corresponding to the main results are rather high (see chart above). Note: a p-value quantifies the degree of statistical confidence one can have in an ‘effect’ of given magnitude; the smaller the value, the more confidence.

Although two out of the three are below the conventional threshold of 0.05, they are only just below it. This raises questions about the robustness of the results. On the other hand, the p-value corresponding to the difference in seroprevalence among those aged 60+ does appear to be quite low.

However, a much more serious criticism is that the study is not actually a randomised trial of mask-wearing. Rather, it is a randomised trial of mask promotion campaigns. This means that, even if the intervention did have an effect, that effect was not necessarily brought about by more people wearing masks.

A true randomised trial of mask-wearing would have had two groups: each receiving the same promotion campaign, except in one group the masks would have been genuine, and in the other they would have been fake (e.g., made out of a more porous material). For ethical reasons, this kind of study might not be allowed.

As an alternative, the researchers could at least have given each of the two groups a ‘COVID-19 awareness campaign’, and then only distributed masks to the intervention group.

Yet in the study itself, the control villages received no interventions. It’s therefore plausible that seroprevalence ended up being lower in the intervention villages not because of greater mask-wearing, but because the campaign made people more concerned about COVID-19 in general, and hence caused them to change their behaviour in other ways.

For example, they might have become more likely to avoid crowds, to ventilate their homes or to stay at home when sick. As a matter of fact, the researchers also measured physical distancing, and found that this was greater in the intervention villages.

One counter-argument is that the greater efficacy of surgical masks than cloth masks suggests the results really are explained by mask-wearing. (Surgical masks appear to be more effective in lab experiments.) However, this result is also consistent with the alternative explanation.

Given that surgical masks are ordinarily worn by healthcare workers or people dealing with hazardous materials, seeing them every day may have prompted greater concern among participants. And in fact, physical distancing increased by slightly more in the intervention villages that received surgical masks. People in those villages may have also changed their behaviour more in ways that weren’t measured.

Some commentators have argued that it doesn’t matter whether the effect came about through greater mask-wearing or other behavioural changes. What matters, they say, is that the mask promotion campaign reduced seroprevalence.

However, this is not a convincing argument. First, while a mask promotion campaign could plausibly affect people’s behaviour in remote Bangladeshi villages, it would be unlikely to have any such impact in countries like the U.K., where the pandemic is already highly salient.

Second, if it was the campaign that mattered, rather than the mask-wearing itself, the same effect could be achieved far more cheaply in the future. For example, rather than distributing thousands of masks to villagers, one could simply inform them about the risks of COVID-19, as well as the importance of social distancing and ventilation.

The Bangladesh mask study could have resolved the debate over whether community masking ‘works’. Unfortunately, it’s a massive missed opportunity. All it really tells us is that promoting mask-wearing while also making the pandemic more salient leads to a reduction in seroprevalence among older people living in rural Bangladesh.