Dr. Tom Jefferson is the lead author of the newly updated Cochrane review on the evidence on masks and other physical interventions for combating respiratory viruses like SARS-CoV-2. Following the publication of the review Dr. Jefferson was interviewed by his colleague, Oxford’s Professor Carl Heneghan. There follows a full transcript of the interview.
CH: Welcome to the Trust the Evidence podcast with Carl Heneghan and Tom Jefferson. We’re coming to you today because last night was the release of a Cochrane systematic review on physical interventions to interrupt or reduce the spread of respiratory viruses, which Tom leads. Tom, I’m just going to come to you with a first question. This is an update of the Cochrane review. So just tell me how long have you been doing this review and which update is this now.
TJ: So, a Cochrane review is a study which synthesises all available studies – all that we can find or identity – on a particular topic. It follows a highly structured format and is always preceded by publication of a protocol. All this is to minimise the bias. Also, it is extensively transparent. In this case we are looking at about 300 pages of review. Now, the review called “Physical interventions to interrupt or reduce the spread of respiratory viruses” is called in code A122 for short and I will be using that acronym simply because it is just too long a title. So the protocol was first published in 2006 and then the first version was published in 2007, updated in 2009, 2010, 2011, and then 2020, so this 2023 is the fifth update of this review. And the reason why we update the reviews is they are soon out of date if we don’t do that, especially in some fast moving topics.
CH: So it’s interesting what’s happening here is that if I look at this you’ve sort of updated it for the swine flu pandemic of 2009-2011, and then there was a large gap until now, when you’ve had two updates within a couple of years. And I think that’s quite interesting because there’s a lot of interest in the interventions within this review. Let me just say what physical interventions are. They are screening at entry ports, isolation, quarantine, physical distance, personal protection, hand hygiene, face masks, glasses and gargling. That’s a lot of interventions but I’m sure most people out of all those will be really interested in terms of the masks results, so I’ll come to them later. But first, what did you include differently in this review compared to just two or three years ago, what new studies are included, and what difference has it made?
TJ: The original review had randomised and non-randomised evidence, but when we got to 2020 we had 67 trials, it had grown exponentially, with all sorts of physical interventions, so we decided there was no point in looking at low quality evidence which was difficult to interpret and no conclusions could be drawn from, if we had this massive wealth of trials. So in 2020 we went forward only with randomised control trials, and we had 67. We’ve since added another 11, so we’ve got 78 in this update. Forty-three trials contribute to the metanalysis, that’s the statistical pooling and analysis of the results. So 11 studies added in this update and the total of the participants in the whole review is 610,872, so it is a very huge dataset of randomised data.
CH: Interesting, so we’ve got a significant amount of randomised controlled data so we’ll come back to your observational data later. But here’s the first result I want to go to, medical and surgical masks compared to no mask. And what you say in the results is that we included 12 trials, and it says that wearing masks in the community probably makes little or no difference to the outcome of influenza-like illness (ILI) or COVID-19-like illness compared to not wearing masks. Now could you just decode what that actually means, because there are lots of Covid-like or influenza-like illness, and what that result means now in the context of this new evidence
TJ. The result means that regardless of what pathogen or what presenting symptom there is no evidence from high quality studies that either medical or surgical masks make any difference to transmission, which is the whole point of wearing or not wearing a mask or any of these other interventions like hand-washing.
CH. OK. And then the other outcome you’ve got here is that wearing masks makes little or no difference to the outcome of laboratory confirmed influenza or SARS-CoV-2 compared to not wearing masks. And the relative risk there is 1.01, right on the line of no effect, with six trials, nearly 14,000 participants and moderate certainty evidence. So when you say moderate certainty evidence, what does that mean also in the context of that what I consider is an important result because it’s got an objective outcome
TJ. The outcome is objective because the trials, such trials, actually did confirm the presence of those particular agents. What the result means is that irrespective of the agent that we are looking at, whether it’s a known agent , or whether it’s within the influenza-like general ILI box, it makes no difference, there doesn’t appear to be any convincing evidence that masks make any difference to transmission. They may do, but the evidence is not present from trials at present. And they may do if mixed up with other interventions but we’ll come to that later because there are some comparisons for that.
CH: OK, and then I’m just going to cover some of the others. You’ve got medical/surgical mask compared to no mask, and it probably makes no difference, so a similar outcome. We’ve got N95 masks, and we’re very uncertain on the effects of those compared with surgical masks on clinical respiratory illness. But let’s just before we get to what this means, hand hygiene compared to control, what did you find there with hand hygiene?
TJ. Well, hand hygiene and disinfection of surfaces are the two most promising interventions. Now by that I don’t mean that people should do backflips – the effect of either is quite small. But for instance, in schools where children at breaks were supervised by teachers in these trials, there was a 10-11% decrease on average of influenza-like illness. However, the problem with schoolchildren and the problem with hand washing is that if you don’t have a programme which institutionalises handwashing, the effect seen in the trial soon fades and children go back to not washing their hands. The same goes for surfaces, for disinfecting surfaces.
CH. Now look, I’m going to take you to task here. In the author conclusions people are going to read this review and start to look at this and say, look, we’ve got the high quality evidence, we’ve got randomised controlled trials and particularly at the mask level they’re going to say, look, you’re showing in the community this lack of effect, but you start with the high risk of bias in the trial, variation in outcome measurement, and relatively low adherence with the intervention during the studies, which hampers us drawing firm conclusions. Now I push that point because the obvious answer then is to go to all the observational studies where people have done systematic reviews and certainly drawn firm conclusions about what to do. So could you just elaborate on what that means in the context of 78 trials – that’s a lot of randomised control trial evidence – can you elaborate on what that means
TJ. It’s called caution, and it’s called being honest with the evidence that we have found. This is the best evidence that we have, but unlike some of the ideologists pushing the idea that non-randomised studies, observational studies could give answers, some of them come up with sweeping answers, sweeping statements, certainties, which simply do not belong to science. Science is not about certainty, science is about uncertainty, it’s about trying to move on the agenda, and accumulate knowledge. The use of non-randomised studies in respiratory virus assessment of interventions with respiratory viruses means that people do not understand, those who did those studies do not understand the play of several factors. For instance the seasonality, for instance the capricious comings and goings of these agents, they’re here one day, and gone the next. If you look at the SARS-CoV-2 behaviour in the U.K. surveillance for the last 12 months its up and down, and it’s just completely independent of any intervention, and also it’s very quickly up and very quickly down. Observational studies cannot account for that. Also, a very large proportion of observational studies are retrospective, and so they are subjected to merciless recall bias; researchers draw conclusions from data that they got from asking questions such as “Can you remember a month ago how many times you wore a mask” or “hat you did on this or what you did on the other day” without keeping a diary. This is just simply not science. Inferring meterage, distancing, when the original studies did no such thing. So this is just an endless list of bias which cannot be taken into account by observational studies. And the only way that we have to answer questions is to run large prospective randomised control trials to answer a specific question in a specific population.
CH: OK well look, when you look at the grey tables – and I find them very helpful – of viral respiratory illness, confirmed influenza or SARS-CoV-2, you said the risk with no mask is 40 per thousand, that’s a study population, and it says the risk in randomised studies with the mask is 40 per thousand, and that is the line of no effect, in effect. But that effect could be somewhere between 29 to 57 per thousand, so it could increase the risk or it could also decrease it by about 10 per thousand, and that’s moderate certain evidence. So let’s say we take the best plausible effect and do more research, and you find out you get 10 per thousand less infections within the course of this – most of this is short term evidence. What that shows to me is that the best you could assume is there is a small potential benefit when you step outside the door for individuals, but at a population level in the community this is not going to change the course of the pandemic. And to be honest that’s exactly what we have seen out there in the world, if you did an observational study that you followed for long enough. And this is moderate evidence, I don’t think you’re ever going to get to high quality evidence because you’re relying on things like adherence, which reflects the real world and not the quality of the outcome. I think people will measure different outcomes, because here you’ve got SARS-CoV-2, influenza outcomes, so you may never get beyond moderate quality evidence and this is as good as it gets, what would you say to that?
TJ: I would agree, but also the underlying principle of wearing masks outside is that we know how SARS-CoV-2 and the other respiratory viruses are transmitting for certain – and we don’t. Readers of our riddles will know that the evidence is complex, it goes back 100 years, it’s sometimes contradictory, and the transmission is probably situational. In some studies you see a quite close pointer towards close contact and droplets, but the so-called airborne aerosol route simply doesn’t have any convincing evidence if you use the same meter to judge it. And it’s not a certainty, it never is a certainty, it cannot be a certainty, it has to be a probability. Or, if you’re using legal jargon, its ‘more likely than not’ or ‘less likely than not’, less likely than yes.
CH: I’m just going to finish with one final question. You end with the need for large well-designed RCTs addressing the effects of these interventions in multiple settings. I’d agree with that totally. Schools for instance is a helpful setting, you’ve got workers in different settings, whether it’s shops or even in certain settings like care homes, right now what difference does it make. The question I have, which is my final question, is: Why have we failed to do those randomised control trials, and for instance here in the U.K. we haven’t done a single randomised study for masks, and I’m asking you why do you think that is?
TJ: I think it’s because there’s a very strange going-on because for years and years and years the Chief Medical Officers and their departments are supposed to have been preparing for the next pandemic, they even had some high-placed people doing that, importantly, and yet they had completely failed to invest in assessment and development of physical interventions, new physical interventions, and even new materials and new technologies. They have completely failed to do that, and the backlash from that has been that those two weeks in March when they all were saying that masks and other forms of personal barriers didn’t work and then they changed their minds and went into this mask mandate mode, which appears to have been instigated by a few very vocal, strident observers, I wouldn’t call them scientists, and instead of trying to answer the question with an emergency protocol and randomising the population, or large chunks of population, explaining the limits of our knowledge, they went for certainty. They went “do this and nothing will happen to you”; well we know what happened. Something did happen to society but it wasn’t the minimising of the circulation of SARS-CoV-2, and this is a story that needs to be told and retold many times and in different way so that next time round maybe somebody will take notice.
Dr. Carl Heneghan is the Oxford Professor of Evidence-Based Medicine and Dr. Tom Jefferson is an epidemiologist based in Rome and lead author of the latest update to the Cochrane review of physical interventions to interrupt or reduce the spread of respiratory viruses. This is a transcript of an interview published on Trust The Evidence, which you can subscribe to here.
Stop Press: For another interview with Dr. Tom Jefferson, see this article by Dr. Maryanne Demasi on her Substack.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.