On February 17th fact-checker Iria Carballo-Carbajal, a neuroscientist by training, but apparently without any education in epidemiology, published a ‘fact-check’ article on the Health Feedback website. In her headline, Carballo-Carbajal makes the following statement: “Multiple studies show that face masks reduce the spread of COVID-19; a Cochrane review doesn’t demonstrate otherwise.”
This article is now being used by social media companies to suppress all references to the Cochrane study. I became aware of this on March 10th when I received a notification that a post by a member of a Facebook group I manage contained ‘false information’. The post referred to an opinion piece in the New York Times, published on February 10th. The ‘independent fact-checker’ resource referred to in the notification was the aforementioned article by Carballo-Carbajal. Getting a fact-check stamp can be a serious issue for a newspaper, no less so for a scientific institution. Therefore, it didn‘t come as a surprise that already on March 10th Cochrane editor Karla Soares-Weiser published a statement trying to downplay the study results and incorrectly claimed the study only aimed at assessing the effectiveness of interventions to promote mask wearing, when, in fact, the clearly stated objective of the study was to assess the effectiveness of the physical interventions themselves, not only the effectiveness of their promotion.
The same day the New York Times published a piece claiming in the headline that masks work, dedicated to smearing Cochrane study author Dr. Tom Jefferson. For example, the article claims Jefferson stated in an interview that there is no evidence the SARS-CoV-2 virus is airborne, when what he actually said is that there are many transmission routes and further evidence is needed to ascertain precisely how transmission occurs.
This chain of events is a shockingly clear example of how the censorship industry works. It is all the more shocking considering how seriously flawed Carballo-Carbajal‘s ‘fact-check’ is, both factually, logically and ethically.
- The strawman
Carballo-Carbajal begins by creating a strawman, in this case a claim she attributes to Dr. Robert Malone, referring to a recent post on his blog. Under the heading ‘Claim’, the purported claim is set out as follows: “Face masks are ineffective at reducing the spread of COVID-19 and other respiratory diseases, a Cochrane review demonstrates.” This claim, quoted beside a picture of Dr. Malone, is nowhere to be found in his blog post.
As if this were not enough, Carballo-Carbajal continues, presenting what she calls a ‘full claim’: “Review ‘failed to find even a “modest effect” on infection or illness rate’; ‘the CDC Grossly Exaggerated the Evidence Supporting Mask Mandates’.”
The problem with this is that while Dr. Malone is correctly quoted in the first part of the paragraph, the second is something he simply does not say in his blog post, in spite of having quotation marks around it.
- The ad hominem
Carballo-Carbajal then attacks Dr. Malone, claiming he has spread “misinformation about COVID-19 vaccines”, referring to another article, also published by Health Feedback. What does the purported misinformation consist of, according to that article? The article is a ‘fact-check’ of a Washington Times opinion piece by Dr. Malone and Dr. Peter Navarro published in 2021, where they argue against the U.S. Government’s universal vaccination policy, saying it is based on four flawed assumptions: first, that universal vaccination can eradicate the virus; second that the vaccines are highly effective; third that they are safe; and fourth that vaccine-mediated immunity is durable.
Carballo-Carbajal could hardly have been less lucky with her reference. It is now abundantly clear that universal vaccination cannot eradicate the virus, that vaccine-mediated immunity wanes very quickly, even to the point of becoming negative, as infection studies and reinfection studies have already shown. The fact that the vaccines are not “(near) perfectly effective”, quoting Malone‘s and Navarro’s article, is now abundantly clear; it is in fact the reason why they cannot eradicate the virus. As for the third point, this is what Malone and Navarro say in their article: “The third assumption is that the vaccines are safe. Yet scientists, physicians, and public health officials now recognise risks that are rare but by no means trivial. Known side effects include serious cardiac and thrombotic conditions, menstrual cycle disruptions, Bell’s Palsy, Guillain Barre syndrome, and anaphylaxis.” In other words, they aren‘t safe, they have many known rare side-effects, and this is becoming clearer as time passes.
In short, Carballo-Carbajal tries to disqualify Dr. Malone by accusing him of ‘misinformation’ about something other than the subject of her article. This is the classic ad hominem tactic almost universal in ‘fact-check’ pieces. Her failure is spectacular, as all the purported pieces of ‘misinformation’ are now already verified facts.
- The argument
Carballo-Carbajal‘s main summary (including “Details” and “Key take away”) goes as follows:
Claims that face masks are ineffective at reducing the spread of COVID-19 based on a Cochrane review didn’t take into account the limitations of the review. While many users presented this review as the highest-quality evidence, the individual studies it evaluated varied greatly in terms of quality, study design, populations studied, and outcomes observed, which prevented the authors from drawing any definitive conclusions.
Randomised controlled trials are considered the gold standard when assessing the effectiveness of an intervention. However, this type of study can vary greatly in quality, particularly in complex interventions such as face masks, affecting the reliability of the results. In this context, many scientists consider that randomised controlled trials should be seen as a part of broader evidence including other study designs. When taking those studies into account, evidence suggests that widespread mask usage can reduce community transmission of SARS-CoV-2, especially when combined with other interventions like frequent handwashing and physical distancing.
I shall break up this statement into parts and then examine the validity of each part. We must keep in mind that the source quoted is Dr. Malone‘s blog post, thus any reference to “claims” must be to Malone‘s blog post, which is the only source quoted. References to unidentified sources, such as “many websites and social media posts” must be disregarded because no references are provided:
- Statement: Dr. Malone claims the Cochrane review shows masks are ineffective at reducing the spread of COVID-19.
Discussion: As shown above, Dr. Malone does not make this claim. Instead he claims the study “failed to find even a ‘modest effect’ on infection or illness rate”. There is a critical distinction between claiming A doesn‘t work and claiming A has not been proven to work. The two do not have the same meaning.
Verdict: Carballo-Carbajal‘s statement is false.
- Statement: Dr. Malone does not take into account the limitations of the review when making this claim.
Discussion: To begin with, Dr. Malone never makes the claim referred to, but a different claim. Notwithstanding this, in his blog post he clearly cites the study authors’ disclaimer on the uncertainty about the effects of face masks: “The low-moderate certainty of the evidence means our confidence in the effect estimate is limited, and that the true effect may be different from the observed estimate of the effect... [T]he high risk of bias in the trials, variation in outcome measurement, and relatively low compliance with the interventions during the studies hamper drawing firm conclusions and generalising the findings to the current COVID-19 pandemic.” It is therefore untrue that Dr. Malone “didn’t take into account the limitations of the review”.
Verdict: Carballo-Carbajal‘s statement is false.
- Statement: [T]he individual studies […] evaluated [in the review] varied greatly in terms of quality, study design, populations studied, and outcomes observed […] prevented the authors from drawing any definitive conclusions.
Discussion: The study results are clear: “There is low certainty evidence from nine trials (3,507 participants) that wearing a mask may make little or no difference to the outcome of influenza-like illness (ILI) compared to not wearing a mask (risk ratio (RR) 0.99, 95% confidence interval (CI) 0.82 to 1.18. There is moderate certainty evidence that wearing a mask probably makes little or no difference to the outcome of laboratory-confirmed influenza compared to not wearing a mask (RR 0.91, 95% CI 0.66 to 1.26; 6 trials; 3005 participants). … The use of a N95/P2 respirator compared to a medical/surgical mask probably makes little or no difference for the objective and more precise outcome of laboratory-confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; moderate-certainty evidence; 5 trials; 8407 participants).“
Those results are repeated in the Authors’ conclusions, adding the disclaimer that “[T]he high risk of bias in the trials, variation in outcome measurement, and relatively low compliance with the interventions during the studies hamper drawing firm conclusions and generalising the findings to the current COVID-19 pandemic.“
This disclaimer is the straw on to which Carballo-Carbajal clings with all her might. But as the lead author of the study has explained, this does not change the results of the study, it only states that the results may be affected by uncertainties arising from limitations in the studies used. In his own words: “It’s called caution, and it’s called being honest with the evidence that we have found. This is the best evidence that we have” (see reference below).
It looks as if Carballo-Carbajal does not understand the meaning of a disclaimer in a scientific paper. Instead, she tries to use this to invalidate the study results and back up her claim that masks work, despite the lack of evidence. A disclaimer in a study does not invalidate its results.
Verdict: Carballo-Carbajal‘s statement is misleading.
- Statement: Randomised controlled trials are considered the gold standard when assessing the effectiveness of an intervention.
Discussion: The reference on which this statement is based is Dr. Malone‘s blog. While this statement may well be true, inferring that something is generally “considered the gold standard“, based on the opinion of one scientist, is a grave error.
Verdict: Carballo-Carbajal‘s statement logically invalid.
- Statement: Gold standard studies vary greatly in quality.
Discussion: This claim is not backed up by any evidence. It may be true, or it may not.
Verdict: Carballo-Carbajal‘s statement is not supported by evidence.
- Statement: Many scientists consider that randomised controlled trials should be seen as a part of a broader evidence.
Discussion: The source for this is an article in The Conversation by three epidemiologists and one expert in primary healthcare. The authors certainly make this claim, but without citing any references. Thus, the statement that it is based on the opinion of “many epidemiologists” is simply false. This is a claim made by three epidemiologists and considering the vast number of people with that training, the word ‘many’ is not warranted. It must be added that appeal to numbers (argumentum ad populum) is a logical fallacy.
Verdict: Carballo-Carbajal‘s statement is not supported by evidence. Its relevance is based on argumentum ad populum, a logical fallacy.
- Statement: When studies that do not fulfil the requirements of gold standard research are taken into account, they show widespread mask usage can reduce community transmission.
Discussion: It is of course true that by lowering the standard you may get different results, but this statement is problematic, for Carballo-Carbajal seems to draw the conclusion from it that despite the result of the Cochrane review, masks do in fact prevent transmission. This is evident from this passage, toward the end of the article: “A growing body of evidence from RCTs and observational studies suggests that consistent mask-wearing can effectively reduce the spread of respiratory viruses like SARS-CoV-2 in both healthcare and community settings. … For the time being, face masks are another layer of protection in addition to vaccination, frequent handwashing, and physical distancing when the circulation of respiratory viruses is high.“
This means Carballo-Carbajal‘s claim is not only that low-quality studies suggest something, but that that what they suggest is actually true. This claim is even clearer in her headline: “Multiple studies show that face masks reduce the spread of COVID-19.” A subtle difference on the surface, but an all-important one. It means it is justified to rephrase the original statement as: “When studies that do not fulfil the requirements of gold standard research are taken into account, they show widespread mask usage can reduce community transmission and this is a valid conclusion.”
This brings us to the question of why the low-quality studies Carballo-Carbajal quotes were not included in the Cochrane review. Luckily, we have a transcript of a detailed interview between the lead author of the study, Dr. Tom Jefferson (JF) and Dr. Carl Heneghan (CH), where this is discussed in detail:
CH. Now look, I’m going to take you to task here. In the author conclusions people are going to read this review and start to look at this and say, look, we’ve got the high-quality evidence, we’ve got randomised controlled trials and particularly at the mask level they’re going to say, look, you’re showing in the community this lack of effect, but you start with the high risk of bias in the trial, variation in outcome measurement, and relatively low adherence with the intervention during the studies, which hampers us drawing firm conclusions. Now I push that point because the obvious answer then is to go to all the observational studies where people have done systematic reviews and certainly drawn firm conclusions about what to do. So could you just elaborate on what that means in the context of 78 trials – that’s a lot of randomised control trial evidence – can you elaborate on what that means?
TJ. It’s called caution, and it’s called being honest with the evidence that we have found. This is the best evidence that we have, but unlike some of the ideologists pushing the idea that non-randomised studies, observational studies, could give answers, some of them come up with sweeping answers, sweeping statements, certainties, which simply do not belong to science. Science is not about certainty, science is about uncertainty, it’s about trying to move on the agenda, and accumulate knowledge. The use of non-randomised studies in respiratory virus assessment of interventions with respiratory viruses means that people do not understand, those who did those studies do not understand the play of several factors. For instance the seasonality, for instance the capricious comings and goings of these agents, they’re here one day, and gone the next. If you look at the SARS-CoV-2 behaviour in the U.K. surveillance for the last 12 months its up and down, and it’s just completely independent of any intervention, and also it’s very quickly up and very quickly down. Observational studies cannot account for that. Also, a very large proportion of observational studies are retrospective, and so they are subjected to merciless recall bias; researchers draw conclusions from data that they got from asking questions such as “Can you remember a month ago how many times you wore a mask” or “What you did on this or what you did on the other day” without keeping a diary. This is just simply not science. Inferring meterage, distancing, when the original studies did no such thing. So this is just an endless list of biases which cannot be taken into account by observational studies. And the only way that we have to answer questions is to run large prospective randomised control trials to answer a specific question in a specific population.
As Jefferson explains here, the limitations of observational studies make it virtually impossible to draw from them the conclusion that Carballo-Carbajal does. Carballo-Carbajal quotes a number of observational studies to support her claim. I will not go through all those here, but looking at a some examples should be sufficient to illustrate some of the problems Jefferson discusses, as well as refuting some of Carballo-Carbajal‘s unsubstantiated conclusions.
For example one of the studies quoted, Wang et al., concludes that face mask use by the primary case and family contacts before the primary case developed symptoms was 79% effective in reducing transmission. This is a retrospective observational study where the evidence for mask usage is purely based on participant‘s after-the-fact own self-reporting.
Another one, Mello et al., shows how viral particles accumulate in masks, but Carballo-Carbajal takes this as evidence that “available data indicates that mask-wearing is more effective when combined with other control measures, such as physical distancing and frequent handwashing”.
To summarise, Carballo-Carbajal claims that since high-quality studies do not prove the effectiveness of masks against transmission, then unreliable observational studies, which are excluded from the “gold standard” meta-review, precisely because of their unreliability, prove what the high-quality studies fail to prove.
Verdict: Carballo-Carbajal‘s (rephrased) statement is false. Without rephrasing it is irrelevant.
- Statement: The effect of mask usage is greater when combined with other interventions.
Discussion: This statement is problematic. It is already clear from the high-quality evidence provided by the Cochrane review that the claim that masks reduce transmission is unproven. This means claiming they add to the protection provided by other interventions must be incorrect.
Verdict: Carballo-Carbajal‘s statement is false.
Summary
Carballo-Carbajal begins by falsely attributing to Dr. Robert Malone two claims which he has never made. Those false claims become the basis of her ‘fact-check’.
She then wrongly accuses Dr. Malone of making false statements regarding a different matter, an ad hominem argument irrelevant to the subject of the article.
Out of the eight claims made by Carballo-Carbajal in her summary, backed up by her main text, four are plainly false, one is logically invalid, one is misleading and two are not supported by any evidence, one of which is based on a logical fallacy.
Considering how this seriously flawed article is now being used to suppress the dissemination of important scientific research, as well as to press the Cochrane Editor-in-Chief into making false claims about the objective of the paper and downplaying its results, and to censor a review of the findings by an important mainstream newspaper, there is clearly an urgent need to push back against the ‘fact-checking’ industry. The level to which this censorship has escalated is a threat to scientific research and development.
This article was first published on Thorsteinn Siglaugsson’s Substack newsletter, From Symptoms to Causes. You can subscribe here.
Update: This article has been corrected. An earlier version incorrectly stated that Carballo-Carbajal had mis-stated the conclusion of Wang et al.
Stop Press: Eugyppius has weighed in on the New York Times article trashing the Cochrane review referred to above.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.