A new paper in the Lancet has attracted some interest, both because it claims to find that the pandemic death toll is over three times higher than official Covid death figures suggest and because it seems to confirm that restrictions made no difference to outcomes. The authors say that while “reported COVID-19 deaths between January 1st 2020 and December 31st 2021 totalled 5·94 million worldwide”, they estimate that “18·2 million people died worldwide because of the COVID-19 pandemic (as measured by excess mortality) over that period”.
However, the paper is heavily dependent on modelling, so despite the welcome implication for the ineffectiveness of lockdowns, caution is needed.
The paper aims to “estimate excess mortality from the COVID-19 pandemic in 191 countries and territories, and 252 subnational units for selected countries, from January 1st 2020 to December 31st 2021”.
The relevant data were not always available, however, so the authors “built a statistical model that predicted the excess mortality rate for locations and periods where all-cause mortality data were not available”.
Not all excess deaths are Covid deaths, of course. The authors say that although they “suspect most of the excess mortality during the pandemic is from COVID-19”, excess deaths also include deaths from lockdown, including “deaths from chronic and acute conditions affected by deferred care-seeking”. However, there are currently insufficient data to distinguish Covid deaths from other excess deaths, they say, and while audits in Belgium and Sweden have suggested that excess deaths and Covid deaths are of a similar magnitude, audits in Russia and Mexico have suggested otherwise, as a “substantial proportion of excess deaths could not be attributed to SARS-CoV-2 infection in these locations”.
The authors used an ensemble of six models to estimate expected and thus excess deaths: “Excess mortality over time was calculated as observed mortality, after excluding data from periods affected by late registration and anomalies such as heat waves, minus expected mortality. Six models were used to estimate expected mortality; final estimates of expected mortality were based on an ensemble of these models.”
These models took account of no fewer than 38 covariates, as listed in the table.
The covariates included the latitude of the country or region, for reasons not fully explained – and as Dr. Clare Craig notes, the study appears to hugely inflate figures for equatorial countries and southern U.S. states.
The authors compare their estimates to those of the Economist, which they say “provides the most comprehensive assessment of excess mortality due to COVID-19 to date”. Although the Economist comes to a similar global estimate of 18 million excess deaths, it gets there in a very different way (suggesting the similarity is something of a fluke) as its estimates for individual countries and regions are frequently dramatically different.
There are dramatic differences in the estimated excess mortality counts between the two studies for many countries. The relative difference between the estimates from each study, defined as the ratio between excess deaths from the Economist study over those from our study minus 1, ranges from −382·7% (Vanuatu) to 2282·3% (China). In terms of absolute relative difference, at least a 25% difference is observed in 129 of 187 countries. Some 23 countries have absolute relative differences between the two studies of higher than 100%… Although the global total produced by The Economist was similar at 18·0 million (95% UI 10·9–24·4) excess deaths, which is about 212,000 deaths fewer than the estimate derived in this study, country contributions to the totals varied. The Economist estimated 192,000 fewer excess deaths for Mexico, 140,000 for the USA, and 140,000 for Peru, and 1·07 million additional excess deaths for India, 409,000 for China, and 193,000 for Sudan. For sub-Saharan Africa, the absolute relative differences range from 0·6% in Gabon to 310·7% in Burundi. The absolute relative difference is at least 50% among 21 out of 46 countries in the region.
These dramatic differences obviously call into question the reliability of at least one of the models. The authors themselves admit that the input into their models “can have a sizeable impact on the estimated expected mortality for a particular location”.
Perhaps though, the more sophisticated Lancet modelling is closer to the truth than that of the Economist?
The problem with this hypothesis is that when we compare the Lancet estimates for excess mortality to those from Our World in Data (OWID) we again find huge discrepancies. OWID uses the expected mortality estimates from the World Mortality Dataset, which fits “a regression model for each region using historical deaths data from 2015-2019”. So still a model, but a more basic one than the Lancet‘s ensemble of six. It is based primarily on the pre-pandemic five-year average, while also aiming to “capture both seasonal variation and year-to-year trends in mortality”.
In the table below you can see the OWID estimate, the Lancet estimate and the ratio between them for each country where OWID has the relevant data up to the end of 2021. I have also included the official reported Covid deaths per million and the ratio of that with the Lancet estimate, which is one of the main outputs of the study.
The differences in many cases are huge. For instance, in many South American countries, such as Brazil and Chile, the Lancet estimate is around half the OWID estimate. In Ireland the Lancet thinks OWID overestimates excess mortality five-fold. On the other hand, in Japan, OWID finds negative excess, whereas the Lancet finds positive excess more than four times higher than the negative figure (a negative ratio in the table means the Lancet has found positive excess where OWID found negative, or vice-versa). In Iceland and Singapore, while OWID reports significant excess mortality, the Lancet says both countries actually have negative excess. This makes little sense. OWID uses a straightforward method for estimating expected and excess mortality, based on the five-year average. Why is the Lancet‘s modelling disagreeing with this so radically? The authors claim to have carried out “out-of-sample predictive validity testing”, which indicated “a small error rate (0·85%)”, but it’s hard to credit this when the results differ so greatly from a much more straightforward estimate of expected deaths.
In the table above, many of the ratios are below one, indicating that the Lancet thinks OWID over-counts excess mortality. Yet overall the Lancet claims that Covid deaths are under-counted by more than a factor of three. Where do all the missing deaths come from? Some come from the non-Covid excess deaths that many countries experienced in the second half of 2021. But many of them come from regions where data are lacking, as depicted below, and the authors use modelling to fill in the gaps.
However, the reliability of this modelling is cast into doubt by the huge differences between countries or states within the same region. For instance, in Africa: “Low excess mortality rates were estimated across sub-Saharan Africa, with the notable exception of four nations in southern sub-Saharan Africa: Eswatini (634·9 per 100,000), Lesotho (562·9 per 100,000), Botswana (399·5 per 100,000), and Namibia (395·6 per 100,000).” Why would these four countries have excess death tolls so much higher than others in the region?
Also in India and Pakistan: “The most extreme ratios in the region were found in the states and provinces of India and Pakistan, ranging from 0·96 in Goa, India to 49·64 in Balochistan, Pakistan.” Why would some states in the region over-count Covid deaths while others under-count them by up to 50 times? In India, the Lancet claims that Bihar under-counted pandemic deaths 27-fold, even though Goa over-counted them. Is this really plausible?
Even in countries where we have reliable data, the Lancet makes extraordinary claims of under-counting. For instance, Japan has recorded 73 Covid deaths per million, and OWID even records negative excess deaths. Yet the Lancet claims Japan has under-counted pandemic deaths more than six-fold, and that it has actually had 441 excess deaths per million – a long way from negative. Similarly in Denmark, OWID has 135 excess deaths per million, but the Lancet thinks the country has under-counted seven-fold and that the true figure is 941 excess deaths per million. That’s over three times as high as its Covid death toll of 296 deaths per million. That fails the smell test.
The whole study seems to be a modelled fantasy. One to be avoided, I suggest, however useful some of the findings might appear.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.