The ONS's New Excess Death Figures Don't Pass the Smell Test

And just like that, 20,000 fewer people died unexpectedly last year, according to the Office for National Statistics. Which is handy for a Government refusing to investigate the excess deaths crisis.

Okay, so we haven’t quite reached North Korean levels of data meddling and history editing. But this does not seem far off.

Under a new methodology, unveiled on Tuesday, the ONS cut excess deaths in 2023 from 31,442 to 10,994 – a 65% drop. This isn’t because the ONS officials suddenly discovered that all these people are actually still alive. The number of people who died last year hasn’t changed. What’s changed is the baseline for determining how many deaths you should expect to happen in a given week, and thus how many that actually occurred are in excess of that baseline.

Previously the ONS used a straightforward five-year baseline, meaning that the expected figure for a given week was simply the average number of deaths in that week in the previous five years (an exception was made following the pandemic, when the ONS dropped 2020 from its five-year averages due to its high death toll).

This simple if crude method is now to be replaced with a complicated model that tries to guesstimate how many deaths you should expect after taking into account factors like the age and size of the population.

Here is what the model looks like as an equation, where each part (i.e., each sigma Σ symbol) stands for a factor that the model is trying to account for:

Not exactly comprehensible to your average politician, journalist, medic or researcher. Indeed, even Professor Carl Heneghan and Dr. Tom Jefferson have confessed that they “do not fully understand the ONS’s new model”.

But is it an improvement?

It comes with an apparent endorsement from Dr. Jason Oke, Senior Statistician at Oxford University’s Medical Statistics Group and a long-time collaborator of Drs. Heneghan and Jefferson. Dr. Oke said:

The excess death statistic rose from relative obscurity to prominence during the pandemic, putting it firmly in the public consciousness.

This, however, also exposed the flaws in the way it had been calculated – using historic averages, taking no account of prevailing trends or changes in the population.

As a result, excess deaths were overestimated before, during and after the pandemic.

Cambridge Emeritus Professor of Statistics Sir David Spiegelhalter was full of praise, calling it a “world-leading methodology, setting an appropriately high standard for national statistics”.

So should we, like these eminent statisticians, welcome the change?

The heavy use of modelling naturally makes anyone who’s been following data in the pandemic highly suspicious. Mathematical models are, by their nature, highly dependent on the assumptions, parameters and inputs their creators feed into them. But perhaps this one is different. Let’s see.

The ONS team has recalculated excess death estimates back to 2011 – the table is shown below. Believe it or not, 2023’s cut of more than 20,000 deaths is not the biggest change, not even close. 2016 was down by over 22,000, 2017 by 21,000, 2018 by over 24,000 and 2019 by – wait for it – over 40,000 deaths. On the face of it it’s hard to credit the claim that the old 2019 baseline was out by more than 40,000 deaths. But let’s continue.

We should put the changes in context. The U.K. sees about 600,000 deaths a year, so a change of 20,000 is in the region of 3% and a change of 40,000 is around 7%. Not huge in relative terms. But equally, 20,000 or 40,000 is not a small number of deaths and can make the difference between a crisis and an ordinary year.

The changes are hugely variable. The years 2011-2013 see small increases in their excess deaths, yet 2016-19 see massive cuts. Then, in the pandemic years, the numbers suddenly stop moving around and stay largely the same (see table below) – which is, again, somewhat handy for the Government as it means it doesn’t need to revise down its scary pandemic death toll by very much.

This raises a key question: why did the underlying population trends that caused the 2019 figure to drop by over 40,000 abruptly (and temporarily) come to an end, so that the 2020 figure drops by just 8,000? Call me cynical, but that’s a big and convenient difference from one year to the next.

The ONS article explaining the change in methodology is helpfully detailed and informative. It anticipates people wondering what’s behind these huge changes and tries to give an answer. Here’s the ONS’s chart of how population size, age and mortality rate – the key variables in the model – have shifted since 2006.

Here we see a steadily increasing population (driven largely by immigration over this period) and a more sharply increasing share of the population aged over 70 – the latter growing by 35% between 2006 and 2023. Note this is a percentage increase since 2006; it does not mean that 35% of the population was over 70 in 2023. In absolute terms the U.K. over-70 population was 11.5% in 2006 and 13.8% in 2023, growing by 2.3 percentage points.

The ONS team explains that in the new model, the ageing and growth of the population counteracts a fall in background mortality rates to produce an overall change in the baseline and excess deaths. The chart below shows the contribution that each factor in the ONS model makes to the changes in excess deaths compared to the current system. The green line shows what this means for the resulting numbers of excess deaths each year.

Here’s the ONS’s explanation of what’s going on:

The U.K. population grew by 12.2% between 2006 and 2023… from 60.8 million to 68.3 million (Figure 3). All else being equal, having more people in the population each year means we can expect more deaths to occur. Furthermore, people aged at least 70 years, the group in which most deaths occur each year, made up an increasing share of the U.K. population (from 11.5% in 2006 to 13.8% in 2023), with the size of this group growing by 35.4% over the period.

Conversely, mortality rates have been decreasing over time. The age-standardised mortality rates (ASMRs) among the U.K. population generally decreased from 2006 to 2011, before levelling off from 2012 to 2018 and dropping again in 2019 (before the start of the pandemic).

These increasing trends in population size and ageing, and the generally decreasing trend in mortality rates, are not accounted for by the current methodology for estimating excess deaths. However, they are reflected in the new methodology.

Interestingly, although the 2023 figure was cut by around 20,500, the 2022 figure actually went up by around 4,500. The 2020 and 2021 figures, on the other hand, were down by around 7,500 and 7,000 respectively. Again, quite a bit of jumping around.

The chart below compares the new and current excess death estimates since the arrival of Covid. It shows that since the second half of 2023 the new estimate has been progressively dropping compared to the current estimate, so much so that an excess has become a deficit. Great news – the excess deaths crisis is over! The Government will be pleased.

While the new methodology certainly looks and sounds sophisticated, taking a step back it’s hard to make any real sense out of it. The changes it has wrought in the excess death figures – represented by the green line in Figure 4 above – are all over the place.

In Figure 3 we saw that the rises in both the general population and the over-70 population since 2006 have been steady; they don’t jump around. Why then do these steady demographic changes result in a jagged and jumping green line of excess death shifts? Why is 2015 adjusted down by 6,000 and 2016 by 23,000? Why is 2018 adjusted by 25,000, 2019 by 40,000 and 2020 by 7,500? These wildly diverging differences from one year to the next make no obvious sense when the underlying population dynamics move only steadily and slowly.

Heneghan and Jefferson in the Telegraph agree that there’s “something fishy about the ONS’s excess death figures”. They note that “while the difference in the total number of recorded deaths between 2020 and 2023 is 27,629, for example, the difference in excess deaths between these years is more than double at 65,418, suggesting that something is amiss with the new estimates”.

They also highlight that for 2022 and 2023, according to the ONS, “by far the biggest contributors to the changed figures are lumped together under the category of ‘other changes’. Quite what’s in this bucket of changes isn’t totally clear, although some of it is to do with the treatment of 2020 data”.

As ever with complex models like this it’s hard to get to the bottom of what exactly is going on, and hence why the model is spewing out such counterintuitive results. But with a string of apparently implausible outputs, it’s reasonable to suspect that the inputs may be faulty. I am wary in particular of the model’s reliance on age-standardised mortality rates (ASMR) and the European Standard Population which I, like statistics expert Professor Norman Fenton, am not convinced are reliable tools for adjusting data.

A further problem is that, despite its complexity, the model fails to take into account one of the biggest factors in play: mortality displacement, i.e., the periods of lower excess deaths that (should) naturally follow on from periods of high excess owing to vulnerable people dying earlier than expected. The ONS article says the team will be looking at including this in a future development of the model. It would, of course, significantly increase the number of excess deaths since 2020.

It should also be noted that the extra deaths in 2023, like in 2022, were not due to the usual diseases of old age, such as Alzheimer’s and Parkinson’s, which have been in deficit, but were largely due to cardiovascular problems. Deaths have also been up in the younger age groups, not just in the elderly. Both these things count against the idea that an ageing population is playing a large role in recent deaths such as would warrant raising the baseline to cover over them.

With the 2023 figure slashed by 20,000 and data in the second half of the year retroactively altered to transform an excess into a deficit, there is a legitimate worry that the excess deaths crisis is being artificially modelled to a close.

Nonetheless, it is worth stressing that even with these changes, there were still 11,000 excess deaths in 2023 and 43,500 in 2022 – years when Covid was only a minor player and everyone was supposedly protected from the virus by their multiple vaccine boosters.

The ONS can fiddle. But even on the new estimates, tens of thousands of people died unexpectedly in the past two years, and the Government is still showing no interest in finding out why.