The new ONS data on deaths by vaccination status seem to have a bias such that deaths in the unvaccinated are more likely to be included in the ONS sample, whereas deaths in the vaccinated have the opposite bias and are more likely to be excluded from this dataset. Oddly, the bias is different for Covid deaths. Differences with the accuracy of record matching could be enough to explain the bias.
The ONS data are based on only a sample of the population, albeit a large one. If the sample were representative we would find the mortality rates in the population included and excluded from the sample to be the same. But it turns out this is not the case.
For simplicity, going forward those excluded from the sample are referred to as the ‘ghost population’. Analysis which compares the mortality rates (per 100,000 people) between these groups shows a systematic bias.
For every age group, the mortality rate of unvaccinated people in the ghost population was lower than the ONS sample mortality rate, meaning the ONS sample over-represents deaths in the unvaccinated. Conversely, the mortality rate for the vaccinated ghost population was higher than the ONS sample for every age group except the over-80 year-olds, meaning the ONS sample under-represents deaths in the vaccinated under-80s.
For example, figure 2 shows the mortality rates in the 40-49 year-old population. Graphs for all age groups can be found here.
Many people have claimed that it is invalid to use the vaccination database (NIMS) as a measure of the whole population because it may contain duplicate records such that the population is overestimated. If that were the case, then the mortality rate in the vaccinated ghost population will have been artificially lowered by exaggerating the size of the vaccinated population. This would show up as a persistently lower than expected mortality rate in the vaccinated ghost population.
However, the mortality rate for the ONS sample and the ghost population (those excluded from the ONS sample) converge in later months to the same figure for the 18-39 year old group (see figure 3 below) indicating that all four populations are in fact comparable, and the vaccinated ghost population (and thus the vaccine database population) is not overestimated. A bias in population size could not disappear for a period of time whereas a bias due to misclassification of records may well vary over time.
The bias between the mortality in the ONS sample and in the ghost population suggests that deaths in the vaccinated have been disproportionately excluded from the ONS data, while deaths in the unvaccinated were disproportionately included.
Data for deaths with Covid show a totally different bias. For Covid deaths, the mortality rate is substantially lower in the ghost population regardless of vaccination status, meaning the ONS sample over-represents Covid deaths for both vaccinated and unvaccinated. There are two important implications of this. Firstly, any kind of human bias in how data were assigned is unlikely to have resulted in a bias one way for all cause deaths and the opposite way for Covid death. Secondly, there must be something different about how all-cause and Covid deaths are being recorded that results in this opposite bias depending on the cause of death.
One major difference between Covid deaths and all-cause deaths is the proportion that occur in hospital, which is 44% for Covid vs 71% for all-cause. The ONS has previously said that 94.6% of its ONS records match to the NHS database.
If we make the simple assumption that a higher proportion of death certificates for in-hospital deaths are correctly matched to an NHS number and vaccination status than for deaths outside hospital, we can recreate this bias. In fact, even an assumption of 95% matching for hospital deaths and 94% for deaths outside of hospital is a sufficient difference to create most of these biases. The only oddity that could not be replicated in this way was having a significantly higher vaccinated mortality rate than unvaccinated mortality rate in the excluded ghost population.
The ONS recorded anyone as unvaccinated if they did not match to a vaccine record. That means there will be a risk of vaccinated deaths being wrongly classified as unvaccinated if their vaccine record is missing, which would artificially increase the unvaccinated mortality rate and decrease the vaccinated mortality rate. While this effect may be small in the large vaccinated cohort, it may make a significant impact in the small unvaccinated cohort.
If we take a hypothetical population with identical mortality rates for the ONS and ghost populations, then the discrepant mortality rates seen above can be largely created simply by having a higher failed match-rate to the vaccine database for in-hospital deaths compared to other deaths. The ranking of mortality rates is shown in table 1. Even only a minimal difference in matching has this effect.
Whatever the cause of the bias, it requires thorough investigation in order for people to be properly informed about the effectiveness of the Covid vaccines. As it stands, the ONS dataset exaggerates deaths in the unvaccinated and underplays deaths in the vaccinated.
Dr. Clare Craig is a diagnostic pathologist and Co-Chair of the HART group. This article was first published by HART.
Profanity and abuse will be removed and may lead to a permanent ban.