The accuracy of any data purporting to show vaccine effectiveness or safety against a disease is critically dependent on the accurate measurement of: people classified as having the disease; vaccination status; death reporting; and the population of vaccinated and unvaccinated (the so called ‘denominators’). If there are errors in any of these, claims of effectiveness or safety are unreliable.
The risk/benefit of Covid vaccines is best – and most simply – measured by all-cause mortality of vaccinated against unvaccinated, since it avoids the thorny issue of what constitutes a Covid ‘case/infection’. In principle, the data in the ONS vaccine mortality surveillance reports should provide us with the necessary information to monitor this crucial comparison over time. However, until the ONS released its November report, no age categorised data were provided, meaning that any comparisons were confounded by age (older people are both disproportionately more vaccinated than younger people and disproportionately more likely to die).
The week 44 ONS report and data release from November finally provided some relevant age categorised data. Specifically, it includes separate data for age groups 60-69, 70-79 and 80+, but there is only a single group of data for the age group 10-59. After the November data release the ONS released further data on December 20th 2021, albeit at a significant lower level of granularity that inhibits cross comparison with earlier data (different age categories; monthly rather than weekly data; age-adjusted mortality rather than raw death and population data; death counts updated; and fractional membership of vaccination category based on time spent in category) and with different categories for vaccine status than those used in November (five categories rather than four with double dose vaccinated split into less than and greater than 21 days).
At first glance the data suggest that, in each of the older age groups, all-cause mortality is lower in the vaccinated than the unvaccinated. In the 10-59 age group all-cause mortality is higher among the vaccinated, but this group is likely confounded by age since it is far too wide for the data provided to be sufficient to draw any firm conclusions.
However, despite this apparent evidence to support vaccine effectiveness for the older age groups, on closer inspection this conclusion is cast into doubt. That is because we have shown a range of fundamental inconsistencies and flaws in the data. Specifically:
- In each group the non-Covid mortality rates in the three different categories of vaccinated people fluctuate in a wild, but consistent way, far removed from the expected historical mortality rates.
- Whereas the non-Covid mortality rate for the unvaccinated should be consistent with historical mortality rates (and if anything, slightly lower than the vaccinated non-Covid mortality rate), it is not only higher than the vaccinated mortality rate, but it is far higher than the historical mortality rate.
- In previous years, each of the 60-69, 70-79 and 80+ groups have mortality peaks at the same time during the year (including 2020 when all suffered the April Covid peak at the same time). Yet in 2021 each age group has non-Covid mortality peaks for the unvaccinated at a different time, namely the time that vaccination rollout programmes for those cohorts reach a peak (see below).
- The peaks in the Covid mortality data for the unvaccinated are inconsistent with the actual Covid wave.
- There are sufficiently serious anomalies in the population and very poor health category data to suggest the data are unreliable.
Whatever the explanations for the anomalies, it is clear that the data is unreliable and conclusions regarding vaccine efficacy specious. Likewise, given the ONS’s suggestion in its December report that the anomalies are the result of vaccinations being denied to moribund or terminally ill patients, or that there is a healthy vaccinee effect, we tested this hypothesis and found it was not plausible. The onus is now on those who propose this explanation to demonstrate empirically how it works. We considered the socio-demographic and behavioural differences between vaccinated and unvaccinated that have been proposed as possible explanations for the anomalies but found no evidence supporting any of these explanations. By Occam’s razor we believe the most likely explanations are:
- Systematic miscategorisation of deaths between the different groups of unvaccinated and vaccinated.
- Delayed or non-reporting of vaccinations.
- Systematic underestimation of the proportion of unvaccinated.
- Incorrect population selection for Covid deaths.
With these considerations in mind, we applied adjustments to the ONS data and showed that they lead to the conclusion that the vaccines do not reduce all-cause mortality, but rather produce genuine spikes in all-cause mortality shortly after vaccination.
There are, of course, some caveats to our analysis. While we have completely ignored the 10-59 age group because it is far too broad so age confounding would likely overwhelm any conclusions, the age groups 60-69, 70-79, 80+ are themselves quite coarse, and there may be some age confounding within these age groups. For example, the average age of the vaccinated 60-69 age group may be higher than that of the unvaccinated 60-69 group and hence the number of deaths would naturally be slightly higher.
We have deliberately chosen not to subject the data to a degree of sophisticated statistical or probabilistic modelling but can readily imagine what might be done. We have carried out some basic computations of confidence intervals to address the fact that at various points the population sizes differ dramatically, and from this the patterns reported remain visible, significant and our analysis credible.
Ultimately, our analysis is hypothetical insofar as it presents two processes, one based on the risk presented by the period before/after vaccination and infection and one based on categorisation, both of which might better explain the patterns in the data. However, we believe it is up to those who offer competing explanations to explain how and why the data is the way it is. We have explained that various social and ethnic factors are very unlikely to explain these odd differences in the ONS data set. Same with the moribund or healthy vaccinee effects. Absent any other better explanation, Occam’s razor would support our conclusions. In any event, the ONS data provide no reliable evidence that the vaccine reduces all-cause mortality.
This article is taken from the paper “Official mortality data for England suggest systematic miscategorisation of vaccine status and uncertain effectiveness of COVID-19 vaccination“, of which it forms the summary and conclusion. Read the full paper here.
Dr. Martin Neil is Professor in Computer Science and Statistics at Queen Mary, University of London. Dr. Clare Craig is a Diagnostic Pathologist. Dr. Norman Fenton is Professor of Risk Information Management at Queen Mary University of London. The paper is also authored by Jonathan Engler, Joshua Guetzkow, Scott Mclachlan, Jessica Rose, Dan Russell and Joel Smalley.
Profanity and abuse will be removed and may lead to a permanent ban.