Over the past 15 months we’ve had a barrage of statistics presented to us shouting about how great the vaccines are at preventing hospitalisation from (or with) Covid. However, these statistics have been light on detail on how they were calculated and we’ve not seen much sight at all of the raw data that the statistics were based upon.
Until now. In April, a paper was published by the UKHSA (currently in pre-print, which means that it hasn’t yet undergone the usual peer-review process) on its statistical analysis of a selection of hospitalisation data by vaccination status. The intent of this paper was to support its statements that the vaccines prevent hospitalisation. However, the paper also includes the raw data upon which the UKHSA statistics were derived, and these data tell a very different story to that presented by the UKHSA. The data show:
- Far higher accident and emergency admission rates for reasons other than Covid in the vaccinated than in the unvaccinated.
- Much higher rates of hospitalisation due to non-Covid acute respiratory illness in the vaccinated.
- Even higher A&E admissions and hospitalisations in the double-vaccinated (not boosted).
- Even where the data suggest that the vaccines offer some protection (the risk of admission to intensive care resulting from Covid infection) the results look like they might be an artefact created by the assumptions used by the UKHSA.
In addition, the data strongly suggest that the UKHSA is using an inappropriate method in its statistical analysis of vaccine effectiveness – the test-negative case-control (TNCC) method. It is likely that it has been significantly overestimating the effectiveness of the vaccines at preventing hospitalisation.
The data released by the UKHSA and expounded below aren’t proof that the vaccines have caused a great deal of harm and injury to the population, but they do raise a huge red-flag that something is amiss. Urgent investigations need to be undertaken to clarify the situation regarding the safety and effectiveness of the vaccines.
A note on the Test-Negative Case-Control (TNCC) method
The basic concept of measuring vaccine efficacy (in a trial) or effectiveness (based on real-world results) is relatively straightforward – simply calculate the ratio of the proportion of the vaccinated that get infected (or hospitalised) with the proportion of the unvaccinated that get infected (or hospitalised). However, while this simple method can work well, it can be affected by differences in the types of people vaccinated and unvaccinated and more powerful methods are preferred.
The ‘gold standard’ for measurement of vaccine efficacy/effectiveness (VE) is the prospective matched cohort design. This is quite simple in concept – you simply divide the study into a number of groups of individuals with similar characteristics, based on the vulnerability to the disease and the typical effectiveness of vaccines. Matched cohort studies nearly always split into groups of similar age and sex, and will usually include many other factors thought to be related to risk of disease. For example, for Covid these might include race, BMI and whether the individual has diabetes – all factors identified in early data as being relevant risk factors.
The ‘prospective’ part means that the individuals are placed into their groups before they are given their doses of vaccine, but this isn’t the only way – it is possible to undertake a retrospective study where people are placed into their different groups after they are vaccinated (potentially some time after).
The problem with matched cohort trials is that they’re rather expensive and also require you to know what factors to control for when calculating vaccine efficacy/effectiveness. This led to the development of the test-negative case-control method. With this method you compare the ratio of positive (have the disease) to negative (don’t have the disease) tests results for a given condition (e.g. admission to hospital). This method, when applied correctly, automatically corrects for many biases, such as propensity to be tested or seek medical care, and is both accurate and removes much of the complexity and costs associated with matched-cohort studies. The important part is the ‘when applied correctly’ – if it is applied incorrectly then you end up with inaccurate and potentially misleading results.
The UKHSA data – Emergency admissions
Let’s apply the TNCC method to the data in the UKHSA paper on vaccine effectiveness at preventing hospitalisation. Fortunately, it offers raw data in its supplementary document. I’ll start with hospitalisations ‘with symptomatic Covid’ for those aged over 65 who presented to A&E for reasons other than accident/injury (Table S12 in the paper), and to keep things simple will look at ‘any vaccine’ (i.e., any number of doses) vs ‘no vaccine’ and only for the Omicron period (the data covers the period from 22nd November to 2nd February).
To show the TNCC method in action we can use the figures in the table above to gain an estimate of VE using the TNCC methodology:
VE = 100 × (1 – (873÷140,931) ÷ (103÷1,705) ) = 90%
Thus even with this simplified case where we only consider the protection offered by the vaccines to the ‘vaccinated group’ (with any number of doses, though most are boosted in the over-65 age group), we can see that TNCC estimates that the vaccines offer significant protection from hospitalisation, around 90%.
But wait – those raw numbers for A&E presentation by vaccination status look more than a little suspicious. We know that during the study period approximately 10 million individuals aged over 65 had been vaccinated with at least one dose of vaccine, and around 600,000 remained unvaccinated. Thus we can present the raw figures above as a ‘per 100,000’ to remove the effect of the size of the vaccinated vs unvaccinated groups.
Wow. According to the raw data the vaccinated are presenting to A&E without having Covid at around five times the rate of the unvaccinated. Sure, there are more hospitalisations with symptomatic Covid in the unvaccinated, but only by eight per 100,000.
In Table 2 above I have also included an estimate of vaccine effectiveness based on these raw data. Now, I’m being a bit naughty here as the data aren’t meant to be used this way – this is why I’ve used the UKHSA trick of greying out the text in the hope that no-one will notice. Nevertheless, for population-wide data this shouldn’t be too far out.
Now, I’m sure that epidemiologists up and down the land are shouting that the data shouldn’t be used in this way – and they’re right. It certainly doesn’t prove that the vaccinated are getting ill because of the vaccine. There are a number of reasons why this result might be found:
- The vaccinated might be much more likely to be hypochondriacs/malingerers and thus be going to A&E even though they’re not ill at all. More realistically, the vaccinated might have a lower threshold for the severity of symptoms required to get medical assistance at A&E. If this was the case then there would be vaccinated individuals presenting themselves to A&E where the average unvaccinated person with similar symptoms wouldn’t.
- The vaccinated might be much more unhealthy in general than the unvaccinated.
However, the sheer scale of the differences between the A&E visits not-for-Covid is huge, and given that these are population-wide figures I’d suggest that it couldn’t all be explained either by health seeking behaviours or because of general health – but I’d accept they they could certainly contribute.
Nevertheless, the TNCC assumption would be that the vaccinated are simply the type of people that are five times more likely to go to A&E (whether because of differences in behaviour or health) and thus they’re also going to be five times more likely to attend A&E with symptomatic Covid. The researchers would therefore adjust the figures to allow for this difference between the groups, boosting VE. I’d suggest that this latter point isn’t necessarily the case – it is very often the case that behaviours aren’t proportional like this, for example, just because an individual chooses to drive at 40mph in a 60mph zone, doesn’t mean he or she will drive at 20mph in a 30mph zone.
The alternative explanation:
- Some of the visits to A&E might be due to a reaction / side-effect / complication of the vaccines.
- The vaccines might have an impact on the immune system for diseases other than Covid, resulting in increased illness and thus presentation to A&E.
Just to be clear – we don’t know whether the vaccinated are seeing much higher admissions rates to A&E due to a vaccine effect or simply because the vaccinated have different behaviours and general health to the unvaccinated. However, anecdotal data on pressures on A&E services and on the general health of the nation (‘worst cold ever’) suggest that the vaccines may be at least partially responsible.
More on the emergency admissions data
The UKHSA paper also includes incidence by vaccination status (Table S12 again). We have to be a bit careful here as we don’t know when the individuals were vaccinated, but we do know that the incidence of Covid varied substantially through the period. Without information on which individuals were vaccinated on which date we run the risk of introducing a bias. However, we do have information about some aspects of the vaccinated population:
- Around 600,000 individuals over the age of 65 remain unvaccinated, and this hasn’t changed much for over six months (this is why it was safe to use this assumption in the prior analysis).
- The vaccination data suggest that around 90,000 individuals over the age of 65 took the first dose of vaccine during spring 2021 but didn’t receive the second dose.
- The vaccination data suggest that around 440,000 individuals over the age of 65 took their first and second doses of vaccine according to the vaccination schedule (i.e., early/late spring 2021) but didn’t receive the booster/third dose.
Table S12 splits out hospitalisation data for those vaccinated with their first dose more than 28 days before their positive test, and vaccinated with their second dose more than 175 days before their positive test. Thus we can tentatively include these specific data in our analysis – individuals that had their first dose (only) or second dose (no booster) some time before the study period started.
Two points immediately stand out.
First, the hospitalisation rate with symptomatic confirmed Covid in those that had a single dose of vaccine ‘some time before’ the study period is similar to the hospitalisation rate in the unvaccinated but their A&E presentation rate for ‘not Covid’ is 2.5 times the rate of the unvaccinated. The TNCC assumption would be that the similarity in the symptomatic Covid rate is a fluke and what’s important is that on average they’re simply the type of individuals that would go to A&E more often and if that group of individuals hadn’t been vaccinated they’d have had 2.5 times more hospitalisation rates ‘with Covid’. I suggest that it is far more likely that the single-dose individuals have no vaccine induced protection against hospitalisation but that they are very much more likely to attend A&E.
Second, the A&E attendance rate of the double-vaccinated (only) without Covid is very similar to the A&E attendance rate of the vaccinated (any dose). However, their hospitalisation rate ‘with Covid’ is 2.5 times greater than that of the vaccinated (any dose) – the double-dosed that didn’t take their booster appear to have the ‘worst of both worlds’: increased A&E attendance (non-Covid) and increased admission rates ‘with Covid’.
Summary so far:
- The UKHSA has provided us with some raw data on hospitalisations by vaccination status.
- Examination of the data suggests that ‘with Covid’ hospitalisation rates in the unvaccinated aren’t too far from those in the vaccinated (any dose). However, non-Covid admission rates for A&E are much much higher in the vaccinated (any dose) than the unvaccinated.
- The TNCC approach would suggest that the vaccinated are simply ‘the type of people’ more likely to attend A&E and that the vaccines really do offer substantial protection against hospitalisation ‘with Covid’.
- Examination of other data suggests that the single dosed have ‘with Covid’ rates similar to the unvaccinated but 2.5 greater A&E attendance (without Covid) and that the double dosed (only) appear to have the worst situation of all – much higher Covid hospitalisation and much higher non-Covid admission to A&E.
Admission rates for acute respiratory illness
Table S7 in the UKHSA paper presents data on hospitalisations after an A&E visit where the individual had symptomatic Covid (again, Omicron, over 65). This sounds like the condition for the previous table, but in that table the ‘Covid negative’ column counted all non-accident or injury A&E visits, whereas the data in Table S7 only consider those that had symptoms similar to Covid.
I’ll present only the rates this time (feel free to look up the raw numbers yourself).
That’s perhaps even more interesting. In terms of the overall ratios it is a similar situation to the previous table – approximately 40% lower hospitalisations with symptomatic Covid in the vaccinated compared with the unvaccinated, but around four-fold higher rates for admission with symptoms that look like Covid, but aren’t Covid. But the interesting part is in the detail:
- Even though the absolute rates are very much lower, the unvaccinated still have the lowest admission rate to A&E. However, the difference for the data in the table above is that a doctor had assessed the individual and determined that he or she was ill enough to warrant hospitalisation. Thus the data in the table above are not influenced by the ‘symptom severity threshold’ that different individuals have before they’ll go to A&E. This is particularly of note because it suggests that the very high rates of presentation to A&E in the vaccinated in Table 1 and 2 are unlikely to be simply because of the vaccinated are more likely to go to A&E for ‘more trivial reasons’ than the unvaccinated – it looks like the vaccinated as a group really are more likely to be ill.
- Over twice as many of the unvaccinated are hospitalised with symptomatic Covid than with a condition that looks like Covid but isn’t (17.5 vs 6.5 hospitalisations per 100,000). However, twice as many of the vaccinated are hospitalised with ‘looks like Covid but isn’t’ than ‘with Covid’ (23 vs 11 hospitalisations per 100,000).
This latter point is important – one of the potential problems with vaccines (in general) is viral interference, that is, that a vaccine changes the immune response to other infectious diseases. Is it the case that the vaccines are significantly increasing the incidence of other respiratory infections? I note that last autumn/winter we had a mini-epidemic of what was popularly called ‘the worst cold ever’ – is this related?
However, the hospitalisation rate for symptomatic Covid might be more complex than it looks. In the UKHSA paper it appeared to define hospitalisation ‘with’ Covid to 14 days after the first positive test; if the individual presented to A&E after this 14 day period his or her data was excluded. It is possible that vaccination delays (rather than prevents) disease progression, in which case some of the hospitalisations with a condition that ‘looks like Covid but isn’t’ might in fact be due to a Covid infection that took longer than 14 days to develop to the point where hospitalisation was necessary. In addition, if the vaccinated were more likely to test themselves earlier in the progression of the disease then they might also be more likely to ‘run out of time’ and present themselves after the 14 day period has finished.
There’s another interesting aspect of these data – overall, the unvaccinated appeared to get around 24 hospitalisations per 100,000 of an illness that ‘looked like Covid’ (whether it was Covid or not) whereas the vaccinated appeared to get around 34 hospitalisations per 100,000. By this measure, vaccination is associated with an increased risk of a serious respiratory illness (whether Covid or not).
Again, the single-dosed appear to have the same risk of Covid as the unvaccinated, but increased attendance with looks like Covid but isn’t, and the double-dosed appear to have the ‘worst of both worlds’ – increased rates of attendance both with Covid and with looks like Covid but isn’t.
One more point – comparing Table 4 with Table 3, there appear to be far fewer admissions to A&E with Covid than without Covid. This indicates the current pressures on NHS A&E services are not related to Covid infections but ‘other things’.
The UKHSA data – Secondary uses data
The UKHSA paper also offers data using the NHS ‘secondary uses dataset’ (Table S10). This covers all hospitalisations and offers more granularity on the reasons for the hospitalisation and the level of treatment offered. The authors use a selection of the full dataset, where the admission was for an acute respiratory illness, and for several levels of seriousness.
First up are the data on hospitalisations for admissions for an acute respiratory illness where the individual was discharged the same day.
Two aspects stand out:
- The rate of serious Covid infection that warranted admission into hospital but that was not serious enough for an overnight stay was higher in the vaccinated than in the unvaccinated. This suggests that the vaccines increase the risk of being ‘somewhat unwell from Covid’. Note that this isn’t simply ‘infected’ – these individuals were deemed by experts to be sufficiently at risk of their illness to be admitted to hospital, even if they didn’t pass the threshold for an overnight observation/treatment.
- The hospitalisation rate of ‘somewhat serious but not Covid’ acute respiratory illness in the vaccinated was around twice as great as that in the unvaccinated, and even higher in those having only one or two doses of vaccine
This time we see that overall the risk of ‘somewhat serious’ acute respiratory illness (whether due to Covid or not) appears to be similar in the vaccinated and unvaccinated, and that those having taken one or two doses of vaccine (only) appear worst off.
What about a more serious symptomatic infection – perhaps acute respiratory illness requiring several days of hospitalisation and supplementary oxygen?
Here, at last, we appear to see some benefit from the vaccines – the unvaccinated appear to be rather more likely to be hospitalised for a few days following Covid infection (acute respiratory illness requiring supplementary oxygen), and even though they also appear to be less likely to be similarly hospitalised without Covid, this isn’t by so great a margin to remove the protective effect of vaccination.
But yet, I keep returning to the matter of the 14 day limit after the positive test. Individuals typically get to the point where they require supplementary oxygen some time after infection. If the unvaccinated aren’t testing themselves at the point where symptoms start but the vaccinated are, then they’ll be relatively more likely to get to the stage where oxygen is required within that 14 day period. Any delay in symptomatic disease in the vaccinated would only make this effect worse. Is the supposed effectiveness of the vaccines at preventing severe disease simply an artefact arising due to the fact that it takes longer to get to the point where the symptoms are severe, or due to the unvaccinated being more test-averse?
The UKHSA doesn’t report on the basis for the use of a 14 day limit – I’d very much like to see supporting data, or a sensitivity analysis in its paper, comparing rates for 14, 21 and 28 days after the positive test.
The UKHSA has at long last published raw data on hospitalisation rates by vaccine status, for those infected with Covid as well as those that aren’t. The results are very concerning, showing significantly higher A&E admission rates in the vaccinated for reasons other than Covid, and much less difference in admission rates for symptomatic Covid in the vaccinated vs unvaccinated than suggested by the estimates of vaccine effectiveness published by the UKHSA.
What I’ve shown here isn’t proof that the vaccines are causing harm – but it is a huge red-flag that strongly suggests that there might be a serious problem, and certainly indicates that a proper analysis of illness after vaccination needs to be undertaken urgently.
Furthermore, the significant differences in the ‘negative test’ arm of the UKHSA data suggest that the test-negative case-control method is not appropriate, and that a full retrospective matched-cohort study into vaccine effectiveness and safety should be undertaken.
Amanuensis is an ex-academic and senior Government scientist. He blogs at Bartram’s Folly.