The report from round 13 of Imperial College’s REACT-1 Covid infection survey was published yesterday, covering the period from June 24th to July 12th, broadly corresponding to the Delta surge.
The press release led with the claim that “double vaccinated people were three times less likely than unvaccinated people to test positive for the coronavirus” (0.4% vs 1.2%). This is clearly misleading as an indication of vaccine effectiveness, however, as younger people were both less likely to be vaccinated and more likely to test positive. As the report itself admits: “These estimates conflate the effect of vaccination with other correlated variables such as age, which is strongly associated with the likelihood of having been vaccinated and also acts as a proxy for differences in behaviour across the age groups.”
Presumably, the headline was chosen by a politically savvy communications officer who did not want to draw attention to the fact that the study found a lower vaccine effectiveness than other studies such as those of Public Health England.
It found a vaccine effectiveness (vaccine type unspecified) among 18-64 year-olds of 49%. However, the 95% confidence interval ran from 22% to 67%, meaning the authors didn’t have enough positive test results to be very sure of their estimate (despite testing nearly 100,000 people, only 527 results or 0.54% came back positive). They couldn’t even be very confident it wasn’t as low as 22%.
This low certainty is probably why they didn’t try to break down the results further by age. However, without that it’s hard to see how the results can be relied upon, as the vaccination rates of 18 year-olds differ hugely from those of 64 year-olds, as do the infection rates, and lumping them all together makes the results almost meaningless. With most infections in the young and most vaccinations in the old, it suggests 49% is an upper bound rather than a central estimate.
The authors note that their reduced estimate for vaccine effectiveness is similar to recent data from Israel, but below PHE’s latest figures:
These estimates are lower than some others, but consistent with more recent data from Israel. Our estimates were higher when we restricted our analyses to people reporting symptoms of COVID-19 in the previous month. These higher estimates were still lower than those reported using a test-negative design for routine testing of symptomatic people presenting for RT-PCR in England. However, our data are based on a random sample of the population and include asymptomatic people, as well as symptomatic individuals who may not present for routine testing, and may therefore give a less biased representation of transmission risk.
Another disappointment in the study – again absent from the press release – is that the ratio of positive test results to both hospitalisations and deaths has narrowed during the recent surge (see below – compare the red and blue lines to the grey line). The widening gap had been taken as an indication of vaccine effectiveness, and the narrowing is understood to mean the opposite.
The authors write: “In our more recent data (since mid-April 2021), infections and hospitalisations began to re-converge, potentially reflecting the increased prevalence and severity of Delta compared with Alpha, a changing age mix of severe cases, and possible waning of protection.”
The relationship has even gone negative in hospitalisations in the under-64s, implying that they are worse off now than before the vaccination programme. Deaths are still trending (slightly) below. The results weren’t broken down by vaccination status, so we can’t calculate vaccine effectiveness here.
