What do you do when people have spotted that infection rates are higher in the vaccinated than the unvaccinated and are spreading this ‘misinformation‘ on the internet?
It appears that you commission the ONS to come up with a model that fixes the problem. Or rather, in this case, three models.
The ONS on Monday published a new ‘technical article‘ based on its Covid Infection Survey that provides “analysis of populations in the U.K. by risk of testing positive for COVID-19”. It covers the two-week period August 29th to September 11th, though regular updates are now promised.
It involves no fewer than three models, briefly summarised as:
Our first model, Model 1 (the core model), predicts the likelihood of an individual testing positive based on general demographic characteristics in order to help identify broad groups where infections are persisting or arising. …
We then built upon Model 1 resulting in Model 2, the screening model. This includes the core demographic characteristics from Model 1 and incorporates other characteristics individually to identify other factors associated with testing positive for COVID-19. …
Finally, Model 3 (the behaviours model) adds behaviour variables to the core demographic characteristics from Model 1 and the screened characteristics that were kept in Model 2.
I’m sure this talk of models built on models is filling you with confidence.
I’m not sure it filled the authors with very much confidence, though, as their main findings are stated without specific figures:
● People who had received one or two doses of a coronavirus vaccine were less likely to test positive for coronavirus (COVID-19) in the fortnight ending September 11th 2021.
● People living in a household of three or more occupants were more likely to test positive for COVID-19 in the fortnight ending September 11th 2021.
● Those in younger age groups were more likely to test positive for COVID-19 in the fortnight ending September 11th 2021.
● People who never wore a face covering in enclosed spaces were more likely to test positive for COVID-19 in the fortnight ending September 11th 2021.
● Those who reported socially distanced contact with 11 or more people aged 18 to 69 years outside their household were more likely to test positive for COVID-19, in the fortnight ending September 11th 2021.
The media made much of the mask finding, with the Mail declaring: “People who don’t wear face masks indoors are up to TWICE as likely to test positive for Covid.”
Here’s the chart with the full ‘Model 3’ results.
And here are the ‘Model 2’ results, which include the vaccinated categories.
There are a number of curious points to these. For example, having a regular lateral flow test (LFT) makes you more than twice as likely to test positive. But why, when the ONS survey is based on a random sample of PCR testing, does your chance of being found positive increase just because you also test regularly with LFTs? Perhaps it’s because those more likely to test regularly, such as school children and healthcare workers, have higher background infection rates? Possibly, but isn’t the model supposed to control for confounders like this? Besides, according to the results above, children and students are actually less likely to test positive, not more.
Which is curious in itself. How can the ONS seriously suggest that in the fortnight August 29th to September 11th, as schools went back, infection rates were lower in students and children? Here are the ONS’s own estimates – from the same infection survey – for different age groups. The far higher rates in those under 24 are clearly visible.
What has gone wrong with the modelling in the study that it produces a piece of nonsense like lower infection risk for children and students? If the higher rates in children and students have been erased by the model, how can the rest of the findings be trusted?
The ONS has provided the raw data in a linked spreadsheet, so we can dig a little deeper.
Here we see that across all categories the prevalence varies little, moving mainly between 1-2%. This suggests that whatever relative risk reductions there are for the various categories, the absolute risk reductions are small, less than 1%.
The highest values are found in the unvaccinated, with 2.8% prevalence in the two-week study period, and children/students, with 2.6%. These will be many of the same people, of course. The fact that the raw data for children/students is up at 2.6 times the reference value (namely 1% for those employed/working) is baffling given we’ve just seen that the model comes up with a reduced risk for that group. That’s one heck of an adjustment.
Yet no such massive adjustment occurs for the unvaccinated, with the modelled risks for the vaccinated categories being little different to the raw data. This indicates that little account has been taken of the fact that the unvaccinated in the sample are mostly young people with high background infection rates, which will skew the implied vaccine effectiveness upwards.
We know from Public Health England data that over this period reported infection rates in the vaccinated over-40s were higher than in the unvaccinated. However, in this study there is no age breakdown for the vaccine categories, so any negative vaccine efficacy like this will be concealed.
One big problem for the study is that the study group is highly unrepresentative in a number of ways. For instance, only 12.8% of it is unvaccinated, compared to 29% in the wider population.
The study group is also an uncommonly conscientious bunch, as befits a group willing to participate in this kind of survey. As well as nearly a quarter doing regular LFT testing, 84% say they always wear a mask in an enclosed space (significantly higher than surveys indicate), while just 8% say they wear one sometimes and 6.5% never. There’s little chance that the researchers will have taken into account all the differences in the way these groups behave, such as never-maskers being likely to be less cautious people. Small wonder then that their results on masks are so different to the RCTs.
Only 6% of the study group has had a previous reported infection, yet PHE reports that around 19% of the population have antibodies from a Covid infection. Perhaps this is why the study shows that previous infection only gives 55% protection from re-infection, lower than from the vaccines even after their efficacy has waned over time and against Delta. However, this finding is based on just 59 positives in the recovered out of 10,055 tested or 0.6%, which is too small to have confidence in generalising it to the population, particularly when you take into account the chance of false positives. It also contradicts recent data from Israel which suggests that natural immunity provides much stronger protection against Delta than the vaccines.
Readers may recall an Oxford University study last month on vaccine efficacy against Delta which also used ONS Infection Survey data and made a similar finding that natural immunity was inferior to vaccine immunity. I pointed out then that the data in the study was all over the place and made no sense at all, and wondered whether the ONS Infection Survey sample was really fit for purpose. Is it too unrepresentative of the wider population to provide the basis for sound and meaningful results, even after adjustments, which seem to be done cackhandedly anyway? After this latest study I am even more convinced that the answer to that question may be yes.
Stop Press: Oxford Professor of Evidence-Based Medicine Carl Heneghan has been scathing about the study on Twitter: “No protocol, no peer review, no accounting for confounders, selection bias and reporting bias you couldn’t make it up – actually, it seems you can.”
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.