The latest UKHSA Vaccine Surveillance report was released Thursday, and its authors are now bending over backwards to keep their critics happy. Following a telling-off this week from the U.K. Statistics Authority, the UKHSA’s Head of Immunisation, Mary Ramsay (pictured above), published a blog post explaining what they’ve done to appease their detractors, while the report now states no fewer than four times, twice in bold typeface, that “these raw data should not be used to estimate vaccine effectiveness”. Ramsay grovels:
To make our data less susceptible to misinterpretation, the U.K. Health Security Agency has worked with the UK Statistics Authority to update some of the data tables and descriptions in the report, specifically around rates of infection in vaccinated and unvaccinated groups. In our commitment to transparent and clear data, we regularly review our publications to ensure they reflect the current situation within the pandemic, and we will continue to work with our partners at the statistics bodies, to ensure our reporting is as scientifically robust as possible.
As I noted last week, the UKHSA does not accept the criticism of its population estimates levelled by, among others, David Spiegelhalter, who declared that using them was “deeply untrustworthy and completely unacceptable”.
The agency instead takes the view that the problem is systemic biases in the data which mean it “should not be used” to estimate vaccine effectiveness. But as I have noted repeatedly, those biases just mean that the estimate will be of unadjusted vaccine effectiveness, which is a perfectly legitimate quantity to estimate and has its uses, particularly when looking at trends or when there is reason to think the biases may be relatively small. (For instance, a recent vaccine effectiveness study in California adjusted its raw data for 22 different factors but in almost all cases the adjustments were tiny.)
The UKHSA report itself correctly gives the definition of vaccine effectiveness: “Vaccine effectiveness is estimated by comparing rates of disease in vaccinated individuals to rates in unvaccinated individuals.” The U.S. CDC, likewise, states the definition as “the proportionate reduction in disease among the vaccinated group”. The CDC distinguishes “vaccine efficacy”, estimated from controlled studies, from “vaccine effectiveness”, which is used “when a study is carried out under typical field (that is, less than perfectly controlled) conditions”. It is therefore not appropriate for the UKHSA, a Government agency, to insist that its data “should not be used” to estimate vaccine effectiveness, which is a false statement and amounts to attempted Government censorship of scientific enquiry.
The report explains that “vaccine effectiveness is measured in other ways as detailed in the ‘Vaccine Effectiveness’ Section.” However, that section is clear that each estimate “typically applies for at least the first three to four months after vaccination”, and “there may be waning of effectiveness beyond this point”. The report discusses this waning, but only for the Alpha variant: “Data (based primarily on the Alpha variant) suggest that in most clinical risk groups, immune response to vaccination is maintained and high levels of VE are seen with both the Pfizer and AstraZeneca vaccines.” What use is data based primarily on the Alpha variant, which went almost extinct around six months ago? There is no attempt to present adjusted estimates of vaccine effectiveness based on the most up-to-date data. Instead, we are just given repeated insistences that the data is not showing what it appears to be showing because it is subject to unquantified biases.
What are those biases? Last week the report claimed that vaccinated people “may engage in more social interactions because of their vaccination status”, which didn’t fit with the more usual idea of unvaccinated people as a less cautious sort. Neither did it fit with the other reason they gave, that the vaccinated “may be more health conscious and therefore more likely to get tested for COVID-19”. This week they kept the latter but changed the former to the entirely ambiguous: “People who are fully vaccinated and people who are unvaccinated may behave differently, particularly with regard to social interactions.”
The other two biases they suggest are that “many of those who were at the head of the queue for vaccination are those at higher risk from COVID-19” and “people who have never been vaccinated are more likely to have caught COVID-19” previously. (The latter they say gives a person “some natural immunity to the virus for a few months”, which seems a very pessimistic view of natural immunity, particularly seeing how optimistic they are about the effectiveness of the vaccines.)
The report asserts categorically that the unvaccinated have higher previous infection rates, but cites no evidence to support this. Why not? Why, almost a year into the vaccination campaign, are researchers still so often waving their hands when talking about the differences between vaccinated and unvaccinated groups? Where is the published data? Precisely how much more likely are the unvaccinated to have had a previous infection? This is a simple data comparison. Why hasn’t it been done? The study in California mentioned earlier found that 2% of the vaccinated had recovered from Covid against 2.3% of the unvaccinated, so not a large difference. Is England similar? Why don’t we know? Likewise, how much more likely are vaccinated people to be tested? This is just a comparison of the testing rates in vaccinated and unvaccinated populations. Why hasn’t it been done? This is not good enough. We want more data from UKHSA, not lectures on how not to use the meagre amounts of data they release.
In her blog post, Mary Ramsay points to studies PHE (UKHSA’s predecessor) has published in the past:
These factors are all accounted for in our published analyses of vaccine effectiveness which uses the test-negative case control approach. This is a recommended method of assessing vaccine effectiveness that compares the vaccination status of people who test positive for COVID-19, with those who test negative.
This method helps to control for different propensity to have a test and we are able to exclude those known to have been previously infected with COVID-19. We also control for important factors including geography, time period, ethnicity, clinical risk group, living in a care home and being a health or social care worker.
While PHE did publish such studies earlier in the year (I analyse them here and here), they have not published anything based on data more recent than May, over five months ago. This was just as Delta arrived, and before infections surged over the summer and the raw data started showing infections in the vaccinated eclipsing those in the unvaccinated.
So where is the update? It’s all very well writing pages at the behest of the U.K. Statistics Authority policing how people use your data, but where are the studies setting the picture straight? We’ve had studies from California, Sweden and Israel using data from over the summer, all showing sharp decline in vaccine effectiveness. Where is the U.K.’s contribution to this emerging understanding of the vaccines?
Yes, we had that dubious study in August from Oxford University based on the ONS Infection Survey. But there’s been no update from UKHSA to its studies based on Government testing data.
Here’s a suggestion. Why don’t Daily Sceptic readers write a (polite!) email to the UKHSA’s Mary Ramsay (address here, Twitter here) asking for an update on their very useful test-negative case control study with data from the summer and autumn. You might say you have been concerned about the data in their Vaccine Surveillance reports showing high infection rates in the vaccinated compared to the unvaccinated, but note they say vaccine effectiveness can only be properly estimated in a study, so would be grateful for an update on this.
Here’s this week’s table of unadjusted vaccine effectiveness and the updated graphs showing how it is changing over time. It shows infection rates currently twice as high in the vaccinated compared to the unvaccinated for those aged 40-79, corresponding to an unadjusted vaccine effectiveness of minus-100% or more. Vaccine effectiveness is negative for all over-30s, and almost zero for those aged 18-29 (and still declining). It remains high for under-18s, and effectiveness against hospital admission and death is holding up. This week the decline appears to have stopped, or at least paused, in most age groups.
Profanity and abuse will be removed and may lead to a permanent ban.