Toby has already gone through in detail the new report from the Science and Technology Committee and the Health and Social Care Committee of the House of Commons on the Government’s handling of the COVID-19 pandemic and taken it apart.
One point worth underlining further is that one of its central conclusions – that “if the national lockdown had been instituted even a week earlier ‘we would have reduced the final death toll by at least a half'” (the report quoting Professor Neil Ferguson here) – is demonstrably false on all the data available. That’s because it assumes that the epidemic was continuing to grow exponentially in the week before lockdown was brought into effect on March 24th, a growth which supposedly only the lockdown brought to an end.
That this is not the case is evident from all the data we have, as has been shown on numerous occasions.
For example, already in April 2020 Oxford’s Professor Carl Heneghan had noted that by projecting back from the peak of deaths on April 8th it could be inferred that the peak of infections occurred around a week before the lockdown was imposed. This early deduction was subsequently backed up by Chief Medical Officer Chris Whitty himself, who told MPs in July 2020 that the R rate went “below one well before, or to some extent before, March 23rd”, indicating a declining epidemic.
Further support arrived in March 2021, when Imperial College London’s REACT study published a graph showing SARS-CoV-2 incidence in England as inferred from antibody testing and interviews with those who tested positive to ascertain date of symptom onset. It clearly showed new infections peaking in the week before March 24th (see below), as well as a similar peaking of infections ahead of the subsequent two national lockdowns.
There follows a guest post from our in-house doctor, formerly a senior medic in the NHS, who draws attention to the errors made repeatedly by the modellers and government advisors and the huge implications of them.
Napoleon Bonaparte remarked that “history is the version of past events that people have decided to agree on”. When the official version of the pandemic is written, I wonder what analysis will be made of the role of statistical modellers and public health experts in driving Government policy over the last 18 months?
To inform this question it may be helpful to examine the recent evidence of how predictions have matched up to real events. For example, on September 8th, SPI-M-O (one of the multitudinous acronym salad bodies advising the Government), produced a paper entitled “Medium-term projections“.
Perhaps mindful of the woeful inaccuracy of previous predictions, the very first sentence heavily caveats the entire document:
These projections are not forecasts or predictions. They represent a scenario in which the trajectory of the epidemic continues to follow the trends that were seen in the data up to September 6th.
If that is the level of confidence the authors of the report have in their own abilities, one rather wonders what value this publication contains – yet this is the level of advice being given to decision-makers.
Firstly, to the “projection” of admissions. The document is in PDF format, so I am unable to reproduce it here, but the graphical representations show a 90% confidence interval fan chart for the period September 12th-28th. Hospital admissions in England are “projected” to be between 600 to 1,200 per day – a fairly wide spread. Graph One shows what actually happened – daily admissions on the blue bars, seven-day moving averages on the brown line.
It is clear that admissions have been consistently below the lower ‘projection’ for the entire period and the seven-day moving average at the end of the month was below 500 admissions per day.
Neil Ferguson’s team at Imperial College London (ICL) has released a new paper, published in Nature, claiming that if Sweden had adopted U.K. or Danish lockdown policies its Covid mortality would have halved. Although we have reviewed many epidemiological papers on this site, and especially from this particular team, let us go unto the breach once more and see what we find. The primary author on this new paper is Swapnil Mishra.
The paper’s first sentence is this:
The U.K. and Sweden have among the worst per-capita Covid mortality in Europe.
No citation is provided for this claim. The paper was submitted to Nature on March 31st, 2021. If we review a map of cumulative deaths per million on the received date then this opening statement looks very odd indeed:
Sweden (with a cumulative total of 1,333 deaths/million) is by no means “among the worst in Europe” and indeed many European countries have higher totals. This is easier to see using a graph of cumulative results:
But that was in March, when the paper was submitted. We’re reviewing it in August because that’s when it was published. Over the duration of the journal’s review period this statement – already wrong at the start – became progressively more and more incorrect:
The report from round 13 of Imperial College’s REACT-1 Covid infection survey was published yesterday, covering the period from June 24th to July 12th, broadly corresponding to the Delta surge.
The press release led with the claim that “double vaccinated people were three times less likely than unvaccinated people to test positive for the coronavirus” (0.4% vs 1.2%). This is clearly misleading as an indication of vaccine effectiveness, however, as younger people were both less likely to be vaccinated and more likely to test positive. As the report itself admits: “These estimates conflate the effect of vaccination with other correlated variables such as age, which is strongly associated with the likelihood of having been vaccinated and also acts as a proxy for differences in behaviour across the age groups.”
Presumably, the headline was chosen by a politically savvy communications officer who did not want to draw attention to the fact that the study found a lower vaccine effectiveness than other studies such as those of Public Health England.
It found a vaccine effectiveness (vaccine type unspecified) among 18-64 year-olds of 49%. However, the 95% confidence interval ran from 22% to 67%, meaning the authors didn’t have enough positive test results to be very sure of their estimate (despite testing nearly 100,000 people, only 527 results or 0.54% came back positive). They couldn’t even be very confident it wasn’t as low as 22%.
In a recent article, we considered the implications of the U.K.’s spring rise in infections, given that before now the assumption has been that coronaviruses are seasonal at northern temperate latitudes. Do we have to dismiss that hypothesis in light of the ‘Third Wave’?
Here we argue that, contrary to Government claims, the British summer is indeed finally impacting viral transmission, with sharp falls in positives reported across the U.K. In England, reported cases have more-or-less halved in a week, from 50,955 to 25,434.
This sharp fall runs counter to all three of the most recent SAGE models driving Government policy, which predict rising infections leading to peaks in hospital admissions in high summer – and by implication falsifies the assumptions upon which these models are based.
Parsimony predicts the summer troughs and winter peaks evident for SARS-CoV-2
In spring and summer 2020 and winter 2020-1, SARS-CoV-2 infections parsimoniously followed the pattern of seasonal respiratory viruses, falling away in the summer months and rising again in the autumn, with peaks in deaths occurring between mid-November 2020 and mid-April 2021 in different northern temperate countries.
Although falling infection levels were sometimes prolonged into early summer or began to rise again in late summer, there were no peaks in fatalities in summer or early autumn 2020.
Most notably, while cases in Sweden rose in a pattern close to the European average in early 2020, they persisted much later, continuing to a plateau in late spring and early summer, before falling away sharply from the end of June. Hospitalisations and deaths fell more smoothly from the mid-April peak, however, and showed no corresponding rise in late spring and early summer.
Similarly, while infections began to rise in late summer in some countries – such as France – there was no substantial increase in deaths before mid-autumn. Summer 2020 appears to have broken the link between infections and serious illness in the absence of vaccination.
Sweden has so far emerged relatively unaffected by the Delta variant. Although this variant was detected in Sweden – as it was in most other European countries – infections in Sweden nevertheless fell with the onset of summer. As Sweden’s State Epidemiologist Anders Tegnell remarked in an interview on June 18th, 2021 (at about 8 minutes 25 seconds), “the number of cases in Sweden are falling rapidly, very rapidly I would say, much more rapidly than we ever thought was possible”. Tegnell also makes some sceptical points about asymptomatic transmission and mass testing, so beloved of the U.K.’s SAGE committees.
Recent peaks attributable to the Delta variant have occurred in countries such as Denmark, Belgium and the Netherlands, but these outbreaks too may have peaked as they appear to have in the U.K.
SAGE scenarios – anything can happen in the next eight weeks…?
Turning to the latest (July 6th) scenarios of the SAGE’s SPI-M-O modelling groups, we find hospitalisations could be between 50 and 10,000 per day by August 31st depending on the R value. SPI-M-O note these scenarios are not forecasts or predictions, leaving open to question their purpose with regard to Government formulation of policy.
Previous over-estimations of hospitalisations are attributed to: 1) the cancellation of ‘Freedom Day’ on June 21st permitting more vaccinations to be administered and transmission to be delayed due to restrictions; 2) less than anticipated mixing between adults since late April to mid-May; and 3) the effectiveness of vaccines against the Delta variant.
There appears to be no suggestion of an emphatic effect of spring and summer on behaviour, the virus or viral transmission, which would have been considered conventional wisdom until mid-March 2020.
The Warwick models predict the current rise in hospitalisations will persist to peak in late summer or early autumn, which may or may not be accompanied by a small wave – based on the mean estimates – from late December 2021 or early February 2022 depending on supposed “precautionary behaviour”.
None of the Warwick models predict a fall in hospitalisations in summer 2021 nor – by implication – a fall in infections.
Imperial College London
Imperial College offers two models, based on optimistic (upper figure) and central (lower figure) estimates of vaccine effectiveness, adjusted according to estimates of the speed of change in behaviour and the R value.
Both models predict peaks in the early autumn, possibly delayed to mid-autumn if changes in behaviour are slow. Using these assumptions, the mean estimates presented for hospitalisations are higher than in the Imperial models.
Central estimates of vaccine effectiveness with sudden relaxation in precautionary behaviour appears to predict mean daily hospitalisations of about 2,500 to about 12,000 per day by the end of September depending on the R value. Imperial have produced a further model based on pessimistic estimates of vaccine effectiveness (not shown).
Of the Imperial models, only the gradual relaxation of restrictive behaviour scenarios indicate a fall in hospitalisations, but in both instances this simply delays a peak in hospitalisations and – by implication – infections until the early autumn. Neither model anticipates an imminent fall continuing into summer, nor a winter peak between December and February.
London School of Hygiene and Tropical Medicine (LSHTM)
LSHTM present similar models based on a further set of assumptions and predict a peak in hospitalisations in mid-summer, varying in size according to the extent of reduction in transmission (five to 20% reduction at medium mobility is shown in the figure).
Again, the LSHTM model precludes the current reduction to a baseline as in summer 2020.
The ZOE Symptom Study, which provides invaluable independent comparator to reported positives figures, appears to show infections to be rising to July 20th, but only since the method of estimation was revised. Comparators such as ONS and REACT-1 are out of date.
Implications of the models
None of the SAGE models predict a sharp fall and summer lull in infections. Rather, the SAGE report states “the prevalence of infection will almost certainly remain extremely high for at least the rest of the summer”.
We are left with two competing hypotheses:
SAGE predict a continued rise in infections, accompanied by hospitalisations and deaths, peaking in mid-summer or early autumn. There may be a further small wave from late December or early February, or none at all.
Parsimony predicts cases will fall to baseline as summer advances, much as occurred in Sweden last year – a late spring or early summer cold that does not cause significant morbidity or mortality. The summer disappearance will be followed by a resumption in the autumn rising to a peak in infections and deaths in winter proper.
Are the SAGE models already wrong?
Although summer peaks in infections in seasonal respiratory viruses are rare, they are not unknown, particularly in novel varieties and, it may be noted that – unlike in Sweden in 2020 – the spring rise in infections in the U.K. arose from a low base and involved a new variant – the Delta variant – and was preceded by the vaccine roll-out.
While vaccination is argued to be the key factor in keeping hospitalisations and deaths figures low, these measures were also low in the late spring 2020 wave of infections in Sweden. It is possible that nosocomial and care-home outbreaks have also been prevented, in part due to the seasonal fall in general demand for hospital beds in the spring and summer. The most recent ONS report shows overall excess deaths in England and Wales to be higher at home than in care homes or hospitals. Nevertheless, it is striking that reported positives in Scotland have been falling since the end of June.
Hospitalisations in Scotland are also falling from a peak approximately a week later.
The rest of the U.K. is now following the trend in Scotland, which showed a rapid fall in infections from the end of spring and beginning of summer, as Sweden did in 2020. Are we simply experiencing a late impact of seasonality on suppression of spread, which has finally taken effect?
Reported positives peaked just prior to ‘Freedom Day’ in England and about three weeks earlier in Scotland. There is no sign of any stall in the falling trajectory of infections in either country, as could be attributed to the relaxation of restrictions on ‘Freedom Day’. This would be not at all surprising to those who observed the lack of impact of ‘opening up’ in Texas and Florida some months ago.
On the basis of current infection data, the SAGE models are already wrong.
So must be the assumptions of virus transmission and effects of Non-Pharmaceutical Interventions – and lack of effect of nature – on which they are based.
It begs the question as to why the Government and media have again so enthusiastically engaged with consistently disappointing predictions leading to such damaging public health policy.
None of this should be a distraction from the point that lockdowns cause a good deal of harm to physical and mental health and to the economy, far outweighing any presumed benefit – if any can be shown. The models, NPIs and lockdowns are about politics, not science.
The co-authors are a PhD epidemiologist trained at a Russell Group University and a retired former Professor of Forensic Science and Biological Anthropology.
The above graph is the COVID-19 epidemic curve for England, reconstructed by Imperial College’s REACT antibody survey by asking those who tested positive in an antibody test when their symptoms began. I’ve added the start dates for lockdowns in red and the end dates in blue.
It’s a very useful graph because it does not involve any PCR tests at all, only lateral flow immunoassay tests, self-administered at home. This means it does not suffer from the problem of detecting non-infectious virus as it is not detecting virus at all but antibodies. (Its specificity is reported as 98.6%, giving it a 1.4% background false positive rate, which the researchers adjust for.) This means, for example, that the epidemic decline is much faster than in the familiar “case” curves, and the curves are more symmetrical.
What does it show? Here’s what I take from it. You might see more.
Firstly, it provides further evidence that SARS-CoV-2 was circulating at low levels in England throughout December 2019 and to some degree also in November. This fits with widespread anecdotal evidence of people falling ill with Covid symptoms in December. It doesn’t fit with the original official timeline of an outbreak beginning in Wuhan in December.
Secondly, despite circulating widely during the winter of 2019-20, SARS-CoV-2 did not undergo fast spread in England until the end of February. Indeed, the winter of 2019-20 was the least deadly on record in terms of age-adjusted mortality, despite SARS-CoV-2 being around and infecting people.
Then, around February 25th 2020, it suddenly launches into a three-week long spike of extraordinary exponential growth. This abruptly comes to an end around March 17th, and after a short plateau till around March 21st it enters just as extreme a decline. This is all ahead of the first lockdown on March 23rd of course.
The mystery is: what happened on February 25th (or thereabouts – we don’t know whether Imperial’s assumptions about the incubation period are exactly right) to cause a virus that had been circulating for at least three months at a low level suddenly to go bang and spread like wildfire? It wasn’t panic – no one was panicking at the end of February. Mobility levels were still normal until around March 12th. There was nothing unusual about the weather. Suggestions on this welcome in the comments below.
There was a good letter in the Telegraph today co-signed by Lockdown Sceptics contributor David Campbell and his colleague Kevin Dowd. It was a pithy summary of a piece they co-authored for Spectator Australia earlier this month.
SIR – Matt Ridleys criticism (Comment, June 21st) of the distorted presentation of scientific predictions in order for those predictions to have political impact identifies the worst feature of current public policymaking.
Amazingly, however, in the case of Covid policymaking his criticism is insufficient. The crucial prediction was that of the Imperial College COVID-19 Response Team, which said that 510,000 deaths would occur in the (unlikely) absence of any control measures or spontaneous changes in individual behaviour. This was misleading in the extreme, for there was absolutely no possibility that the outbreak of this disease would not be met by widespread spontaneous changes in behaviour, or that the Government would not take extensive measures to support them.
The world has been turned upside-down by an absurd, alarmist prediction of what was always a zero-probability event, as it was this prediction which panicked the Government into adopting a suppression policy.
Professor David Campbell Lancaster University Law School Professor Kevin Dowd Durham University Business School
Glen Bishop, the second year maths student at Nottingham who was the first to spot that none of the modelling teams feeding into SAGE had taken seasonality into account last February, has taken a look at the new, improved models from Imperial, Warwick and the London School of Hygiene and Tropical Medicine that led to headlines earlier this week saying SAGE was no longer predicting an apocalyptic ‘third wave’. (Yipee!) The good news is, the teams have corrected their seasonality mistake when modelling the likely impact of the lifting of restrictions and now graciously allow that summer sunshine will ameliorate the spread of the virus – one of the reasons their latest projections are less gloomy. But there’s also plenty of bad news, as you’d expect.
Here is an extract:
A rational group of scientists would advise that risks are now within the normal accepted range and thus the end of restrictions is nigh and normal life will return. Unsurprisingly, that is not what these three modelling teams have done. Their models have failed to deliver the pessimism and danger craved by scientists clinging on to power, but a new obsession is taking over – the danger of variants. Imperial elaborates: “preventing the importation of variants of concerns (VOC) with moderate to high immune escape properties will be critical as these could lead to future waves orders of magnitude larger than the ones experienced so far.”
Previous Imperial models have made only passing reference to new variants and never tried to model them, yet Imperial’s latest paper, which shows (even with their modelling) the risk from covid to now be incredibly low, is half filled with predictions of theoretical super variants. The most pessimistic of the predictions entails an imaginary ‘high escape’ variant, which, if we stick to the current roadmap, would lead to a peak of over 4,500 deaths per day and a total of 225,000 deaths this summer. To put this into perspective, it would mean a death rate this summer of 3,300 per million, that is double the death rate in Florida since the pandemic began of 1,669 per million despite Florida being near fully open for the last eight months. It’s a higher total than anywhere in the world since the pandemic began. This is void of reality, but even if it weren’t, what is the proposal? Lockdown for another year until a vaccine for this new variant can be distributed, by which time even more variants will have appeared? One might as well include in the modelling a super infectious variant of Ebola or a new improved laboratory leak from our friends in the Wuhan Institute of Virology.
Phillip W. Magness in AIER has crunched the numbers and shown how poor Imperial College’s modelling has been at predicting the outcomes of the COVID-19 pandemic under different policy responses in every country in the world (well, 189 of them). Yet for some unexplained reason Neil Ferguson and the rest of the Imperial team remain respected authorities on epidemic modelling and management. Magness writes:
COVID-19 has produced no shortage of doomsaying prophets whose prognostications completely failed at future delivery, and yet in the eyes of the scientific community their credibility remains peculiarly intact.
Assessed looking backward from the one-year mark, ICL’s modelling exercises performed disastrously. They not only failed to accurately forecast the course of the pandemic in the US and UK – they also failed to anticipate COVID-19’s course in almost every country in the world, irrespective of the policy responses taken.
Magness has put together a table of all the countries with the predictions ICL made for them and their actual outcomes. The results should be fatal for the reputation of anyone whose job it is to make accurate predictions of the future course of events. But not ICL it seems, whose credibility appears to be invulnerable despite repeated and consistent failure. Magness wonders why.
Why is Ferguson, who has a long history of absurdly exaggerated modeling predictions, still viewed as a leading authority on pandemic forecasting? And why is the ICL team still advising governments around the world on how to deal with COVID-19 through its flawed modeling approach? In March 2020 ICL sold its credibility for future delivery. That future has arrived, and the results are not pretty.
When Professor Neil Ferguson and his team at Imperial College London have been challenged on their model’s miserable failure to predict the pandemic death toll in Sweden they have always pushed back saying they didn’t model Sweden, disavowing the work of the team at Uppsala University which adapted their modelling to the Swedish context. But it turns out this is not exactly accurate. Phillip W. Magness explains on AIER:
In the House of Lords hearing from last year, Conservative member Viscount Ridley grilled Ferguson over the Swedish adaptation of his model: “Uppsala University took the Imperial College model – or one of them – and adapted it to Sweden and forecasted deaths in Sweden of over 90,000 by the end of May if there was no lockdown and 40,000 if a full lockdown was enforced.” With such extreme disparities between the projections and reality, how could the Imperial team continue to guide policy through their modelling?
Ferguson snapped back, disavowing any connection to the Swedish results: “First of all, they did not use our model. They developed a model of their own. We had no role in parameterising it. Generally, the key aspect of modelling is how well you parameterise it against the available data. But to be absolutely clear they did not use our model, they didn’t adapt our model.”
The Imperial College modeller offered no evidence that the Uppsala team had erred in their application of his approach. The since-published version from the Uppsala team makes it absolutely clear that they constructed the Swedish adaptation directly from Imperial’s UK model. “We used an individual agent-based model based on the framework published by Ferguson and co-workers that we have reimplemented” for Sweden, the authors explain. They also acknowledged that their modelled projections far exceeded observed outcomes, although they attribute the differences somewhat questionably to voluntary behavioural changes rather than a fault in the model design.
Ferguson’s team has nonetheless aggressively attempted to dissociate itself from the Uppsala adaptation of their work. After the UK Spectator called attention to the Swedish results last spring, Imperial College tweeted out that “Professor Ferguson and the Imperial COVID-19 response team never estimated 40,000 or 100,000 Swedish deaths. Imperial’s work is being conflated with that of an entirely separate group of researchers.” It’s a deflection that Ferguson and his defenders have repeated many times since.
In fact, though, as Phillip points out, it is not true to say that the Imperial team never estimated 40,000 or 100,000 Swedish deaths. Hidden away in a spreadsheet in the appendix to Report 12, published on March 26th 2020, are the team’s estimates for other countries including Sweden. The projections are expressly intended to encourage those countries to follow suit with social restrictions. They write:
To help inform country strategies in the coming weeks, we provide here summary statistics of the potential impact of mitigation and suppression strategies in all countries across the world. These illustrate the need to act early, and the impact that failure to do so is likely to have on local health systems.
The predictions for Sweden are up to 90,157 deaths under “unmitigated” spread (Uppsala projected 96,000) and, under “population-level social distancing” (lockdowns), 42,473 deaths (compared to Uppsala’s 40,000). So, contrary to their repeated denials, Ferguson’s team did make predictions for Sweden very close to those made by the Uppsala team who adapted their model, and those predictions were just as way off. Sweden’s Covid death toll at the end of the first wave, on August 31st, was 5,821.
Phillip summarises further failures of the Imperial modelling in a table showing four non-lockdown countries (Sweden, Taiwan, South Korea, Japan) and the United States (most of whose states imposed a lockdown in the spring) with their one-year death toll and how it compares to Imperial’s projections.
It’s worth saying, though, that the models for the ‘unmitigated’ scenarios predicted the deaths to occur over the course of a few months, not a whole year including another winter flu season. There will be another ‘wave’ of deaths every winter, possibly from (or with) COVID-19 if it remains the dominant respiratory virus (and if we keep on testing for it). If we keep on adding the deaths over several seasons then of course they will eventually reach the predicted figures. But that wasn’t what the models were claiming to show and would be a case of making the evidence fit the model.