Testing – Muddle and Myth – The Daily Sceptic

by Gordon Hughes

In BBC Radio 4’s More or Less programme broadcast on September 23rd Sir David Spiegelhalter tried to row back from some of his previous explanations of the potential scale of false positives in testing for COVID-19. At the same time, the Huffington Post published an article making essentially the same arguments, though in a somewhat muddled fashion. In both cases, the logic is what I would term accurate nonsense – i.e., an argument that is formally correct but which misses the point of the debate by focusing on a small part of a much broader issue. In doing this, the presentations illustrate the dangers of attempting to communicate complicated issues by breaking them into small portions judged suitable for journalistic discourse.

Spiegelhalter’s reasoning was that:

(a) the false positive rate for PCR tests was much lower than 1% because ONS tests found much lower rates of infection during the summer; and
(b) the current infection rate in those being tested now was much higher than the prevalence of virus infection in the general population because many of those being tested had COVID-19 symptoms – i.e., the population being tested is not a random sample of the whole population.

He produced no evidence to support either assertion and there are strong reasons to believe that both are wrong. In any case, the argument entirely neglects the fact that the point of testing is nowhere near as simple as getting a Yes/No answer.

To understand this we must take a step back. For politicians and the bureaucracy we have a simple chain of logic:

Positive PCR test => Individual is infectious => Individual must be isolated.

None of the current restrictions on personal liberty and all of the associated economic costs can be justified without this logic. On the other hand, any knowledge of the distribution of the virus in the population tells us that it is patent nonsense. At the simplest, it reflects a confusion between “infected” and “infectious”.

In the manner essential to proper probability analysis but likely to put off most people, we must think about test outcomes in much greater detail. A PCR test is simply an attempt to determine whether the COVID-19 virus is present, usually but not always at some minimum level, in the sample taken. No test is completely reliable, but, more important, there may be large variations between tests according to how the sample was taken, how the sample was handled and how it was processed. The assumption of constant – and low – error rates is contrary to all experience when carrying out decentralized testing unless there are very strict quality controls in place. It is not possible to process 200,000 or more PCR tests per day in different labs while maintaining identical and low error rates.

Next, suppose that a PCR test has accurately found the presence of the COVID-19 virus in a sample. Clearly, the person tested has been exposed to the virus at some point in the last six or nine months – i.e., they have been “infected” at some point. However, this tells us almost nothing about whether they are “infectious”. The window of time during which a person who has come into contact with the virus is actually infectious is quite limited. In very stylized terms, the average period from contact to becoming infectious seems to be 4-5 days, while the average period during which someone is infectious is 8-10 days, and the average period from becoming infectious and the virus being eliminated from their system is 28-35 days. These numbers vary greatly across the population and, in particular, some people seem to retain low levels of the virus in their systems for three months or longer.

Once it is recognized that there may be large numbers of people who may have the virus (i.e., who return valid positive test results) but who are not infectious, the logic underpinning current policy must be seen as extremely questionable. While it is very difficult to make reliable estimates, there is a strong likelihood that at least 80% of those who return positive tests are not in reality infectious if we test the population at random. To emphasize, this is not a consequence of the technical meaning of the false positive rate – i.e., the proportion of all tests that return a positive result when no virus is actually present. Instead, it reflects the fact that most of those who have the virus are not, in fact, infectious.

How did we get to this position? We can blame the usual combination of incompetence and sloppy thinking among politicians, bureaucrats and journalists. At the outset of the epidemic, the proportion of the population who had been exposed to the virus was practically zero. People were being tested only if they had recently developed significant symptoms or were in occupations that were likely to expose them to the virus. In such circumstances, almost everyone who had a positive test would be infectious. The false positive rate was not important because the prevalence of the virus in the population being tested was relatively high. But none of this context is true today, so the error rate in making the equation between “infected” and “infectious” is now very large.

Both Spiegelhalter and the Huffington Post rely on the idea that the people being tested now are not a random sample of the population but are largely those with Covid symptoms. This is clearly wrong. All of the public evidence points to a desire by institutions – schools, universities, care homes, companies, professional sports teams – to test as many people as possible to insure themselves against liability for spreading the virus. This is made worse by the seasonal resurgence in the occurrence of non-Covid virus symptoms which overlap with Covid virus symptoms. In any case, it is certainly not the guiding assumption behind the pressure to increase the number of people being tested. The goal is to ensure that a large proportion of the population can be tested daily or weekly. That is the route to disaster – of policies founded on an erroneous interpretation of results from mass testing.

Being generous we might acknowledge that policymakers are operating on the assumption that most, but not all, positive tests are for people who are infectious. The difficulty of identifying who among those infected are actually infectious might warrant imposing costs on all of those who are infected for the overall social benefit of preventing transmission. Even so, this is dangerous territory; in other areas of public policy we tend to veer in the opposite direction – better N guilty people go free than one innocent person be convicted.

The question is what should the number N be in this context? Is it reasonable that four non-infectious people be isolated for every one infectious person who is isolated? What if the number is not four but 10 or 20 once we allow for family and contacts? Some honesty and careful thought is required to address such questions, not just a set of arbitrary rules that appear to have no logical foundation and cannot easily be enforced without applying draconian penalties. It seems very unlikely that such honesty prevails, even in private discussions of current policies. Further, the ratios are going to get much worse as either (i) the proportion of the population who have been infected but are no longer infectious rises, and/or (ii) the rate of testing of asymptomatic people increases.

The core problem is that the UK is relying upon a set of policymakers who appear to be neither competent nor willing to be honest about the choices that are being made. Selling policies to credulous journalists is simple. Convincing a wider public that contains a significant number of people who have the expertise and willingness to challenge what appears to be irrational policy is a different matter. The UK Government appears to believe that honesty will further reduce already low levels of willingness to comply with current policies. What they – and their public supporters – should recognize is that the unwillingness to admit and discuss the complexity of the cost-benefit calculations that surround testing merely fuels the lack of trust in current policies. Many of the public statements are so obviously wrong that it is hard to give credit to the more generous view that the policymakers are not really incompetent but merely dealing with a very difficult situation.

That, it seems to me, is the real lesson from Sweden. There were many people in the Swedish medical establishment who profoundly disagreed – and still disagree – with the policies recommended by the State Epidemiologist. There was an open debate and individuals were allowed to make their own choices. The outcome was mixed and the sources of failure have been identified and followed up. In the UK – across all of the nations – the attitude is that we (the “experts”) know better than everyone else and you must just do what we say. That is never a good story in an educated democracy. It is especially weak when the self-declared experts are easily challenged on a string of assumptions and issues so that their credibility cannot be sustained. All that is being achieved is a polarization of trust and belief in the competence of both politicians and those who advise them.

I would make a final remark. The Huffington Post article applies the logic: why would any politician damage the economy on the basis of willfully misinterpreted evidence? There is a simple answer. How many politicians – or bureaucrats – would not do that in order to avoid having to admit that they had got things badly wrong? Admitting one’s mistakes comes hard to everyone. John Maynard Keynes is notable for his willingness to change his mind when the evidence changed. He was not a politician and he was a remarkably intelligent and open-minded individual. None of that can be said of our current policymakers.

Gordon Hughes is a former Professor of Economics at Edinburgh University.