Tom Chivers at UnHerd has published an article headlined “PCRs are not as reliable as you might think“, sub-headlined “Government policy on testing is worryingly misleading”. The core argument of the article is that due to high rates of false negatives, a positive lateral flow test followed by a negative ‘confirmation’ PCR should be treated as a positive. I pass no comment on this. However, the article makes a claim that itself needs to be fact checked. It’s been quite a long time since PCR accuracy last came up as a topic, but this article provides a good opportunity to revisit some (perhaps lesser known) points about what can go wrong with PCR testing.
The claim that I want to quibble with is:
False positives are so rare that we can ignore them.
Claims about the false positive (FP) rate of PCR tests often turn out on close inspection to be based on circular logic or invalid assumptions of some kind. Nonetheless, there are several bits of good news here. Chivers – being a cut above most science journalists – does provide a citation for this claim. The citation is a good one: it’s the official position statement from the U.K. Office for National Statistics. The ONS doesn’t merely make an argument from authority, but directly explains why it believes this to be true using evidence – and multiple arguments for its position are presented. Finally, the arguments are of a high quality and appear convincing, at least initially. This is exactly the sort of behaviour we want from journalists and government agencies, so it’s worth explicitly praising it here, even if we may find reasons to disagree – and disagree I do.
Please note that in what follows I don’t try to re-calculate a corrected FP rate, alternative case numbers or to argue that Covid is a “casedemic”.
Let’s begin.
Lab contamination. The ONS’s first argument goes like this:
We know the specificity of our test must be very close to 100% as the low number of positive tests in our study over the summer of 2020 means that specificity would be very high even if all positives were false. For example, in the six-week period from July 31st to September 10th 2020, 159 of the 208,730 total samples tested positive. Even if all these positives were false, specificity would still be 99.92%.
This seems quite reasonable at first, so I don’t blame Tom Chivers or the ONS for making this point; indeed, I also accepted it for a while. However, it rests on an assumption they spell out explicitly later in the document:
Assuming that false-positives… occur at a roughly similar rate over time…
Taking the lowest rate of positive reports as the FP rate requires modelling FPs as a form of uniformly random noise, but this isn’t valid.
Contamination is a very serious issue in PCR labs. We know this because in 2018 the WHO published guidance on how to run a PCR testing lab. Recently, a Chinese biotech firm called Bioperfectus published an article with very similar requirements. The advice is extreme and demands conditions comparable to the cleanliness of a semiconductor fab:
- Use four physically separate rooms for different stages of the testing process.
- Positive/negative air pressures should be maintained between the rooms. The rooms should each have dedicated air ducting such that they exhaust to the outside world separately. Note: this requires specifically constructed buildings and specialized air handling equipment.
- Each area requires not only separate air handling but also separate lab coats and gloves. Staff must use a one-way system in which neither people nor equipment ever go “backwards” during a day’s work. If it becomes necessary to do so, decontamination procedures are required.
- Equipment must be regularly cleaned with ethanol, sodium hypochlorite and de-ionized water. There must be 10-15 minute gaps between these cleaning stages.
And so on. Contaminants don’t have to come from the environment. Research into how the PCR process itself can create false positives due to “carry over contamination” was being done as recently as January.
Given the major contribution contamination makes to PCR FPs, it is logical that at any given lab there is no one fixed value but rather that the false positive rate is actually a multiplier of the true positive rate. With low real prevalence there is hardly any virus to contaminate the lab. With high prevalence not only does contamination become intrinsically more likely, including from other possibilities that the WHO advice ignores completely like infected lab workers, but the workload increases and pressure on the lab to cut corners grows along with it.
Still, the low base rate does at least indicate that at least in summer months when not under pressure and when contaminants are rare, there is very little “noise” in the test.
The other arguments presented by the ONS likewise rest upon this assumption:
…high rates of false-positives would mean that, the percentage of individuals not reporting symptoms among those testing positive would increase when the true prevalence is declining because the total prevalence is the sum of a constant rate of false-positives (all without symptoms) and a declining rate of true-positives
External validity. The ONS argument is based on data submitted by the labs themselves. However, this isn’t the FP rate that citizens actually care about. They care about the end-to-end performance of the entire system, and that captures false positives introduced by bureaucratic failures of various kinds – for example, workers deciding to report an inconclusive test result as a positive because ‘better safe than sorry’, results getting sent to the wrong person and someone being told they tested positive even when they didn’t take a test at all. There have been many reports of problems like this happening. I myself have received a PCR certificate for the wrong person (who shared my last name), and I know how it can happen. After my fiancé’s test results were “lost” I watched as the site workers fought with a badly written web-app that randomly auto-completed other people’s details into the form they were trying to fill out.
To determine the actual false positive rate of Covid testing an end-to-end study that includes the actual citizen-facing sites and systems would be required. Governments appear to have never done this.
Occasionally, scientists do perform lab challenges, sometimes called “external quality assessments” (EQAs). Artificially created samples are submitted that are known to contain certain types of viruses or not, and the lab results checked. Although this doesn’t verify the infrastructure of a mass testing programme, it is nonetheless more informative than just checking the lab’s own testimony. A meta-analysis from 2020 looked at the history of EQAs for RNA viruses post-2004, and then looked at how much those observed FP rates could affect claims about Covid:
Review of external quality assessments revealed false positive rates of 0-16.7%, with an interquartile range of 0.8-4.0%. Such rates would have large impacts on test data when prevalence is low. Inclusion of such rates significantly alters four published analyses of population prevalence and asymptomatic ratio.
The study also concluded that reliability didn’t improve over time.
Cross-viral confusion. To what extent can PCR tests mix up different pathogens? Scientists make extremely strong assertions that this cannot happen under any circumstances. However, in the past there have been incidents where this did in fact happen.
In 2015 a large scale lab challenge was performed. Labs were sent samples they were told contained MERS-CoV, but some contained other coronaviruses like those that cause common colds. Many yielded no false positives at all, but around 8% of labs incorrectly detected MERS-CoV. NB: 8% of labs reporting false positives is not the same as an 8% false positive rate.
In 2003 an outbreak of a common cold virus (HCoV-OC43) occurred at a Canadian nursing home. Because this occurred during the SARS-1 epidemic in Asia the samples were subjected to routine PCR and serology testing for SARS. Unexpectedly, both tests indicated a SARS outbreak. Because this was implausible further PCR testing was done, which confirmed the far more likely cause of an OC43 outbreak.
In 2006 an epidemic of whooping cough was announced to be occurring at a U.S. hospital. Over 1,000 staff were furloughed and quarantined, ICU units were closed and 142 people PCR-tested positive for the disease. On further investigation using a simpler and more trusted but slower type of test (culturing the bacteria), all 142 results were found to be false positives. The event was dubbed a “pseudo-epidemic” and reported on by the New York Times. I wrote about this event in more detail last year. The staff concluded they had misdiagnosed whooping cough when in reality it was just an outbreak of a normal respiratory virus because they didn’t want to argue with the apparently more ‘scientific’ PCR results, even when their own clinical experience raised doubts.
This highly heterogenous pattern of false positive rates in which some labs yield none and others yield a large number is a common outcome of lab challenges. It isn’t surprising assuming the primary driver of FPs is contamination, yet means any attempt to characterize the FP rate of an entire mass testing programme at a single point in time is guaranteed to be incomplete. To detect and fix labs generating high FP rates would require a continuous challenge programme, yet governments – misled by scientists claiming that the technique is inherently immune to FPs – haven’t done this.
Logic errors. Many claims about the accuracy of Covid testing turn out on close inspection to be logically invalid. Trying to calculate a ‘true’ FP rate for Covid testing is extremely difficult due to the ubiquity in public health of circular logic that goes like this:
- A PCR test returning positive implies a Covid case.
- A Covid case implies a positive PCR test.
This situation means that in many discussions Covid PCR testing cannot have false positives by definition. Too often, claims that the tests are FP free or have nearly no FPs must therefore be discarded, because they are merely measuring tests against their own output – an approach which has no scientific validity.
This problem also affects many arguments based on symptoms. A typical dictionary definition of disease is as follows:
a disorder of structure or function in a human, animal, or plant, especially one that produces specific symptoms or that affects a specific location and is not simply a direct result of physical injury.
Oxford English Dictionary
The symptoms of Covid were in the very beginning quite specific, involving a new and unusual type of pneumonia. Once mass PCR testing began the list of symptoms expanded to include whatever seemed to be wrong with people who tested positive. The current definition of Covid includes every symptom you might find in the general population, including no symptoms at all, which is what you’d expect to see happen in an environment where a test that has false positives is treated as if there are none.
Circular logic has corrupted or destroyed attempts to determine the accuracy of Covid testing on a truly staggering scale. In 2020 a meta-analysis study published in the Journal of Infection looked at every paper published up to that point on the accuracy of Covid tests. Of the 43 studies they located, none of them was methodologically valid. They concluded:
Current studies estimating test performance characteristics have imperfect study design and statistical methods for the estimation of test performance characteristics of SARS-CoV-2 tests
…which is quite the understatement:
Critical study details were frequently unreported, including the mechanism for patient/sample selection and researcher blinding to results
In other words, the scientists didn’t explain how they identified people as having Covid, then compared test results to their sample anyway (=results are useless). But in most cases the problem was circular logic:
Eight studies… attempted to determine the accuracy of rRT-PCR by comparing the initial rRT-PCR result to the result after multiple repeated samples from the patient… Suo et al. considered a positive result of either repeated measurements of rRT-PCR or serology to indicate a positive test according to the reference standard… Three studies determined the accuracy or agreement of rRT-PCR or automated rRT-PCR platforms/instruments compared to a reference standard based on the results of several tests as a “composite reference standard”.
etc., etc.
Testing a test against itself is not a valid way to measure its accuracy. The results from this sort of approach are actually a measure of test-retest reliability, not sensitivity or specificity. The problem is compounded due to incorrect interpretation of test flips. A test that returns both yes and no in quick succession should have its output discarded. Instead, scientists routinely classify these as false negatives (because FPs are “impossible”). They then report that Covid tests have a high rate of false negatives but no false positives, which re-assures scientists that FPs are impossible: yet more circular reasoning.
Conclusion. The belief that Covid testing has such low FP rates as to be ignorable is based on a combination of highly heterogenous lab accuracies, an incorrect assumption that PCR FPs are characterisable as random noise, and rampant circular reasoning within public health research.
This essay has some limitations to bear in mind:
- There are other sources of PCR false positives that aren’t mentioned because they’re well discussed elsewhere already, like how PCR tests are routinely interpreted as evidence of infectiousness when that isn’t what they actually test for. The topic of cycle thresholds is related to this point. I make no claim to be comprehensive.
- Terminology can prove difficult. “False positive rate” can be used to mean the fraction of all tests performed that are incorrectly positive, or to the fraction of all positive results that are false, depending on context and author. These difference can result in wildly different numbers for what is actually the same claim.
- Most of the studies relied on were published last year. There may be newer studies with better methodologies I’m not aware of, including studies that contradict the arguments presented here.
- To repeat for clarity: these problems do not imply the false positive rate is very high. They imply we do not seem able to characterise Covid test accuracy rigorously, even nearly two years into the pandemic.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.
I know of 3 people in my circle of 100 or so friends who have had false positives (a positive test followed immediatly by one or more negative tests) Therefore it can’t be that rare.
For your claim to be in context you ought to also say how many of those 100 friends have ever taken tests at all, and how many have certianly had covid, preferably also breaking down which if any who had it got tested at the time.
But no, false positives won’t be rare. One of the most telling examples is to look at people who’ve returned positive PCR’s many times, if it was truly testing for disease then any given person would only be able to get a true positive once in a period of several years or more. Given how good natural immunity is someone with multiple positive PCR tests from intervals of weeks to many months apart will have either had covid at one of those tests and false positives at all others, or never had covid at all.
I “think” it tests for a fragment of a molecule. If that is correct how can we be sure of the fragment being sufficiently unique.
Also, Kary Mullis was scathing about using PCR at high cycle thresholds
Very few people test positive multiple times after a gap without testing positive. It does happen but it’s a tiny fraction of the total.
The most we can really say about false positives is that they must happen given some of the known facts, and the argument that the ONS presents for them being an ignorable problem isn’t really valid. It’s very difficult to present alternative calculations because the data you’d need to generate them isn’t available.
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
Yes, I’ve read it already. Which bit specifically are you interested in?
The fact that the government clearly states that (through this document) that the PCR test ‘is not able to distinguish whether infectious virus is present.’
Sort of negates the discussion really. Further more, questions the whole narrative, lockdown agenda, case rates etc.
Yes, I know. Last paragraph:
What the government doc means is it can also detect destroyed virus, not that the results are entirely disconnected from viral presence. So unfortunately it doesn’t quite negate the discussion.
Although infectiousness matters for quarantine/pinging, test data is used for lots of other things too. For deciding on travel “red lists” or anything else where the user only wants to measure general prevalence, the distinction between “infectious at time of test” vs “infectious prior to test” doesn’t matter much. But we are still interested in the accuracy or not of those use cases!
Plus even for quarantine purposes, although a positive result doesn’t always imply infectiousness, it’s obviously not true that therefore a positive result implies non-infectiousness. The question is how often people are both positive and non-infectious, and what the costs of those quarantines are vs the “benefits” of forcing the non-infectious to isolate “just in case”, which is a question of priorities and costs rather than a slam dunk debate-ending problem – that is, even if e.g. 70% of all PCR positives are non-infectious, that may seem like a no-brainer for ending self-isolation policies to us, but to someone with a really distorted idea of costs and benefits it may still make sense
What the fuck are you going on about?
PCR tests are not able to detect infectious virus.
What are your medical qualifications?
What last paragraph are referring to?
You seem to be confused. If a pathogen is non-infectious, what’s the problem?
The issue of false results is a key issue that is explored in standard statistical texts – and it’s not some obscure, aberrant phenomenon.
who the fuck are you never been here before? I went to a party of 22 persons 20 got the plague 2xjabbed 2 others unjabbed me included, caught it, one carrier and one previously infected person unaffected. Put that in your pipe and smoke it
How many are now dead and buried?
I have driven through the Taunton Crematorium grounds regularly since the Covid outbreak was declared open last year, and have yet to have seen a single hearse, let alone a queue of hearses. Where are the piles of bodies that Whitty and Co said would be here with half a million dead in the first three weeks, unless we did as we were told and don’t argue?
The whole thing is a scamdemic and the government are lying through their teeth at us.
Yes. Someone posted here a while back the results of an information trawl of burials and cremations in certain authorities.
It would be interesting to see a wider sampling.
PCR tests do not detect live infectious SARS-COV-2 virus.
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
Anyone who’s had a cold will test positive.
Surely the issue with the PCR tests is that they don’t test for Covid-19. They test for the presence of RNA sequences that we’re told identify SARS Cov2. That’s not the same thing.
Even if the virus is present, how many other viruses are present in a given human body? But that’s almost beside the point. The point is that Covid-19 is supposed to have completely unique disease symptoms for serious infections requiring hospitalisation. It goes from a respiratory viral replication stage (common to all respiratory viruses), to a cytokine stage where the immune system attacks the lungs and other organs (some doctors have identified this as an anti-phospholipid condition) and finally you get thrombosis and death. If a test doesn’t definitively identify that disease from those symptoms, it can’t seriously be considered a Covid-19 test.
This is exactly it. It’s a farcical test that serves no propose but to generate “cases” to create the illusion of a pandemic.
The late President of Tanzania, John Magufuli, submitted samples from a goat, a pawpaw and motor oil for PCR testing, all came back positive for the virus.
The entire rationale for mass testing is the myth of “asymptomatic infection”. This lie first surfaced in March 2020 when Christian Drosten (also author of the current testing protocols in use) published a fraudulent paper on it. The media ran with it and it’s been stuck in people’s psyches ever since.
Also let’s not forget that back in 2019 it was Drosten who “invented” these protocols in Germany, using a computer model for the proteins, and sent them to the Chinese to test their population with. So we have a test that was computer generated, for a scenario based on computer models. If that isn’t technocracy, what is?
I believe Drosten invented the protocols after a WHO specification was issued on the subject; and so it came to pass that over a three day period in January 2020 a process was published, “peer reviewed” in Eurosurveillance – at least one person in the “review” process was conflicted, adopted and mandated as the Gold Standard mass testing regime throughout the world by the WHO ( who then recanted in part from this decision, twice ).
Reference :ILLA; The PCR Disaster.
One member of the UK MHRA was part of the WHO specification process and “she” also signed off the adoption of the Gold Standard regime; did not recuse herself from the approval thereof….
“Chivers – being a cut above most science journalists”
I choked on my drink at that one! Even given the low bar, he’s pretty below average in the thinking area if you’ve been following his stuff on Unherd.
… as evidenced by that statement. The phenomenon of false positives is well known – especially in low incidence situations, as now.
He volunteered for another Covid vaxx trial, after having been selected for and participating in one already and was disappointed when he wasn’t chosen.
I think that tells you all you need to know about this chap and his intelligence in particular.
Unherd in general does at least have some balance unlike most of the usual MSM, Unherd has run plenty of anti-lockdown anti-authoritarian-policy articles alongside the odd stupidly pro-lockdown piece, whereas the MSM has only run pro-lockdown lies (and by liars who don;t even have the decency to try to back up their claims, just expect them believed by syaing “THIS is the science”).
Again – I wouldn’t disagree that Unherd has had some good articles.
But comparing anything to the generality of the MSM is setting the bar about as low as it gets.
As to Chivers – his record is pretty consistent – Poor.
all buddies at the same club mate. Toby’s bar.
He doesn’t automatically believe anything academics say, and understands how to do back of the envelope calculations using things like Bayes theorem, which is better than the vast majority of science journalists.
If journalists all nailed those two things we’d be in a much better place, even though it doesn’t mean you’d always agree with them.
That said my favourite science journalist remains Sarah Knapton at the Telegraph.
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
Bollocks! This article and the premise it’s set against is pure, unadulterated gaslighting.
Sure one could argue about contamination rates etc but this is not the issue at all or if it is only compounds.
The issue is and always has been; as per Kerry Mullis said, “you can find anything you want to with this test”.
This article completely ignores anything to do with the cycle threshold of the tests carried out in the U.K. and worldwide.
Anything about 25-28 makes all the data pure rubbish and the U.K. rate is 40-45 so it tells you NOTHING!! And certainly nothing about actual infection. The test is totally fraudulent as has been known from day one.
When the double poisoned started testing positive via this test the WHO quickly changed their tune to suggest for this cohort (but not any other, no fucking surprise) the test should be limited to a ct no higher than 28.
In effect, this article is merely a regurgitation of that awful Panorama bullshit put out last year, total deflection.
Sorry, but to publish this without the counter argument strengthens my belief this site is little more than controlled opposition.
Articles like this really grip my piss!
Hey, my first ever thumbs down, hiya Toby;-))
Or should that be, Corporal?
Cycle thresholds are mentioned in the final section:
The main reason the article doesn’t discuss Cts further is because the claim that high Ct = useless results is a weak argument that would get you slaughtered if you went up against a competent opponent in public debate.
Kary Mullis was exaggerating for effect. Think about it. If it were true that at high cycle thresholds the test can find “anything” then the tests would return positive nearly all the time. Any argument about false positives has to handle the fact that positivity rates never go above 15% any more, even in the middle of the biggest waves, and during summer months the positivity rate is often <1%. Additionally, excess death curves follow test curves reasonably well, and test positivity follows the wave patterns that are normal for epidemics.
If the results were genuinely noise 100% unrelated to actual viral presence, then it’s hard to explain these things. That’s the point the ONS and Chivers are making.
I might be wrong but I believe you’re referring to the CDC not the WHO, and this turned out in the end to only be the criteria for breakthrough cases they wished to investigate further, not for the classification of any positive at all, which remains the same as before.
Consider that if the criteria were truly very different then it would be impossible for raw/unadjusted vaccine effectiveness to go negative, as it has done in the UK.
Lol. If you figure out who’s controlling me please remind them to send the cheque, because it never turned up.
Mr Hearn, your article is well written as a constructive crticism of another writers piece. It has the ‘feel’ of a personal construct as if two people were having a friendly banter in the pub, a bit of one-upmanship to prove the superior intellect.
Of course we know the ONS and the CDC have been paragons of virtue in how they have handled the statistics in this SARS2/covid business. And all the readers on here are delighted in your explanation of how we might have misinterpreted their findings.
I know, sarcasm is the lowest form of wit, but sometimes it has its place.
Look, it’s frustrating for everyone me included, but COVID measures persist because lots of people believe they make sense and support them. The only way for this to change is to make and win arguments with the open-minded-but-government-supporting section of the population. To do that you need arguments that make sense and don’t over-reach.
People can view this site in a couple of ways. One is scepticism as a noble quest for truth, however convenient or inconvenient that truth may be. And the other is simply strategic – you want life to go back to normal and you want ammo (arguments, news). People in the latter camp will inevitably sometimes see stuff written by the former camp as “controlled opposition” because where’s the ammo? But regardless of which camp you’re in, what you don’t want are faulty arguments that blow up in your face the first time you try to use them to convince an open minded audience.
Last year I got an ‘inside view’ as some naturally sceptical people in power were pushed away and back towards SAGE by claims that over-exaggerated the scale/impact of PCR false positives. People who argued COVID was 100% a ‘casedemic’, that the test results were completely meaningless, ended up actually convincing important people to follow the government instead, because those arguments were wrong.
So. If you want to argue test results are meaningless noise with a near 100% FP rate, fine, go ahead and do that (somehow). But you cannot then turn around and make common sceptical arguments like “test data proves lockdowns/mask mandates don’t work”. To be consistent you’d have to say something like, “we think the test data is 100% wrong and therefore we don’t know if NPIs work or are worth it, because we have no data on which to base that decision”. Also if you think the data is faked by the government you’d have to explain why that data so often seems to disprove the government’s own narratives.
Here’s what I think:
These are real problems! Fixing them wouldn’t change much because government policy is only tenuously connected to reported data anyway, but pointing them out is (yet another) way to make people realize that even the most trusted institutions and segments of society don’t seem to be doing their jobs correctly when it comes to COVID.
If you don’t care for these arguments, fine, don’t use them. Go ahead and keep claiming that every test is a false positive, more power to you. But I think your impact will be low.
I understand the thrust; “construct your case empirically and not by any other method” I think is what you are , in effect saying.
Surely part of the issue is that HMG has never “properly” established the FP rate; if that is the case, there appears to be no stated allowance for what a non statistician might deem as a “margin for error” in these “case” numbers – I acknowledge I might be wide of the mark.
The strategy adopted by SAGE and foisted onto a bunch of non scientist politicians who seem incapable of countering the likes of Ferguson, Michie, Vallance , Whitty and others with vested interests ( NHS/PHE etc) appears entirely based on “cases”; cases determined by a very dubious mass testing regime which has been heavily criticised by specialists in this field. Such people have a lot to lose by voicing their concerns because of the WHO/Governmental gagging of MSM – worldwide – and the vilification they receive at great personal cost. Why would they do so, to their own detriment – for publicity, “anti establishment kudos”, money, fame….?
Lets not forget; Johnson & co stated that their strategy was to save the NHS; well I’m no genius very self evidently, but by not mandating and facilitating early treatments to keep patients OUT of hospital, Johnson/Hancock et al have literally broken the NHS in less than 2 years; if you think that is over the top, suss out the scale of career clinicians leaving the NHS, never to return, if you can. I have information that Trusts are already scaling back capacity to cope with the current – low – levels of ICU demand and we have not yet entered the Winter Horror Story of 2021/22; staff shortages are legion, be it from retirements, CV19 infections or the elephant in the room – CV19 induced workplace stress.
Your argument is a classic case of a victim of misdirection.
“The only way for this to change is to make and win arguments”
Unfortunately, no. Although I don’t disagree with the need for that process, most people have been programmed to accept falsity – not argued into that position. And I agree about counter over-reach.
However, the PCR argument is a distraction, because the test has been used to stimulate panic through the creation of ‘testdemics’ that disguise the fundamental fact that there never was any health ’emergency’. This is the foundational untruth about a non-epidemic, non-pandemic (only called such by removing the key definition of mortality).
You can argue until the cows come home about false positives, but it’s not an argument that is essential to the key issues of the scamdemic, and the programming of the population into a state of hysterical hypochondria.
fair enough Mike. Here in hard border Western Australia we haven’t “yet” had very many cases (or “cases”). I did think the false positive rate in the context of low prevalence should have created many more false positives and false alarms. I’d be surprised if the labs were super efficient at clarifying false positive test as genuine negative test so quickly. It has made me question some of what i’ve read which is not to say i’m any the wiser either way
Just had a chance to see your response after travelling from Panama to Florida on the back of an antigen test which was negative ( as most antigen tests are).
I think Rick H response to your response sums up my feelings well. nothing much to add. You are conducting an intellectual argument whilst Rome burns. We are talking about a fascist biosecurity state being erected whilst you talk about whether RT-PCR tests are accurate. Personally I am well beyond that stage, by at least 12 months.
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
WS, keep going, the penny might drop; it is amazing that when the Government publish articles which everyone can access stating, and I am paraphrasing for sure, that a RT-PCR test result on its own cannot be relied upon for “accuracy”, and must be followed up with other “assessments” ( not verbatim), how long is it going to take for people to realise the extent of this mass testing fraud. They ( .gov.uk) also have some “interesting disclosures” about the effectiveness of CT rates.
May I again (YAAAAAAAAAWN) recommend the ILLA document investigating the Cormen Drosten RT-PCR scam, pages 57-70 for a detailed explanation how this fraud came to be. And guess what, they even quote Drosten confirming a PCR positive test on its own proves precisely NOTHING!!!
May I also remind folks that several FoI requests for the CT rate used for every “positive” PCR test has been denied by HMG, including my own, saying that information is not held but some information might be available from “our academic partners”. SO, the greatest reliance is placed on RT-PCR testing outcomes, and the acknowledgment of the importance of the applied CT rate by HMG, BUT ..they don’t keep track of the numbers ( strange that, don’t you think?) and they even say that different assay regimes from different testing regimes applied CANNOT be compared; yet the results from all these “different assay regimes” ARE combined and pushed out via the MSM as “cases”.
FFS how much more world class evidence – from HMG’s own website FFS!!!! – do “you” need before “you” say – “Oh, I get it NOW”
“ the claim that high Ct = useless results is a weak argument “
Not at all, since real world infection relates to viral load. High Ct means difficulty in detecting even tenuously related RNA, let alone an infection.
Thus the ‘casedemic’ problem, which is not contradicted by RNA fragments roughly tracking real infection.
My bullshit antenna got me no further than the first few lines of it. Just rabbit hole rubbish
Just anecdotal, but shows the power of the positive test. A work colleague and her son were ill with exactly the same symptoms, although she was rather worse than him. But he tested positive and was isolated in his room for a week. She tested negative. In normal times we would have assumed two members of a household with the same symptoms had caught the same infection and got better after a few days and that was the end of it.
If the incidence of virus is low in the population – say 4 cases per 100 – and the PCR false positive rate is anything from 0.8% to 4%, as the government told us last year…
…If you test 1000 random people you’d expect 40 to test positive based on the prevalence of the virus. But in reality you would get anything from 48 to 80 positive PCR test results – 40 true positives and between 8 and 40 false positives (we’ll ignore the impact of false negatives for simplicity but they are less of a problem than false positives with PCR).
Believing you have twice the cases that you actually have seems pretty significant to me. Chivers is talking bollox.
As an aside I remember Chivers from his days blogging on the Telegraph. His articles were savaged by readers below the line. He’s not a credible journalist.
You have missed Chivers’ point. Yes when prevalence is low then a greater percentage of the positives will be false positives. That is well known and accepted. Chivers is simply trying to set a limit on the rate of false positives. So if the rate of positives increases then it is almost certainly down to true positives. It is a perfectly reasonable argument as Mike Hearn himself admits.
Another way to phrase it is the positive rate increased from about 2000 a day in the summer to well over 30,000 a day at the moment. Can you reasonably expect that increase to be down to an increase in false positives? A sudden outbreak of mass incompetence at the labs?
“So if the rate of positives increases then it is almost certainly down to true positives. ”
Yes all other things being equal. But all other things aren’t equal.
“Another way to phrase it is the positive rate increased from about 2000 a day in the summer to well over 30,000 a day at the moment.”
But how many tests are being done now compared to the summer? More. Many more. But the share of tests which are positive has remained under 5% – little more than the upper bound of the PCR false positive rate – for nine months. See graph attached.
So, as I said, the background rate of infection is low, and a false positive rate of between 0.8 and 4% can therefore have a big impact on exaggerating case numbers.
But how many tests are being done now compared to the summer? More. Many more.
I am afraid that is false. Just look at https://coronavirus.data.gov.uk/details/testing July was much the same testing rate as now. It dipped a bit in August but not nearly enough to explain the difference between 2,000 and 30,000 positives.
Your chart is misleading because it includes the 30% rate early last year (when there was very little testing being done and that was concentrated on the people who were getting ill in the early stages of the epidemic). This peak is so high you can’t see the big shift in positivity from the summer to the autumn. I attach a chart showing just that period and the shift is clearer. We went from almost no positivity to about 5%
Blah blah blah
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
Read this you twat
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
What defines a false positive %?
Is it % of positives that turn out false, in which case how was that determined or is it a % of total tests that are false positive and again how determined?
This document explains more on the topic It’s also where the reference to a FPR of between 0.8 and 4.0% for PCR comes from.
https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/895843/S0519_Impact_of_false_positives_and_negatives.pdf
its a test that indicates someone is infected when they are not. Look in your underpants/knickers what do you see???? M/F Penis or vagina Left or right. Its binary. There is nothing in the middle, infected not infected
And don’t forget: infected & infectious?
PCR doesn’t determine that does it?
This test is so easily manipulated to give any desired outcome it’s beyond a joke but it suits an agenda hence it remains.
Yes, during the summer months it’s possible that ~all positive results are false. The ONS survey pilot paper stated exactly that last year:
Chivers is well aware of Bayes theorem and uses it in the article in question. The problem here is the assumption that the FP rate is extremely low. People who consider COVID to be a major crisis don’t care about small absolute numbers of FPs in summer even if the relative rate is very high, as it is perceived as a trivial cost relative to the benefit of having the tests.
Fundamentally the issue is an epistemic one. We don’t actually have any idea of what the true FP rates are. The ONS argument is only valid if you assume FPR is static and universal, i.e. that FPs come from the tests randomly breaking. The moment you use a more realistic model in which FPRs are correlated to TPRs then their reasoning falls apart.
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
Read it
An interesting and thoughtful article, Mike.
But you are obviously a much nicer person than me, for you obviously believe that those working in the testing industry and the ONS are saintly people who are doing their very best.
In very many cases, I’m happy to acknowledge that this is indeed the case.
But, unfortunately, politics rears its ugly head.
Many other commenters here have pointed out the obvious deficiencies in the testing industry.
The ONS have shat on their own doorstep often enough, by blatantly producing bogus data supporting HMG’s chosen “science”, notably in the field of Climate. As usual, policy based evidence making at its finest.
That isn’t a criticism of many hard working and sincere statisticians. But there are also the other kind, and especially those selected by HMG to head it up.
The fact that HMG has hand picked the likes of incompetent Professor Pantsdown Ferguson and Communist Party stalwart Susan Michie (and many others, notably Farrar, Vallance et al) and annointed them as “The Science” really tells you what you need to know.
The fact that, today, someone being run over by a bus 27 days after a positive Covid test (any test, false or not), is still a Covid “death”, proves that HMG wants to inflate ‘deaths’ and continue Project Fear.
The fact that Her Majesty’s Loyal Opposition and all the other parties, regional Governments are even dafter and less competent than Boris’s team, is no excuse.
Sorry to criticise your piece on grounds you chose to exclude, but it is essential that people recognise exactly what is going on.
There’s a lot to agree with here, except do we actually need to assign any sinister motives to people? Almost everything can be explained by refusal to admit past mistakes + doing whatever seems emotionally most appealing at the time (usually ‘saving lives’, or ‘keeping people safe’).
Public health is not a field I’d like to work in, because if you make any mistakes at all – or even just difficult tradeoffs – then you have to go to bed at night knowing your decisions led directly to people dying. Moreover there will never be a shortage of people reminding you of that, if you let them. Yet, make decisions you must.
The natural defense is to ignore any evidence you were wrong (“conspiracy theories”), ignore any people who claim you made mistakes (“conspiracy theorists”) and try to avoid changing direction at almost any cost. Especially don’t do anything that could be presented as not putting human life first, because saving lives overriding all else is a universal cultural norm.
I’m of a libertarian bent partly because of problems like this. Public health as a field just has way too much power. It’s not even that abstract power is corrupting – people corrupt themselves as part of trying to maintain self-esteem in the face of the terrible magnitude of their decisions. The only way to truly fix it is to stop imposing decisions on other people in the first place.
Yes exactly this Mike. It’s all in Bayes Theorem. First year undergrad stuff.
https://www.gov.uk/government/publications/cycle-threshold-ct-in-sars-cov-2-rt-pcr
WS, keep persisting mate!
A simpler conclusion from the same data would be: All so-called COVID tests are unreliable and given how many of them are being performed all the time, have certainly led to a lot of people who should have been isolating not isolating because they wrongly believed to be free of Sars-CoV2. Nothing particulatly noteworthy has happened because of this. Ergo: both testing and isolation are demonstrably useless and should be discontinued.
I got tested going into hospital, next day they had to test me again as there was something wrong with the test, I am going to find out what the problem was, as I am sure it was a negative test
While false positives undoubtedly occur and may well be a significant issue, isn’t the much more fundamental issue, as others point out here, that the test doesn’t really have much meaning. There’s only any point in testing for something if you then base some different course of action on the result, but the PCR test as done says nothing of significance. The whole thing is nonsense from the ground up. Arguing about false positives is yet another rabbit hole.
“The whole thing is nonsense from the ground up.”
That – detail apart – is also true, and hardly news to anyone that is awake and a ‘good journalist’. It has also been known for a long time .
Freddie Sayers has done some really good interviews – credit where credit is due, but Unherd has failed spectacularly to build on this in relation to the Covid scam.
The test should be performed by a doctor who has diagnosed COVID and wants to verify and it should be <28ct
absolutely
You provide a genuine, smart analysis of the fine print here.
But the big elephant in the room was, is and remains that the PCR test’s false positive rate is much closer to 100% than 0% IN PRACTiCE and under all conditions and scenarios, as it was, is and will likely remain (ab)used through the total lack of standardization, in particular with regard to the max. ct number.
Everything else is just noise, I am afraid.
And as this particular issue is widely known since June 2020 the latest, and as the only change in that regard since then has been the most outrageous and obvious fraudulent instruction to cap it at 28 for the vaccinated only in the US by the CDC, it follows quite clearly that the PCR test is solely a tool used very deliberately to engineer and uphold a fraud- the biggest, most costly and deadly fraud which mankind has had to endure sofar.
Chivers’ LFT vs PCR argument is about as opposite to the truth as possible, LFT has a small possibility of false negatives but relatively fewer false positives, PCR has huge numbers of false positives and virtually no false negatives. LFT is what might be called slightly under-sensitive, PCR is seriously over-sensitive. This is a consequence of how they search for evidence of the virus, in PCR anything that looks like a tiny bit of the virus gets amplified until it looks just like the virus, in LFT the virus is only found if the sample contains a large enough piece of viral code in perfect condition which by luck binds to the right chemical at the right point. And if LFT ever says positive but PCR says negative then the chances of contamination or human error somewhere on the chain look very high, when an under-sensitive test says infected but an over-senstivei one says uninfected you are going to be looking for some mistake as to why the tests would disagree in this fashion.
I would note here that today’s DS article seems to have forgotten about the argument from basic statistics as to why even if tests themselves have relatively low false positive rates, any given person taking the test may still have a greater likelihood of false than true positive, if they got a positive, for rare enough (and even 1% of the population is “rare” enough) disease being tested for. This is somewhat connected to the mention of different meaning for false positive rates, but it means that the interpretation of the accuracy of a given test result can often involve a different number from that which you use when describing test results across a whole population.
If a lab has been created to detect COVID-1984, and its funding and existence depends on continuing to detect COVID-1984, it will detect COVID-1984.
Someone’s missing the fundamental point. There is no such thing as a Covid test. There is a PCR process that can amplify a sample and then detect very small amounts of the RNA it’s been programmed to find. As The SARS cov-2 virus has never been isolated there is no way to ‘prime’ for it. However, let’s be generous. Let’s stipulate that by insane chance Drosten’s primers approximate SARS Cov-2. Now the PCR will detect microscopic amounts of it. How old? Unknown. Dead or alive? Unknown. Possibly the only way to make this ‘test’ at all relevant is to restrict the Ct to 30 at the most. Add lab contamination and the whole process is a joke.
False negative? False positive? Irrelevant. Pointless test? 100%
Exactly my point, well said.
PCR test is an intentional fraud designed by a fraud with a record of being so.
Correct. There is a whiff of political abuse around it, especially the way in which it is being used. Whenever something physical is measured, there is always an element of accuracy, and the tolerance of a given level of accuracy.
Just before the panic, I used one of the health screening programmes to identify potential cancer in certain age groups. The results were good (from my perspective), but the letter reporting the results was much more circumspect than the trash that is being issued re PCR for alleged sars-cov-2 infections. In short, they said something like “we didn’t find anything, but that doesn’t prove it isn’t there.” Followed by a list of dietary precautions etc, and do it again in 5 years time.
PCR does not test RNA substances; it has to convert the RNA sample to DNA by a reverse transcriptase process which is then used to amplify by the CT rate to determine a”positive” outcome – using computer software in the “final” analysis.
And is it not strange that those who refute that these mRNA jabs cannot effect your DNA – “impossible” – yet the very same result – conversion of RNA to DNA – is achieved during the assay process to determine a “positive” test.
Dr Richard Fleming and others have pointed to studies showing a flat lining of “results” from a PCR assay test using CT rates just over 25; WHO mandated Gold Standard is 40, and I have seen one FoI response from a NHS Trust ( Manchester if I remember correctly) which confirmed their testing used a CT rate of 41.
“Not a scientist” and appreciate this is a simplification of a complex process.
Mass testing of non symptomatic individuals by this WHO stamped Gold Standard RT-PCR regime appears to be 100% worthless.
And to MTF et al , “amendments” to the use of PCR in this setting might well have been made – does not alter the fact that this process does not test for infectiousness, cannot test for the whole gene sequence of SARS COV2, cannot distinguish sufficiently if a fragment of RNA>DNA genetic code exclusively comes from SARS COV2, no matter how good your primers etc are etc. I wonder what the incidence is of all the positive results of RT-PCR tests from their introduction to date has been successfully cultivated post CT amplification in a petrie dish and THEN was identified CONCLUSIVELY as SARS COV2 without recourse to the computer software manipulation – anyone know what that % is? What are the chances of assay tests from these “factories” being uncompromised?
Isolated: https://theconversation.com/i-study-viruses-how-our-team-isolated-the-new-coronavirus-to-fight-the-global-pandemic-133675
Isolated: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7045880/
Isolated: https://wwwnc.cdc.gov/eid/article/26/6/20-0516_article
Isolated: https://wwwnc.cdc.gov/eid/article/26/6/20-0516_article
You may not want to believe it, but at least have the decency to call everyone who claims it a liar, rather than pretending ignorance.
How is a false or true positive validated?
Side question: Why does this site communicate constantly with various payment processors such as Stripe and PayPal and also use Google Analytics? Daily Sceptics trusts Google!
Said it more than a year ago in BMJ Rapid Responses – the world’s shortest letter:
Operation Moonshot or How to Shut Down Society and the Economy Forever
Dear Editor
How many false positives will 10 million tests a day generate?
Good read:
https://citizenjournos.com/2021/10/06/confirmed-the-mater-hospital-was-not-full-of-unvaccinated-20-30-year-olds-on-ventilators-on-the-22nd-july/
Tom Chivers is a lying ,bought off cretin. There is NO GOLD STANDARD to measure CONvid against
Rather than getting distracted by the issue of false positives and diving down that rabbit hole, it’s far better to take a step back and ask “who cares?”. Why should anyone give two hoots about who and how many people are positive because we know that, with every vulnerable person having being offered the vaccine, cases are irrelevant. All I’m interested in is how many people are dead or languishing in ICU. Which, by any measure (including some false positives), is not very many.
Time to move on.
It’s not a case of false positives. It is a case of cycle thresholds.
If you test a drink to find out if it is poison and you found it containing 1% arsenic, it is poison, but find it has 0.001% arsenic and its apple juice.
The poison is in the dose. Pcr positive at 45 cycles is the same as a false positive. This cycle limit is the issue and always has been.
After nearly two tears of this nonsense and no one can agree on the use of PCR/lateral flow tests which were never meant to be used for the purpose they are being used for, diagnosing COVID19. This is according to Kerry Mullins the inventor of the PCR test. However, he died, conveniently (a healthy man died with pneumonia) almost immediately before the plandemic, November 2019. So it is no good trying to get his opinion. However, Drs. Ryan Cole, a US pathologist and Clare Craig a UK pathologist have plenty to say regarding these tests.
The PCR test is highly accurate at detecting SARS-CoV-2 if the testing lab has scrupulous controls to stop contamination resulting in false positives.
What it doesn’t detect is if the virus is viable or not and that can only be done in a wet lab and cell culture.
It has never been shown to test positive for inanimate objects in a scientific test, it is only hearsay.
The amplification or CT rate stops immediately when a positive result is achieved.
They may be designated to go up to 45 amplifications but taking the UK ONS Infection Survey as an example the vast, vast majority of positive results are achieved before 35 amplifications and there are NO positive results after 37 amplifications.
The first 10% of cases are positive at around an 18 CT rate, 25% of positives at around a 24 CT rate, 50% around a 31 CT rate, 75% around a 33 CT rate and 90% around a 34 CT rate.
It is governments who decide not to differentiate between the infectious and the possibly 60% who are not infectious, it is not a fault with the PCR test or most scientists. The scientists advising governments are the ones who are corrupt.
The Corman-Drosten et al paper was rushed through at the end of January 2020 during the initial panic of the “pandemic”. It was soon improved making it even more accurate at detecting SARS-CoV-2 and no other coronavirus.
SARS-CoV-2 has been shown to exist by “whole” gene sequencing over 4 million times.
Covid is real but has been blown out of proportion to bring in a Global Elite money and power grab and possibly something more sinister.
Could you please explain precisely how the virus was initially isolated and then proven to be the cause of the disease?
I am not seeking to be argumentative, it’s just a layman’s curiosity.
Viruses are complex entities smaller than the wavelength of visible light (400-700nm) which are only evident when they attach to another cell and therefore virologists have always known they cannot be “isolated” in the dictionary sense of the word.
Partial computer generated sequencing was done early on but since then the full 30,000 bases of SARS-CoV-2 have been laboriously “whole gene sequenced” many times around the world. To date virologists have (now) taken over 4 million human samples of the virus, gene sequenced them in labs using different types of machines and uploaded the results, which are the same gene sequence as the original computer enhanced ones (apart from the variants) to the GISAID Initiative. https://www.gisaid.org/
The virus in many cases can be shown to infect another person by cell culture in a lab.
May I ask a question ( genuinely) ? How does the original RT-PCR mass testing regime compared with the improved versions test for the full 30,000 bases of the entire genetic code of SARS COV2 if I have understood your point above? ( I am not one who believes it does not exist..)
The original Corman-Drosten et al paper was improved soon after it came out so the vast majority of mass testing was with the latest protocols.
The original Corman-Drosten et al PCR test paper had been criticized by some people. “Since then, a consortium of over forty life scientists has petitioned for the withdrawal of the paper, writing a lengthy report detailing 10 major errors in the paper’s methodology.”
On February 4, 2021, Eurosurveillance (where the paper was first published) published its long-awaited response to the Corman-Drosten Review Report, after a two-month period of review by five external experts.
Within one week of the receipt of the Report, and after a discussion with the editorial board members, it was decided that scientific misconduct or conflicts of interest were a non-issue.
They were also happy with the peer review.
https://www.eurosurveillance.org/content/10.2807/1560-7917.ES.2021.26.5.2102041
The Review Report and the Addendum on the Borger-Kämmerer team against the Corman-Drosten et al paper was criticised by people like Prof. Andreas Beyer who states ….
“The Borger-Kämmerer text is pseudoscience, it is full of misconceptions, errors and flaws. Therefore it is ignored by experts for good reason. The impact it had in public consciousness, however, is fatal. Borger reported on Twitter more than 30 Million views of his “Report” (March 2021) [now 50 million]. Hence I ask all colleagues please to spread this essay for at least a little bit of counterbalance.”
https://www.researchgate.net/publication/351286220_Borger_Kammerer_Corona_qPCR_Pseudoscience_Conspiracy_Theory_Revisited_-_an_Analytical_Essay_-
WTGR you answer a question I did not put – have another go please ( hopefully this will stay posted unlike the previous attempt from me)
I answered your question in the first sentence “The original Corman-Drosten et al paper was improved soon after it came out so the vast majority of mass testing was with the latest protocols.”
Being a simple soul i have always been worried about how false positives/false negatives are actually measured in medical matters. Surely there must be some absolute 100% guaranteed accurate test out there against which the (in this case ) PCR’s are measured?
If indeed there is, then why is that not used?
If not, then err…
That’s the ‘reference standard’ question.
Normally the reference standard (at least in theory) is trained doctors making decisions using all the available evidence. If you introduce a newly developed test and it disagrees with doctors almost all the time, it’s considered to be a failed test. This assumes that doctor’s decisions aren’t constantly wrong, but that’s a good bet at scale.
With COVID governments didn’t want to wait for doctors because they felt that was too slow and wanted mass testing at airports etc, so they needed a mechanical doctor. PCR is considered by scientists to be the “absolute 100% guaranteed accurate test”, so, that’s what ended up being used. Everything else gets calibrated against it, including doctor’s own diagnoses. The usual approach has been inverted.
That’s one of the points made in the article. You cannot actually determine the accuracy of PCR tests in any rational way, because the PCR test is itself the reference standard, hence the epidemic of scientists doing things like trying to calibrate the tests against themselves (which is meaningless), or measuring the test against “suspected COVID” without explaining how the doctors arrived at that suspicion in the first place.
Thanks.
I understand things a bit more now.
Your last paragraph explains it beautifully. It’s a bit like Catch 22 – in an inverted way.
On a similar vein I wonder what Joseph Heller would have made of this.
Hi Mike
I think you are being unreasonable here.
I don’t think it is true that PCR is considered by scientists to be the “absolute 100% guaranteed accurate test”. Even the ONS accepts that the sensitivity of PCR tests is less than 100%.
The review paper you referenced was published in Aug last year – so it necessarily looked back at the early days of the epidemic and testing. In any case it was only saying that the reviewed studies were unsatisfactory and needed improving and standardising. This is not the same as saying there has to be an independent gold standard for PCR tests to compare to in order to assess their accuracy. Chivers himself references two methods of doing this:
This ONS paper compares PCR test results with LFT results and works out the best combination of spec and sens for each test to explain conflicting results.
This paper compares PCR test results with the results from three different PCR assays looking at different parts of the virus genome.
I am sure assessments of the sens and spec of different PCR tests can be improved but there are methods of assessing them without disappearing up your own behind.
(By the way – I also plan to respond to this very interesting and challenging comment but it will take some time. I hope you are still in the loop when I finally manage it)
But positive for what?
Here Kary Mullis (PCR inventor) explains it to us himself. It is an amplification research tool, not capable of diagnosing disease –
https://youtu.be/iWOJKuSKw5c
And even the package inserts clearly state that “Positive results do not rule out bacterial infections or co-infection with other viruses. The agent detected may not be the definitive cause of disease”
And yet THIS is driving the numbers of Covid “cases” “hospitalisations” & “deaths” on which Governments are basing their unprecedented restrictions and impacts on our ways of life.
Kary Mullis has been quoted out of context. It can test for a virus but doesn’t confirm whether the virus is alive or dead.
The PCR test is incredibly accurate if there is no contamination in the lab but even if the virus is detected the patients disease may ALSO be due to bacteria or ANOTHER virus.
Both highly accurate and incredibly accurate (is this supposed to be something different from credibly accurate?) don’t really mean anything except It’s inaccurate but the error rate is very small compared to [not mentioned]. On its own, it could also mean but I’m convinced the error rate is so low that it doesn’t matter.
if there is no contamination in the lab is an impossible qualification. There’s always going to be some contamination in the lab because humans are going to make mistakes and will try to cover them up by pretending they didn’t happen. With a lot of (relatively) poorly trained operators having to do lots of tests and employed by private government contractors whose only efficiency drive is provide the cheapest, that is, worst quality of service the government is still willing to accept, it’s very likely significant. More importantly, the amount of lab contaminiation for each individual test is unknown and can’t be determined by the people handling the results. In other words, they’re garbage.
Considering this and the wider implications, there’s obviously no dangerous pandemic ongoing. Otherwise, test failures alone would be sufficient to start exponential growth which then cannot be stopped anymore, as is claimed. This story must either be inherently wrong, ie, this would never happen, or not applicable to COVID.
Surely the reason governments won’t accept ‘recovery from Covid’ as qualification for a “passport” is because secretly they know that there are a large number of false positives (people with the common cold, sore throat, etc, etc).
They do accept it, at least in Europe. Only for six months, though.
The six month limit seems to be a consequence of papers claiming that people can be easily reinfected with COVID. I read one of those papers last year. It was simply looking for people who had two PCR+ within some (I think six) months of each other. There were relatively few, and thus they knew that FPs could mislead them because an FP rate of only (iirc) 0.15% would be sufficient to eliminate their finding. But they simply stated that FPs were “widely believed” to be some incredibly low number so it wasn’t due to that. Their citation was another paper that was (what a shock) a modelling paper, except the number wasn’t even a model output. It was just a made up number used as a scenario, justified by saying that it’s well known that labs are very rigorous. And that was that.
This is the sort of thing that makes reasoning errors about false positives dangerous. They compound over time into more and more erroneous or unsupportable policies.
Nailed it.
But, it must be very inconvenient for the bastards in charge of this.
But, no matter ,they’ve nailed the sheep.
Is Tom Chivers Right ?
NO
About fifteen years ago I had real ‘flu, with such awful swirling pits that for three days all I could do was to remain immobile in bed, not daring to move lest I stir it up again. Thus, since all ‘flu is of the Caronavirus family, I have the Caronavirus footprint in my DNA. If I was to be tested the sirens would ring, security would be summoned and I’d be another danger to the public. Yet I am perfectly fit and well, no danger to anyone, except the government’s agenda of control freakery.
ratcheting up the reading cycles of these spurious ‘tests’ is Westminster/Whitehall’s evil means of frightening the Branch Covidians, who listen to and believe politicians, forming a new and scary cult.
The contamination issue is well identified in this piece, but the issue that was the big problem with PCR testing was the number of cycles undertaken. It was considered that if sufficient of the virus was not found after about 20 cycles or magnifications of the sample, then it should return a negative result. The problem was and possibly still is that 45 cycles were being undertaken, which increases the possibility of contamination being identified as being in the sample and even if the uncontaminated sample when magnified to that level shows traces of the virus they would not be sufficient to cause active disease or symptoms and may be at a level that even low levels of immunity could deal with and neutralise. Of course, they could also identify early stages of the disease, but it is by no means certain, and it could also pick up remnants, after being dealt with by the immune system, of the end of an asymptomatic infection. As I understand it this is another element of PCR test inaccuracy.
What would be useful data would be the proportion of positive PCR tests where the tested individual develops symptoms needing treatment. This would give us a better idea of the usefulness of random PCR testing, which I suspect is much lower than its proponents suggest. In my opinion, PCR testing should have been limited to its original purpose of use in hospitals to confirm the disease in patients presenting with symptoms. That would have saved us millions in taxes and reduced the effectiveness of our government’s fear campaign and the resulting general psychological damage, both of which we are paying the price for now and will continue to for some time.
That Chivers article, classic junk maths. Oh look, he used the term Bayesian so he must know mathematics. The guy is a Philosophy grad who did a Masters in something called Medical Law. Looks like zero background in hard science or mathematics.
The prevalence numbers he quotes are just models, nothing else. And if you look at the huge disclaimers below the model numbers he quoted, no values for sensitivity / specificity or even the mix of test results used to generate the models they are quite simply – Made Up Numbers. After a quick tutorial in using the R software package in Python anyone can produce models just like these ones all day long. And just a valid.
The actual plausible values for community spread prevalence are < 0.3% for the last year plus. Typical community HCOV infection rates are 1%. Which gives a Type II error rate of 90% plus for PCR with typical screening test sample swabs when passed though high throughput testing machined. This is very basic maths given actual field use sample error rates and plausible community prevalence numbers.
The PCR test used is not a clinical test for active infection. Which should have Type I Error Rate 10%, Type II Error Rate 5%. The PCR test for active SARS 2 infection has a Type I Error Rate of 50% plus and a Type II Error rate of 90% plus when used as a screening test or with non-specific symptoms. There is a type of PCR test that can be used to distinguish active infection. A strand specific test. But thats not the one that is used. Too expensive, too slow, and too insensitive. It has a clinical acceptable Type II Error rate for a diagnostic test but the Type I error rate is even worse that standard PCR due to sensitivity issues.
The PCR test when used as a screening test or early stage clinical test with purely non specific symptoms is about as scientific as using crystals as a clinical diagnostic tool.
The prevalence numbers he quotes are just models, nothing else. And if you look at the huge disclaimers below the model numbers he quoted, no values for sensitivity / specificity or even the mix of test results used to generate the models
The prevalence numbers Chivers quotes are from the ONS infection survey. As I am sure you know, these are based on samples which are then used in “models” to estimate prevalence. (You always need a model of some kind to estimate a population value from a sample – even if it is just that the value is normally distributed in the population). But there is certainly data underlying the models, it is just not true to say the values could be generated entirely from a python programme. The ONS documents the methodology in some detail including assessments of specificity and sensitivity and provides the data. You could hardly ask for more documentation or transparency, They then do the calculation including 95% credibility intervals.
On the other hand you have quoted prevalence and error rates without any documentation or references at all.
Setting all that aside – what Chivers did was quote figures from government sources for prevalence, sensitivity and specificity and make a good point about the interaction of lateral flow tests and PCR tests. His logic is impeccable. He makes it clear what his sources are and they are supposed to be authoritative. If the ONS is at fault (and I doubt it) then he is not to blame..
I have been following the published clinical data (NOT models) since Jan 2020. I read all the published clinical literature on SARS CoV 1-03 in early Feb 2020. I have been reading hard science papers with dodgy maths for many decades. The ONS methodology and data is just far too typical bio-science dodgy maths.
The prevalence numbers I quoted are from general population seroloigal studies in a bunch of countries. Discussed here in previous threads. Not models based on inferred values from testing results. Non clinically valid tests I might add. In adult populations the general HCOV infection rate is around (seasonal) 1%. For children it can be up to 10%. Given that the SARs CoV 2 has a much lower attack rate that the other general circulation HCOVs (229E/OC43/HKU1/NL63) the prevalence numbers modeled by the ONS are junk. A much lower attack rate infectious agent, even if novel, cannot have a much higher prevalence in a greatly impaired population transmission rate situation than a much higher attack rate infectious agent with transmission in a normal unimpaired circulation population.
You cannot have a low attack rate SARs CoV 2 prevalence of 4% in an impaired circulation population when the total general circulation moderate much higher attack rate HCOV infection rate is 1% in a general population with unimpaired circulation. This is very basic maths. And physics.
If the ONS data was based on a clinical valid diagnostic test on done on a rolling basic on a longitudinal population sample then it would have some validity. In fact that would be a fantastically useful epidemiological data set. But it isnt so it just junk data with fancy graphs. I can produce just as plausible (and valid) data and graphs in half an hour with R. Give me an hour or so with one of the PDE solver packages I can produce really impressive BS model data. Like so many of the models published in the last 18 months. To those of us who have to make very fancy maths work in the real world the hand waving junk data like the ONS data is very easy to see through. Its a GIGO situation. No matter how narrow the CI’s claimed.
As for PCR v immunochromatography. At least immunochromatography tests for the antigen or antibody. Which in the case of the antigen is a test for active infection. Sure the sensitivity is only good enough for casual screening not clinical confirmation but the PCR test as used since March 2020 has no medical validity of any form. The test which so much of the ONS model data is based on.
Chivers claimed PCR has low Type II error rates. Almost zero. That in itself invalidates any claims he might having an informed opinion in what he published. In fact a quick look at the FDA sensitivity / specificity data on the various EUA PCR reagents currently used would soon disabuse him of just how accurate the PCR tests are even in the most idea lab situation when tested against reference samples. The specificity numbers are fine, its the sensitivity numbers that should give pause. All the EAU reagent (100 – S) values greater than even the most louche ONS prevalence values. Which in itself puts the Type II Error rate close to 90%.
The prevalence numbers I quoted are from general population seroloigal studies in a bunch of countries.
I assume by serological you mean testing for antibodies – or do you have some different meaning for serological? As far as I know there is only one up-to-date survey of the presence of Covid antibodies in the UK and that is the ONS Infection Survey! In any case the presence of antibodies is not at all the same as currently having the virus.
Not models based on inferred values from testing results. Non clinically valid tests I might add.
As I said, if you want to estimate a population parameter such as prevalence, you have to take a sample and infer population values from the sample using a model (unless you survey the entire population!). This is also true of serological studies (and indeed is the basis of much of statistical inference).
In adult populations the general HCOV infection rate is around (seasonal) 1%. For children it can be up to 10%. Given that the SARs CoV 2 has a much lower attack rate that the other general circulation HCOVs (229E/OC43/HKU1/NL63) the prevalence numbers modeled by the ONS are junk.
You accuse the ONS of modelling! They used real data (I gave you the link to it). You make conjectures based on hypotheses about how SARS Covid 2 compares to other coronaviruses.
I can produce just as plausible (and valid) data and graphs in half an hour with R.
I dare say you could do it with Excel. But the ONS study is based on data. What data would you use?
the PCR test as used since March 2020 has no medical validity of any form. The test which so much of the ONS model data is based on.
Well that is debateable to say the least.
Chivers claimed PCR has low Type II error rates. Almost zero. That in itself invalidates any claims he might having an informed opinion in what he published.
Chivers claimed said that false positives are so low we can ignore them and gave the link to the evidence from a highly reputable source. I assume by that he meant that the specificity is very high (of course the actual false positive rate can be quite high even with a very high specificity if the prevalence is low). Anyway that seems to be what you are assuming him to mean because you accuse him of low Type II error rates which is the same as high specificity. You may dispute the ONS paper (that is the topic of this thread) but it is hardly poor journalism to accept the judgement of the ONS for what is a relatively minor point in his argument.
The specificity numbers are fine, its the sensitivity numbers that should give pause.
It is the specificity that is in dispute not the sensitivity. Chivers made it clear that there are a wide range of estimates of the sensitivity of PCR tests as low as 66%. He opted for 95% for the purposes of his example but did not argue that it was better than the others – it was just illustrating a principle.
All the EAU reagent (100 – S) values greater than even the most louche ONS prevalence values. Which in itself puts the Type II Error rate close to 90%.
I don’t understand this. But a specificity as low as 10% seems most implausible. Can you expand on what you mean by reagent (100-S) values? Or give a link.
They key point on which the whole essay turns is whether the specificity can change with prevalence because contamination can act as a multiplier of prevalence. So how big could this effect be? And how well could it explain what we observe?
I did a little crude maths to assess the possible impact of a contamination factor. It has got to be pretty massive to have a significant effect on the proportion of positives that are true positives (the positive predictive value). For example, if the reported positivity is 10% (as it was in July) and 10% of all true positives lead a false positive via contamination then it is still true that 90% of reported positives are true. Of course you can assume any contamination factor you like – but 10% strikes me as extremely high.
There is also the pattern of positives. The current wave began with a surge of positive results in a few selected location such as Bolton. A proportion of these were sequenced and found to be the new Delta variant unlike the rest of the UK at the time. Are we to believe that cross contamination favours specific locations and variants?
Mike’s case is
these problems do not imply the false positive rate is very high. They imply we do not seem able to characterise Covid test accuracy rigorously,
which is fair enough. But it is not a complete free for all.
Thanks for your thoughtful analysis.
One of the (many) problems with discussions about FPs is at some point you reach the question of what level is too high. Sometimes the level is fixed by the use case. The paper I describe above that claimed natural immunity doesn’t work with Covid would have been invalidated by a FP rate as low as 0.5% (I think – it was pretty low but I read it a while ago). For such a study the acceptability threshold is bounded by the nature of what it’s doing: if the true averaged FP rate is 0.5% then the study is junk, but even if it’s a little less than that, it still means most of the data has to be discarded and the resulting conclusions may fall outside the threshold of significance.
In other cases like trying to contact trace and stop an epidemic before it starts (NZ approach), you could argue that nearly any level of false positives is acceptable because the cost of the epidemic is high and the number of people inconvenienced is low, as you aren’t testing everyone. In theory you could do QALY analysis to make that argument rigorously but nobody ever does, not even public health professionals, so …. whatever.
OK, so above, you say if 10% of TPs cause a single FP then 10% of reported positives are false, with the implication that it’s not a big deal. It may be, it may not be (for monitoring the shape of the pandemic, who cares), but 10% is a heck of a lot higher than zero which is what governments are told to assume today!
Anyway this is why I don’t try to compute the “real” values. You say 10% sounds high but in the Dartmouth-Hitchcock whooping cough incident the lab yielded 14% positivity on a true base rate of zero, so, is it really so implausible? In reality none of us, including apparently the ONS, has sufficient real world data to characterize the true FP rate. We can’t even define what a positive means. It’s a downward spiral of bad logic and missing data.
That would presumably depend on how samples are load balanced across labs? If samples from Bolton get sent to the nearest lab and a few samples of Delta contaminate the lab somehow then yes, you’d see a spontaneous large local outbreak that in reality was much smaller.
Mike
I have been thinking about this on and off most of the day. I concluded it is almost impossible to prove or disprove the presence of a hypothetical false positive multiplier such as your proposed contamination factor based on test results alone. However, surely there is some kind of quality control, at least a comparison of results from different labs? If contamination is an important factor, it seems almost certain that some labs would be more prone to contamination than others and this would show up in inconsistent positivity rates. (I would imagine that there are EQAs from time to time as well).
Syphisus: “hold my beer”
Still flogging a dead horse. About a year ago, anyone who could read knew the PCR test was a scam. The bigger issue is why the hell did we accept a bogus test for the healthy?
Chivers has shown himself to be a vaccine cultist. FoI requests to the NHS show the cycle count to be 40, way above the recommended maximum of 10. Not to mention that the guy who invented PCR testing was clear it was not appropriate for the detection of Covid-19. Chivers, as someone once said of another football manager, “He’s gone down in my estimation”. Musk had four tests in one day. 2 positive 2 negative. Their only use is to ramp up and perpetuate a state of fear.