Hacker News new | ask | show | jobs
by mike_hearn 1073 days ago
That's unfortunately not the case. PCR false positives are a massive problem. Scientists liked to claim during COVID that PCR tests don't have false positive because they use an obscure definition that doesn't match how people normally use the term.

I wrote about this problem back in 2020 when it started becoming apparent that the public health establishment wasn't attempting to stop or even measure COVID PCR test false positives, because they had simply defined the problem out of existence [1]. Yet dig into the literature a bit and you discover that FPs are a well known problem that have occurred many times before.

There are at least three problems and arguably more:

1. Scope limits. Scientists have an extremely myopic view of what counts as an FP somewhat akin to "that's not a bug it's a feature" in the software world. Sample contaminated somewhere along the line from collection to machine? Maybe even when the sample brushed up against the outside of the machine itself, or the skin of a lab worker? Not a false positive in their view, because the PCR test itself is finding the RNA fragment it's looking for. PCR detects virus your immune system already destroyed? Not a false positive, because the test is actually looking for RNA so begins with a lysis stage that breaks open the viral capsid anyway. Results got mixed up in a shoddy IT system? Not a false positive because the "test" is just the machine they operate, not the whole infrastructure the public interacts with.

2. Over-sensitivity. COVID PCR tests were run at thresholds normally considered way too sensitive for normal usage, which is why they were so frequently triggering for weeks after a patient seemed to have fully recovered. Even with CTs ~25 these tests were pre-2020 run in extremely controlled labs with specialized air handling equipment and the like, to try and avoid FPs, but COVID tests were being routinely run in ad-hoc labs with CTs of over 40 and each additional cycle is twice as sensitive as the last!

3. Circular logic. Normally medicine is careful to keep pathogen and disease separate. It's important because some diseases ("collections of symptoms") can be caused by multiple pathogens, or no pathogen, and many people become infected with a pathogen yet never develop disease. In the very early days of COVID the distinction between disease and causative agent was kept properly separated because diagnosis was done by doctors, but this didn't survive contact with the public health establishment. They wanted a mass-scale system in the incorrect belief that they could slow down spread this way (there's no link in the data between testing levels and outcomes). Once mass PCR testing started the definition of COVID became circular: A positive test means you have COVID, but COVID was defined as having a positive test.

It's obvious what happens if you define COVID this way: the tests can by definition never have false positives. On what basis can you dispute the accuracy of the test if the test and not disease is defined to be ground truth? Using this approach, false positives always become "asymptomatic cases" that inconveniently never become sick and thus are not a case under classical medical reasoning. This is what led to bizarre claims coming from health authorities, like the claim that the tests had FP rates of zero. But it's not reality.

As I document in the essay, we know PCR results have FPs because there have been "pseudo-epidemics" in the past where ordinary coughs and colds were mis-diagnosed as outbreaks of dangerous diseases due to bad PCR results, and because lab challenges have often returned failures i.e. labs were submitted samples known to contain just ordinary rhinovirus or whatever and came back with positive SARS-CoV-2 results. There were also cases where reagent mixups caused false results, or where PCR tests returned results that switched between negative and positive then back to negative from samples taken back to back and so on. None of these were ever really investigated or root caused.

And all that's before you get into the systems that surround the labs. I got a PCR certificate for someone else's name at one point, almost certainly a form autofill mistake (I saw workers wrestling with this problem when they lost my wife's results). When I complained I got a new certificate in under 5 minutes for my own name this time but now the results were signed off by a different doctor i.e. the signatures supposedly demonstrating expert supervision of the test were just being randomly picked by software.

[1] https://blog.plan99.net/pseudo-epidemics-7603b2da839

3 comments

But aren't we talking about rapid tests? There is a big difference between false positives when it comes to PCR vs rapid tests. False positives with rapid tests are known to be far less prevalent.
Other types of test are calibrated against PCR tests. That's why the article says:

"[...] finds the largest study to compare home rapid tests with gold-standard PCR tests"

PCR is assumed to be the truth to which rapid tests should aspire.

> False positives with rapid tests are known to be far less prevalent.

That's probably true if you use a normal definition of false positive, but under the intellectual framework public health uses you can't say this because you can't have less prevalent than zero.

The subtitle of your Jul 2020 essay criticizing PCR tests is "Why COVID-19 is guaranteed to never end". But in fact, judging by confirmed (by PCR, or by other tests grounded in PCR as you say) COVID-19 deaths, the pandemic has ended--the daily count has decreased by a factor of ~50 from the peak.

Does that not suggest to you that maybe you were wrong? It's not just the decrease in testing, since test positivity has decreased from the peaks even as total test count plummeted.

There is no evidence that PCR tests greatly overcounted SARS-CoV-2 infections:

1/2. The specificity of commonly-used PCR tests for SARS-CoV-2 is clearly very high, >99.9%, since there have been regions (e.g. Australia) where total positivity was <0.1% for months at a time. It's true that other PCR tests (like your whooping cough tests) may have much worse specificity, especially in the early days before primer design was as well-understood; but we know empirically that these ones doesn't.

Sloppy lab work could result in false positives by cross-contamination, but contamination has to come from a true positive sample; so this is a problem only when the true positivity is high. Sloppy clerical work could result in any possible incorrect result, but you've presented no evidence that this is a significant problem in aggregate.

Lots of papers studied the relationship between PCR positivity and culturable virus, for example

https://academic.oup.com/cid/article/72/11/e921/5912603?logi...

It's true that at high Ct, the cultures are often negative; but (a) the relationship between Ct and culturable virus may vary dramatically with small changes in PCR protocol, (b) a patient early in their infection may have high Ct now but lower Ct later, and (c) it's not clear whether all naturally infectious patients would be positive by artificial culture anyways. Based on these factors, most public health authorities chose not to use an artificially lower Ct. So a positive PCR test doesn't necessarily mean that a patient is infectious now, but it does mean that they are or were recently (weeks to months) infected.

For emphasis, it is absolutely expected that some samples will be positive by PCR but negative by cell culture--the PCR is testing for any RNA matching the primers, but the culture is testing for replication-competent whole virus. Presence of the latter implies the former, but the converse isn't true. The public messaging on that wasn't very good, and perhaps they initially misinformed you; but no one with any background in molecular biology was confused here.

3. Your idea that the PCR test is simply assumed to be accurate is completely false. The point of papers like the one I linked above is to compare two test methods, find the places where they disagree, and try to judge--from the experimental results and from our understanding of the biochemistry of the test methods--the reasons for those disagreements. They're not assuming either method to be perfect. Based on many such comparisons--against cell culture, against electron microscopy, against antibody tests, against excess mortality once that got big enough to distinguish statistically (which it mostly wasn't by Jul 2020), etc.--public health authorities judged PCR to be the best available gold standard, and good enough for many statistical purposes.

The public messaging was again pretty bad there, and if you looked only to that then I understand why you'd feel misled. The actual scientific literature is there for anyone who wishes to read it, though.

> test positivity has decreased from the peaks even as total test count plummeted.

Where do you see that? Where I live that isn't the case and according to official data the pandemic has never ended:

https://www.covid19.admin.ch/en/epidemiologic/test?epiRelDev...

As recently as April the positivity ratio was as high as it was in December 2020 (~16%). I have no idea who these people are who get tested 10,000 times a week but they do.

Now, I don't think this is just due to false positives. It might be - my point was and still is that we can't ever know due to these logic problems - but it might also be that Omicron mutation rates have just settled down into a background pattern that these tests still pick up or something. With no real definition of what COVID is beyond test results this question is unanswerable.

> contamination has to come from a true positive sample; so this is a problem only when the true positivity is high.

So? FPs are a problem regardless of when they occur or why.

> Sloppy clerical work could result in any possible incorrect result, but you've presented no evidence that this is a significant problem in aggregate.

How could I? Governments were told by scientists that PCR testing has an FP rate of zero, so they just didn't build any processes to recognize the possibility or measure the error rate. I had direct experience of clerical errors twice, and we didn't get tested much. And that's in Switzerland which has a relatively strong culture of quality control and carefulness. During COVID there were many viral social media posts and videos where people did things like submit a swab that had never been used, or that was dipped in a puddle, or where they swabbed their dog, and got a positive result. There were also lab challenge studies which showed some labs failed badly (although others didn't). To me these things are good enough evidence that FPs were a real problem.

> a positive PCR test doesn't necessarily mean that a patient is infectious now, but it does mean that they are or were recently (weeks to months) infected ... perhaps they initially misinformed you; but no one with any background in molecular biology was confused here

You're supporting my points here. Nobody cares about detecting people who were infected months ago. Yet governments reacted to any positive test by assuming they were highly infectious and immediately forcibly quarantining the unlucky person who got it. Even when people said "yeah I had it weeks ago but am fine now", that counted for nothing, because the test was taken to be the gospel truth. Anyone who pointed out that this was nonsense got shouted down as a "denier" (see responses to that blog post, or even my comments here on this thread in 2023!).

> Your idea that the PCR test is simply assumed to be accurate is completely false ... good enough for many statistical purposes

Statistical purposes!? People were locked in hotel rooms for weeks on end on the basis of these tests, they were prevented from taking vacations no refunds allowed, they were fined, they were locked in their homes. The Swiss government had a web page that stated point blank that PCR tests had no false positives at all. If this assumption is completely false, how could someone appeal a false positive, in your jurisdiction? Because I'm sure it was impossible in mine.

> The public messaging was again pretty bad there

The public messaging came directly from scientists. Unless you think politicians had opinions on the accuracy of PCR tests predating the pandemic, of course. I even ended up arguing about this with an immunologist acquaintance who used PCR testing for his job, he was also under the impression that PCR tests couldn't have false positives and anyone who thought they could was the victim of misinformation. When I told him COVID tests used a Ct>40 though, he turned pale, suddenly lost his confidence and said they shouldn't be doing that.

> Where do you see that? Where I live that isn't the case and according to official data the pandemic has never ended:

If those data don't show the end of the pandemic, then what would? Your confirmed deaths are around 1/100 of the death peak. Positivity fluctuates much less, in part because there's a feedback loop--authorities will test more if it gets too high (subject to resource availability), and less if it gets too low. It's well below the positivity peaks, though. In any case, if the plummeting death count were due primarily to plummeting testing, then we'd expect the test positivity to increase, and it definitely hasn't done that. I'm in the USA myself, with broadly similar trends.

> Governments were told by scientists that PCR testing has an FP rate of zero, so they just didn't build any processes to recognize the possibility or measure the error rate.

I don't know what the Swiss government did, but globally scientists certainly considered that possibility--for example, that's basically the point of all the papers comparing excess mortality against confirmed deaths. The agreement isn't perfect, but the trend is pretty clearly there.

> So? FPs are a problem regardless of when they occur or why.

Non-contamination false positives show up like an additive offset to the positive count, while contamination false positives show up more like a multiplicative scale. They're both bad, but the latter is usually less bad, since relative error is usually what matters.

> Nobody cares about detecting people who were infected months ago. Yet governments reacted to any positive test by assuming they were highly infectious and immediately forcibly quarantining the unlucky person who got it.

All governments that I dealt with were clearly aware that a positive PCR test didn't mean you were infectious now. For example, travel guidance generally permitted a recent (but not too recent) positive PCR test as an alternative to a very recent negative PCR test, since they knew that recovered patients might test PCR positive for a long time.

For people newly testing positive, governments didn't advise or mandate isolation because they thought the tests perfectly predicted whether a person was infectious now. They did so because those tests were the best tool available, and they judged the harm of missed isolation of infectious people to be greater than the harm of unnecessary isolation of non-infectious people. I agree that in many cases, that was a policy mistake. That problem is with the policy, though--the tests were performing exactly like any competent microbiologist would expect.

I'm not sure why your immunologist turned pale, since Ct cutoff around or slightly above 40 isn't unusual. For example, here's a pre-pandemic test for infectious salmon anemia virus that chose Ct = 42 in certain cases:

https://journals.sagepub.com/doi/10.1177/104063871102300102?...

In general, Ct values aren't comparable between different protocols. Sensitivity and specificity are judged from results on positive and negative controls, not from absolute Ct. There's no single best Ct; it's always a tradeoff between false negatives and false positives, and the weighting of that tradeoff varies with the intended use of the result.

> The Swiss government had a web page that stated point blank that PCR tests had no false positives at all.

That's clearly false. I don't think the American government said that, though they made other false absolute statements (if you get the vaccine then you definitely won't get sick, etc.). The consensus of the scientific literature told a much more cautious story, though.

It seems like your complaints are about the use of the PCR results, not the accuracy. If the PCR test correctly reports the presence of viral RNA, and based on that a patient is forced to isolate based on the x% chance they'd infect someone else, and you believe x is too low to justify that, then that's not a false positive; it's just a policy position that you disagree with.

It's probably best to leave it here, because you seem to just be arguing the position that I was describing ("If the PCR test correctly reports the presence of viral RNA ... then that's not a false positive").

This thread is sufficient to prove my point - the idea that PCR tests have very low or even zero false positive rates is essentially misinformation spread by scientists who are unwilling to consider anything about how the tests are actually used. Your whole post boils down to, "we only did PCR tests for statistical purposes so FPs didn't matter as long as they didn't affect the trend, if governments used them for other things well that has nothing to do with scientists".

Sorry, but like hundreds of millions of other people I simply don't accept this perspective. We don't think politicians had much agency because they just followed instructions from scientists at every point. Therefore the science community owns the COVID response. They don't get to now dissemble and engage in blame deflection when people point out what a catastrophe it was.

> We don't think politicians had much agency because they just followed instructions from scientists at every point.

No matter what your desired policy, you could find a scientist to support it--some scientists were advocating for Chinese-style lockdowns, while others were drafting the Great Barrington Declaration. So how do you think the politicians chose which scientist to listen to, or to promote in the mass media? It's not like they took a poll--the vast majority of scientists expressed no policy opinion at all, and simply concentrated on their actual work.

Politicians are the ones with the power to make laws, and direct the police to enforce them, if necessary with physical violence. It's amazing to me that their attempt to cloak their policy decisions in "trust the science" worked so well on you that you've absolved the people with actual power of their actions, instead directing your anger at some abstract "scientists".

no.