[awrence]

This is a classic statistical fallacy. A test that is 50% accurate serves 0 purpose. You might as well consider coin flipping a valid test. If you tested America with this test you’d end up thinking half the country is infected. And within the ones that tested positive only half would actually be and within the negatives half would be and you would have missed them.

A test needs to be materially more accurate than the odds of having the disease to be worth anything.

Classic thought experiment: if 0.0001% of the population has a disease and a test is 99% effective and you test positive, what are the odds you have the disease? Answer: 1%.

Complimentary thought experiment: if an expensive preventive drug was available in limited supply (for 0.1% of the population only) and the earlier you took it the more preventive it was should you give it blindly to as many positives as you can? Probably not because 99% of that would be going to waste. And you would run out of drugs to cover the actual sick. Only 10% of the sick would end up actually getting the drugs.

[DangitBobby]

Isn't the false positive rate the percentage of positives that should have been negative? E.g. if I test 10000 people and 100 of them test positive, but really only 50 of those are sick (and almost none of the other 9900 people are), then I have a false positive rate of 50% and a very useful test. Obviously, this is very different from flipping a coin. And flipping a coin could also have a very high false negative rate.

[usrusr]

It's the rate of actual negatives that are perceived as positives.

I make that mistake myself all the time, What helps me avoid it is this: you want a number that is a property of the test method alone, independent of case distributions. A ratio between misidentified positives and identified positives (or true positives) would depend on sample distribution.

[monktastic1]

The metric you're talking about is called the "false discovery rate": https://en.wikipedia.org/wiki/False_discovery_rate

I also have a hard time keeping these all straight: https://en.wikipedia.org/wiki/Sensitivity_and_specificity#Ap...

[sliken]

50% accurate I'd agree is useless.

But the parent said 50% false positive, presumably with close to a 0% false negative would be VERY useful and save potentially millions of lives. We need enough tests yesterday or so to avoid a repeat of Italy and a 50% false positive rate (with a very low false negative rate) could help do that.

[makomk]

Even the properly-conducted version of this test has a fairly substantial false negative rate, somewhere around the 30% mark, and if it's done by students using samples that might not be taken correctly on a rigged-together testing setup that's going to get worse. Seriously, you might as well flip a coin.

[leereeves]

A test with a 50% false positive rate, administered to 100 million people who aren't infected, would say 50 million people were infected.

False positive rate is a confusing term; it means the % of tests that should be negative that report a false positive.

I think the term for what you're probably thinking of (% of positive results that are false positives) is the false discovery rate.

[awrence]

That’s fair, if the assumption is that false negatives are very low or 0 (in which case it’s a super high accuracy test at low contamination rates and super useful).

And yeah obviously we desperately need a decent test like three months ago.

[dschuler]

A test might have a high false positive rate, ie “low specificity”, but can still have “high sensitivity”.

That would mean that while you catch most cases of the virus, you’d also get a bunch of false positives. Flipping a coin as a hypothetical test would give false positives and negatives, or low specificity and low sensitivity.

[prepend]

Isn’t false negative specificity and false positive sensitivity?

https://en.wikipedia.org/wiki/Sensitivity_and_specificity

[boyd]

No, sensitivity is TPs / all positives (detected TP and FN). It’s the green half of the diagram in your link. The language is very easy to get tripped up by though. :)

[prepend]

I think I’m getting tripped by false positive, true positive.

A test with high sensitivity will have a low false positive rate.

A test with high specificity will have a low false negative.

So having a test with high sensitivity but low specificity will result in trust in the positives, but not trust in the negatives?

[dschuler]

For me it’s easier to think of sensitivity as being “sensitive to the true positive” without saying much about false positive or negative.

A sensitive test will catch many positive cases. It may or may not have false positives though, eg a test that’s always gives the right answer vs a test that always returns positive no matter what.

A specific test will give you few positive results when the true answer is negative. You could use it to rule something out. One test might say “patient has A or B condition”, and a second test with high specificity may then rule out A or B, leaving B or A, respectively, as the probable condition.

[kortilla]

I think you need to learn the difference between false positives and false negatives. A 50% false positive rate with no false negatives is absolutely better than 50% false negative and positive.

False negatives miss infected people and you fail to quarantine. False positives just mean you quarantine too many.

[gbrown]

You’re not differentiating between sensitivity and specificity - in this case the test is highly sensitive but not specific. That means that it could be effectively hard to rule out cases while failing to provide strong evidence of actual infection in individual cases.

[lopmotr]

You're applying conventional knowledge to an unusual situation where it doesn't apply. The current test in my country is something like "was sitting within 4 rows of a known patient on a plane". That's obviously got a massive false positive rate but we do it anyway. The action you take when officially tested positive is the same as you take when learning you had contact with a patient - isolate yourself at home. There's no wasted drugs because there's no treatment for it! At worst, you might do an official test, but if those are in short supply, they're not going to be given to anyone who just walks in with a home-made result.

[hnaccy]

>A test needs to be materially more accurate than the odds of having the disease to be worth anything.

Shouldn't it just need to be better than 50/50?

>Classic thought experiment: if 0.0001% of the population has a disease and a test is 99% effective and you test positive, what are the odds you have the disease? Answer: 1%.

My odds still changed dramatically and the cost of a false positive is just me hanging out at home and not visiting my parents. I'm not getting a biopsy or taking expensive medicene.