[halpert]

It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity

[mnw21cam]

No, the article isn't talking about sensitivity. We don't actually know what the sensitivity is from the data the article gives us. We are told that lots of people were screened and a small number had a positive result, of which a proportion were actually positive. You can't calculate sensitivity from that because you don't know how many actually positive cases were missed.

This article is talking about precision, which is the proportion of positive results that are true. And it's okay for precision to be awful, especially when the condition is so rare. But it's only okay if the result is communicated alongside a statement saying what the precision is, which it seems these were not.

[halpert]

Yes you are correct.

[adjkant]

While the author may not be well versed or focusing on the stats side, you're missing the human side here I think.

> the tests are inaccurate, when in reality the tests are accurate

If the test make someone consider terminating a pregnancy or even considering it, that's a lot of pain. So for that human, the test is failing its purpose potentially, depending on the value calculation of terminating a viable pregnancy vs the severity of the issue if it comes to term.

For a human, accuracy as you defined it means little to nothing. Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those, which is what the article is highlighting.

[andreilys]

Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those

How exactly do you plan on codifying usefulness and helpfulness?

A high false positive rate is not necessarily a bad thing and may instead be the catalyst for additional tests to confirm the first one. The tests accuracy may actually be 100%, which is great because it avoids a child being born with a fatal genetic disease. Would you prefer a high false negative rate that misses these diseases instead?

[halpert]

Or maybe you’re missing the human side of having a child born with a serious genetic defect?

[mcguire]

Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do? At what point is it not better to terminate 100% of pregnancies?

[loeg]

> Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do?

Yes, it’s absolutely better to do that. Of course, the actual ratio is much better than that because we do follow-up tests after the screen.

[Pigalowda]

Your first question and answer is against medical ethics. If you’re a physician you know that terminating pregnancies based on a non-specific but sensitive screening test is inappropriate (do no harm).

However, you are correct in your second statement - there are much more specific follow up tests which improve diagnostic accuracy.

Medical utilitarianism should never be so cruel and gruesome as to eliminate 85 healthy and wanted pregnancies in order to find and eliminate 15 genetic aberrations.

[paulryanrogers]

> At what point is it not better to terminate 100% of pregnancies?

Everyone should decide for themselves. Having seen the long term consequences I would rather err on the side of caution, even if it were difficult to become pregnant.

Such diseases are often incurable and significantly degrade the quality of life of not only the person to be born but the whole immediate family. At least in the US the there isn't enough social safety net or support too offset the crushing costs.

[ramraj07]

Did they use the word accurate? You used the word accurate and then you yourself are going on a tirade about how that’s not correct?

It’s clear the article is talking about why sensitivity is important in layman’s terms and while it could use better writing it’s a real problem in diagnostics. This is why you don’t ask men to take a pregnancy test to check for prostrate cancer. It is accurate but not sensitive.

[halpert]

They used the word “wrong”. Whether or not they used wrong to mean inaccurate, or wrong to mean not sensitive is up to the reader.

[wizee]

The issue is that the tests portray themselves as being accurate (in the sense of low false positive rates), and portray the result as “your baby has XYZ rare syndrome” instead of “your baby has a 15% change of having XYZ rare syndrome”. If the test providers stated the false positive rate for their results more clearly, parents would be in a better position to make informed decisions.

[rflrob]

The larger issue as I see it is that the medical system around these screenings are not well versed in the statistics and able to communicate that to patients. "Eight [patients] said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive." It's hard to know what happened in the room when the doctor spoke with them or what was on those particular patients tests, and that's (one hopes) the worst medical news those people will receive for a long time so listening comprehension is understandably impaired, but there needs to someone available who can help them interpret, even days or weeks later, and these people were let down by the entire system, not just the test manufacturers.

[whakim]

I don't think that it's useful for articles like this to try to educate readers on the way that a precision-recall curve works (and how that differs from the statistical definition of accuracy). Honestly, that would just confuse the vast majority of readers when it's simpler to point out that the tests produce more false positives than they might otherwise expect. Also note that even if we want to be incredibly pedantic, the article never calls the tests "inaccurate" and instead uses a layperson term without a hidden definition ("wrong").