My wife and I went through this a couple of years ago, with a 10 week NIPT calling a rare trisomy (chr 9), which is always fatal within a few weeks of birth.
It was absolute hell. The key problem here is the waiting and uncertainty. You have the NIPT at 10w, but you can’t have the amniocentesis until several weeks later. When that came back fine, there were questions about whether it was a “mosaic” meaning only a small proportion of cells are effected. We were only really in the clear after the 20 week ultrasound.
That’s a lot of weeks to be consumed by wondering about whether to terminate the pregnancy, or wait it out for more information. I have a masters in bioinformatics (in genomics!) and my knowledge of stats and the science was next to useless in the face of these decisions.
I know of couples who simply couldn’t deal with this uncertainty and chose to terminate on the basis of this test alone.
Fortunately for us our child was fine and is a perfectly healthy 18 month old now, but I wouldn’t do the rare trisomy test again.
Having gone through two twin pregnancies (where the odds of these tests being correct are especially low) we declined all of them. Anecdotally, I know of several parents who had a positive test for genetic disorder, went ahead with the pregnancy anyway and children were perfectly healthy. Until these tests are close to 100% reliable I don’t see the point.
"The good news is that the invasive test proved the screener was incorrect. The bad news is that it looks like you've now lost your baby. It was fine though!".
The risks associated with this extended testing are just not worth it, perhaps aside from Down's (from a numbers point of view). Even then, there are many completely gorgeous children and people with Down's .. chances are you'll have a curveball in life one way or another at any rate.
I have two children, one diagnosed with ADHD/ASD, the other likely not too different but too early to tell. Apple doesn't fall too far from the tree. Wouldn't change a thing, other than to avoid the ABA services companies like plague - they prey on your insecurities and you might face financial ruin for possibly no real benefit to the child if you go along with their spiels.
So, to see medical companies exploiting vulnerable new parents who will do anything for their children? I am shocked. (/S...)
You are free not to take the test, but I think we would have taken the 0.3% risk to see if it is really Down's. While children with Down's can be gorgeous, I am not up to the task.
We did get the test for it for my two; can't recall if we did any extra ones. Believe we skipped them, at least on the 2nd.
I've come around to maybe change my mind since then, however I'd need to be in a very good position to be able to be up to the task.
I truly wouldn't have been, the ASD diagnosis was hard enough - and made magnitudes harder due to the manipulations of the "autism industry". Do this, do that, or else - you only have one chance for an early intervention, so better throw your own life away or you'll be a bad, bad parent.
No. I think this is a cruel and ignorant thing to say.
Wanting to have a child who is not special needs is not an evil thing. Choosing to not bring a a child with special needs to term is not evil.
Having a special needs child can dominate your finances, your life, and the lives of your family members and already existing children. People have a right to choose what what they want out of life, especially in the context of before a child is born.
“ Wanting to have a child who is not special needs is not an evil thing. ”
No its not.
“Choosing to not bring a a child with special needs to term is not evil.”
A huge evil. Just consider your next paragraph explaining why its not evil; it centers around how having a disabled child affects you and others. I do apologize, but I find your last paragraph frighteningly narcissistic .
Why stop there? It can be a challenge to have a kid who is ADHD, or is susceptible to depression or mental illness, or a different sexual orientation, etc. It’s not wrong to abort those children either, right?
The alternative is to take the same energy you'd spend on that one unlucky human being and spend it raising 3 healthy children who will go on to live full lives.
Parenting is an enormous time investment and families that take on the burden of raising disabled children almost universally reduce their family size. This is not a decision without cost, to those who lose their chance at life.
It's fine for you to live in whatever fantasy world that allows you to never have to make any hard life choices, but you shouldn't go out of your way to try to make others feel guilty for having the unfortunate fate of having to live in the real world.
Many of these conditions would in the past have been fatal in infancy, and it is only through modern medicine that children can survive them at all. It's fair to expect medicine to provide the solution to the problem it has created. There's nothing natural about children having to live their lives in and out of a hospital.
Depending on what state/country you live in, care for a disabled citizen may be partly or wholly paid for by the state. The parents may also be able to become qualified caregivers, in which case they can be paid to take care of the child.
It's still a full time job that precludes you from doing a great many normal family activities for the child's entire lifetime. There is also the danger that your state may elect budget hawks who decide to cut funding for the programs halfway through your child's life, leaving you on the hook for ruinous medical expenses. It can also be very unfair to your other children who are going to basically lose out on activities because their disabled sibling requires too much care.
I agree completely about all of the challenges, hardships, lost opportunities, and unfairness. It is an awful situation to be in for everyone involved and no amount of money in the world could ever make up for that.
Our experience was kicked off by a troublesome ultrasound and then confirmed by amniocentesis.
The tragedy of receiving news like this is probably fathomable, but I think it may be hard to grasp the emotional and intellectual agony of deciding whether to terminate a pregnancy based on a set of probabilities.
It breaks my heart to think that parents face this decision with erroneous data.
Why was there doubt surrounding an entire chromosome trisomy? My understanding was that it is easy to have high confidence about that since allele frequencies in the sequence reads are skewed across the entire chromosome.
IMHO, some of those criticizing the article for failing to understand statistics are missing the point.
The point is that people who get a "positive" result on these tests are often put through terrifying levels of anxiety when there is no actual problem; this anxiety is often exacerbated because they aren't informed of the false positive rate. This clearly has a harmful emotional effect on people, and explaining the false positives in Bayesian terms, or reframing it in terms of sensitivity and specificity, doesn't undo that damage.
That potential harm needs to be explained to patients, and it needs to be weighed carefully against the potential benefits of the test (as is done for PSA tests for prostate cancer, which also have a high false positive rate). Given that potential for harm, it's not unreasonable to ask that these tests be more tightly regulated.
To quote the OP:
> In interviews, 14 patients who got false positives said the experience was agonizing. They recalled frantically researching conditions they’d never heard of, followed by sleepless nights and days hiding their bulging bellies from friends. Eight said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive.
If you get a positive for a horrid cancer with a 90 percent false positives you should be afraid. Its lunacy for tests to be regulated beyond requiring rough false positive false negative rates, and if anything smacks of "I dont understand statistics and therefore have to protect my children from understanding statistics." The article is most likely written by some anti abortion idiot.
> if you get a positive for a horrid cancer with a 90 percent false positives you should be afraid.
No you should not panic, anymore than you should celebrate and buy a yatch if you think you have 10% chance of winning a lottery.
The doctor should phrase it as "the test indicates you have a 10% possibility of cancer. The majority of people who test positive do not have cancer. Further testing is required to confirm or rule it out."
Going with the expected value of 10% of a horrid thing is still bad, or 10% of $100m is still $10m, is not applicable to a single non repeated event.
The GP wasn’t saying go out and kill yourself because you are definitely condemned. They were highlighting that you still have an extremely elevated chance of having cancer. If you got into your car and found out you had a 10% chance of being in a fatal accident, would you not worry? Sure, there is a 90% probability you’ll be fine, but the probability for most other people is 99.999%. Pretending like there isn’t a potential issue is dangerous and willful ignorance.
> Its lunacy for tests to be regulated beyond requiring rough false positive false negative rates
There is a good reason tests are described in terms of sensitivity and specificity ("if the answer in reality is yes, how often will the test say no?") rather than in terms of false positive or false negative rates ("if the test says no, how often is the true answer yes?"). The sensitivity and specificity are facts about the test which can conceptually stay constant[1] as you apply the test to different people. False positive and false negative rates do not have that property; they are facts about the group you're performing tests on just as much as they're facts about the test.
[1] This is not to say that the sensitivity and specificity of a test do stay constant as you apply the test to different populations. Often they won't. But it is a theoretical possibility, and even that isn't true for false negative rates.
For a test manufacturer you are right, and they aren't responsible for accounting for general population frequencies, or conditional frequencies based on things like some sibling has it etc. But there is no need to regulate that. But the serivice in question isnt a test manufacturer, its diagnostics, and they are responsible for this. I think a company selling a diagnostics service should be required to prominently display reasonably accurate FP,TP,FN,TN rates for gen pop, as the next example will show.
This is is not a legal problem, its a customer experience one, and as such its very much what the free market does well. A clever company could say that the tests service we provide works in a two stage process where a preliminary test is used to identify which conditions need to be investigated further, the results of which are presented in a meeting with one of our doctors were the results are explained and additional tests are performed to resolve any ambiguity. This combines the benefits of sensitivity, and specificity in a resource efficient way, and does not provide the user with any scary results before a certain answer is available. The FP,FN,TP,TN probabilities for gen pop for such a company would be much better than those of companies trying to do so in a single pass. But for these figures to mean anything regulating that they are available and reasonably accurate is key to ensure an efficient market.
Another problem is of course that the tests themselves are shit, or the rates based on poor data. The requirement on reasonably accurately reported gen pop rates help with this however, and I see no way around legally forcing diagnostic services providers to prove this. And until someone does, my patent pending algorithm(){return false} tests for all rare genetic disorders with excellent accuracy^^
I'm not sure GP was intending to make that distinction (rates globally for the test vs. the sample population being tested by.. what a given doctor/hospital?) - I haven't come across that before.
If the population is the same then your changed-order definitions are just inverses, and they're just different terms for the same thing.
> I'm not sure GP was intending to make that distinction (rates globally for the test vs. the sample population being tested by.. what a given doctor/hospital?) - I haven't come across that before.
> If the population is the same then your changed-order definitions are just inverses
No, you just haven't understood the concept.
Let's assume some condition has a prevalence of 20%, and a test for it will correctly identify presence of the condition 95% of the time, while correctly identifying absence of the condition 90% of the time. We can immediately answer the first question: when the answer is "yes", the test will say "no" 5% of the time.
You have proposed that when the test says "no", the answer is "yes" a share of the time that might be the inverse of 5%, or perhaps 5% itself. I have no idea what you meant -- and I suspect you didn't either -- but the correct rate of false negatives is not 5%, 95%, nor 2,000%.
In a model population of 10,000 people, we will see this:
From this table we can see that the false negative rate is 100/7300 or 1.4%. The false negative rate looks much better than the sensitivity and specificity figures because the condition is rare. The corollary to that is a horrific false positive rate of 800/2700 = 30%.
I don't think I have, it sounds like you think I'm saying false negatives are the inverse of false positives? Not at all, that's obviously not true.
I was surprised at 'sensitivity and specificity' (jointly) being considered different from 'false negatives and false positives' (jointly).
The given reasoning was about population differences, which.. fair enough, I understand that makes a difference, I just wasn't aware that was a standard difference in definition (if it is) and suggested the up thread commenter wasn't (or wasn't meaning to use it) either.
> correctly identify presence of the condition 95% of the time, while correctly identifying absence of the condition 90% of the time. We can immediately answer the first question: when the answer is "yes", the test will say "no" 5% of the time. You have proposed that when the test says "no", the answer is "yes" a share of the time that might be the inverse of 5%, or perhaps 5% itself. I have no idea what you meant -- and I suspect you didn't either
10%. 'inverse', as I called it, of 90%, not 95%.
(That's why I think you think I think (..!) that false negatives/positives rates are derivable from one another. Sorry if not and I'm just still not getting it...)
I've worked clinically and taught this in professional training programs. It's a classical clinical decision making problem.
The solution is fairly straightforward I think: require the test companies to report the posterior probability of the illness given the test result. Probably with some kind of uncertainty interval.
Generally speaking in clinical settings there are rough estimates of base rates of diseases available, either through biological theory (eg, mutation rates) or empirical studies or both. Not always to be sure, but often enough to mitigate against some of the problems discussed.
My 2nd daughter was flagged during our 20 week for something having to do with the way her skull was forming and they wanted to do a series of genetic test. They charged us through the wazoo and everything came back negative. She arrived 3.5 weeks early and contracted bacterial meningitis shortly after birth. We found her code blue in the crib. She ended up having a bilateral craniotomy to relieve the empyema that had formed. CP, CVI, global TBI - every day is hell on earth. This was 2019, so the nightmare of the last few years started early for our family. We've had a number of medical professionals drop hints at the fact there might be something wrong from a rare disorder perspective but we're in a league of our own and that is hindsight - the damage and trauma are non-stop. Anyone trying to shickle a few dollars from the medical system to provide "pre-natal diagnosis" without sound science - they can come burn in the same hell I live in every day.
I don’t really understand what your implication is. Are you wishing you had not done the genetic tests? Suggesting you should have aborted the 20-week fetus based on unusual skull appearance? Suggesting that the meningitis was a result of some malpractice? Mad at medical professionals who are now “dropping hints” without saying something more substantive?
The problem with these tests highlighted (confusingly) by the NY Times article is that they test for extremely rare conditions but have a high enough false positive rate (by my napkin math, on order 0.05% = 1/2000 false positive rate) that they end up dramatically over-diagnosing these conditions.
That doesn’t necessarily mean the people designing the tests were lacking in “sound science” or were behaving maliciously. Their tests just need to be almost perfect to not overdiagnose rare abnormalities, and perfection in medical tests is a very high bar. Arguably we shouldn’t do tests for such rare conditions unless the false positive rate can be dropped by another order of magnitude or more, and if we do such tests communication to those who test positive should be much clearer. On the other hand, since genetic abnormalities are so life altering, even a 10% or 20% chance of an accurate positive might be grounds to abort a very early fetus.
It sounds like your specific problem was that the genetic tests did not return a positive result, even though it seems plausible that your child had some genetic abnormality. Not picking up every possible genetic condition is a different problem from the false-positive problem highlighted by the article.
I've always wondered-- is the error between tests correlated? Why does it not suffice to gather all the positive outcomes and test them again, then again and again until you have sufficient power depending on the rarity of the disease in the population? I know that for covid tests for example the test outcome errors aren't independent, which is why even taking several tests you can't be confident you're negative even if you tested negative. But I figured that was a special case and it was because their error comes from variance in viral load or something. But would that apply to these genetic marker tests?
Unlike a false negative where there isnt enough to trigger the test, false positive implies that something triggered a positive.
You have something, just not what the test thinks you have. For example, men with testicular cancer, apparently, show up as pregnant in pregnancy tests. Clearly thats a false positive for pregnancy.
How did this article, written by someone who clearly lacks an understanding of basic statistics, make it into the Upshot? They try to make it seem like the test is wrong 85% of the time, but that's not necessarily the case. All we know from the article is that 85 / 100 positive results are false positives, which means the test could actually be quite accurate. If the test correctly identifies 100% of real cases, then that sounds like an excellent test. Just as an example, if 1/4000 people have the disease, and the test identifies 100% of these cases, then around 0.14% of test takers will get a false positive.
Their infographics convince me that they understand the statistics. But one of the key issues here is that the statistics are radically counterintuitive in a way that most people don't understand - the patients, the testing companies, and even some medical staff all incorrectly believe that a positive test for a rare condition means you probably have the condition.
Their graphics say the tests are “84% wrong.” Do you really feel that’s an accurate description? That doesn’t feel like an accurate description to me, and their usage of “wrong” in this context highlights that they don’t understand the distinction and importance of true positives, false positives, true negatives, and false negatives when measuring accuracy.
Going through something like this is very VERY stressful. When you get a negative you immediately forget about it. When you get a positive you die inside. Speaking from experience here.
84% wrong sounds, to me, as an accurate description. Experiencing this from the inside out, only the false/true positive ratio matters. (Given sufficiently low false negative rates, of course)
84% of people whose world is turned upside down are actually getting a wrong diagnosis.
You’re talking about precision (true positive / true positive + false negative) but that’s only one part of the story.
There is a real human cost to having a child born with a rare genetic disease (and I would argue is immensely more stressful). You can easily adjust the sensitivity to the test but at the cost of detecting actual true positive cases. The correct response to receiving a positive is to do another test to ensure it’s not a false positive.
To say 84% wrong is clickbait and used to elicit a legislative response (FDA regulation), which will help the reporters career.
The actual ratio to tell if something is “wrong” is accuracy
(True positive + true negative) / (true positive + true negative + false positive + false negative)
I really feel it's an accurate description. If you get a positive result on the test, there's a 16% chance your fetus has a 1p36 deletion and an 84% chance they don't.
As you said “if you get a positive result”. It’s true, if you ignore the 99.9% of the time the test is correct (true negative result), then you can say the test is 84% wrong.
84% of people who got a positive test result will end up telling their family "it's OK, the first test was wrong, my baby doesn't have a 1p36 deletion after all". The 99.9% of other people who got true negatives are important from a test design perspective, because specificity is closer to the actual levers you can pull on, but it's not super relevant to the decisionmaking process of someone who gets a positive result.
Ignoring all the true and false negatives which themselves are markers of how accurate the test is.
16% precision is the correct statement, saying the test is wrong 84% of the time implies that those getting negative results might actually have positive results.
He framed his statement correctly, limiting his observation to the condition that the test returned a positive result. Saying that 84% of positive results are false is correct if only 16% are true. You'd need to know false negative rates and base occurrence rates (modified by whatever other factors are unique to your situation) to inform the nature of information you get by performing the test.
I disagree. It is clear from the title, “When They Warn of Rare Disorders, These Prenatal Tests Are Usually Wrong”, and the lead that they’re focusing on false positives.
It's true they are focusing on false positives, but the authors are using the ratio of false positives to true positives to paint a picture that the tests are inaccurate, when in reality the tests are accurate. What this article is looking at is called the "sensitivity" of a test: https://en.wikipedia.org/wiki/Sensitivity_and_specificity
No, the article isn't talking about sensitivity. We don't actually know what the sensitivity is from the data the article gives us. We are told that lots of people were screened and a small number had a positive result, of which a proportion were actually positive. You can't calculate sensitivity from that because you don't know how many actually positive cases were missed.
This article is talking about precision, which is the proportion of positive results that are true. And it's okay for precision to be awful, especially when the condition is so rare. But it's only okay if the result is communicated alongside a statement saying what the precision is, which it seems these were not.
While the author may not be well versed or focusing on the stats side, you're missing the human side here I think.
> the tests are inaccurate, when in reality the tests are accurate
If the test make someone consider terminating a pregnancy or even considering it, that's a lot of pain. So for that human, the test is failing its purpose potentially, depending on the value calculation of terminating a viable pregnancy vs the severity of the issue if it comes to term.
For a human, accuracy as you defined it means little to nothing. Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those, which is what the article is highlighting.
Usefulness and helpfulness are far better metrics, and such a high false positive rate is clearly causing issues in respect to those
How exactly do you plan on codifying usefulness and helpfulness?
A high false positive rate is not necessarily a bad thing and may instead be the catalyst for additional tests to confirm the first one.
The tests accuracy may actually be 100%, which is great because it avoids a child being born with a fatal genetic disease. Would you prefer a high false negative rate that misses these diseases instead?
Is it better to terminate 85 pregnancies which do not have a serious defect in order to catch 15 which do? At what point is it not better to terminate 100% of pregnancies?
Did they use the word accurate? You used the word accurate and then you yourself are going on a tirade about how that’s not correct?
It’s clear the article is talking about why sensitivity is important in layman’s terms and while it could use better writing it’s a real problem in diagnostics. This is why you don’t ask men to take a pregnancy test to check for prostrate cancer. It is accurate but not sensitive.
The issue is that the tests portray themselves as being accurate (in the sense of low false positive rates), and portray the result as “your baby has XYZ rare syndrome” instead of “your baby has a 15% change of having XYZ rare syndrome”. If the test providers stated the false positive rate for their results more clearly, parents would be in a better position to make informed decisions.
The larger issue as I see it is that the medical system around these screenings are not well versed in the statistics and able to communicate that to patients. "Eight [patients] said they never received any information about the possibility of a false positive, and five recalled that their doctor treated the test results as definitive." It's hard to know what happened in the room when the doctor spoke with them or what was on those particular patients tests, and that's (one hopes) the worst medical news those people will receive for a long time so listening comprehension is understandably impaired, but there needs to someone available who can help them interpret, even days or weeks later, and these people were let down by the entire system, not just the test manufacturers.
I don't think that it's useful for articles like this to try to educate readers on the way that a precision-recall curve works (and how that differs from the statistical definition of accuracy). Honestly, that would just confuse the vast majority of readers when it's simpler to point out that the tests produce more false positives than they might otherwise expect. Also note that even if we want to be incredibly pedantic, the article never calls the tests "inaccurate" and instead uses a layperson term without a hidden definition ("wrong").
This article is a confused mess. It's something of a Gish gallop in conflating all the different issues they could come up with, while leaving out all the necessary vocabulary (C-f "Bayes" "posterior" "decision theory" [Phrase not found]) making it almost impossible to consider each issue in adequate detail.
It mixes up poor communication (reporting false-positive/negative rates as if posterior probabilities, & exaggerated confidence thereof), arbitrary-seeming decision thresholds (but their hyperventilating over '85% wrong' notwithstanding, many are probably too conservative, if anything, given how devastating many of these problems are, there should be more false positives to trigger additional testing, not less), costs of testing (sure why not but little is presented), tests which they claim just bad and uninformative (developed based on far too little _n_, certainly possible), implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)... Sometimes in the same paragraph.
Plenty of valid stuff could be written about each issue, but they'd have to be at least 4 different articles of equivalent length to shed more light than heat.
> implicit calls for the FDA to Do Something and ban the tests (not an iota of cost-benefit considered nor any self-reflection about whether we want the FDA involved in anything at all these days)...
This is true in so many areas of journalism but lately seems especially egregious in the NYT. And I don't really blame them, as the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter.
I basically can only use these types of articles as a jumping off point for my own research, as I usually find the moralizing conclusion the article comes to as unsupported.
"the incentives for any individual reporter are just too great - having the government make a major policy change based on your article is basically the brass ring for an investigative reporter"
Yep, this is the framing I came here looking for.
Investigative journalists live in the same asymmetrically-incentivized world as social science researchers. If the reporter had looked into the phenomenon and concluded "yeah, boring technical logic pretty much works as expected here" then there's no story.
I wonder if the NYT editorial staff receives zillions of these pitches from their reporters purporting to reveal nefarious phenomena, and most of them turn out benign. It's fun to imagine that the NYT editorship is actually exceedingly good at detecting these outrage-false-positives before publication, but the base rate of outrage-true-positives is just so low that you have to expect some to make it to publication.
I'd like to imagine there's an investigative-journalism-editor-news somewhere, and they're discussing this discussion saying "bah, these clowns are making sweeping generalizations about editorial standards based on only a false positive; this is totally specious with no mention of the prior distribution or sensitivity vs specificity trade-offs"
Specificity and sensitivity are two dimensions that you can measure tests in. You can claim your test is 99% accurate if you mean that "if the test says you don't have the disease, there is a 99% chance that you don't have the disease". That same test can still be 85% wrong if it says you DO have the disease, though.
I doubt that hyping one side of this equation is fraud. Pushing the error in this direction seems like a good idea, anyway. If you have some weird illness, and the test comes back as a false positive, at least you'll continue to explore that possibility for a while. If it comes back as a false negative, then you'll spend a ton of time exploring alternatives which will be true negatives. Probably infuriating.
I'd appreciate it if you could point out where any of them walk the potential customer through sensitivity, specificity, and the fact that if they test positive, there is an 80-90% chance that their will not be affected. I can't seem to find any of that.
When I got my test results, they were clear that the odds of having a disorder were (for example) 1/144, even with a ‘positive’ result. This was through Natera. The problem is that this information is sent directly to the provider in most cases, so parents are left interpreting someone else’s interpretation of statistics. My midwife specifically told me that the test isn’t often wrong, even though the actual odds were there in the fine print.
> “The chance of breast cancer is so low, so why are you doing it? I think it’s purely a marketing thing.”
This mindset is ingrained in every doctor I speak to, but I think it's just so wrong.
Take DiGeorge syndrome. You have a 1/4000 chance of having it, and the test carries an 81% chance of a false positive. The above doctor calls this "marketing"? Foolishness. That's an incredibly useful test. The downside is small, and the upside is asymmetrically large.
We need far, far better screening for all sorts of things. Adult cancer and heart screens once a year, prenatal screening, and on. We do a good job with breast and prostate screens, but for rarer conditions our current approach of waiting for the disease to be symptomatic makes no sense. Part of that will be driving the cost down. There is so much market need for a legitimate version of Theranos and I'm glad there are some companies working on these things.
This article seems a bit deceptive. We are going through NIPT soon and our doctor went over false positive and false negative rates for the common screens. Our doctor has pointed out some of the screens (esp for rare conditions) are not that accurate. The only procedure with high accuracy, amniocentesis, has a slight risk of miscarriage (our provider quoted 0.3% ) so its still statistically better to take NIPT and then only consider amniocentesis with a positive result since there is no risk from NIPT.
You are supposed to treat a positive on NIPT as “there’s a chance your baby has this, need a more accurate procedure to confirm”.
It sounds like their ob gyn wasn’t able to explain results to them or they didn’t understand the probabilities. To be fair our provider didn’t even suggest tests for the disorders in the article, probably because of the false positive rates and rarity. Sounds like these extra screens shouldn’t be offered.
We have been undergoing IVF with my wife since 2019. (Covid made a huge mess of those plans...) One of our embryos tested as a possible positive (but only slightly) for aneuploidy of one chromosome.
The doctor, a veteran of IVF, looked at the results and said "my experience is that this is either a very small mosaic error, which tends to be utterly invisible in real life, or a computer artifact. I have never seen embryos with those borderline results develop any serious problems later. Things would be different if the aneuploidy signals were clear, but definitely do not discard this embryo".
I am not a parent, but the criticism of the article appears to be around a misunderstanding of statistics, or at least how to apply them. While I agree that criticism is completely correct, it overlooks the human nature of the people receiving the tests. At an already-stressful point in someone's life, it seems almost like bad bedside manner for the medical community, even if in an automated fashion, to tell people that there might be a complication looming.
This _does_, however, seem like a framing issue, more than a utility issue. If the tests are 100% accurate at detecting true positives, they're a great aid. But rather than framing the tests as a be-all, end-all source for information, why not frame them as "a test that suggests whether or not you should get other tests"? That simple wording change would save a great deal of added stress on someone starting or growing a family.
I totally agree with this. Managing perceptions and expectations is super important here.
Having been on the receiving end of a false positive, I'd still do the test again for a hypothetical future pregnancy. Even though it was hell for a couple of days.
I recall a period in the early 2000s when unindicated whole-body CAT-scans were being advertised on television.
That got knocked down pretty quickly but wow a lot of folks picked up a big chunk of their lifetime radiation allowance because of that.
These tests seem to operate under a similar model, disregard the risks of unnecessary testing in return for information of limited utility that may cause material harm.
I think you're conflating "these tests cause harm" e.g. radiation and "the information gleaned from these tests could cause the patient to make poor decisions". Having a regulatory body make this value judgement for people has quite a bit of disadvantages. See "DON’T TRY THIS AT HOME: THE FDA’S RESTRICTIVE REGULATION OF HOME-TESTING DEVICES" https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article....
This isn't really a fair criticism. I could be wrong, but I believe your comment reflects a bit of naivete about the current state of evidence-based medicine.
To evaluate the value of performing a diagnostic test as an intervention, you DO have to look at final actual patient outcomes at an appropriate end target which includes sending people unnecessarily down different treatment paths, including additional testing with additional risks. And most importantly is that, in fact, mere knowledge of diagnostic results has been PROVEN to cause harm in many scenarios.
Now... if a patient WANTS that test, I think it should be available. But whether or not it should be performed routinely without prompting is an appropriate question for regulatory bodies.
'For a disease w a 1-in-20,000 risk, a test w a false positive rate of 1% and a false negative rate of 0%—an insanely accurate test—would identify 1 correct case and 200 false positives every time. Or would be wrong 99.5% of the time.
Medical professionals are often shockingly bad at statistics. My wife and I were talking about birth control to an RN* after our first child was born. The RN mentioned cooper IUDs were 95% effective. He asked what timeframe that was measured over and she couldn’t answer. Not only did she not know, but she couldn’t even understand why we were asking the question.
*) My wife insists that it was a doctor, not an RN, but my brain won’t let me process that possibility.
Before my daughter was born I sometimes felt like it was the doctors job to scare us with every worse case scenario possible. It was quite stressful and upsetting.
When my then-pregnant wife called our fertility clinic with a worry the nurse calmed her down but also basically told her “welcome to the rest of your life.”
I've heard that in the early days of HIV, the tests were (e.g.) 95% accurate, and when patients saw their positive results and the supposed 5% chance it's wrong they'd sometimes kill themselves.
They revised the tests so the first test would say Inconclusive rather than Positive, and ask them to repeat it. This saved some lives.
Maybe this a UX failure? Shouldn't the test designers present the results like this, even to doctors?
Absolutely a UX failure here, one that it seems some doctors translate for patients while others are left in the dark on. From the way people are responding on here about the use of statistics in the article, it's clear that a big portion of the techo community I think is undervaluing that often UX is far more important than it is treated.
I had a friend tell me their daughter tested positive for something and they didn’t do any follow-up testing because someone else they knew also had a positive result and their baby was fine, so “the test is worthless.” Luckily their daughter seems to be healthy herself, but it clearly wasn’t explained well to them by their doctor.
This seems to miss the point entirely. Even for their worst example the odds of the fetus having it go from 0.005% to 7%. That's valuable information even if it's not perfect or somewhat hard to understand.
This would be valuable for running some extra tests (possibly more expensive, but more accurate), but not for, say, decision to abort the kid, which is what usually "hangs in the air" after such a test result.
"A 2014 study found that 6 percent of patients who screened positive obtained an abortion without getting another test to confirm the result."
Maybe people aren't informed enough. It is my experience that some doctors tend to cut conversations short and some people are shy/insecure enough not to pry answers out of them.
In this case, that would be a tragedy, given that statistically 5 of those 6 aborted fetuses were healthy.
Edit: I found the following comment in the comment section of this article, which appears to address the same issue:
I am a physician with a PhD in Biomedical Informatics. Most patients who receive these tests do not see a maternal fetal medicine doctor or genetic counselor, and no one actually explains that the tests they are receiving are “screening” or “diagnostic.” Your opinion that this article does a disservice to patients reflects your unrealistic assumption that most of the doctors ordering these tests are actually communicating effectively with patients (or frankly, even understand the tests themselves). In my experience, they usually aren’t/don’t. Articles like this “fill the gap” on patient education when doctors are unable to explain math and risk (i.e., most of the time).
That’s a tragedy. Maybe there needs to be regulation requiring results are delivered by genetic counselors rather than physicians. Or maybe this is willful patient error.
That depends on the person though doesn't it? I'm not sure what I'd do in that situation. But 7% seems awfully bad odds for painful and debilitating life.
I guess that depends on the exact scenario. There are likely people with a variety of conditions who enjoy their lives vs having not been born. It brings up a seemly logical contradiction that we terminate fetuses (potentially viable in some cases) on the assumption that they don't want that life yet we don't allow people who want to kill themselves to do so.
There's a lot of sibling comments going on about whether the value they're looking at is the right one. What the Times is showing as their headline number is Positive Predictive Value (True positive/(TP+FP)), which depends on the prevalence in the population. The "methods section" here is a little vague, but given the low prevalence I'm willing to accept on face value that it's basically accurate (i.e. that it's not assuming that the families getting these tests are not orders of magnitude more likely to be positive for these diseases). If the test result truly said one patient's 'daughter had a “greater than 99/100” probability of being born with Patau syndrome', then that's concerning, but given the fairly narrow quotes around the number, I'd suspect that what is actually on the test result is not inconsistent with the fairly low PPV on these screens.
We were told our son had a high chance of being born with down syndrome. It was quite stressful to hear this as we weren't going to do anything about it regardless (he was born with no issues whatsoever and is now a thriving young adult).
I was in this exact situation. I received a phone call from my midwives, saying that my son had tested positive for one of these disorders, and that these tests aren’t usually wrong. Fortunately I had done my research and knew that the false positive rate is high. But the entire system is set up to provide a terrible experience.
Your results are sent directly to your provider, so you can’t read the fine print yourself. And if you do get access to the results, the wording implies that a null result (not enough DNA collected) actually means you’re likely to have some disorder. In fact the wording here actually got worse in the three years between my two (healthy) births.
Ideally these companies should require genetic counseling before you take the test. Parents should understand that these tests are for screening purposes only, and that a definitive diagnosis can’t be gotten until 16-20 weeks. Unfortunately these companies have found a niche- parents wanting to know the sex and health of their children as soon as possible- and have no real reason to improve their practices.
Isn’t that often true with screens in general? The threshold often allows a good number of false positives in order to minimize false negatives. The goal is to know when to seek further diagnostics. Communicating that to patients can be a challenge but it doesn’t mean the screens were designed incorrectly.
Edit: They kind of do this farther down in the article.
Considering this as a UX challenge - imagine a grid of 10,000 dots (100x100).
Draw one box around the base rate - the rate at which you expect to find the problem in the population. If the base rate is 1%, then the box is 10x10 = 100 dots.
Then color in the dots for the test positive rate (not false positive, just all positive tests) False positives would be the colored dots outside the box.
Next to that, put strikes through the dots corresponding to your expected false negative rate.
This is an example of a problem that is so hard to explain. The vast majority of folks getting these tests will get a true negative. Such that for most people, this is not an issue. So I get that it takes effort to make people care.
That said, I do feel that pulling in abortions to the debate is specifically to trigger a set of readers. But to what aim? They have not established that the tests could be better. Just that when they say yes, they are still not perfect.
The state mandated tests in California are far worse. At least with NIPT tests, if you get a negative, it's fairly certainly a negative. The state tests have all kinds of unnecessary false positives, and if you don't have the NIPT to negate them, you are in for a lot of worry.
http://web.archive.org/web/20220102044133/https://www.nytime...