A very close relative of mine passed away 2 weeks ago. She had cancer for 2 years and when she was first diagnosed the doctors would give her 8-10 “good years”. Well...
When she died, I was shocked to learn that very few data points of her sichkness and treatment history would be preserved for later analysis. Doctors work almost entirely on their gut feeling and probably some clinical studies with n being very small. I hate that she died, but it’s even worse to know that all her data died with her and won’t help any other patient. I think this is one of the rare cases where collecting more data would help protect people
I get the impression that much of medicine is one of those strange practices that looks kinda like science, but really isn't. Medical practice appears to work more like a highly evolved guild system than any other kind of recognizable scientific activity. It's probably far beyond the point where its necessary to bring in actual scientific practices, so I can't say it's on the cusp of a sweeping change.
One of the problems that really prevents the change is that analysis of patient data is so highly protected that it's virtually impossible to do proper studies. We assume that people with such access are performing them, but I know from second-hand experience that actual time spent on real treatment studies is very small and most of the time is spent dealing with policy and patient privacy issues.
Large insurance and government-based (VA, CMS, etc.) providers have the most data, but either what they're looking for isn't exactly aligned with what patients need, or the organizations are so fubar (e.g. the VA) that real studies are unlikely to happen.
There really needs to be some kind of research exception/feedback loop on massive scale data analytics that allows some group of objective researchers to mine patient data and put in place actionable recommendations that work their way into medicine rapidly -- perhaps with some penalizing stick insurance providers can leverage against practitioners to make sure they adopt newer better practices.
> I get the impression that much of medicine is one of those strange practices that looks kinda like science, but really isn't.
It is very hard to escape from the fact that there are patients who are so difficult to keep alive that the rational choice is probably to let them die. The exact circumstances will depend on your value structure, but once that is settled any scientific approach that optimises for greatest net good or quality of life will involve letting some patients go without really trying to save them.
There is a pretty substantial lobby (no particular affiliation) who just won't accept that attitude being out in public.
I watch the struggles of data science to take root in our medical system with great interest in light of this. Anyone using a statistical argument to justify inaction when confronted with a patient is running a risk of being pilloried. Pictures of ancient grandmas on the front page of a newspaper style stuff. And that is if they get the stats right - get something wrong and then you feel like a loser on top of running that risk.
Between that and the serious privacy concerns, I can believe that the medical world will be mysteriously disorganised and ad-hoc in their decision making no matter what statistics available. I don't see what the real incentive is to look at and use the numbers, without having a statesman like dedication to the greater good, come what may. Such persons combined with leadership and statistical knowhow are rare.
The simple fact that "evidence-based medicine" needs a special name to describe it, rather than all medicine being evidence-based, is still absolutely mindblowing to me.
No, this term was proposed as an "emerging new paradigm" in 1992 (!!) focusing on "the examination of evidence from clinical research" as our most reliable source of information.
"Western standard medicine" was still largely based on intuition and woo less than 30 years ago.
Yup. Doctors interested in the philosophy of science have moaned about this sort of thing before. A more typical example goes like this:
I'm a GP, and I see dozens of patients every year with cases of Foozle disease. I was taught in medical school to give them orange pills, but I follow developments in general medicine and I know the new purple pill has seen good results for Foozle too. Nobody has published even a small study about which is better.
The law says since a responsible doctor wouldn't know one is better I can choose either. I could give all my female Foozle patients the orange pills and prescribe the purple for everybody else, or I could just pick whichever is cheapest, or I could even pick randomly for each patient. All legal.
But, if I pick randomly AND record the results, thus doing science to find out which was best, that's an unauthorised medical experiment and I could lose my job or even go to jail.
» But, if I pick randomly AND record the results, thus doing science to find out which was best, that's an unauthorised medical experiment and I could lose my job or even go to jail.
Perhaps we can try to cut down the red tape a little but we can't eliminate the idea of informed consent. If you're doing a study on patients, they absolutely have a right to know and a right to refuse to be a part of your study without any repercussions.
I think informed consent is something we can't get rid of even if we do something drastic and eliminate medical licensing altogether.
As a researcher, this is absolutely true and incredibly frustrating.
I am trying to make the survival data of a clinical trial combined with sequencing data of the patients publicly available. Nearly all the patients have died by now, as the patients in this cohort patients have a very poor prognosis (median survival of about 24 months). It is still almost impossible to release clinical data (diagnosis, number and location of metastasis, age of patients, etc) combined with sequencing data without violating privacy laws.
In the meantime, there is almost no career incentive for me to make the data available. I do not have, and am unlikely to get a permanent contract. In my next job interview I will be judged by the number of publications on my CV.
When it's the other way around and someone survives 10 years on a "2 years left to live" diagnosis, we seem quick to credit their strength, determination, spirituality, etc., rather than looking at how inaccurate the original prediction might have been.
Thanks. Absolutely. And of course, I’m aware that I’m quite emotional about this right now. But I think the other way around like you describe is better because you’re forced to say goodbye, and can enjoy whatever extra time comes afterwards. I have the feeling that I’ve missed that opportunity (which is of course my fault, but still)
Thank you. Well, the way I understand it: her treatment history won’t be considered by any other doctor than her own. The reason for that is that she wasn’t part of a proper medical study.
However, I believe there is value in her data. The last days she was treated with an experimental drug. If there was an open database on all cases with treatment history I could have looked into that and get a feeling what to expect. These things often come with huge side effects, so you want to do it only if there is a slight chance.
Also, if this data was collected globally, the number of reference cases would be much larger.
This is partially true. Her data can't be used willy nilly (sp?). Hospitals and doctors offices can use this data for their internal operations, to improve efficiency, quality, etc and (honestly) profit. They can also use this data in research, but only with IRB approval for exemption from consent. Her data should be protected!
Also, I would add that the data collected for routine patient care is often of significantly lesser quality than that collected as part of organized research studies. This is because in a research study, all patients get the same studies and tests whereas in clinical care, they only get what they need, so there is significant amounts of 'missing' data.
but if it could be anonymized (i.e., the medical data such as blood tests, history of diseases etc disassociated with any names), then wouldn't it make sense to have this data be used for research?
Uses cox regression model which is a survival regression model.
Also the base model aka previous model they're comparing it to is a logistic regression and the link leads to a pdf about how to increase hospital efficiency it seems like. This sounds stupid and heartless.
In statistic we got survival analysis, a whole branch that is focus on patient and their survival rate for the medical field. Google chose to compare to a paper and algorithm that focus on what seems like making hospital more money instead.
I've seen a lot of data science people goes into different field and just telling people they can make money for them. It's great but with healthcare I don't think people should be treated as dollar signs.
If anybody up and coming wants to use data science in bio, I would encourage them to look into statistic and biostatistic. We have tons of stuff already and then branch out to ML later. But at least know what's out there and there are establish organization, nonprofit out there too that all they do is biostat and build model. My friend works at a nonprofit child oncology.
I just want to point out there are people that's building model to help patient with terrible sickness out there to survive. We're not diddling our thumbs trying to make other people richer.
Their model was not logistic regression (the networks may have had a few logistic units in it, but it's hard to call that a logistic model). The logistic models they compared against were published models from the academic literature. I'm not enough of an expert in this specific subdomain to comment on whether these were the benchmark papers or not.
I don't know if you're trying to imply that the authors of this paper didn't know/know of survival analysis, or if it was a general rant. Looking at the names I know on the paper and the affiliations/backgrounds of the others, it's safe to say they are aware of proportional hazards models.
Survival analysis is not called for when predicting the outcome variables of interest in this study, and that seems to be your primary beef - that they chose the wrong outcomes to model in order to "make hospitals money". I would think that being able to predict outcomes help hospitals plan and manage their resources effectively. From your high horse this may appear to be a wasteful endeavor, but controlling costs will do much more to save lives by making healthcare accessible, rather than building survival analysis models for rare diseases that affect some trivially small portion of the population.
The truth is outside of tech, statisticials (or data scientists) are way underpaid relative to the training and specialization demanded of them. This is true for non-profits and academia. Note that administrators in both these fields are not underpaid to the same degree. Instead of money, they are expected to pay their bills with warm fuzzy feelings of doing good for the world, because of attitudes like the ones expressed in your comment.
Also, fun fact: survival analysis was developed for actuarial use to make ugh money, not bio/medical statistics.
That reply is a bit factual... I want to talk about why it came to be this way.
Medicine has advanced to a point where some serious illnesses can be kept at bay for decades. People with some formerly mortal diseases can get medicines that cost $100k/year and/or professional help every day, and live well for thirty years.
This means that each of rich societies has to choose:
1. Raise health insurance rates until the budget covers everything that's possible medically. However, in some rich countries, people's after-tax income is only a few doublings above the current health care plus pension costs...
2. Decide that some treatments aren't worthwhile, ie. everyone with Hyperthis or Abnormalthat Syndrome gets cheap palliative treatment and a peaceful, gentle death.
3. Decide on a per-patient basis, often involving numbers such as "cost of treatment" and "years left of productive life".
Option 1 is the humane way, but slightly impossible. Options 2 and 3 involve treating people like dollar signs, one way or another. It's an unpleasant choice, not an avoidable one.
There are many minor variations. Another common one is the variation where you you pick an insurance plan before your illness, and the insurer later decides on the details on behalf of all of its insured individuals. And... but they're all minor variations.
Decommodify every aspect of healthcare, from the education of doctors to the hospitals themselves. Invest massively in basic medical science research and provide the corresponding advancements and technologies free of charge to the healthcare system.
These treatments cost 100K because of the economic arrangements from which they emerge, not because that’s some inherent (or even sane) price. For a more concrete contrast, Japan has a healthcare system based almost entirely on government set prices for procedures and has better outcomes than the US.
€100k per year isn't very many hours of professional attention per day. A nurse for an hour or two per day, a physician once or twice per week, an occasional consultation with another, some expensive equipment, and of course education and general facilities for all these professionals... it adds up.
How much is the fully loaded cost of one of your own your own working hours?
I think a more useful question is how is it that developed countries like the US and UK have widespread unemployment, but no shortage of need of this kind of work? Perhaps the economy is massively misaligned in who and what it serves.
> It's great but with healthcare I don't think people should be treated as dollar signs.
These kinds of models are also great for triage. Healthcare is a limited resource, especially in trauma situations which have been using models to measure survival for decades.
Even when we have whatever technology would make heathcare less limited, there will need to be ways to measure the prognosis of the patient. Otherwise we would be giving very unpleasant treatments to patients who only need palliative care (like chemo for terminal cancer patients).
As much as 80 percent of the time spent on today’s predictive models goes to the “scut work” of making the data presentable, said Nigam Shah, an associate professor at Stanford University, who co-authored Google’s research paper, published in the journal Nature. Google’s approach avoids this. "You can throw in the kitchen sink and not have to worry about it,” Shah said.
Alarm bells start ringing right about here.
A solid 90% of AI is data munging, so that work isn't going away even a little bit; historically, Google throwing the kitchen sink into healthcare problems with lots of noisy data leads to flu seasons being correlated with softball victories or other wildly spurious correlations.
I'm serious - I've got a better-than-average understanding of statistics, and everything I've seen referred to as "AI" seems to really be just a statical model with a thin layer of code over the top to make decisions based upon it on the fly. Is there more to the state of the art that isn't apparent from the outside?
In fact, the only thing that comes to mind that doesn't fit that description would be genetic algorithms, but I don't hear much about them these days.
I hate to pour cold water on this, but medical researchers have been predicting mortality for years.
For example, here's a paper from the 1980's also predicting when a patient will die, also using a couple of thousands of patients' data: http://europepmc.org/abstract/med/3816253
And, Google's paper wasn't published in Nature, it was in a new open access journal owned by Nature. In academia, that is like the difference between a really fancy porsche, and a really cheap volkswagen (two very different things, both owned by the same company).
Google's innovation isn't predicting mortality. It's predicting mortality based on data that was hidden from previous models. That's very likely where all of the improvement is coming from, but since it's hard to write a good headline or lede based on that, we get this instead.
"...the novelty of the approach does not lie simply in incremental model performance improvements. Rather, this predictive performance was achieved without hand-selection of variables deemed important by an expert, similar to other applications of deep learning to EHR data. Instead, our model had access to tens of thousands of predictors for each patient, including free-text notes, and identified which data were important for a particular prediction."
So it sounds like the advance here is actually in the following: "a generic data processing pipeline that can take raw EHR data as input, and produce FHIR outputs without manual feature harmonization".
The article doesn't explain this very clearly. Yay!
>In academia, that is like the difference between a really fancy porsche, and a really cheap volkswagen (two very different things, both owned by the same company).
I think you want to say Skoda or Seat. Both owned by VW and the cheap cars.
I would feel quite unconfortable if every action taken (or not taken) by medic staff has to be second-guessed simply because there are financial incentives derived from this research.
Customer A does not receive full medical support because "he's dying anyway"?
Is there a way to ensure that such cruel use of the research is prohibited?
It's better if this becomes an ethical/moral problem about when to care for patients and to what degree, rather than a guessing game problem while anybody's guess is as good as anybody else's.
Heroic efforts on futile attempts to keep dying patients alive have a huge cost in human suffering and we, as a society, really ought to stop making them.
>Customer A does not receive full medical support because "he's dying anyway"?
Unless the patient consents to stop or reduce care, this would be medical negligence, which is illegal (in the US). Medical providers are required by law to provide medical treatment with all the knowledge and skill they possess.
> Medical providers are required by law to provide medical treatment with all the knowledge and skill they possess.
No, to avoid liability for malpractice, they need to meet the “professional standard of care”, which is assessed by general practice in the medical community, not “all the knowledge and skill” the individual provider possesses.
I wish that were the case. They'll treat you in the ER, but not make sure you can actually purchase antibiotics to cure the thing that brought you to the ER to begin with, nor make sure you can afford the bone specialist you need to see for your surgery.
Anything short of an emergency and you can be denied care due to finances. Even if your problem is cancer or they know it will eventually be an emergency or life-threatening. Money is part of the reason folks will use the emergency room instead of urgent care: Urgent care requires upfront payment, the emergency room does not. Many doctors will refuse to see someone if they do not pay upfront.
Luckily, different agencies tend to step in with end of life care.
What is the ER going to do about the fact that you can't afford a prescription or treatment/surgery from a specialist? Their job is pretty much to stop you from dying right now and direct you to another doctor when you're stable.
You can also present insurance upfront but same difference I suppose. A world where you didn't have to pay upfront would make care even more expensive because doctor's offices have to essentially become banks.
Most hospitals have pharmacies in-house. Would it be too much to ask that the hospital provide you with the prescription or pain meds that will take care of the problem that brought you there and include that in the bill? As far as the specialist is concerned, they can easily refer people to places that help the folks that can't afford such things. All they'd need to do is give a piece of paper with information on it.
It would essentially be this: YOu break your leg and go to the hospital. When you leave, you have not only a cast, but pain meds and crutches if required (many hospitals do not offer them). If you can't breathe and you need medicine for infection, you leave with the medicine. If you can't afford a follow up with a doctor, they refer you to folks that can help you.
Insurance doesn't help if you are near-broke and two days away from payday. Lots of folks can't afford their deductible plus the yearly out-of-pocket.
Huh? The doctor is still getting guaranteed payment but by your government instead of you. I mean the world in which doctors had to accept patients regardless of their ability to pay and have to try and recover later.
This was a year ago so things may have changed, but a senior exec at one of the nations largest health systems told me they were looking to change their end of life care program and looked at dozens of AIs that predicted when patients would die. None of them came close to just asking doctors which patients they thought would die within 12-18 months
The point of the project was to make end of life easier for patients and families. Many patients get transferred between multiple facilities in their last few months, and few end up passing in a place and manner they'd like. This project aimed to ask patients and their families how they'd like to spend their last few months.
Initially the health system resisted this bc they wanted to keep patients in the hospital to make money off them, but then they realized these patients were in the hospital so long they weren't profitable. Then the plan was adopted quickly
> None of them came close to just asking doctors ...
at a conference, i once heard that the best single predictor of ICU patient mortality was reduction of doctor attentiveness. in other words, patients in ICUs whom doctors began to visit less, treat less, basically start to ignore -- those patients were the ones for whom death was imminent.
the challenge is that it's hard to untangle cause and effect. was it a self-fulfilling prophecy? etc.
Maybe this could be useful in countries with nationalized healthcare, but it seems a terrible idea to deploy in the for-profit monstrosity of American healthcare. Institutions will immediately begin to deferring to the “objective” judgements of such systems to either pull care to save costs, or, as was the case with that hospice provider in Texas, kill sick people for more money:
With deep learning, actuaries were inevitably, mostly obsolete, at some point. It’s unpleasant to think, that in a way, tech can now readily disrupt nearly any arbitrary office-worker knowledge industry at will, destroying businesses and specialties and concentrate both capability and wealth in ever fewer hands. Coding isn’t a “forever” specialty either... self-coding machines will create languages, protocols and systems humans will likely be unable to understand, limit, modify, monitor or audit.
Uncomfortable, but over all a good thing as long as the system uses this information properly; offering doctors additional supportive training, if appropriate. (And not forgetting that the doctors with the the worst recovery rates may be the ones predisposed to take the trickiest cases).
>(And not forgetting that the doctors with the the worst recovery rates may be the ones predisposed to take the trickiest cases).
IIRC there are a lot of surgeons not taking on cases already because of the risk involved to their career. This might exacerbate it if the data is used blindly.
The UK had a big case where the question asked was why they didn't spot the doctor with the unusually large number of patients who died unexpectedly, when a cursory examination of his approach to their medical records might have uncovered evidence that many of those deaths weren't natural.
I'm sure it's an interesting technical challenge with huge amounts of complex, nuanced debate involved, and it could have a huge impact on health care.
That said, my wife is an oncology nurse, and I will bet everything that no machine will ever get better at predicting than an experienced, skilled nurse. Humans are built to read humans.
Have you ever asked her if she wants to do the job of predicting when a patient will die? How many hours would she need to spend reading the patient's history to come up with a prediction she feels satisfied to use to decide what service the patient will get? I'm not a medical professional but my gut reaction is I don't want to do anything with predicting or sentencing if some machine can do it almost as well as I can.
I remember this story of a radiologist who told me he thinks we spend too much money for too little during the last about six months of a patient's life. If we had better information on when the last six months starts, maybe we could reduce the cost of healthcare? Apparently, we spend cost to 18% of GDP on healthcare in the US? Iirc, most of Europe is closer to 12?
I would be very willing to take that bet on a 30 year horizon
Just think about the machines/algorithms your wife already (indirectly) uses to predict outcome for patients.
- An MRI scan, which is created from physical laws (Maxwell's law together with a quantum mechanical understanding of hydrogen atoms) and reconstructs a 3D-image of the different tumors. Knowing if and where metastasis are present have an enormous impact on the prognosis.
- The algorithms used to align sequencing reads from biopsies to determine the mutation status, which are critical to determine the prognosis of some tumors.
In fact, I'd wager that if your wife did not have access to these algorithms, she would already perform worse. And, in my opinion, the distinction between machine learning and the algorithms used to reconstruct 3D-images or align sequences is somewhat arbitrary.
Why do you say that? Take insurance in its most basic form: a company does its best to determine how much money they're likely going to pay out over the lifetime of a policy and spreads that cost out for their customer who is then insulated from unexpected expenses or non-uniform cost schedules. It seems perfectly suited to medicine.
Insurance doesn't solve the problem of a population where the total cost of care is more then they can collectively afford or when an individual's treatment has known costs higher than what they can afford but nobody has a good answer for that.
Insuring everyone from birth can be a solution for the individual but not the population.
Insurance works when you incur a lot of one time costs, it works much worse when you start incurring a lot of expenses over a long period of time. Ideally you could buy insurance that would cover all of your long term costs once you contracted some disease, but this doesn't exist in the US. And doubly doesn't exist when you realise that there are a bunch of forces incentivizing people to get their insurance from an employer.
But at the earlier point in time, when people are buying insurance, these are unknown future costs.
And this is the whole problem, insurers don't want to pay those costs because that would increase the premiums they need to charge, making them uncompetitive (and then you're into the realm of behavioural economics), but what people actually want is something that can insure against spending on chronic diseases, which accounts for 90% of the spending on healthcare in the US.
And IMO, to the extent that insurance doesn't cover chronic conditions, it is a bad model for healthcare.
You just need regulation in place to prevent this. In the Netherlands, it's illegal to deny someone healthcare. It's also illegal not to have health insurance. The system works, Insurance is around ~100 euro per month, covers almost all treatments and deductibles are between 350 and 850 euro's per month.
I work for a cancer research hospital. I'm training an LSTM today using TensorFlow to determine months left to live for specific patient conditions given previous outcomes.
I'm programmer and I'm always saying (something like this) - the program / AI can be able to do things the best way how it can be done the best way by its programmer(s)
My point is - this is completely nonsense. IT world and Real world are two difference worlds.
Which is why positive AND negative feedback loops from your users/customers/<people whose lives your software will affect> are vital to building great software.
The trolley problem here already exists regardless. Just because making difficult decisions is uncomfortable doesn't mean the most moral outcome is to leave it to chance.
When she died, I was shocked to learn that very few data points of her sichkness and treatment history would be preserved for later analysis. Doctors work almost entirely on their gut feeling and probably some clinical studies with n being very small. I hate that she died, but it’s even worse to know that all her data died with her and won’t help any other patient. I think this is one of the rare cases where collecting more data would help protect people