Hacker News new | ask | show | jobs
To improve medical trials, justify exclusion criteria (trevorklee.substack.com)
83 points by klevertree 1521 days ago
8 comments

>These unjustified exclusions can have real clinical implications, too. For example, most asthma studies exclude morbidly obese people, as morbidly obese asthma is notoriously resistant to treatment and there aren’t good explanations as to why. However, once asthma drugs are approved, they’re approved for all asthmatics equally. As a result, morbidly obese people get prescribed asthma drugs that were never tested on people like them [1].

>[...]

>[1] This is literally going on today by the way. The FDA approved Tezpire as a breakthrough drug for asthma in December 2021. Tezspire excluded morbidly obese people from their efficacy trials. This fact is not mentioned anywhere in Tezspire’s labeling.

Wow, I had never heard of anything like this before. Does the FDA have a justification for why there isn't a requirement to mention significant exclusions like this?

As someone who has both designed and run clinical trials as my job, this assertion is likely* nonsense. First, the division you submit to will reject your recruitment criteria if they feel it does not accurately reflect the treatment population within the United States. They also reflect the specialty of the particular division (e.g. Oncology is allegedly far less worried about most side effects than, say, the Dermatology division, due to the kinds of indications they deal with).

Second of all this kind of info is on the label (prescribing information) even if it doesn’t make it into the short summary (package insert, typically only a dozen pages or so) given to patients. Doctors do read those, you know, and within their specialities know what kinds of things to look for.

There have been some notorious cases, but by and large I’ve found the people I worked with at the agency to be professional and solid. I’m no longer in that business and have no reason to say anything I don’t believe.

* I didn’t bother to look up the label for this drug but they are all public info on the FDA web site and In the USP.

Why on Earth did you not bother to look it up before asserting your opinion...?

https://www.accessdata.fda.gov/drugsatfda_docs/label/2021/76... Body Weight Based on population pharmacokinetic analysis, higher body weight was associated with lower exposure. However, the effect of body weight on exposure had no meaningful impact on efficacy or safety and does not require dose adjustment.

Assuming that grandparent comment is correct about morbid obesity exclusion, then you were the one spouting nonsense right? Doctors will definitely give this to patients regardless of weight with a note like that.

Both my note and the download you discuss are clear about this.

You have linked to the the package insert, not the label, and it clearly states at the very top

    HIGHLIGHTS OF PRESCRIBING INFORMATION
    These highlights do not include all the information needed to use
    TEZSPIRE safely and effectively. See full prescribing information for
    TEZSPIRE.
As I wrote in my comment:

> ... this kind of info is on the label (prescribing information) even if it doesn’t make it into the short summary (package insert, typically only a dozen pages or so) given to patients. Doctors do read those, you know, and within their specialities know what kinds of things to look for.

You are simply quoting the short summary and drawing a conclusion based on the limited information that appears on it. Perhaps the author of the blog post made the same error. The doctor reads the actual prescribing information and the evaluation population must be specified there.

The second page starts the "FULL PRESCRIBING INFORMATION"; the body weight quote above comes from section 12.3 of it and there is no mention of a weight exclusion in the discussion of the clinical studies in section 14. AFAIK, "label" typically refers to this sort of ~20 page prescribing information, but is there a different label you have in mind? I believe the one-page package insert is the last page, page 17.
Am I losing my gdmn mind here? You say look at the label not the insert, the FDA will have it. I link you to the FDA's label. The link literally had the word 'label' in it drugsatfda_docs >>> label <<<

Inside the document there are 3 things the 'HIGHLIGHTS OF PRESCRIBING INFORMATION', AND the 'FULL PRESCRIBING INFORMATION' AND finally 'PATIENT INFORMATION'

I cite the relevant information about weight from them 'FULL PRESCRIBING INFORMATION: CONTENTS: PART 12 CLINICAL PHARMACOLOGY and you act like I not you have performed some sort of bait and switch...

You are the one claiming expertise here, so enlighten me, where is the the carve out for "we didn't test this at all on the morbidly obese" in place a doctor will find it if not in 'FULL PRESCRIBING INFORMATION'?

While you may not have financial incentives, the psychological incentives are often very strong to maintain a positive idea of an industry one was a part of.
That's a reasonable concern. All I can say is that I left that field for a reason (well reasons) and have some serious concern about some ethical issues and attempts at gaming the system.

However by and large my concerns aren't around science (though there are exceptions, cough alzheimers), they are mostly around the marketing, pricing manipulation with regards to medicare (ever wonder why drug companies give everyone coupons?) etc. The FDA has very little to do with some of these issues and none with others (e.g. scamming the taxpayer)

In general I don't think pharma execs are necessarily nice people (though some are!) but most are not evil like the Sacklers.

Oh the irony that you’ve excluded an investigation of the relevant example here, but made broad claims nonetheless.
My note was clear that the package insert is insufficient for prescribing. It is explicitly marked so -- see a comment I wrote to another reply to my GP comment.
Totally unrelated, but just briefly to say, thanks for your work with Cygnus.

I doubt I'd have been able to build a career for myself in software engineering without Cygwin being available and lively back in the days when I was required to use Windows fulltime, and along with that I learned a lot from working with it that's been of great help to me ever since. And I'm to this day running Cygwin on the one Windows box I still maintain!

So, thanks for whatever hand you had in that. If you ever find yourself in Baltimore and thirsty, hit me up and I'll buy you that beer or other suitable beverage I owe you. :D

Much like sleep apnea in the morbidly obese, this is likely just a different condition entirely with a physical cause.

As to why it is prescribed anyways - like the MD who taught me about sleep apnea said, morbidly obese people get offended when you suggest that there aren't drugs that you can give them and they just need to take some extra weight off the strained organs. It's easier to give a person who is certain that it's not the weight, but some condition solvable with drugs, a drug.

I feel like I have to play devil's advocate here a bit.

Obesity is an easily treatable condition. One might try to treat them in order: Obesity first, then the asthma. That would justify excluding that particular complication from the study.

> Obesity is an easily treatable condition.

Is it? I thought long-term success rates were very low and that prevalence was increasing all over the globe.

Also a person who has trouble breathing tends to not do significant exercise because they run out of breath too quickly.
While exercise is healthy, it isn't a great way to lose weight. Eating a healthier diet, in particular lower calorie, is how you lose weight.
prevalence != difficulty of treatment

people's unwillingness to cooperate != difficulty of treatment

Asthma also isn't the only condition that is hard to treat in morbidly obese people. For many conditions it will be too late to start paying attention to one's calorie intake, but asthma is one that is survivable in the meantime. Severe discomfort and possible death tends to be a good motivator.

Researchers generally analyze intervention success by intention to treat.

If a treatment is telling people to "start paying attention to one's calorie intake," and that doesn't have the desired effect, whatever the reason, I think it's fair to say that intervention isn't useful.

Success rate also isn't the same as difficulty.

It's very easy to fill out a lottery ticket, but success rate is very low.

Success is a boolean condition that often first needs to be defined, while difficulty is a spectrum and a more rigid concept. Really the only complication is subjective vs. objective difficulty (what is objectively difficult may be subjectively easy to someone practiced).

Difficult things require hard skills. Reducing someone's calorie doesn't require any hard skill that I am aware of.

I believe the FDA relies on the "maximum expected utility principle" - a cornerstone of free-market economic theory.

> "By combining the concept of utility with the notion of rational decision making, economists in the mid-twentieth century established a basis for the maximum expected utility principle. This principle is a key concept behind the creation of autonomous decision-making agents."

https://algorithmsbook.com/files/dm.pdf

This has been expressed in the past as "Each portion of wealth has a corresponding portion of happiness, and of two individuals with unequal fortunes, he who has the most wealth has the most happiness."

A good way to accumulate wealth and maximize happiness is to sell drugs, and preventing the sale of drugs because of concerns over ill effects reduces wealth and brings sadness to the pharmaceutical corporation and its shareholders and board members; such sadness prevents them from hiring ex-FDA employees as consultants or managers, thus defeating the principle of maximum expected utility.

The autonomous decision-making agents at the FDA therefore have no choice but to rubber stamp everything that comes across their desk. Doing anything else would be irrational.

> The autonomous decision-making agents at the FDA therefore have no choice but to rubber stamp everything that comes across their desk. Doing anything else would be irrational.

You have clearly never tried to get a drug or device approved nor have you looked at the number of drugs that fail, expensively, in Phase 2 or even Phase 3. Your statement is utter nonsense.

> I believe the FDA relies on the "maximum expected utility principle" - a cornerstone of free-market economic theory.

This has nothing to do with 'free-market economic theory'. It's about decision under uncertainty. The concept was expressed for the first time by Von Neumann and Morgenstern in a book that was supposed to explain how to play poker.

> "Each portion of wealth has a corresponding portion of happiness, and of two individuals with unequal fortunes, he who has the most wealth has the most happiness."

I have never ever read that anywhere. One could argue that rich people provide more value to society and then should be prioritise in some circumstances but what you are writing seems very unfounded.

You can get that quote from the linked text. Notice I'm not actually attacking 'free-market economic theory' per se - but we could adjust the behavior of FDA regulators and pharmaceutical corporate boards by (1) banning FDA regulators from ever taking jobs or gifts from the entities they're supposed to be regulating and (2) enforcing criminal penalties for fraud and deception in the pharmaceutical sector.

There's nothing like a 5-10 year term in an American prison to reduce happiness...

> You can get that quote from the linked text

True. Utility is increasing with revenue. I misunderstood this part of your comment. I read it as : 'if we want to maximum social utility let's prioritise rich people'.

> I have never ever read that anywhere. One could argue that rich people provide more value to society and then should be prioritise in some circumstances but what you are writing seems very unfounded.

Nor have I. It's such a remarkably stupid statement that it strikes me as stupid in itself to think anyone should be so stupid as to believe it.

It's also nothing whatsoever to do with expected utility theory (I don't know where they did get it from). 'Utility monsters' - per Rawls - are a valid objection; 'money monsters' are not, for the very reason that marginal economic gain is not equivalent to marginal gain in happiness/utility, nor would anyone think it is.

> Why no smoking?

Smoking can be an appetite suppressant: https://www.npr.org/2011/06/09/137085989/the-skinny-on-smoki...

> Why no diabetes, given that this is an overweight subject group?

Given the goal of intermittent fasting is to get insulin levels low enough that your body burns fat instead of sugar, and based on this article at least IF requires closer monitoring of your diet, maybe they wanted to eliminate a variable/variables.

https://www.verywellhealth.com/diabetes-and-intermittent-fas...

maybe they wanted to eliminate a variable/variables.

There's an even simpler explanation: Fasting can provoke dangerous hypoglycaemia in diabetics. The first priority for trials is the safety of the participants.

>The FDA approved Tezpire as a breakthrough drug for asthma in December 2021. Tezspire excluded morbidly obese people from their efficacy trials.

This statement is very misleading because morbidly obese people were only excluded from the Phase 2 PATHWAY trial[1] and were not excluded from the larger Phase 3 NAVIGATOR trial[2]. The FDA approved Tezpire based on both trials demonstrating efficacy, not just the PATHWAY trial.

[1] https://clinicaltrials.gov/ct2/show/NCT02054130

[2] https://clinicaltrials.gov/ct2/show/NCT03347279

I think the author makes a fair point, but the analogies and examples in this piece offended my aesthetic sensibilities to the point that I find myself wanting to disagree with him purely out of spite.

> Some of these exclusion criteria are pretty straightforward. […] you’d want to avoid any participants with conditions that make them predisposed to gain or lose weight, as that would make an apples-to-apples comparison across groups more difficult.

Absolutely!

> However, there are other exclusion criteria that I don’t understand at all. Why no oral contraceptive use? Why no smoking? Why no diabetes[…]?

Smoking makes you lose weight. Diabetes is literally characterized by abnormal metabolism.

> There might well be good reasons for these exclusions that I’m not aware of. […] However, those reasons are not outlined here.

Agreed. There should at least be a sentence in a supplement somewhere.

One things about trials and extrapolation is that trails are tightly controlled and will never reflect real world conditions in terms of the patient population, the intervention, or the context in which the intervention is performed.

This is a double edged sword. It’s good because you can get “cleaner” measurements and data by getting rid of cofounders and such. But it’s bad because your trial scenario may be unrealistic and your results may not translate into the clinic.

If anything good has come out of Surveillance capitalism and those shitty EPIC / Cerner EHR systems that everybody (except for hospital quality people) hates, it is a drastic improvement in capabilities for post market data collection (i.e. real world data).

RWD is increasingly a thing at FDA. I think the big issue there which needs public discussion is the extent to which post market data collection should either replace or augment clinical trials. If you replace clinical trials with RWD then you are essentially running mass experiments on sick people with untested drugs, which is monstrous. If you purely augment, then it is basically like a tax and you are driving up the cost and complexity of drug development.

>There might well be good reasons for these exclusions that I’m not aware of.

Is it possible that these reasons are more obvious or known to some folks and they just didn't list them, or they just wanted to see what happens without those groups, and it's not about shenanigans?

I do think there's a reasonably compelling case that drug companies could make for wanting to measure/demonstrate their drug's performance in an ideal population first - now the fact that they then go on to market the product with little to no care for the populations excluded from the trials undercuts this significantly. But it is at least possible that its not entirely shenanigans...
Based on the record of the pharmaceutical industry and their entirely captured regulatory agency, the FDA, it's not entirely unwise to wait about ten years after the introduction of a 'breakthrough drug' to see if it actually has negative side effects that were not discovered in the clinical trials. See Vioxx, etc.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC534432/

> "Dr Graham, associate director in the FDA's Office of Drug Safety, said an estimated 88,000 to 139,000 Americans had heart attacks and strokes as a result of taking rofecoxib. The number, he said, far exceeds earlier disasters such as the 100 children killed in the United States by an elixir of sulfanilamide in the 1930s and the 5,000 to 10,000 children born in the 1960s with birth defects related to thalidomide. Both events led to sweeping regulatory changes in the United States."

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC534432/

This is an unfortunate situation as relatively safe and effective medicines (i.e. Sars-CoV2 vaccines) end up mixed in with ineffective and even dangerous ones, and the public has no real way of distinguishing between them. As the whole opiate epidemic (driven by pharmaceutical corporations pushing their FDA-approved products via shady doctors and pill clinics) demonstrates, these outfits only care about profit margins, and since there are no criminal penalties and any fines are sure to be much less than their profits, they have no incentive to change their behavior.

I just want to add some anecdotal evidence and what my experience has been like with a popular antibiotic. Cipro has wrecked my connective tissue and nervous system. This family of drugs has been around for a long time. The side effects have been discussed in medical studies for years. The drug has a black box warning label. Yet it gets prescribed very often against the current FDA recommendation to only be used for life threating infections. I have talked to top physicians in the US and they literally just ignore the problem like it doesn't exist. I have connected with physicians, chemists and other people with PhDs from all over the world who suffered the same. A pharmacists I know was called crazy by her colleagues when she said what happened to her.

My primary physician has stopped prescribing the medication, I send pages of research to him, which he probably didn't have time to read but it validated what is going on with me. He was surprised I actually took the covid vaccine, as not only did cipro screw me up but the whole medical community also completely ignored me and kept asking about anxiety. When I clearly had mechanical issues with tendons. They ran a bunch of tests and told me I was healthy.

Recently, Dr. Stefan Piper released a book on FQAD (flouroquinolone associated disability) and discussed pathogenesis, possible causes and therapies he has been using on hundreds of patients by now. He is an actual MD. From his experience a subset of people who are diagnosed with fibro, cfs developed it after taking the an fq antibiotic. Currently, there is no way to test for it. However, Mayo clinic is doing a study right now, to see if flouroquinolones are causing damage to mitochondria.

> relatively safe and effective medicines (i.e. Sars-CoV2 vaccines)

By your own logic, we don't know this, because these vaccines haven't been in use long enough.

Exactly this. In this case it does seem like emergency speed was justified. However, every year over 1,500 drugs are pulled off the shelves because, like vioxx, they were discovered later to have terrible side effects. Maybe some of these would have been found with different study demographics. Maybe not. Drugs are dangerous.
I personally went and got vaccinated the second month the Pfizer vaccine was released (but not the first) and even so I viewed myself more as a guinea pig in a clinical trial than anything else. As I had no immediate negative symptoms, I then got the second dose on schedule. Had some minor muscle stiffness in the arm after that one that persisted for a few months. I then got the booster when it came out, as at that point the data seemed pretty clear: side effects were minimal, and the risks of hospitalization and long Covid justified vaccination.

Notably however I certainly didn't expect that the vaccine wouldn't be 'sterilizing' and at the time the FDA was keeping quiet about the fact that vaccinated people could be asymptomatic carriers and spreaders of Sars-CoV2, which now seems widely accepted. The clinical data from the original trials has yet to be released as I understood it, and it likely showed that as well.

You may not have 10 years to wait if you're dying of some disease.
You dying to a disease is nobody's fault (usually). You dying to a bad drug is arguably someone's fault, and I suppose there are probably not any number of documents you can sign that will completely abdicate the responsibility for your death from the corporation that created the chemical compound that resulted in your demise.

And probably for good reason? I really don't want to live in a society that gives people the ability to sign their life away so corporations can run scientific experiments on them that have a real chance of death.

I think there are some exceptions to this, but generally it makes sense to not let people go from "dying of a disease" to, "dying of a drug".

The proper, moral, and correct thing to do is to allow each individual to decide for themselves after being presented with the facts on the risks, probabilities, and benefits.

> I really don't want to live in a society

In a free country, you don't really have the moral authority to make such decisions for other people. After all, you aren't wearing their shoes.

There is no moral way to let people decide for themselves if they want to be experimented on by pharmaceutical corporations.

In a free country, you absolutely have the moral authority (and imperative) to protect those who cannot protect themselves from being taken advantage of.

Hilarious how you say companies have entirely captured the regulatory role of the FDA.

Then you say look at Vioxx, which went through a full phase 3 study.

And then say “but don’t worry, Covid vaccines were know are safe”.

You’re contradicting yourself.

Other questionable exclusion criteria: removing patients from trials of a prophylactic drug if they are seen by medical staff outside the trial for any reason, especially for the condition being studied.
How is that questionable? Isolating to one relevant treatment per group is a standard procedure.
It gives study staff the ability to exclude arbitrary sick patients from the trial. Combine this with a little harmless unblinding and the whole study outcome can be fabricated with plausible deniability.

Notice that I'm referring exclusively to studies of prophylactics. The patient getting sick is the data point.

But trial arms need to be balanced, so excluding sick patients would stick out like a sore thumb.

Of course you don’t allow patients to seek care elsewhere during a trial. If they went to their doctor and were prescribed some alternative therapy it would ruin all the data from that patient.

How can their being prescribed treatment retroactively ruin the data point that they got sick in the first place?

And moreover, unless trials are just standing by and watching patients die, the fact that a patient received X intervention and lived (or X and died) is the only kind of data trials of prophylactics can collect anyways. It doesn't really matter whether the treatment is administered by the trial since the treatment is not being studied.

Because interventions beyond those being tested are closely monitored. Many are excluded, diagnoses need to be confirmed by specific tests, certain treatments may be required to stay in the trial.

Trials usually run for a set period of time. If a patient gets outside treatment half way through, all data beyond that point (including the final analysis) is tainted. So likely the patient would be disqualified from the study and all their data discarded (and they would just show up as "n=1 patient withdrawn from study due to outside treatment for X").

A good example is an eczema trial. Patient gets drug and is monitored for 1 year. If that patient goes to an outside doctor because the rash got worse and gets a steroid cream, and get better, now you don't know - did the experimental drug help? or the steroid? or both? Was the rash actually worse? Or did the patient just think so? The treating doctor never saw the rash prior, so has no baseline. And they likely didn't score the severity of the rash so another data point could be captured.

It's all very standard a part of controlling variables in a trial.

> While they have utility as tools to make running trials on interventions easier or more straightforward, they’re too often used to run different trials altogether than what’s promised in the abstract.

Yes, this makes it a lot easier to find something to publish. If they stop doing this they hurt their career prospects, it might be good for science but it isn't good for the individual.

In order to improve science you need to see it from a capitalist perspective where the researchers are desperately fighting over resources. As long as that fight is won by performing bad science we will mostly get more bad science. The most critical part to change is peer review, today we treat peer review as the main cornerstone of the scientific method, even though it is very unrelated, it is only to help filter out the worst of spam, it doesn't say much at all about the validity of the paper.