[dahart]

> No reasonable person would get one absent demonstrated benefit.

But if we insist on measuring demonstrated benefit by factoring in participation rates, then it’s a catch-22. What if given a prior distribution of 100% opt-in, colonoscopies are effective? And you ignored my question about what if we made colonoscopies more convenient, less intrusive and time consuming, which is becoming possible with new tech. These things can change the participation rate, which in turn can flip the outcome from little demonstrated benefit to high and conclusive demonstrated benefit.

> this study did the fairest possible test for effectiveness of colonoscopies

“Fair” is a subjective term, and it depends on what question you’re asking. I agree with your statement if the question is how effective is the current system of recommending colonoscopies. It’s not the fairest test of how effective a colonoscopy screen could be if everyone shows up for the screen. Colonoscopies might be not better than genetic tests because participation rates for genetic tests are higher, as opposed to colonoscopy screening being less effective on their own.

I understand your point that the total probability is important. But so is understanding the Bayesian factors, it’s equally enlightening and important to separate and understand the effectiveness of the screen given participation, from the likelihood of participation. And you effectively cemented how important this point is by clarifying that people use knowledge of these outcomes in order to choose whether or not to participate, so framing them incorrectly can and likely does lead to unnecessary loss of life.

[timr]

> What if given a prior distribution of 100% opt-in, colonoscopies are effective?

That is what the "per-protocol" analysis in this RCT estimates. They considered only those people who had a colonoscopy. This completely breaks randomization and is subject to investigator bias, but at least it gives you an idea of the best you could possibly do if you lived in a world where everyone was forced to get one.

And that is: a drop of 0.15% in colo-rectal death. They then attempt to extrapolate this to estimate the effect on all-cause mortality if you somehow forced everyone to get a colonoscopy, and come up with a number of 10.88% (vs. a baseline of 11.03%). So, even if you forced everyone to get a test, you'd need to test 667 people to save one life.

That's a lot of pain for very little gain.

[dahart]

> a drop of 0.15% in colo-rectal death.

Where’s that coming from? I didn’t read the paper yet, but the article says: “A secondary analysis also offers another silver lining, Gupta said. When the investigators compared just the 42% of participants in the invited group who actually showed up for a colonoscopy to the control group, they saw about a 30% reduction in colon cancer risk and a 50% reduction in colon cancer death.“

> This completely breaks randomization and is subject to investigator bias

Yes, true. I agree with you. This is important when asking the question “how effective is recommending screenings”. But, voluntary opt-in also breaks randomization, and is subject to patient bias.

> you’d need to test 667 people to save one life. That’s a lot of pain for very little gain

Assuming that’s accurate (the article appears to disagree), I’d still be very hard-pressed to agree with this conclusion. Why is a once or twice a decade doctor’s visit spread out over many people amounting to enough pain to let someone die? How much is a life worth? Is the colonoscopy really so bad that you’re willing to risk a greater than one in a thousand chance of dying? That risk is comparable to many extreme sports.

The problem with this kind of sum up the cost and make it look large is that you didn’t do this for anything else and compare it for reference, which makes this framing prone to cognitive bias. We spend billions and have laws for keeping drivers safe, for example. You could make it look insane by adding up the billions of dollars and people-years people spent bucking and unbuckling seatbelts, but the truth is that it’s a teeny inconvenience per person for a sizeable gain in safety and reduction in the accumulated secondary costs of accidental death.