[haldujai]

It’s actually quite a lot worse than even doctors in training except for highly constrained experimental settings and a few very nice applications that are mostly too tedious/impractical for a human to do or are very basic detection tasks.

I am a radiologist and researcher predominately focused on AI.

[yread]

I work with pathologists and radiology is way ahead of us with AI use in clinical setting (but still not very far). Only things that get serious use are lab-developed (ie not commercial) image analysis algorithms for very limited (tedious, error-prone and ultimately not that often used) biomarkers. Don't believe the hype.

You could also look at the market, one of the biggest players, Paige, was acquired for about 30% of the money they raised.

[marcus_holmes]

Thanks for the informed take :)

Do you think this will result in more routine/boring/tedious tests? Is the bottleneck on these things the human time to analyse them?

[haldujai]

I don’t think so, not beyond the current trend in medicine which is going up anyway.

For some things, like 3D volume segmentation of structure or disease (e.g. CVA/stroke volume, cardiac muscle mass, iron quantification) the bottleneck is the time it takes so we currently use approximations like single longest dimension, circular regions of interest, etc. AI will dramatically increase accuracy allowing for more accurate treatment and easier large scale research with quantitative endpoints.

Other things people think of like detection of aneurysms, fracture, lung nodules are not “hard” but AI has already added and will continue to add the second-reader benefit which will reduce detection errors. For this category the clinical benefit is as of yet unclear and we know that increased detection does not necessarily translate into improved patient outcomes and can in fact make them worse from over-diagnosis which means investigation related harms and over-treatment.

We were already in a phase of “over detection” in much of radiology with advances in imaging technology so the incremental benefit of current AI remains to be seen and I personally think is going to be much more limited. I had a case recently where a 2 mm brain aneurysm was missed on 3 CT scans over 10 years but was picked up by AI so now is being followed annually. This is too small to treat considering the risks and a serious argument could be made that 10 years of stability is proof enough that this is almost certainly clinically irrelevant for this patient.

Far more interesting areas of AI in imaging are in acquisition of acceleration (i.e. the medical equivalent of upscaling) which can dramatically decrease costs and increase accessibility as well as analyzing imperceptible features.

It may not be a popular take here but in my opinion the future of radiology is like what we see in software engineering today - a skilled human equipped with AI will outperform humans without AI and AI without humans, the latter of which we are still several years away from prototyping due to various technical hurdles.

[marcus_holmes]

Thanks :) Very interesting.

> in my opinion the future of radiology is like what we see in software engineering today - a skilled human equipped with AI will outperform humans without AI and AI without humans

I suspect this will be the case across the board. It's a useful tool, but it's just a tool. It's not a replacement.

[nesk_]

A friend of mine, a dermatologist, told me that LLMs are quite performant for melanoma analysis. Based on their own statistics, LLMs are able to beat humans with ~10 years of experience in the field.

They will never beat the human instinct tho, but they can be great tools sometimes. Unfortunately, LLMs mostly produce garbage.

[haldujai]

Whenever it comes to medical diagnosis I would caution anyone to be careful with what “beat humans” really means.

In real life pathology is a spectrum not a binary and physicians are not trained to be 100% accurate instead optimizing sensitivity and specificity considering pretest probability as well as the harms of overdiagnosis and under diagnosis for a given scenario.

For something like melanoma which is relatively easy to diagnose with a superficial, extremely low risk skin biopsy and where early staging dramatically improves outcomes you would want to design around overcalling (high sensitivity) rather than maximize accuracy given the significant harms with false negatives and minimal harms with false positives.

An AI may be more accurate at classifying melanoma/not melanoma but if it does not meaningfully improve on the clinical threshold of biopsy/no biopsy or result in less biopsies that accuracy is wasted and may even be detrimental.

Note: I am just using this as an example to illustrate the considerations.

[not_that_d]

I don't think your friend understands Large LANGUAGE models.

[stingraycharles]

You do realize that today’s multi-modal LLMs are actually able to understand images? They’re tokenized very much like language is.

[the_sleaze_]

First I'm hearing that is a lot worse - do you have sources? Genuinely curious

[haldujai]

The better question is are there any sources that AI is better than human readers? I haven’t heard anyone make this claim outside of single/few disease classification tasks and even those are mostly 2D.

Anecdotally, my practice has most FDA approved AI deployed as we are an evaluation site and very rarely is the AI result useful. Over the past few months we have been cancelling contracts as these cost quite a lot of money (in some cases eating >50% of the study interpretation cost) for little to no benefit and a LOT of noise.