[lurking_swe]

if the hospital IT system is temporarily down, i certainly expect my doctors to still be able to do their job. So it is a (small) problem that needs addressing.

Perhaps a monthly session to practice their skills would be useful? So they don’t atrophe…

[sothatsit]

If the AI systems allow my doctor to make more accurate diagnoses, then I wouldn't want a diagnosis done without them.

Instead, I would hope that we can engineer around the downtime. Diagnosis is not as time-critical as other medical procedures, so if a system is down temporarily they can probably wait a short amount of time. And if a system is down for longer than that, then patients could be sent to another hospital.

That said, there might be other benefits to keeping doctors' skills up. For example, I think glitches in the diagnosis system could be picked up by doctors if they are double-checking the results. But if they are relying on the AI system exclusively, then unusual cases or glitches could result in bad diagnoses that could otherwise have been avoided.

[randcraw]

Or, each time the software is updated or replaced, the human in the loop will be unable to... be in the loop. Patients will be entirely dependent on software vendor corporations who will claim that their app is always correct and trustworthy. But what human (or other external unbiased trustworthy authority) will be available to check its work?

[stavros]

Do you expect doctors to be able to image your insides if the X-ray machine is down too?

[lurking_swe]

An x ray machine can be used “locally” without uploading the images into the IT system. So i don’t understand the question. If it was designed to be cloud only then that would be horrendous design (IMO).

The x ray machine would still work, it’s connected directly to a PC. A doctor can look at the image on the computer without asking some fancy cloud AI.

A power outage on the other hand is a true worst case scenario but that’s different.

[stavros]

I'm not talking about the IT system, I'm talking about when the X-ray machine breaks, same as how we're talking about when the colonoscopy diagnosis machine breaks.

[fzeroracer]

How often do you think the x-ray machine breaks vs how often software shits the bed?

Like one of the biggest complaints I've heard around hospital IT systems is how brittle they are because there are a million different vendors tied to each component. Every new system we add makes it more brittle.

[d3vnull]

Seems like a pretty easy fix in treating the system that runs the cancer detection algorithm as an hospital machine and not as part of the IT system. It can be an airgapped system that runs just the needed software in a a controlled configuration.

This is not new, lots of mission critical software systems are run like this.

[yread]

If the hospital IT system is down, hospital stops. That's why hospitals spend resources so that it doesn't happen

[oliwarner]

A lot of imaging and note-taking is digital now, so AI aside, they probably won't meet your expectations.

[akoboldfrying]

> if the hospital IT system is temporarily down

I think we have to treat the algorithm as a medical tool here, whose maintenance will be prioritised as such. So your premise is similar to "If all the scalpels break...".

[tylerrobinson]

I’m think you might agree, though, that the likelihood of one premise is significantly greater than the other!

[akoboldfrying]

Sure. "MRI machine" would have been a better metaphor than "scalpel".

[azemetre]

Which is easier to build resilient systems for: the one where you have a few dozen extra scalpels in a storage closet or the one that requirements offsite backups, separate generators, constant maintenance?

The premise is absolutely not the same.

[akoboldfrying]

s/scalpel/MRI machine/. How about now?

[azemetre]

Sounds like a great system that benefits from having lots of money. IDK how such a thing can last in rural areas where there may be one single MRI machine to use in a 100 mile radius.

[add-sub-mul-div]

"People will practice their skills" is the new "drivers will keep their attention on the road and remain ready to quickly take the wheel from the autonomous driver in an emergency."

[xorbax]

It's like research. People had encyclopedias. If they wanted to know real, deep information about subjects they'd have to specifically spend effort seeking and finding books or papers about that specific subject (which are typically just distilled papers in a far wider range and number than an encyclopedia would be)

Then we could just go Google it, and/or skim the Wikipedia page. If you wanted more details you could follow references - which just made it easier to do the first point.

Now skills themselves will be subject to the same generalizing phenomenon as finding information.

We have not seen information-finding become better as technology has advanced. More people are able to become barely-capable regarding many topic, and this has caused a lot of fragmentation, and a general lowering of general knowledge with regard to information.

The overall degradation that happened with politics and public information will now be generalized to anything that AI can be applied to.

You race your MG? Hey my exoskeleton has a circuit racer blob we should go this weekend. You like to paint? I got this Bougereau app I'll paint some stuff for you. You're a physicist? The font for chalk writing just released so maybe we can work on the grand unified theory sometime, you say you part and I can query the LLM and correct your mistakes

[salawat]

>Then we could just go Google it, and/or skim the Wikipedia page. If you wanted more details you could follow references - which just made it easier to do the first point.

Except at this point, market forces and going whole hog on neural networking and such instead of sticking with just reflective, impartial indexing of the digital medium has made it nigh impossible for technological aid to actually improve your ability to find niche things. Search Engine Optimization, plus the interests in shaping narratives, have made searchability take a plunge. Right now the unpolluted index may as well be a WMD for how hard it is to find/keep operating one.

[seanmcdirmid]

This already happened in aviation a long time ago, they have to do things to keep the pilots paying attention and not falling asleep on a long haul where the auto pilot is doing most of the work. It isn't clear at what point it will just be safer to not have pilots if automated systems are able to tackle exceptions as well as take offs and landings well enough.

[accoil]

Does cancer progress that fast?