[pjc50]

Yes. It's also worth thinking about the sharp cliff effect. Things either fall into the category of "medical device" (expensive, heavily regulated, scarce, uninnovative), or they don't, in which case it's a free for all of unregulated supplements and unsupported claims.

The home brew "automatic pancreas" by making a bluetooth control loop between a glucose monitor and an insulin pump counts as a "medical device". Somehow a computer system that encourages people to take bromide isn't. There ought to be a middle ground.

[bigbadfeline]

> Somehow a computer system that encourages people to take bromide isn't. There ought to be a middle ground.

Yes, there is a very effective middle ground that doesn't punish anybody for providing information. It's called a disclaimer:

"The information provided should no be construed as medical advise. Please seek other sources of information and/or consult a physician before taking any supplements recommended by LLMs or web sites. This bot is not responsible for any adverse effects you may think are due to my information"

When an LLM model detects a health related question - print the above disclaimer before the answer.

There is no need for dictatorship in order to save people from information.

[qwertylicious]

It's also called "liability".

"Warning, this washing machine might burn your house down" is not sufficient to escape punishment. Why should digital technology get a pass just because the product that's offered is intangible?

[api]

Learning to innovate steadily and responsibly without just stopping is one of the things I'd put on my list of things humanity needs to figure out.

Individuals can do it, but as I said it doesn't scale. An individual can carefully scale a rock face. A committee, political system, or corporate board in charge of scaling rock faces would either scale as fast as possible and let people fall to their deaths or simply stand at the bottom and have meetings to plan the next meeting to discuss the proper climbing strategy (after discussing the color of the bike shed) forever. Public discourse would polarize into climb-fast-die-young versus an ideology condemning all climbing as hubris and invoking the precautionary principle, and many door stop sized books would be written on these ideas, and again either lots of people would die or nothing would happen.

[bigbadfeline]

From the OP:

> "There may have been multiple factors contributing to the man’s psychosis, and his exact interaction with ChatGPT remains unverified. The medical team does not have access to the chatbot conversation logs and cannot confirm the exact wording or sequence of messages that led to the decision to consume bromide."

Any legal liability for providing information is wrought with opportunities for abuse, so bigly so that it should never be considered.