[drw85]

But those are obvious errors. What if the AI tells you to up your intake of X and it seems plausible? So you up your intake of X by taking some supplement, but upping the intake of X makes your body deplete more of Y and now you have a new or compounding problem.

A doctor might have never recommended upping X, because they would know what it does to your body. Or they might have suggested additional supplementation to avoid this.

The fact that LLMs are trained on all public knowledge is a huge red flag, because there are more wrong infos out there than right ones. Especially about health, diet, etc.

[simianwords]

I was about to respond but your last statement implies fundamental misunderstanding of how LLMs work. I don’t think you even know about RLHF and you think good and bad ideas are spread in proportion to how much they are seen on the internet.