[Aurornis]

From what I’ve observed, people are very good at getting LLMs to tell them what they want to hear.

Someone I know didn’t believe their doctor, so they spent hours with ChatGPT every day until they came up with an alternate explanation and treatment with an excessive number of supplements. The combination of numerous supplements ultimately damaged their body and it became a very dire situation. Yet they could always return to ChatGPT and prompt it enough different ways to get the answer they wanted to see.

I think LLMs are best used as typing accelerators by people who know what the correct output looks like.

When people start deferring to LLMs as sources of truth the results are not good.