[raydev]

We need it to actually be correct 100% of the time, though. The current state where a chat interface is unable to say "I don't know" when it actually doesn't know is a huge unsolved problem. Worse, it will perform all the steps of showing its work or writing a proof, and it's nonsense.

This revolution is the wrong one if we can't guarantee correctness, or the guarantee that AI will direct the user to where help is available.

[bsenftner]

I've been having luck with framing the AI's role to be a "persistent fact checker who reviews work more than once before presenting." Simply adding that to prompts improves the results, as well as "provide step by step instructions a child can follow". Using both of these modifying phrases materially improves the results.