[PaulHoule]

Papers on arXiv where people try prompt engineering systematically find that chatbots, by short-circuiting, gets the answer right from 30-90% of the time depending on if the problem is terribly difficult or relatively easy.

You’re very likely to get good results with the first example that comes to your head because it will be something conventional for which the short-circuiting gets the right answer. Once you sample the problem space fairly you find the glass is maybe half-full or half-empty but it’s clear to me that one of the competences the chatbot has is getting people to give it more credit than it deserves.