[roncesvalles]

I think it is widely known by now that instructions to alter the LLM's "tone", things like asking it to adopt a persona ("you are the world's best programmer"), and overly broad directives ("make no mistakes") always gives poor results. Just state directly what you want. If you want something very specific, add more information. "Prompt engineering" is pseudoscience.

To put it another way, you will only get the benchmarked performance if you let it talk the way it talks by default. Trying to modify this neuters the model's IQ.

[davedx]

It's a bit more nuanced than that. Earlier models definitely benefited a lot more from prompt engineering. I remember this distinctly from building data pipelines to do things like extract data from PDFs over the last year or two - there are numerous "tricks" like negative prompting, including the right number of examples, massaging the mock data in the JSON examples so it wasn't "too realistic", and so on. I saw how this impacted recall by running evals, so it wasn't pseudoscience.

But what has happened is the models have gotten better - which OpenAI is making explicit for some cases in this release. You need that stuff less and less as they become more human and better at inferring what's required implicitly.

You still do need to be explicit, and you probably always will, but you don't need as much "engineering" of the way you're asking for things with more recent models.