[simonw]

Is the "act like a" pattern still necessary?

I've stopped using that with GPT-4, since in my experience the default "persona" (for want of a better word) answers most of my prompts well enough already - saying "act like an expert in..." doesn't seem to get me notably better results.

The tooling I most want is some kind of lightweight but effective way of trying out and comparing multiple prompts with small tweaks to them to get a feel for if one is an improvement over the other. Anyone seen anything good like that?

[jairuhme]

Are you looking to test different prompts on a set of questions? I have not found anything that specifically does this. When I have tested different prompts, it has just been with a short script iterating through the questions/prompts. Could be a fun project to build something where all you do is add the prompts you'd want to test, but the challenge that first comes to my mind is creating the question set to be used.

As I write this, when you say you want to test prompts, are you looking to test the system prompt on a set of questions or is the prompt the question and just asked in different ways?

[simonw]

I want a simple, reliable way to confirm if putting "OUTPUT IN JSON PLEASE!!" in capitals with two exclamation marks is more effective than just "output in JSON".

[iambateman]

1. I don't think that pattern is all that useful anymore.

...and...

2. I'll make that tool and DM you on Twitter when it's ready.