[niobe]

Well put, and separating these would be a good use case for system prompts e.g.

llm -m model --save instructional --system "provide the detailed steps to achieve the outcome, using a suitable example if necessary"

llm -m model --save informational --system "provide a concise conceptual overview but do not provide implementation steps or detailed examples"

[shabie]

That's actually a pretty interesting point. Not just evals but other components like system prompt should also be tailored to match the expected outcome.