[demosthanos]

You use a two-phase prompt for this. Have it reason through the answer and respond with a clearly-labeled 'final answer' section that contains the English description of the answer. Then run its response through again in JSON mode with a prompt to package up what the previous model said into structured form.

The second phase can be with a cheap model if you need it to be.

[faustocarva]

Great, will try this! But, in a chain-based prompt or full conversational flow?

[demosthanos]

You can do this conversationally, but I've had the most success with API requests, since that gives you the most flexibility.

Pseudo-prompt:

Prompt 1: Do the thing, describe it in detail, end with a clear summary of your answer that includes ${THINGS_YOU_NEED_FOR_JSON}.

Prompt 2: A previous agent said ${CONTENT}, structure as JSON according to ${SCHEMA}.

Ideally you use a model in Prompt 2 that supports JSON schemas so you have 100% guarantee that what you get back parses. Otherwise you can implement it yourself by validating it locally and sending the errors back with a prompt to fix them.

[faustocarva]

Thanks!