[SparkyMcUnicorn]

I've found that giving agents an "opt out" works pretty well.

For structured outputs, making fields optional isn't usually enough. Providing an additional field for it to dump some output, along with a description for how/when it should be used, covers several issues around this problem.

I'm not claiming this would solve the specific issues discussed in the post. Just a potentially helpful tip for others out there.

[bjorgen]

Do you have an example of this in practice? I'm having a hard understanding this and have a very similar problem of the agent wanting to give a response on optional fields.

[SparkyMcUnicorn]

A (hopefully) clear and probably oversimplified example:

Query -> Person Lookup -> Result-> Structured Output `{ firstName: "", lastName: "" }`

When result doesn't have relevant information, structured output will basically always output a name, whether it found the correct person or not, because it wants to output something, even if the fields are optional. With this example, prompting can help turn the names into "Unknown", but the prompt usually ends up being excessive and/or time consuming to get correct and fix edge cases for. Weaker models might struggle more on details or relevance with this prompt-only approach.

`{ found: boolean, missingFields: [], missingReason: "", firstName, lastName }`

Including one or more of these additional text output properties has an almost magical affect sometimes, reducing the required prompting and hallucinated/incorrect outputs.