[Oras]

The problem is that, regardless of how you try to use "micro-agents " as a marketing term, LLMs are instructed to return a result.

They will always try to come up with something.

The example provided was a poor one. The comment from LLM was solid. Why would you comment out a step in the pipeline instead of just deleting it? I would comment the same in a PR.

[SparkyMcUnicorn]

I've found that giving agents an "opt out" works pretty well.

For structured outputs, making fields optional isn't usually enough. Providing an additional field for it to dump some output, along with a description for how/when it should be used, covers several issues around this problem.

I'm not claiming this would solve the specific issues discussed in the post. Just a potentially helpful tip for others out there.

[bjorgen]

Do you have an example of this in practice? I'm having a hard understanding this and have a very similar problem of the agent wanting to give a response on optional fields.

[SparkyMcUnicorn]

A (hopefully) clear and probably oversimplified example:

Query -> Person Lookup -> Result-> Structured Output `{ firstName: "", lastName: "" }`

When result doesn't have relevant information, structured output will basically always output a name, whether it found the correct person or not, because it wants to output something, even if the fields are optional. With this example, prompting can help turn the names into "Unknown", but the prompt usually ends up being excessive and/or time consuming to get correct and fix edge cases for. Weaker models might struggle more on details or relevance with this prompt-only approach.

`{ found: boolean, missingFields: [], missingReason: "", firstName, lastName }`

Including one or more of these additional text output properties has an almost magical affect sometimes, reducing the required prompting and hallucinated/incorrect outputs.

[ffsm8]

Likely because it's temporary?

It takes less effort to re-enable if it's just commented out and its more visible that there is something funky going on that someone should fix.

But yeah, even if it's temporary, it really should have the rationale for commenting it out added... It takes like 5s and provides important context for reviewers and people looking through the file history in the future.

[pancsta]

By splitting prompts into smaller chunks you effectively get “bias free” opinions, especially when cross-checked. You can then turn them into local reasoning, which is different from “sending an email to the LLM” which seems to be the case here. Remember, LLM is Rainman.