[cjonas]

The problem is really more getting the agent to reliable relay a UUID. For example, we were creating files for visualizations and having the agent reference them in there response with a custom <visualization file=UUID /> and found that it would often fail to accurately return a UUID from a tool response it was previously provided (running sonnet 4.6).

For this use case, our solution was just to use a slug for the filename, but we can control the uniqueness constraint on our backend.

[mrweasel]

Except that we don't yet know what would need in all cases, this seems like something that should be provided by the environment.

It feels much like the random number generators in your operating system. The OS is responsible for providing applications with a source of entropy. In the same line of thinking maybe IDEs, agent frameworks, whatever you want to call it, should be responsible for providing some base functionality.

[cjonas]

Not sure I understand. If you generate a random string to use as a reference for something that the LLM interacts with... and the LLM cannot reliably recall the reference, then it's a problem that needs to be solved by simplify the random string.

[mrweasel]

This might be my understanding that's wrong, but I assumed that the LLM itself actually can't produce like a UUID, but it can "predict" one, hence why it sometimes hallucinate IDs. So my thinking was, strip that bit out of the AI prompt and output and leave it to the "wrapper" e.g. Claude Code or your IDE to insert the actual ID.

So in the same way that your crypto library don't have its own randomness generator (ideally) and rely on the operating system to provide an API, the agents would rely on their "operating shell" / IDE / application to provide functionality that lies outside the score of an LLM.