[emilsedgh]

Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the massive token count it was so slow, the experience was _never_ going to be useful.

Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema.

One problem that is not resolved yet is that it still hallucinates a lot of attributes. For example we have tool that allows it to create contacts in user's CRM. I ask it to:

"Create contacts for top 3 Barcelona players:.

It creates an structure like this"

1. Lionel Messi - Email: [e-mail] - Phone Number: [phone] - Tags: Player, Barcelona

2. Gerard Pique - Email: [e-mail] - Phone Number: [phone] - Tags: Player, Barcelona

3. Marc-Andre ter Stegen - Email: [e-mail] - Phone Number: [phone] - Tags: Player, Barcelona

And you can see it hallucinated email addresses and phone numbers.

[pluijzer]

ChatGPT can be usefully for many things, but you should really, not use it if you want to retrieve factual data. This might partly be resolved by querying the internet like bing does but purely on the language model side these hallucinations are just an unavoidable part of it.

[Spivak]

Yep, it's always always write code / query / function / whatever you need that you would parse and retrieve the data from an external system.

[037]

I would never rely on an LLM as a source of such information, just as I wouldn't trust the general knowledge of a human being used as a database. Does your workflow include a step for information search? With the new json features, it should be easy to instruct it to perform a search or directly feed it the right pages to parse.