[rodoxcasta]

Wait, this 'Agents' thing seems to be just a way to couple a system prompt and temperature to a model, that's it?

What's the difference from sending the system prompt in the api call, as usual?

Edit: Oh, missed that: "We’re working on connecting Agents to tools and data sources."

[refulgentis]

There's this massive gap between those who can call API and those who can't. If you can't, then you get the same aspirational-AGI chat UI as everyone else.

I agree with the implied statement that 'Agents' doesn't feel right. Reminds me more of the projects that put the model in a loop.

It does feel to me to be a really tough thing to name & market, I'm about to release an app for this across all providers, I call it "Scripts" with "Steps" like chat, search, retrieval, art...

[DebtDeflation]

I implemented a number of enterprise Conversational AI tools for customer service back before the GenAI craze started and we used to just call it service orchestration and data/application integration. The chatbot was used to figure out what the customer wanted to do and then from there it was just about automating some business workflow. Customer wants to pay their bill, the bot needs to pull their current balance, get their payment information, process the payment. Customer wants to return a product, the bot needs to retrieve the order info, initiate an RMA, process a refund, etc. These were all well established business process that the bot would execute by making API calls or kicking off an RPA routine. The "agent" talk sounds to me like "let the LLM figure out what it needs to do and then do it" which I'm not even sure is the right approach for most enterprise use cases, it's how you get people tricking chatbots into selling them a new car for $1.

[TechDebtDevin]

Why is tool picking such a hard functionality for these vendors to implement.

Seems like a lot of the heavy lifting will come from 3rd parties making their APIs compatible with llms.

There should be some sort of extension type app where people can build extensions or "tools" for llms and share them (I guess openAI sort or attempts to do this). Say I want to build one for Toast to order food. I can collect the info needed to run that tool (toast account info or whatever) and an API key for an appropriate llm and then use this configuration info for Toast to build out a middleware that can use natural langauge to build out an order and send the request to Toast via some function call.

This seems very doable and I don't understand why there aren't a million of these "tools" already built into some LLM centric tool aggregator/ web store. What is the hold up? Is it just 3rd parties not wanting to hand out API access for things that require payment to applications controlled by llms? Would these 3rd parties rather have their own assistant tool they run? I'd imagine that some central llm-extension aggregator could have a central mechanism for payment methods that the llm had access to that could be used to implement safegaurds.

Or is it simply that any assistant type tool that could be easily generalized like ordering food, booking a flight or inputing calender events is simply easier to handle doing yourself than asking an llm to do for you?

[wkat4242]

A lot of models are hit and miss when it comes to invoking tools. I have llama 3 8b with a weather tool but half the time it will just hallucinate giving me made up info instead of running the tool.

I imagine the big sites have similar issues and it undermines customer trust when they're given false information.