[raw_anon_1111]

One of my specialties is AWS Connect based call centers

https://news.ycombinator.com/item?id=47241412

I use LLMs to determine what a caller’s “intent” is. I do my best with my initial prompt and then I have the “business” test it and I log phrases that they use.

I then make those phrases my scripted test suite. Any changes in prompts or models get put through the same test suite. In my case, I give my customers a website they can use to test new prompts and takes care of versioning.

I also log phrases that didn’t trigger an intent and modify the prompt and put it back through the suite.

[yelmahallawy]

Do you play with the temperature/top k parameters at all?

[raw_anon_1111]

No. I also use the least sophisticated but fastest model that Amazon hosts - and it hosts all of them except OpenAI models - Nova Lite

Going from free text to tool call with parameters in the grand scheme of things is one of the easiest things to do especially when you only have a limited number of tools.

[yelmahallawy]

Makes sense, simpler=better. Thanks!