| I've done this a bit I primarily use ChatGPT through the API now. And I regularly train it to use structured commands, which I can then parse and act on For example, prompting it that it is a personal assistant that can save long term data by replying with "!remember <some_key> <some_value>" and can also request a list of all values with "!recall" if it suspects a "!remember" command would aid it in answering a request, then "!recall <some_key>" if it would like to get a value. When you ask it to remember your groceries, it replies with "!remember grocery_list milk, eggs" as a response. When you mention you're headed to the grocery store it replies with "!recall", then "!recall grocery_list" then returns - I've also done a few really open commands. I gave one prompt a "!request <method> <host> <body>" command that would be parsed and turned into an HTTP request automatically. I asked it for the weather in Bali and it took a few tries, but it eventually got to https://wttr.in/ (a site I was completely unaware of) and crafted a query: https://wttr.in/Bali?format=%25C+%25t+%25h%7C%25C%7C%25t%7C%... Now that might sound like a worse version of Siri, but unlike Siri you don't actually have to ask for the weather. One example I tried was: My mom's friend lives in Bali => I'm actually visiting her today, but I'm having some trouble figuring out what kind of clothes to wear => requests weather in Bali and suggests a light jacket. The biggest issue was API keys, I struggled to prompt it out getting caught in loops when an API Key was required (it'd keep trying different variations of the same host instead of moving on) It was a little eerie in cases where I didn't expect it to make an HTTP request, but instead suddenly it'd try 20 different ones to craft an answer. Stuff like stating the "friend's mom lives in Bali" part, and having it try to come up with a query to "learn" more about Bali to give an optimal response. That was likely because I told it to always trust the result of the request command over its own intuition I also gave it a Wolfram Alpha "!info <query for Wolfram Alpha>" command that worked pretty nicely, and a "!time" command to model a home assistant |