I'm curious who the ideal customer of this should be. If we're a startup with our own harness, are we a good fit? What would qualify us or disqualify us from being a good user?
I think startups are a great fit. Getting a really good agent out of the box lets you scale and give your customers value fast. All you need to think about is the business logic: system prompts, tools to give the agent, skills, etc. You won't need to spend time on building the infra layer, orchestration loops, memory, implementing automations, etc.
Yeah I know some of my team members have invested a lot of time in this. Could definitely be worth chatting with them on what improvements could be made here. We're starting to deprioritize our consumer facing agent harness, in favor of more infrastructure level improvements we are making.
Developers with customer-facing chat products are the ideal customer.
If a startup has a specific flow they want the agent to take and their traffic is bursty, then I'd recommend using a framework like Mastra and deploying onto a sandbox.
For long-running always on agents where it's important to learn the users preferences overtime, our approach is the highest ROI.
Interesting. We definitely have long-running agents where certain preferences are key. However, some of the preferences are likely going to be shared universally across our customers. Is there some way of triaging this feedback into permanent improvements in agent performance?
It is possible to do it (ex: run a background process that analyzes memories across customers and updates the system prompt based on the findings). A specific implementation would depend on your application. Feel free to email me rajit@prismvideos.com if you'd like to talk further.