[justanotherunit]

Interesting, would you mind sharing your architectural setup? How does your index communicate to your agent server, what is the main agent framework/engine used?

Sounds like a cool concept to speak into your watch/wearable which automatically saves or performs tasks on the fly.

What is the general execution time from:

Prompt received -> final task executed?

[stavros]

So basically there's a /chat endpoint that goes to the LLM (a Pi agent), which has access to call specific tools (web search, SQL execution, cron) but doesn't have filesystem access, so the only thing it can do is exfiltrate data it can see (pretty big, but you can't really avoid that, and it doesn't have access to anything on the host system). There's a Signal bridge that runs on another container to connect to Signal, a Telegram webhook, and the other big component is a coding agent and a tool container. The coding agent can write files to a directory that's also mounted in the tool container, and the tool container can run the tools. That way you separate the coder from everything else, and nothing has access to any of your keys.

You can't really avoid the coder exfiltrating your tool secrets, but at least it's separated. I also want to add a secondary container of "trusted" tool that the main LLM can call but no other LLM can change.

This way you're assured that, for example, the agent can't contact anyone that you don't want it contact, or it can read your emails but not send/delete, things like that. It makes it very easy to enforce ACLs for things you don't want LLM-coded, but also enables LLM coding of less-trusted programs.

[stavros]

And now it can even make private (and public!) dynamic websites that have access to data from your database, while exposing only the data you want exposed.

I'm really liking it, I created a page to show my favorite restaurants per city, for example:

https://stavrobot.home.stavros.io/pages/restaurants

That's dynamic, loading from the database, and updating live when the assistant creates new entries.

This page was created just by telling the assistant "make me a page to show my favorite restaurants, with their ratings, groupped by city".

[justanotherunit]

Fascinating, thanks for responding. If k may ask, what is your monthly (or any other interval) token usage? And are you finding a pi to be a bottleneck in regards of any performance?

[stavros]

You mean with the bot, or with developing the bot? The bot's token usage is fairly small, but it's only a few days old so I don't know. Pi hasn't been a bottleneck that I can see, my bot is much faster than OpenClaw was when I tried it.

[justanotherunit]

Was thinking about the bot. I stared your repo and will be checking it out later!

[stavros]

Thanks, let me know if you have feedback! I've made it fairly easy to set up, I think, you don't even need a separate server since it's all sandboxed, you can try it on your PC.