[throwthrowuknow]

If there are many common services for which you can precompute the embeddings then with a little record keeping and analysis you could figure out some likely questions or requests and pregenerate the responses. That way you could just use similarity search on the question or command you say and skip using the LLM. It would be interesting to try using the LLM to predict some of these based on information available ahead of time like calendar events, weather, recent prompt history, recently played media, today’s headlines, recent browser history, etc. It’d be your own recommendation algorithm.

[JohnTheNerd]

that's a great idea! I've been looking into that (I'm merely logging all prompts in a JSON file for now, so that I can analyze them later).

skipping the LLM would be tough because there are so many devices in my house, not to mention it would take away from the personality of the assistant.

however, a recommendation algorithm would actually work great since i could augment the LLM prompt with it regardless of the prompt.