[gabev]

This is sick! We've built a similar product for LLM-powered browsing at https://zenfetch.com

Admittedly, we use cloud hosted providers since we found the quality to be much higher, though awesome to see local versions as well.

[andrewnguonly]

Very cool! Have you considered local LLMs for a free/lite tier?

I do agree that 3rd party LLM services (e.g. OpenAI) are much more powerful. For basic tasks (e.g. summarization, Q&A), local LLMs seem to perform decently.

[gabev]

We have considered it, though most of our users use Zenfetch for things like

1. Research across an industry 2. Generating new content 3. Connecting ideas / building networked thought

For that reason we were deterred from using local models. Hopefully the Mistral team and similar keep pushing out new developments to get comparable with GPT-4 levels