[humbleferret]

Great job!

I played around with this on a couple of small knowledge bases using an open Hermes model I had downloaded. The “related notes” feature didn't provide much value in my experience, often the link was so weak it was nonsensical. The Q&A mode was surprisingly helpful for querying notes and providing overviews, but asking anything specific typically just resulted in less than helpful or false answers. I'm sure this could be improved with a better model etc.

As a concept, I strongly support the development of private, locally-run knowledge management tools. Ideally, these solutions should prioritise user data privacy and interoperability, allowing users to easily export and migrate their notes if a new service better fits their needs. Or better yet, be completely local, but have functionality for 'plugins' so a user can import their own models or combine plugins. A bit like how Obsidian[1] allows for user created plugins to enable similar functionality to Reor, such as the Obsidan-LLM[2] plugin.

[1] https://obsidian.md/ [2] https://github.com/zatevakhin/obsidian-local-llm

[dax_]

Yeah, this is exciting - I'd much rather have it as a plugin for Obsidian though! I have my workflow with that, all the features I need. Having some separate AI notes app isn't what I would like to use.

[samlhuillier]

Thank you for your feedback!

Working hard on improving the chunking to improve related notes section. RAG is fairly naive right now, with lots of improvements coming in the next few weeks.

[monkmartinez]

I left an issue to explain this in more detail, but I don't think the problem is chunking. The issue is the prompt. The local LLM space does itself no favors by thinking about and using prompts as an after thought.

IME, the prompt should be front/center in terms of importance and the key to unlocking the models potential. It's one of the main reasons why Textgen-Webui is sooooo good. You can really dial-in the prompt, from the template itself to working with the system message. Then begin futzing with the myriad of other parameters to achieve fantastic results.

[ilaksh]

Which model exactly did you use and how large? I feel like even the best 7b models are just a bit too dumb for most things that I have tried. A 70b model or Mixtral or sometimes 34b seem to be adequate for some things. But those are several times larger and don't run on my oldish hardware.

[humbleferret]

Openhermes 2.5 Mistral 7b