[monkmartinez]

I left an issue to explain this in more detail, but I don't think the problem is chunking. The issue is the prompt. The local LLM space does itself no favors by thinking about and using prompts as an after thought.

IME, the prompt should be front/center in terms of importance and the key to unlocking the models potential. It's one of the main reasons why Textgen-Webui is sooooo good. You can really dial-in the prompt, from the template itself to working with the system message. Then begin futzing with the myriad of other parameters to achieve fantastic results.