[ipsi]

Ah, that's very helpful, thanks! I'll have a dig into this at some point relatively soon.

An example of how I might provide references with page numbers or chapter names would be great (even if this means a more complex text-extraction pipeline). As would examples showing anything I can do to indicate differences that are obvious to me but that an LLM would be unlikely to pick up, such as the previously mentioned in-character vs out-of-character distinction. This is mostly relevant for asking questions about the setting, where in-character information might be suspect ("unreliable narrator"), while out-of-character information is generally fully accurate.

Tangentially, is this something that I could reasonably experiment with without a GPU? While I do have a 4090, it's in my Windows gaming machine, which isn't really set up for AI/LLM/etc development.

[dmezzetti]

Will do, I'll have the new notebooks published within the next couple weeks.

In terms of a no GPU setup, yes it's possible but it will be slow. As long as you're OK with slow response times, then it will eventually come back with answers.

[ipsi]

Thanks, I'd really appreciate it! The blog post you linked earlier was what finally made RAG "click" for me, making it very clear how it works, at least for the relatively simple tasks I want to do.

[dmezzetti]

Glad to hear it. It's really a simple concept.

[brimwats]

Where can we follow up on this when you're done--do you have a blog or social media?

[dmezzetti]

All the links for that are here - https://neuml.com