[squarefoot]

Looks like this is not easy at all for a non ML expert. And probably the required computing power is still out of reach for mere mortals. I'd have a similar use case to the parent: technical books. I'd love to be able to ask where a certain topic is discussed in my pdf archive and have the AI reply with references and possibly a significant piece of the relevant articles, with images (or local links to them).

[gzer0]

This is already a feature of adobe pdf reader professional (called index mode). There’s also an app on macOS called “pdf search” which does quite a good job. I use it for the exact reasons you describe; I’ve got a repertoire of technical books on AWS and Azure and I reference them all the time via my local search engine via these apps.

[deet]

The computing power is definitely not out of reach of mere mortals. I'm working on software that does this for emails and common documents, generating a hybrid semantic (vector) and keyword search system over all your data, locally.

The computing power we're requiring is simply what's available in any M1/M2 Mac, and the resource usage for the indexing and search is negligible. This isn't even a hard requirement, any modern PC could index all your emails and do the local hybrid search part.

Running the local LM is what requires more resources, but as this project shows it's absolutely possible.

Of course getting it to work *well* for certain use cases is still hard. Simply searching for close sections of papers and injecting them into the prompt as others have mentioned doesn't always provide enough context for the LM to give a good answer. Local LMs aren't great at reasoning over large amounts of data yet, but getting better every week so it's just a matter of time.

(If you're curious my email is in my profile)

[siquick]

It’s not as difficult as you think with libraries like Llamaindex and Langchain.

Both have extensive examples in their documentation for almost identical use cases to the above.

[squarefoot]

Thanks for all replies. It would probably be worth creating a HOWTO or something like that aimed at non ML experts or complete AI illiterates like myself to help putting together something that works in simple steps (assuming this is possible), from procuring the hardware offering the minimal requirements to organizing data in a way that can be used for training, and finally using the right tools for the job.

[infecto]

There is zero training evolved. The workflow is no different really compared to normal search. Compare to the high level flow of implementing elastic search. The only difference is you are encoding the data using vectors based on a model that best meets your criteria. Tons of howtos out there already on generating embedding a search for LLMs. I think even openai has some cookbooks for this in their docs.

[infecto]

Definitely accessible to anyone who can write code. Very little ML knowledge is necessary. And all of this can be done reasonably on a laptop depending on how large your corpus material is.