[reacharavindh]

I’m curious to try it out. There seem to be many options to upload a document and ask stuff about it.

But, the holy grail is an LLM that can successfully work on a large corpus of documents and data like slack history, huge wiki installations and answer useful questions with proper references.

I tried a few, but they don’t really hit the mark. We need the usability of a simple search engine UI with private data sources.

[haizhung]

Differentiale search indices go into this direction: https://arxiv.org/abs/2202.06991

The approach in the paper has rough edges, but the metrics are bonkers (double digit percentage POINTS improvement over dual encoders). This paper was written before the LLM craze, and I am not aware of any further developments in that area. I think that this area might be ripe for some break through innovation.

[verdverm]

https://www.kapa.ai/ seems to be the most popular saas for developer tools & docs. I'm seeing it all over the place

[victor106]

Used it, it’s just glorified marketing and among all the solutions we tried it ranked in the bottom three.

The best at least for now is to just use OpenAI’s custom gpt and with some clever (but not hard) it’s quite good.

[verdverm]

If you want to allocate resources to building out the AI, connecting and ingesting sources, setting up rag, fine tuning and hyper param optimization...

Most companies lack the expertise and resources. Kapa means they get a docs bot while maintaining focus on what they do best.

Kapa must be doing something right since they seem to be growing. Having used it in a few discords, it's what I'd expect for quality for a saas product built on current ai capabilities.

[krageon]

> Kapa must be doing something right since they seem to be growing

It's marketing. The person you responded to said they're all marketing. Saying they "must be doing something right" because other people are also falling for it is how you get scammed.

[verdverm]

That's what they say. What I see is high engagement of users in discord channels.

Even OpenAI, GP's alternative, is listed as using Kapa... and no public sign ups available yet either

I saw a glimpse of the internal dashboard companies get. It's much more than just question answering. Another big piece is the feedback, seeing user interaction, and being able to improve things over time

[CharlesW]

Assuming the word “promoting” got lost, can you share more about this?

[snowfield]

Rag is limited in that sense. Since the max amount of data you can send is still limited by the token amount that the LLM can process.

But if all you wanted is a search engine that's a bit easier.

The problem is often that a huge wiki installation etc will have a lot of outdated data etc. Which will still be an issue for an llm. And if you had fixed the data you might as well just search for the things you need no?

[boredemployee]

I think it depends of what they want. Like a search is indeed an easy solution, but if they want a summarization or a generated, straight answer so then things get a little bit harder.

[cyanydeez]

A solution that combines RAG and function calling could span the correct depth, but yeah, the context depth is what will determine usefulness for user interaction.

[HeavyStorm]

The LLM would have to be trained on the local data. Not impossible, but maybe too costly?

[snowfield]

It sounds nice in theory but your dataset is most likely too small for the LLM to "learn" anything.

[IanCal]

I'd like to play with giving it more turns. When answering a question the note interesting ones require searching, reading, then searching again, reading more etc.

[rgrieselhuber]

This gets to the heart of it. Humans are good at keeping a working memory, as a group or individuals, as lore.

[cyanydeez]

If anyone wants down this golden path, I'd recommend forking open search server. It's quite a feat and does the crawling part well.

https://www.opensearchserver.com/

[andeee23]

https://storytell.ai seems to be doing what you’re looking for, especially the part with the linking to proper references

[oulipo]

Have you tried https://markprompt.com/ ?