[bglazer]

These are the best results that I've gotten from an AI research assistant.

I really don't mind the long latency, in fact, I think it's a fundamentally better way of interacting with this kind of LLM based tool.

Like the latency is necessary for the LLM to actually interact with the content, rather than just doing a Bing or Perplexity style RAG+summarization workflow that delivers very uneven results.

I also really like the use of longer prompts, as it encourages a full description of your topic, rather than keyword fiddling trying to make the RAG system pick up the right signifiers.

The "Discovery Progress and Exhaustiveness" section is a bit confusing as a user. Like, ok we have 23.6% of the relevant papers? Why not 100%? What am I supposed to do with that information? Can you give me any information about the missing papers?

Overall, very nice work, I'll be using this in the future.

[tomhartke]

On the discovery progress, we can't look at all 200M papers with the LLM, so we prioritize some of them (the first 100 most promising) for deep analysis. Within those, we find a few that are relevant. But the rate at which we discover these tells us roughly what will happen if we read the next 100 (if we're discovering new relevant papers all the time, we will likely continue). We need a better explanation on the website, but we can statistically model this to quantitatively predict how many papers we would find if we exhaustively searched the whole database.

More info here: https://www.undermind.ai/static/Undermind_whitepaper.pdf

[bglazer]

That makes sense, I figured it was related to some of the statistics work in ecology estimating species count from a limited sample.