[anon373839]

This is true, but good LLMs (even local, consumer-sized ones these days) can do this much better than … whatever it is that Google’s AI overviews use.

Think about your coding harness: the model reads a bunch of files into the context, and it generally doesn’t forget/hallucinate which lines came from which file.

[captainbland]

I'd be surprised if it were anything other than just an LLM only equivalent of Gemma some 4B param model, they can't be spending much money on it because each query essentially needs to be cheaper than the ad revenue per search.

[anon373839]

Could they even afford 4B dense parameters touching every token? That seems like a LOT of compute compared to generating the classic Google SERP.