[lxgr]

LLMs are much better at summarizing textual content or extracting specific pieces of information from it rather than answering complicated/niche queries from their weights alone, and that's likely (part of) what is happening here (i.e. fetching Wikipedia articles and cross-checking before answering).

Arguably this is using Wikipedia exactly for what it's designed for, although in an unexpectedly resource-intensive way. I bet just adding a web query cache for most frequently visited URLs on the side of the LLM provider could mitigate most of the negative effects here.