[xscott]

All the terms are squishy, but being sloppy about it, I think there's intelligence that needs to be in the model, and knowledge that could live in a database. Right now, models are memorizing a lot of stuff they don't need to. We know how to index lots of information on (comparatively slow) SSDs or across a network.

See Karpathy's "Cognitive Core" idea. (I don't have a good link)

[t-3]

I was recently skimming through some 20-30 old books about Information Retrieval (https://en.wikipedia.org/wiki/Information_retrieval), and it was immediately quite striking to me that LLMs are the culmination of the integration and development of many ideas in IR that were impractical or theoretical back then. Decoupling the database from the search interface would make the relationship between modern AI and IR even clearer.