you forgot that to not have a knowledge cutoff and fall behing, you need to always be training new models. It matters jack shit if inference is cheap, if you are forced to do training anyway to stay "competitive"
Depends on the application, of course. For "google replacement" they are trying to sell it as - it's absolutely essential, but even then it works like crap. For coding… Maybe it's not so essential? Yea, it would not know about latest libraries, but maybe that's not much of a problem? Then, of course, it is a big question if LLM code generation is worth it at all.
The AI companies are attempting displace Google for that sweet ad revenue. Relying on another company's search index means they can squeeze you later for a bigger slice of the pie