[mattnewton]

Oh I totally see that, the issue I'm talking about isn't one of compute, but of high quality ground truth. This machine can hallucinate all kinds of information in perfect English already. The difficulty is that a good search engine needs to return more than just information that matches my query, it should return information that matches the objective reality people (and currently not the machine) inhabit. The machine needs text input to learn about the world; is the future going to look like companies hiring people to write essays about the world for machine consumption?

I think it's a similar problem we see today with ad-supported news being indexed by search engines, but taken to another magnitude when those articles need to be scanned by a model only once to have near perfect recall of the details.