I already find original ChatGPT (free version based on GPT-3.X) being "smarter" than Bing's one based on GPT-4. The latter much easier gives up just saying that it could not find anything, while ChatGPT replies based on its "knowledge" even on quite advanced science questions. Having a LLM trained on content generated by other LLM may be a way for disaster in future: filtering true from false (but convincing) information on scale would be hard to say the least. In light of recent troubles with Internet Archive, AI companies, like OpenAI, are of the beneficiaries of the Internet history "made on Earth by humans" :)
You cant prevent an LLM from having a hallucination.
The current state of the art is retrieval + summarization, all that knowledge it was trained on still exists. When performing a search having no reference for the knowledge is a decent signal that it may not exist at all and you may be talking to a liar.