[simonh]

The problem is such an approach is limited by the content of the training texts. As I mentioned elsewhere, our written texts assume huge swathes of contextual and experiential information and knowledge that LLMs don’t have. It’s possible some of it might be inferred from the texts, but not all of it by a long shot.

If somehow you could generate a training text encoding a complete and thorough understanding of the physical world, human psychology and sociology, and reasoning then that might get you quite far. But the existing it even near future human textual corpus isn’t really that. Even then I still think you’d hit the limitations of the LLM cognitive architecture pretty hard.

[giardini]

simonh says >"The problem is such an approach is limited by the content of the training texts."<

Aside: I would like to see ChatGPTs with distinct training texts, e.g., a ChatGPTs trained on the "great books" of Western philosophy and science knowledge up to the time of Victorian England.