[ummonk]

I’m sure a stochastic parrot model could be trained to exhibit reasoning, but the issue is that there isn’t any automated loss function which can discern whether the output of a large language model exhibits reasoning or is illogical. When you train based on text similarity, it will have a hard time learning logic, especially given the amount of illogical writing that is out there.

[AnimalMuppet]

Hmm. That's classically the job of an educator - to choose good training material, not just to let students read the internet all day. Would a ChatGPT trained on a carefully curated reading list do better than one trained on a wider reading list?

But it's more than that. A good educator teaches students to evaluate sources, not to just believe everything they read. As far as I can tell, ChatGPT totally lacks that, and it hurts.