[geraneum]

> … but never any data to substantiate the claim

There is!

https://arxiv.org/abs/2301.06627

From the paper:

  > Based on this evidence, we argue that (1) contemporary LLMs should be taken seriously as models of formal linguistic skills; (2) models that master real-life language use would need to incorporate or develop not only a core language module, but also multiple non-language-specific cognitive capacities required for modeling thought.

[jointpdf]

Thanks for this, I’ll give it a read.