[lr4444lr]

I don't know what his other bad arguments are, but nothing you're describing disputes the point about formal reasoning, which is that getting it wrong is susceptible to parameter fitting. This has been a problem with AI models ever since the perceptron, which can still converge to the wrong classifications even when it's fed enough training data.

[hackinthebochs]

Formal reasoning is reasoning with the "form" or shape of an argument while being agnostic to its content. But LLMs can do this in principle, for the aforementioned reasons (moving data around, applying context-sensitive rules). The practical issues of the current architectures and training paradigms are legitimate. But Gary Marcus's claims generally are a complete rebuke of LLMs as a class being capable of reasoning in any capacity. That's where his arguments fail. But he doesn't give interlocutors a fair read, completely ignores counter-evidence, and is generally dishonest in promoting his viewpoint.