[incrudible]

The problem is in being able to discern reasoning from patterns that happen to exist in the training data. There are plenty of tricks you can play on an LLM by subverting the expectations it must necessarily have due to its training data. A human might fall into the same trap, but can then reason themselves out of it, whereas an LLM tends to double down on its mistake.