[soulofmischief]

I'd recommend educating yourself on the last few years of neural network development before spreading misinformation.

LLMs have been shown to be capable of embedding logic and reasoning. There is nothing preventing a sequence generator from learning stochastic reasoning skills. Modern LLMs don't even just output language tokens. The latent information stored in the hidden layers is far more abstract than just language.

[turtlesdown11]

did you not read https://arxiv.org/pdf/2410.05229

"We hypothesize that this decline is due to the fact that current LLMs are not capable of genuine logical reasoning; instead, they attempt to replicate the reasoning steps observed in their training data."

[soulofmischief]

They've presented a hypothesis warranting further study.

Here is an article where a model's learned ability to carry out modular addition is recovered. https://arxiv.org/abs/2301.05217

As you can see, the jury's still out.

[joquarky]

Didn't Godel demonstrate that language can only approximate reality?

[soulofmischief]

LLMs are a special type of transformer model, however several recent models have been multimodal, allowing for a variety of input and output type. So the question is if a neural network can approximate reality, not LLMs.

Though really, all embedded models of reality are approximate by nature, and based on stochastic empirical data. The real tradeoff with probability-based neural networks is whether a rigid algorithmic approach or a more flexible, but stochastic, approach solves the problem at hand.