[dartos]

You’re talking specifically about perceptrons and feed forward neural networks.

LLMs didn’t exist in then. Attention only came out in 2017…

[tossandthrow]

Yes? Are you saying that attention is less expressive?

[dartos]

I’m saying that LLMs (models trained on language specifically) are not automatically capable of the same generic function solving.

The network itself can be trained to solve most functions (or all, I forget precisely if NNs can solve all functions)

But the language model is not necessarily capable of solving all functions, because it was already trained on language.