| >An LLM will learn what it CAN (and needs to to reduce the loss), but not what it CAN'T. How difficult is that to understand?! Right and the point is that you don't know what it CAN'T learn. You clearly don't quite understand this because you say stuff like this: >Chess is a good example, since it's easy to understand. The generative process for world class chess (whether human, or for an engine) involves way more DEPTH (cf layers) of computation than the transformer has available to model it and it's just baffling because: 1. Humans don't play chess anything like chess engines. They literally can't because the brain has iterative computation limits well below that of a computer. Most Grandmasters are only evaluating 5 to 6 moves deep on average. 2. We have a chess transformer playing world class chess (grandmaster level) - https://arxiv.org/abs/2402.04494. You keep trying to make the point that because a Transformer architecturally has a depth limit for some trained model, a, it cannot reach human level. But this is nonsensical for various reasons. - Nobody is stopping you from just increasing N such that every GI problem we care about is covered. - You have shown literally no evidence that the N even state of the art models posses today is insufficient to match human iterative compute ability. GPT-4o instant shots arbitrary arithmetic more accurately than any human brain and that's supposedly something it's bad at.
You can clearly see it can reach world class chess play. If you have some iterative computation benchmark that shows transformers zero shotting worse than an unaided human then feel free to show me. |
Why don't you write Sam Altman to tell him the good news ?
Tell him there's nothing stopping him from "increasing N" until the thing get up and walks out the door.