[HarHarVeryFunny]

Yeah, it's not clear what companies like OpenAI and Anthropic mean when they predict AGI coming out of scaled up LLMs, or even what they are really talking about when they say AGI or human-level intelligence. Do they believe that scale is all you need, or is it an unspoken assumption that they're really talking about scale plus some set of TBD architectural/training changes?!

I get the impression that they really do believe scale is all you need, other than perhaps some post-training changes to encourage longer horizon reasoning. Maybe Ilya is in this camp, although frankly it does seem a bit naive to discount all the architectural and operational shortcomings of pre-trained Transformers, or assume they can be mitigated by wrapping the base LLM in an agent that provides what's missing.