[grishka]

Or we might not. LLMs are remarkably dumb and incapable of reasoning or abstract thinking. No amount of iterative improvement on that would lead to an AGI. If we are to ever get an actual AGI, it would need to have a vastly different architecture, at minimum allowing the parameters/weights to be updated at runtime by the model itself.

[TeMPOraL]

Right. But there's so much effort, money and reputation invested in various configurations, experimental architectures, etc. that I feel something is likely going to pan out in the coming months, enabling models with more capabilities for less compute.