[whakim]

I’m very skeptical of any future prediction whose main evidence is an extrapolation of existing trendlines. Moore’s Law - frequently referenced in the original article - provides a cautionary tale for such thinking. Plenty of folks in the 90’s relied on a shallow understanding of integrated circuits and computers more generally to extrapolate extraordinary claims of exponential growth in computing power which obviously didn’t come to pass; counterarguments from actual experts were often dismissed with the same kind of rebuttal we see here, i.e. “that problem will magically get solved once we turn our focus to it.”

More generally, the author doesn’t operationalize any of their terms or get out of the weeds of their argument. What constitutes AGI? Even if LLMs do continue to improve at the current rate (as measured by some synthetic benchmark), why do we assume that said improvement will be what’s needed to bridge the gap between the capabilities of current LLMs and AGI?

[_heimdall]

I'm similarly skeptical of any prediction that ignores the fact that human intelligence and consciousness is emergent. LLMs don't seem particularly intelligent to me today, but how can I trust that their perceived limitation today won't lead to intelligence tomorrow or next year?

More generally, how do we even define or recognize general intelligence or consciousness? And if we recognize intelligence or consciousness does that come with legal rights and protections equal to what we offer people today?

[robwwilliams]

There is a critical focus in the article on algorithmic improvements. Much harder to measure and predict, but I think there is a good case to be made that recent progress has not just been quantitative.

[whakim]

I agree that there's a focus on algorithmic improvements, but what is the basis for assuming that we'll be able to continue to make algorithmic improvements on the same scale? The argument feels exactly backwards - if you had a deep understanding of the field, then you'd be able to discuss the untapped areas of potential algorithmic improvement and use that to predict future progress. The argument in TFA uses the trendline of past progress to predict untapped areas of algorithmic improvement.