[dreambuffer]

FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid.

I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.

[dannyobrien]

So, this is not quite right: Alexander contributed to the report, but his personal opinion is more like the mid-2030s[1]. Freddie feels like this is him backing down from the original statement, but in fact he said this at the time the report was published, and in fact pointed out a graf below the quote that Freddie claims does tie him to 2027:

> Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would be annoying to change). Other members of the team (including me) have medians later in the 2020s or early 2030s, and also think automation will progress more slowly. So maybe think of this as a vision of what an 80th percentile fast scenario looks like - not our precise median, but also not something we feel safe ruling out. [2]

I don't think this changes your observation that he is "personally invested" (i.e. believes this trendline will continue), but I'm pretty sure when AGI doesn't appear in 2027, many people will believe that this invalidates the arguments being made here (or in the report). The actual report was intended to give a feel for what a near-future "disaster" AGI scenario, and settled on a date to give that some concrete immediacy. The collective review that gave that as a possible, but not inevitable date is still ongoing (they originally pushed their best estimate out a bit further, but now they think, judging by the goals that are being hit, their scenario was a little too conservative). [3]

[1] https://freddiedeboer.substack.com/p/im-offering-scott-alexa... [2] https://www.astralcodexten.com/p/introducing-ai-2027 [3] https://blog.aifutures.org/p/grading-ai-2027s-2025-predictio...

[grey-area]

AI boosters really are detached from reality.

LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text.

Apparently that is enough to fool many people but it’s nothing close to AGI which would require internal models of the world, reasoning etc.

We are nowhere close to AGI and the fools who predicted we were will unfortunately keep lying about their stated timelines when it inevitably doesn’t arrive. You’re already hedging and trying to caveat previous predictions, as OpenAI did with their AGI predictions which they’re now furiously back-pedalling on.

[naasking]

This is all speculative. We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation", especially once you add feedbacks loops.

[RaftPeople]

> "We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation""

Do you mean "token" as in the LLM sense?

Or are you thinking that thoughts in the human brain are also constructed out of some sort of underlying "token" even though the abstract thought happens and is held before any words are used to try to communicate that thought to an external party?

[ToValueFunfetti]

LLMs also don't run on tokens internally, they're just the inputs and outputs. The reasoning models do operate (partially) in the token space, but then so do I.

[RaftPeople]

LLM's generate their output words sequentially based on probability (from learned stats).

Human's don't operate the same way, the thought happens and then the words are generated to reasonably describe that thought.

[grey-area]

We understand it enough to see the obvious massive deficiencies in LLMs.

They can predict likely sentences but not evaluate truth or logic. They can fairly reliably record facts about the world but not construct internal models of the world.

[naasking]

> They can predict likely sentences but not evaluate truth or logic.

They do probabilistically. So do humans as a matter of fact. The best of us are better at it than LLMs, but that's not persuasive evidence of anything meaningful really.

> They can fairly reliably record facts about the world but not construct internal models of the world.

You don't know that, unless your presuppose a very specific definition of world model that necessarily precludes emergent ones.

[grey-area]

Humans do not reason by guessing the next most likely token/word. They use logic, morality and systems of thought they have constructed and shared to help them reason and don’t in any way predict tokens in a sequence - we use words to represent our thoughts and feelings about the world, not to construct them.

You’re constructing a post-hoc fantasy of human thought based on how LLMs work because you are desperate for some reason to believe that they are thinking like humans, but they are not. The process is very different and the results are also different.

[par1970]

> LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text.

Argument?

Are LLMs close to being able to significantly help AGI researchers?

[throwawayk7h]

Mind you, he is only personally invested insofar as he's staked his reputation on it. Throughout his writing, he expresses the same point over and over again: desperately wants AI to slow down, advocates for politics that would slow it down, and most likely nothing would bring him greater peace than to see a sigmoid curve appear.

[bombcar]

How convenient; when AGI doesn’t appear in 1-2 years his reputation is pristine because he slowed it down.

[achierius]

What do you want? This sounds like you have something against people making a claim in public, at all, on any topic of importance.

[wmf]

To make that argument you'd want to show some causal link which so far we haven't seen.

[ToValueFunfetti]

This is incorrect as written. The author contributed writing to AI-2027 but distanced himself from the underlying model. That model had 2027 as the modal year of AGI, not median or mean. The authors of that model revised it to a later date shortly after and (if I recall correctly) have since done so again.

It is broadly true that Scott believes that AGI will come in the near future and from LLMs, although his reputation runs a ways deeper than that.

[scott01]

If I'm not mistaken, he's either affiliated with or otherwise connected to the effective altruist movement, hence he can't be unbiased. I find this article tells an interesting perspective on it: https://www.noemamag.com/the-politics-of-superintelligence/

[goosejuice]

The whole rationalist movement is super bizarre to the "normies".

It's not at all surprising that they are increasingly getting labeled a cult (they aren't by traditional definition but there are a lot similarities). I'm really surprised it hasn't hit the mainstream yet given the connections to Elon, Thiel, frontier labs, dark crypto funding, FTX/SBF, some suicides and some murders. It's all a little nuts.

Meanwhile you got all the anti-democratic NRx people on the other side of it.

I suspect this new doc coming out on HBO will spark a media frenzy.

[naasking]

Nobody's unbiased.

[scott01]

Obviously. I responded to the commenter with some context, but your comment doesn't bring anything to the discussion.

[paulpauper]

He only has 1.5 more months. If he's wrong he needs to own it. Same for Eliezer Yudkowsky. But these people have too much riding on their brands. No one has the courage to fess up to being wrong. Given how many podcasts he and others have been on professing this belief, it will be hard to just pretend otherwise.

[hollerith]

Yudkowsky has never predicted that "AGI" will be here in 1-2 years. He has been saying frequently for years that it is easier to predict how the AI juggernaut will turn out (i.e., very badly for us) than to predict when the very bad things will happen.

(I don't know about the other guy mentioned above.)

[quickthrowman]

Eliezer Yudkowsky, now there’s a name I haven’t heard in a while.

What has he been up to since finishing the finest work of literature ever produced, Harry Potter and the Methods of Rationality? I’ve been patiently awaiting a sequel!

[arjie]

His new book has billboards along the highway to the Bay Bridge. Surprised you haven’t heard of him recently. His fame has skyrocketed. If anyone builds it, everyone dies.

[woeirua]

Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!

[yorwba]

The METR task set contains no tasks with a duration greater than 32 hours (conservatively eyeballed from Figure 3: https://arxiv.org/abs/2503.17354 ), so any prediction that naively forecasts a longer time horizon is trivially incorrect. I guess that won't lead to a sigmoid-looking graph though, since METR will likely switch to a different evaluation methodology at that point and stop updating the old curve.

[woeirua]

METR themselves say that any estimate >16 is highly suspect because there are too few tasks.

I expect benchmarks like ProgramBench will replace METR this year.

[sigmoid10]

AGI has become such a meaningless nondescript term, arguing when or how it is here has become pointless. Even OpenAI caved in and removed their AGI clause from their contract with Microsoft because they weren't fully sure that we are not there yet. The original ARC AGI was hailed as proof that AGI is not here yet, but now that ARC 1 and 2 got saturated, noone wanted to consider that perhaps we crossed the point where average humans are getting left behind. Frontier models are primarily limited by context and modality at this point, not by intelligence.

[Auracle]

To your point, if we had truly unlimited context to the point where at least that instance of a model could “learn” and have what seems like a continuous “consciousness” I think many of us would think that we’ve attained AGI.

Right now we have an incredibly smart thing with severe short term memory loss, and it’s hard for us to reconcile that as it’s so different from us.

[sigmoid10]

Quite a few people were already led to believe that these models are conscious when we had a fraction of current context lengths. Right now the biggest problem is that the "session" info in form of the current conversation gets lost too quickly, but that has become largely an implementation detail. You could fit an entire life's story into modern context windows. With some clever context management, you could probably build something that feels like what you describe. If we truly had this sort of short-term to long-term memory (i.e. from prompt context to weights) system on a technical foundation, we'd probably be closer to runaway superintelligence than mere AGI that could beat most humans on most tasks.

[Sniffnoy]

> FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid.

I mean, that's called "having an opinion".

[dreambuffer]

He co-authored a report, which is something more than an opinion. It may be used to inspire policy. There should be greater reputational consequences for publishing something you spent a few months studying and writing about along with several experts. Just my opinion.

[Sniffnoy]

I don't understand what you're trying to imply here. Yes, he co-authored a report. What is supposed to be dangerous or suspicious about this? What does your statement about "reputational consequences" have to do with your original comment, which implies that this some indicates a bias on his part?

It seems to me like you're trying to somehow imply that writing things to convince people of what you believe is somehow nefarious? It isn't! It's what we're all doing here right now! Putting it in a format that certain people will take more seriously doesn't make it nefarious either. I am quite confused by your point of view here.

[dreambuffer]

There was no implication of anything you're suggesting. It's a question of correctness (bias vs facts, predicting the sun will rise vs predicting the end of the world), whether you think it's important to be correct as a matter of reputation, and how correctness should be weighed if it is indeed important to one's reputation (a once-off comment vs a full report).

Not interested in further arguments about this.

[DonsDiscountGas]

And now he's publishing more information about that same opinion he still has. How horrible.

[paulpauper]

He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy. He's taken a side. if he's wrong I would hope he owns up to it

[Sniffnoy]

> He's taken a side.

Yes, that's called "having an opinion". Typically people writing argumentative pieces are doing so because they have a belief about the matter. I'm not sure what exactly you expect here.

> if he's wrong I would hope he owns up to it

I think Scott Alexander is pretty good about that.

[boxed]

> He wrote articles arguing that pro-AI people are dismissive of risks or even suggesting they are intellectually lazy

I mean.. this is 2026 right? You're not writing that comment from 2024 or something?

We see massive problems already where photos are just not believable anymore, nor is audio, and not even video actually with many people falling for AI fake image clips from the Gaza war for example. And since then these tools are MASSIVELY more powerful. Disinformation is essentially free, and the cost of truth has been static. Meaning the "buying power" of truth has collapsed and is falling faster and faster.

Anyone who dismissed AI risks a few years ago IS ALREADY PROVEN WRONG.