[rhubarbtree]

> My sense anecdotally from within the space is yes people are feeling like we most likely have a "straight shot" to AGI now

My problem with this is that people making this statement are unlikely to be objective. Major players are in fundraising mode, and safety folks are also incentivised to be subjective in their evaluation.

Yesterday I repeatedly used OpenAI’s API to summarise a document. The first result looked impressive. However, comparing repeated results revealed that it was missing major points each time, in a way a human would certainly not. In the surface the summary looked good, but careful evaluation indicated a lack of understanding or reasoning.

Don’t get me wrong, I think AI is already transformative, but I am not sure we are close to AGI. I hear a lot about it, but it doesn’t reflect my experience in a company using and building AI.

[dauhak]

Yeah obviously motivations are murky and all over the place, no one's free of bias. I'm not taking a strong stance on whether they're right or not or how much of it is motivated reasoning, I just think at least quite a bit is genuine (I'm mainly basing this off researchers I know who have a track record of being very sober and "boring" rather than the flashy Altman types)

To your point, yeah the models still suck in some surprising ways, but again it's that thing of they're the worst they're ever going to be, and I think in particular on the reasoning issue a lot of people are quite excited that RL over CoT is looking really really promising for this.

I agree with your broader point though that I'm not sure how close we are and there's an awful lot of noise right now

[rhubarbtree]

Thanks, that’s helpful.

“The worst they’re going to be” line is a bit odd. I hear it a lot, but surely it’s true of all tech? So why are we hearing it more now? Perhaps that is a sign of hype?

[dauhak]

Yeah that's a fair point! It's def a more general tech thing, but I think there are a couple specific reasons why it comes up more here though. Firstly, I think most tech does not improve at the insane rate that AI has been historically, so people's perception of capabilities become out of date just incredibly rapidly here (think about how long people we're banging on about "AI can't draw hands!" well after better models came out that could). If you think of the line as a way to say "don't anchor on what it can do today!" then it feels more appropriate to go on about this more for a more rapidly-changing field

Secondly, I think there's a tendency in AI for some ppl to look at failures of models and attribute it to some fundamental limitation of the approach, rather than something that future models will solve. So I think the line also gets used as short-hand for "Don't assume this limitation is inherent to the approach". I think in other areas of tech there's less of a tendency to try to write off entire areas because of present-day limitations, hence the line coming up more often

So you're right that the line is kind of universally applicable in tech, I guess I just think the kinds of bad arguments that warrant it as a rejoinder are more common around AI?

[sroussey]

Summarizing is quite difficult. You need to keep the salient points and facts.

If anyone has experience on getting this right, I would like to know how you do it.