[addisonj]

Do I think that we are likely to suddenly see a massive massive leap to AGI/ super AGI? Not really... But does it still seem like enough of a long tail risk in my/my kids lifetime that I hope someone is getting resources and access to think about this? I sure do.

I know comparing AGI development to nuclear weapon development is a trope at this point, and in general, not that that useful of comparison... But I do think having a group with a diversity of how to think about the impact of desired outcomes is a good thing to have whenever you are working in the realm of world changing outcomes, and it seems like the changes at openAI are eroding that to some extent, even if it is really really unlikely to come out of this specific company at this time.

[j45]

I like how you put this, things go slow before they go very fast.

Also, competition search engine style.. this feels is getting ahead and staying ahead.

[nicklecompte]

I don't think any of us will live to see AI smarter than a rat. So I am not concerned whether this team was going to superalign anything.

The problem is that OpenAI's software, especially GPT-4o, is primed for dangerous misuse, and the demo videos of GPT-4o seemed "misaligned" with any reasonable standards of AI safety. Even if Leike/Sutskever have delusions of grandeur about AGI, at least they cared about the idea of AI safety. It seems like pushing the team out meant getting rid of a lot of internal critics (and implicitly threatening anyone else who might speak up).

[me_me_me]

> I don't think any of us will live to see AI smarter than a rat.

Rat, dog, human, bee. Doesn't matter. The moment AI is able to incrementally improve a bee intelligence will evolve into rat, blink again and its dog, another blink and its smarter than us.

That is the danger. The issue of control and danger is not linear but exponential. And without a plan to deal with issues things can get out of control exponentially faster.

> The problem is that OpenAI's software, especially GPT-4o, is primed for dangerous misuse, and the demo videos of GPT-4o seemed "misaligned" with any reasonable standards of AI safety.

I forgot who said it, maybe Sam Altman. The reason for releasing the current models was to show public what the current AI can do and how far it progressed. To get people to understand where we are and where we can get to with AI

Sort of like giving people muskets so they adapt to it before machine-guns appear.

It forces other AI research to also show their stuff and not keep everything secret, until one day out of the blue we have pocket nukes available at everycorner for low low price of $10

[soist]

There was one such group but they determined it was impossible because of Rice's theorem and other limitations of formal systems for computation. Logical incompleteness, Tarski's theorem, and Rice's theorem are the main meta-theoretical results that make alignment fundamentally unsolvable. If you're really concerned about robots taking over the world then understanding basic computational theory should be a prerequisite but most people are not willing to spend the time to learn the theory and instead focus on vague and ill-defined science fiction concepts which are very unlikely to be actually physically possible/implementable because of various physical and formal limitations of computers.

I've decided anyone concerned about these issues knows almost nothing about computability theory so their theories are either nonsensical or just outright crazy. Very few understand the required formal concepts to have any useful ideas about how computers should be programmed to prevent "unsafe" results (which is often left just as ill-defined as most everything on AI safety and alignment research).