[circuit10]

It’s arguable whether it’s actually improbable. Many people, including (and seemingly especially) experts think it’s likely or even inevitable. This video might be interesting to see some arguments for why people think it’s important: https://youtu.be/9i1WlcCudpU

[throwaway115]

I know someone who fancies himself an expert, and indeed, by comparison to the average tech worker, he is an expert in LLMs and AI. He vehemently believes AGI is coming this year. At least, that's what he said last year. I suppose he will probably feel sheepish if I bring it up now that we're at month 6.

My point is that the only thing that seems to convince the pro-AGI crowd is having them make specific predictions, waiting until the deadline, then asking them why they didn't happen yet.

[JKCalhoun]

I feel like we'll know right away though. We can then act right away.

I'm not sure what you would do now that is different because you are anticipating such a thing.

[circuit10]

It would be far too late by that point, as it can already take measures to stop us. This is a difficult problem and we really should have started putting a lot more effort into this a lot earlier. Of course we can’t test a superintelligence directly yet, but we should do as much theoretical work as we can

Actually the video addresses this point, the comparison it makes is waiting until we’re already on Mars before we start thinking about spacesuits and airlocks

[Aloisius]

Inevitable? Sure assuming we don't extinct first. In the near future? No.

This is just another in a long line of technology panics. Unfortunately, there always seems to be some "concerned" experts that are both overly optimistic about the speed of technological progress and overly pessimistic about where that progress will lead adding fuel to the fire.

Eventually some other new technology will incite a new panic and this one will become another footnote in history like the fears over grey goo or genetically engineered superhumans.

[circuit10]

What about when people pointed out problems like leaded fuel and climate change and we didn’t take action for far too long? We shouldn’t dismiss it just because of things it sounds similar to. In fact you could always dismiss any concern that could lead to human extinction like that, because if there was an existing example of human extinction then we wouldn’t be around to talk about it; that doesn’t mean it can’t happen.