[voitvod]

There is an even more important lesson in this in that the people who worked on the bomb thought the world would be over in short order and were totally wrong.

Feynman talked about how there was an idea of "normal" people walking around not knowing they were basically doomed and going to die in a certain nuclear holocaust in a few years.

Von Neumann thought the U.S. should launch a nuclear first strike at Moscow. Obviously, if war is inevitable then you don't have to be the father of game theory to figure out you should strike first.

It was just a year ago that literally everyone was predicting we would be in a recession right now. We can't predict that but we can predict how AGI plays out even when we haven't bothered to define a measure of what AGI even is. Even people who grew up "knowing" we would all have sentient robots by 1997. I can't think of a single prediction I have heard in my lifetime that has turned out to be true other than the government debt going up.

[bnralt]

Not only that, but go back a decade ago and look at what the singularity people (which the AI alignment crowd comes out of) were saying that. When employment was struggling to recover after the great recession, the singularity folks said that it wasn't going to recovers, because tech was replacing humans, and that this was just the beginning of mass unemployment (one of the reasons why UBI got so popular).

CGP Grey's popular "Human's Need Not Apply" video is a good example of this kind of thought:

https://www.youtube.com/watch?v=7Pq-S557XQU&t=199s

Claims that self-driving cars are already here and already better than human drivers, and that the only question is how quickly they replace humans. He argued that Baxter, the general purpose robot, could already copy the tasks of a human worker and do the work for much cheaper. Baxter was discontinued in 2018 because of low interest.

These people have a horrible track record when it comes to technology predictions, and it's unnerving that, instead of reflecting on how wrong they've been, they're doubling down and trying to slow technological advancement.