[DuskStar]

> This is a flawed analogy. The conceptual basis of nuclear weapons was well understood as soon as it was learned that the atom has a compact nucleus. The energy needed to bind that nucleus together gives a rough idea of the power of a fission weapon. If that energy could be liberated all at once, it would make an explosive orders of magnitude more powerful than anything known.

Extrapolating as you seem to be here, when should I expect to see a total conversion reactor show up? I want 100% of the energy in that Uranium, dammit - not the piddly percentages you get from fission!

Seriously, I think you overestimate how predictable nuclear weapons were. Fission was discovered in 1938.

[laughinghan]

If you read your own Wikipedia link, you'd see that Rutherford's gold foil experiments were started in 1908, his nuclear model of the atom was proposed in 1911—we even split the atom in 1932! (1938 is when we discovered that splitting heavier atoms could release energy rather than consume it.)

We haven't even had the AGI equivalent of the Rutherford model of the atom yet: what's the definition of consciousness? What is even the definition of intelligence?

[orbifold]

You might not need a definition of consciousness. Right now it looks like you can get quite far with „fill in the blanks“ type losses (gpt-2 and Bert) in the case of Language understanding and Self-Play in the case of Games.

[laughinghan]

We are indeed getting impressively far. Four decades after being invented, machine learning went from useless to useful to enormous societal ramifications terrifyingly quickly.

However, we are not getting impressively close to AGI. That's why we need to stop the AGI alarmism and get our act together on the enormous societal ramifications that machine learning is already having.