[idlewords]

This is a flawed analogy. The conceptual basis of nuclear weapons was well understood as soon as it was learned that the atom has a compact nucleus. The energy needed to bind that nucleus together gives a rough idea of the power of a fission weapon. If that energy could be liberated all at once, it would make an explosive orders of magnitude more powerful than anything known.

It was hard to predict when or if such a thing could be made, but everyone knew what was under discussion.

Compare this to AGI, some vaguely emergent property of a complex computer system that no one can define to anyone else's satisfaction. Attempts to be more precise what AGI is, how it would first manifest itself, and why on earth we should be afraid of it, rapidly devolve into nerd ghost stories.

[jlawson]

  1932 neutron discovered
  1942 first atomic reactor
  1945 fission bomb

Now for AI

  1897 electron discovered
  1940's vacuum tube computers
  1970's integrated circuits
  1980's first AI wave fails, AI winter begins
  2012 AI spring begins
  2019 AI can consistently recognize a jpeg of a cat, but still not walk like a cat
  ???? Human level AGI

It doesn't seem comparable one way or the other, in many ways. But if we do compare them, AI is going much slower and with more failure, backtracking, and uncertainty.

[throwawaywego]

    1943 First mathematical neural network model
    1958 Learning neural network classifies objects in spy plane photos
    1965 Deep learning with multi-layer perceptrons

    2010 ImageNet error rate 28%
    2011 ImageNet error rate 25%
    2012 ImageNet error rate 16%
    2013 ImageNet error rate 11%
    2017 ImageNet error rate 3%
    2019 Pre-AGI

[idlewords]

Beer * beets * bears * Battlestar Galactica

[polm23]

what?

[idlewords]

BEER * BEETS * BEARS * BATTLESTAR GALACTICA

[DuskStar]

> This is a flawed analogy. The conceptual basis of nuclear weapons was well understood as soon as it was learned that the atom has a compact nucleus. The energy needed to bind that nucleus together gives a rough idea of the power of a fission weapon. If that energy could be liberated all at once, it would make an explosive orders of magnitude more powerful than anything known.

Extrapolating as you seem to be here, when should I expect to see a total conversion reactor show up? I want 100% of the energy in that Uranium, dammit - not the piddly percentages you get from fission!

Seriously, I think you overestimate how predictable nuclear weapons were. Fission was discovered in 1938.

[laughinghan]

If you read your own Wikipedia link, you'd see that Rutherford's gold foil experiments were started in 1908, his nuclear model of the atom was proposed in 1911—we even split the atom in 1932! (1938 is when we discovered that splitting heavier atoms could release energy rather than consume it.)

We haven't even had the AGI equivalent of the Rutherford model of the atom yet: what's the definition of consciousness? What is even the definition of intelligence?

[orbifold]

You might not need a definition of consciousness. Right now it looks like you can get quite far with „fill in the blanks“ type losses (gpt-2 and Bert) in the case of Language understanding and Self-Play in the case of Games.

[laughinghan]

We are indeed getting impressively far. Four decades after being invented, machine learning went from useless to useful to enormous societal ramifications terrifyingly quickly.

However, we are not getting impressively close to AGI. That's why we need to stop the AGI alarmism and get our act together on the enormous societal ramifications that machine learning is already having.

[thrax]

I think there is a lot of evidence that explosive progress could be made quickly. Alphago zero, machine cision, sentiment analysis, machine translation.. voice.. etc etc etc

All these things have surged incredibly in less than a decade.

It's always a long way off until it isn't.

[nradov]

Those are all impressive technical achievements to be sure, but they don't constitute evidence of progress toward AGI. If I'm driving my car from Seattle to Honolulu and I make it to San Diego it sure seems like I made a lot of progress?

[ekianjo]

> I think there is a lot of evidence that explosive progress could be made quickly. Alphago zero, machine cision, sentiment analysis, machine translation.. voice.. etc etc etc

Not at all, these are all one-trick poneys and bring you nowhere close to real AGI which is akin to human intelligence.