[st1x7]

This is just science fiction. To mention "recent developments" in the introduction is somewhat misleading considering how far the current state of technology is from their hypothetical superintelligence.

We don't have superintelligence, we don't have the remote idea of how to get started on creating it, in all likelihood we don't even have the correct hardware for it or any idea what the correct hardware would look like. We also don't know whether it's achievable at all.

[plutonorm]

That's the mainstream opinion on every. single. revolutionary advance. That you and everyone else believes it's not going to happen ever has almost no predictive power as to whether it actually will.

[semi-extrinsic]

It's not so much "opinion on a revolutionary advance". When it comes to AGI-related stuff, we are quite literally like contemporaries of Leonardo da Vinci, who have seen his plans for the helicopter and are postulating that helicopters will cause big problems if they fly too high and crash into the mechanism that is holding up the sky above us.

Also, this is not the mainstream opinion on e.g. fusion, or electric cars and smartphones (20 years ago), or a computer in every home (50 years ago). Those have been arguments about money and practicality, not about "we don't even know how such technology would look or what it would be based on".

[peteradio]

I think we fear ourselves (rightly) and super-fear a super-self (rightly). If this theoretical thing has the capacity at all to interact the way humans do then it will probably be brutal to at least someone and possibly most or everyone.

[plutonorm]

Said just about everyone before the Wright brothers made their first flight.

[alisonkisk]

No one thought that flight was impossible. Birds were known in the 19th Century

[plutonorm]

eh? No one thinks AGI is impossible, brains do it. What's your point?

[semi-extrinsic]

Difference is, people were able to propose detailed mechanisms for realistic flying machines long before we actually achieved powered flight - that was mainly a matter of increasing the power to weight ratio of the propulsion system. For AGI, do you really think there are detailed proposals out there today that can achieve AGI but are only missing the computational power?

Actually the existence of human brains with their (comparatively) extremely low power consumption, indicates that we need something radically different from current silicon-based processors to achieve AGI.

[peteradio]

I'm sure there's a fallacy in the following, but here goes:, Who could have predicted the improvements in computation in the last century? Would someone a century have extrapolated sun-sized machines need to compute a nations taxes based on current SOA? We don't have it and then all of the sudden we will. Its worth recognizing the potential harnesses before the beast is born.

[UnFleshedOne]

Well, we do know that normal intelligence (superintelligence from chimps point of view) is achievable just fine.