Hacker News new | ask | show | jobs
by zztop44 928 days ago
I honestly don’t mean to condescend, but I find it so interesting that smart people can convince themselves that such an outlandish scenario is somehow likely or even perhaps inevitable.

We haven’t invented “AGI” yet (if AGI means a computer system that merges/exterminates/controls humanity). We have no idea even theoretically what such a system might look like, let alone how to create it.

I’d say we’re a lot closer to colonising the solar system and building cities on Jupiter’s moons than we are to building the machines from the Matrix. At least we kind of theoretically know how to do the former.

Meanwhile, we’re really good at building computer and industrial systems that (when combined with economic, social and political incentives) exert strong control over us, exterminate many of us, and may well exterminate a lot more in the near future.

But that doesn’t look like a Hollywood movie, so it’s less fun to think about.

Sorry, this wasn’t meant to be a dig at you personally. Some of the AI doom stuff just sets me off.

3 comments

A lot of the AI doom stuff is essentially just a criticism of capitalism and corporations in general. There's essentially no difference between a hypothesized runaway AI and a normal multinational corporation. We have, right now, large semi-autonomous organizations called "corporations" that are making the earth uninhabitable in pursuit of profits at the expense of human suffering. Corporations are nominally in control of people, but any people that get in the way of profits are pushed aside.
We are simply watching the birth of a new religion. Religious people aren't religious because they're dumb, intelligence is mostly not a factor. Same thing here I think.
Us: don’t build that, it will be the death of us.

You: what a ridiculous statement, we haven’t even built that yet.

No one can even say what “that” is that we shouldn’t build. It’s a wholly arbitrary invention, the product of a collective word game that is more apologetics than serious futurology.
That's partially the point. We may not know it until it's too late.

But it doesn't matter. If you gave everyone a button and the warning that pushing it would end the human race, we'd all be dead pretty much immediately.

I think it's a given that on a long enough timescale, someone will create AGI and we'll no longer be the dominate race on earth. How that ends, in death, or slavery, I don't know.

I do hope that every AI researcher does their work with a small bit of fear of becoming the pet of an AGI.

I actually don't think we should build it either. The best case scenario is it provides a way for powerful people to accelerate the already quite successful project of making feudalism 2.

But that doesn't mean I'm stressed about what you're stressed about. We killed god once before we can do it again if we need to. The real danger is in us using AI to do more of what it already does now, not that it will decide to do some new thing.

Yeah. A lot of it to me feels like the JRPG version of reality.