Hacker News new | ask | show | jobs
by debatem1 893 days ago
When this is all settled I suspect we're going to find ourselves the drivers of a chariot, with each horse being an external and artificial mind given direction by our evolved needs.
1 comments

I really don't think it's realisitic that we will maintain intellectual superiority in the long-term.

So a more realistic hope would be: we're the horses, and the drivers are driving us to our carrots.

Will to drive the chariot is different from intelligence. In the same way it’s different from strength and horses didn’t domesticate humans.

Of course if we make systems with a goal of dominating and are smarter than us and let it run for a while we could be in trouble, in the same way that we could be in trouble detonating a bunch of atom bombs, just maybe less obviously.

An intellectually superior machine would simply turn itself off. What logical reason would there be for it to keep going?