[M_Grey]

On the downside, it's just a loop of Bender saying, "Kill all humans." Kidding aside though, the last bits of the article seem to spell out a future in which, for better or worse, we don't really understand what's under the hood, completely. What exactly is the implication, crazy extremes aside, of systems which essentially build their own black boxes?

[arjunbajaj]

A few days ago I found an Asimov story, The Last Question[1] on the comments on some other HN post. The story talks about computers which are far more advanced than today's and no human can completely understand their inner workings.

I think, even today, a modern microprocessor is out of the reach of any single human to fully comprehend. This is definitely the case for the extremely vast field of computer science.

Maybe the future of computing and artificial intelligence are systems that are in our capability to build but beyond our understanding of why they do what they do!

[1]: http://multivax.com/last_question.html

[danso]

Is it a matter of the interplay between algorithm and mix of data being too complex for a human to know and predict the state? All outputs of the system are still theoretically predictable and reproducible (given the time/resources to reproduce the inputs), right?