[jack_quack]

He lacks humility about minds that humans cannot comprehend

[goatlover]

Same argument can be applied by believers to God. If we can't comprehend something, then we should be careful when talking about it.

"That whereof we cannot speak, thereof we must remain silent" ~ Wittgenstein

[dwaltrip]

First we must demonstrate that the alleged deity is indeed present before we worry about understanding it.

[goatlover]

In context of this discussion, that would be future Superintelligence.

[dwaltrip]

Fair point. Although, there are some key differences. We have a trend of machines with increasingly intelligent capabilities. So we must ask: Is it reasonable to extrapolate? How far? What are the risks? Difficult questions.

I'm personally not terribly worried, as I think we have a ways to go before creating human-like general intelligence, let alone super intelligence. It is also not clear at all to me that an exponential increase in intelligence is likely or feasible after some threshold. Still, treading carefully seems prudent. There are some serious risks, and even if the likelihood appears low, there are a lot of unknowns.

[drdeca]

I am likely mistaken, but my impression was that that quote was after arguing that we cannot really refer to certain things in a coherent way, and that the quote advocates that we therefore do not attempt to do so. How far off am I?

[goatlover]

My understanding is that Wittgenstein likely had metaphysics and certain philosophical problems in mind. I don't know what he would have to say concerning speculation about super AIs and their potential capabilities.

[diegoperini]

Reading the article, I thought it's all about the humility we lack in the industry.