[crispy2000]

The article seems to assume that there will be one researcher or research organization producing one AI. This makes it reasonable to propose asking it a super-special question that reduces the risk.

It's more likely that armies of researchers will be competing to create quantities of ever-smarter AI's: for trading stocks, diagnosing illness, or managing enterprises. We have AI systems now which try to detect fraud--conceivably fraudsters could create AI systems to more cleverly hide fraud. Any of these efforts could lead to a "chain-reaction" of AI extension, which the author terms an explosion.

[Houshalter]

That is the premise of the Intelligence Explosion hypothesis: http://intelligence.org/ie-faq/

It's likely that once we get to the first reasonably powerful AI, it will only be a short time before it is able to improve itself and go significantly beyond human level.