[TheOtherHobbes]

Well, yes. That's the point. This form of AI will automate existing prejudices and misunderstandings. We're not really dealing with AI in any useful sense, but with automated bad policy biases, and the products and systems this generates should be labelled as such.

Corollary - Big Data is political, but it can hide behind the pretence of objectivity.

This shouldn't surprise anyone, but for some reason it does - perhaps because we're used to thinking of AI as the automation of the scientific method, when in fact it's just as likely to be the automation of all kinds of other things, some of which are nasty, stupid, and wrong.

A hypothetical smarter-than-human AGI may be able to talk back and say "You're being stupid because you're not modelling the correct problem, and so your attempted solution is wrong."

But that assumes hypothetical smarter-than-human AGI is also more-ethical-than-human - which unfortunately might be a bit of a reach.

Bottom line - the limits of AI are set by the limits of human political intelligence. And since most human political systems operate at pre-scientific policy levels - the exception being the abuse of social and personal psychology to gain power - this isn't encouraging.