[austerity]

Good article. However I don't understand why so many smart people fear decision-making machines. In my experience the decision makers in bureaucracies, government and private alike, never ever show the "humanity" ascribed to them in arguments about this issue. Instead they follow their instruction (which is actually a program written in human language) to the letter and are usually so obtuse and unempathetic it seems even the present day primitive AI would be much smarter and more flexible. The reason of course is not that people are dumb or mean, but that they are driven by incentives and incentives in these positions are never right. Machines on the other hand are not afraid to lose their job and are (potentially) capable of much deeper and broader analysis of any situation than humans while simultaneously being less prone to error.

[kijin]

It is difficult for a machine to be accountable for its decisions. It's not as simple as holding the manufacturer responsible, either.

With human decision-makers, at least you have a bunch of douchebags that you can point your fingers at and attach epithets like "obtuse and unempathetic". And we can hope that if we collectively do something, we'll be able to replace them with less dysfunctional folks.

With complex decision-making machines, it's not even clear whether a subtle bias it exhibits in its decisions is a sign of a bug or intentional design. The machine, of course, is capable of deep and broad analysis, but only of data that some group of engineers decided would be appropriate for it to analyze, using algorithms that were probably hand-tweaked by another group of engineers with their own individual quirks and unconscious biases.

Even the idea that a machine can be "smart" or "flexible" is based on a particular definition of "smart" and "flexible" that other people might strongly disagree with. And yet the machine presents an image of perfect objectivity, and its complexity makes it nearly impossible for outsiders to figure out exactly who or what is responsible for the many assumptions that underlie its design.

Moreover, the possibility of losing one's job is prettty much the only thing that keeps human decision-makers accountable in this world. Take that away, and we've got a benevolent dictator at best and a mechanical tyrant at worst, with the exact same quirks and biases, only hidden better.

[qznc]

People do point at Facebooks news filtering algorithm, which is "obtuse and unempathetic". People do complain about Googles algorithms flagging stuff as spam in an "obtuse and unempathetic" way.

[finance-geek]

Very very true! This comment sums up my fear exactly. I'll take it further, not only losing one's job -- but the fear of litigation is also a big deterrent. Now imagine in court you just say...Google Tensor Flow made me do it (or Theano or whatever...) I

wonder what would happen, but given my experiences to date (call center rep: no, the systems says you didn't pay, so obviously you did not [despite the cancelled check.])

[sevensor]

The dream of decision-making machines has been with us for 70 years, since Wiener wrote Cybernetics in the '40s, but it keeps falling flat. Ultimately, computers remain a tool, no matter how sophisticated, and they will never be capable of reason. Tasked with implementing a policy, they will implement it deterministically, based only on the available input data, without judgment or context. You may implement a very complicated policy by training an artificial neural network, but that doesn't make it any less deterministic or provide it with powers of reason. It just means that you've succeeded in creating a policy you don't understand.

As for bureaucrats, like everyone I've had my share of bad experiences. But it's just not the case that they're uniformly bad. If you've ever had an airline agent find a creative way to get you on a flight, or a doctor prescribe medicine for a needed off-label use, or someone at the motor vehicle department decide you don't need to fill that form out again, you've seen human judgement and context in action. Even bad bureaucracies have good bureaucrats sometimes.

[parasubvert]

What is reason, beyond sensing, sense-making, logic, verification of facts/premises, and response/conclusion?

Computers absolutely will be (and are today!) capable of reason - but as with humans it will be limited and imperfect - just in different ways than humans. Having worked in the semantic web space for example, some logical inferences are computationally way too expensive to bother with, and a human will often be better. Other cases are better. Research on building reasoning systems for medical diagnosis , prescription and contra-indication are getting very sophisticated for example.

I suspect what computers will long be bad at are imagination, curiosity, ambition, emotion, and intuition... All of which relate to your examples of weaving through the bureaucracy.