[dictum]

> This isn't technology, it's politicians and bureaucrats deciding policies.

The underlying argument of critics of algorithms that make decisions, in government and in private companies, is that this technological apparatus becomes a justification in itself: if a mistake is made, blame the algorithm; if the algorithm says so, it must be right — we can't make exceptions.

Over time, people and institutions begin to shape their own actions so they don't fall afoul of the criteria that can be reverse-engineered out of a black box decision-making system.

Regarding "this isn't technology", this doesn't invalidate the criticisms: part of the argument is that, indeed, what gets called "technology" often shouldn't be simply described as technical tools and processes. They're often the opinions and goals of individuals (politicians, bureaucrats, and other wielders of power) transformed into code, data and other technical artifacts; sometimes this transformation merely hides the underlying sources of these opinions and goals.