[Unkechaug]

> My guess is that some point there will be an antitrust case featuring the "algorithm defense," which basically says: "Hey, I just set up the smart learning algorithm and let it run. How could I know that it would interact with other smart learning algorithms in a way that led to collusion?" And the antitrust authorities (or other law enforcement) will need to argue that when a guy named Bob sets up and signs off on an algorithm, Bob needs to be personally responsible for what that algorithm does.

How do you guys think this is going to be handled? It seems very complicated, because how are we even going to know which algorithm is at fault, or even IF any can be conclusively determined to be at fault?

Furthermore, is he talking about Bob the implementer or Bob the approver? In many organizations the people actually writing the code are not the same people who determined the specs and asked for it. Bob the implementer may have been given a limited spec that is followed, unknowing there is potential that an outside influence can manipulate the results. At that point who holds the responsibility?

Maybe it's just me, but recently I have noticed increased discussion regarding ethics in tech. It seems like there is growing awareness that all the focus on "how" to do something has eclipsed the fundamental question of "if" we should do it. This isn't to say we should just shy away from learning algorithms and complex systems, but maybe we should be put more of an effort to discuss the outcomes and how we agree to handle them before charging full steam ahead into uncharted territory.

[dllthomas]

Liability should start with the people deploying the system. They're the ones who have the fullest picture of how it's to be used. Contracts should be able to move that around, to some degree, but must do so loudly so that everyone knows what they're getting into - not buried in the fine print in "the standard boilerplate".

[dogruck]

What about a more socially sensitive domain, such as college admissions, hiring, or setting pay? What if you put such an algorithm on said task -- to avoid human bias -- and later observe that the algo, say, does not hire <pick your group>?

[maksimum]

Instead of Institutional Review Boards approving experiments we could have Algorithm Validation Boards. Hold out a portion of your dataset for validation, and have the algorithm designers examine the outcomes in communication with ethicists, lawyers, and business strategists.

I don't think the two hands can properly "validate" algorithms without communicating. The algorithm designer can maximize AUC, but what if one <group>'s class is 95% label A; the designer always predicts A for <group>. How bad is ALWAYS missing 5% for label B? If you can put a price on it, then the developer can build it into the algorithm. But if the price is difficult to accurately estimate, or non-monetary qualities are desirable, it may be hard to build them into the classifier ahead of time. On the other hand if the cost of perfect <hard to quantify criterion> reduces AUC significantly, algorithm designers need to communicate that...

[andreareina]

That's already happening. Can't remember which article to link to, but machine learning algorithms make decisions that reflect pre-existing biases (e.g. harsher sentences for black people).

Humans produce the data that the ML algorithms train on, so without whitening the data somehow (and there's another contentious issue) we should expect the result to be biased as well.