[cafebeen]

I think this is a major challenge too, although it depends. Some models are easy to reason about, e.g. Bayesian graphical models, while black-box approaches like deep neural networks are not.

One especially problematic issue is: if a model is too complex for humans to reason about, then a business could encode any kind of illicit behavior in the form of model parameters they like. Even if someone could prove that the model is biased one way or another, there is complete plausible deniability for the business, i.e. "I didn't make that choice, the learning algorithm did". We're in for some very interesting legal battles related to this, I think.

[hx87]

> if a model is too complex for humans to reason about, then a business could encode any kind of illicit behavior in the form of model parameters they like

If a model is too complex for humans to reason about, how would a business encode illicit behavior, unless the AI itself was running a significant portion of the business? Even in that case, someone or some group is responsible for setting the initial parameters of the model, and they can be held responsible for its decisions.

In the end, I think the legal solution would be put less emphasis on mens rea and more on actus reus. In other words, if your AI does something wrong, you are in the wrong, regardless of your intentions.

[joe_the_user]

In the end, I think the legal solution would be put less emphasis on mens rea and more on actus reus. In other words, if your AI does something wrong, you are in the wrong, regardless of your intentions.

But that would be a large change in the way the legal system works. Fewer people than you'd expect understand that today, a large portion of crimes, if not most crimes, require both intentions and action for the legal system to judge guilt.

"A fundamental principle of Criminal Law is that a crime consists of both a mental and a physical element. Mens rea, a person's awareness of the fact that his or her conduct is criminal, is the mental element, and actus reus, the act itself, is the physical element."

http://legal-dictionary.thefreedictionary.com/mens+rea

[cafebeen]

I think there are many companies currently or on their way to using AI to run a significant portion of their business, e.g. financial and advertising sectors. The hard part is demonstrating intention in that scenario, as many learning algorithms are stochastic. Even if there is some group to be held responsible and they give you the training data, the model will slightly vary when run repeatedly. It seems likely that people could use that fact for things like money laundering and insider trading, for example, using biases in the model that are hard to detect.