[singham]

Consider the following case.

Let's say we ban development of self-driving cars since the algos can't be explained. Then we will never know the benefit we might reap from adopting them. We know humans make mistake. But with self-driving cars, the error rates in future might be very low as compared to human. We as a human society will never experience this future because we had this silly idealism that all algos must explain themselves in human terms.

[gaius]

Let's say we ban development of self-driving cars since the algos can't be explained.

That's a false dilemma since it's by no means proven that the algos CAN'T be explained - just that we don't know how to extract the explanation from the model yet. It's entirely reasonable to suspend real-world use until the maths catches up.

[singham]

But we would never reap the benefits in the meantime. That was my main point. And this becomes crucial if the saving of deaths from accident is substantial as compared to deaths that happen due to AI errors.