[JohnStrangeII]

I'm working in formal ethics, mostly on formal aspects of value structure, and agree with you. This article is a hodgepodge of many different ideas, some of them interesting, others a bit naive, and has almost nothing to do with formal models for ethics. There is plenty of research in formal ethics such as deontic logics, abstract argumentation frameworks for preferences and norms, nonmonotonic logics for value-based reasoning, normative systems, input/output logics and formal axiology.

One thing that the author does not address in enough detail is the simple fact that there are many different ethical traditions that come to different conclusions about particular normative issues, and that there are plenty of authors in ethics who (still) consider their business a normative one. A "bottom up" machine-learning based approach to this would invariably fail and miss the whole point. There are some ethicists who consider it mostly a descriptive endeavor - Schopenhauer was one of the first, for example -, but they are in a minority.

As long as experts in ethics cannot agree what the "right" ethics is, it's hard to see how we would be able to teach it to machines. Many meta-ethicists including me would even deny that there can be an "expert" about moral and particularly about ethical questions at all. However, I have no doubts that various robotics companies will implement those ethical rules and approaches that best serves their interests as companies.

That's why I think robot ethics is kind of misguided. What we need is laws that regulate AI and put its use into a legal framework and closes loopholes. This is a political, not just an ethical issue.

[renee_sanscarte]

Hello, I saw the pingback for this thread today. I'm surprised (and a bit giddy) that my obscure little blog wound up on HN, so I hope you don't mind my coming in a little late.

The purpose of this post was more exploratory, rather than for the sake of drawing specific conclusions, which I am aware can be frustrating. Thanks for taking the time to read it anyway.

You are correct that I was not very interested in formalizing ethics. I'm not so sure why OP chose to use that to title the thread, because only a part of the post was oriented towards overviewing the AI safety discourse, and within that discourse, the development of a complete formal system for ethical reasoning is considered a naive way to attack the problem. I never wrote about the formalization angle directly.

The problem, such as it is, is that it's troublesome to encode what we want of an AI agent, even more so to encode how it should reason about what we want of it under unforeseen circumstances, no matter what it is that we want. Ultimately, in order to successfully implement regulation, an agent must be "taught" (in whatever sense) to make judgments according to what is wanted of it. Technically speaking, this is a hard problem. However, there does not necessarily need to be an absolute answer as to what the correct ethics is, in order to teach a machine methods of reasoning about ethical problems, nor for us to reason about what sorts of imperatives a machine should be reasoning about.

The thrust of that section, however, was to raise an issue with the (in my opinion) narrow way in which that problem is typically formulated, to which you have also alluded. Specifically, my beef was ontological (which, for the sake of communication, was distilled into the bottom-up/top-down distinction), but there are conceivably other gaps.

This could have been elaborated; ontological approach is one of the ways in which ethical traditions are distinguished, but far from the only one. Any distinguishing factor may have been missed by an approach like IAD.

> That's why I think robot ethics is kind of misguided. What we need is laws that regulate AI and put its use into a legal framework and closes loopholes. This is a political, not just an ethical issue.

This much I agree with wholeheartedly. I made the case that we shouldn't shaft the ethics for the politics, but the same goes for the reverse. Where how we consider ethical problems comes into play again in the second case, is that putting the use of AI into a legal framework and "closing loopholes" has roughly the same shape as the problem of AI ethics, it just chooses potentially different parties to determine what it is we want of the agents.

Anyway, thanks again! If you have a moment, I'd be interested in reading what you thought was naive in my approach.

[JohnStrangeII]

Sorry, I've seen your post only now, so my reply is a bit late. I think we mostly agree, but it appears to me that you might not be fully aware of the magnitude of the metaethical problem and persistent disagreement about them. To give you an example from formal ethics, according to Temkin's Spectrum arguments strict "better than" comparisons are not transitive. Some authors agree with him, others disagree. Some want to give up completeness instead of transitivity, others opt for lexicographic value hierarchies. Others deny having the intuitions and argue for the status quo. Even this one simple issue has far-reaching normative consequences, though. If Temkin is right, then even if we all agreed to be classical utilitarians, the position would be infeasible and any account based on utility functions would be wrong from the start.

If at all, we could try to implement conforming with systems of laws as hard constraints, so at least robots would not openly break the law. The rest of moral behavior could then be learned. Or, so one might think. However, even that is not possible. It is well-known from the philosophy of law that systems of law are not contradiction free. There are conflicts between opposing laws. This is studied in normative systems research and there are solution to it (essentially, by logicians in the computer science tradition). However, these require some form of defeasible rules, and among the myriad of nonmonotonic logics that can express some form of nonmonotonic reasoning, not a single one has a normative justification. So again, as long as human standards are not coherent enough, it's going to be impossible to make a machine conform to them in a way that is satisfying.

Related to that, I believe there are two main issues that you haven't addressed in your post:

1. Different standards for blame and accountability: The standards for blame and correctness of decision making are completely different between machines and humans. Even very intelligent AI would be judged at much higher standards as humans. I simply don't believe that we would accept robots that commit murder, just as long as they commit murder less often than humans. By the same token, we do expect machines to make substantially less errors, and in certain areas would not allow errors at all. So it's not just about teaching them to follow human ethical standards - they need to excel at this and may not break the law. However, as I tried to show with the above examples, we don't know and agree on our own standards well enough to be able to start making sure AI fulfills these high expectations. There is also a much higher need for transparency of decision making for machines than from humans. An idiot driver is an idiot driver. If a car is driving like an idiot by itself, however, you'd expect to be able to at least retrospectively find out why it did what it did.

2. The political dimension: Laws are the result of a political process. Appointed judges resolve potential conflicts between laws and interpret them. More broadly, whatever standards we expect AIs to fulfill hinges on ethical positions and personal preferences. It can ultimately only be decided as the outcome of a political process as well. For example, a safe a self-driving car needs to be, how it makes decisions (e.g. "Should I try to save my driver or save the pedestrian, or not have any priority at all?"), and which standards it needs to fulfill is up for debate. That is not decided by moral philosophers. It needs to be decided by the publicly accountable, democratically elected representatives of the people. The idea of teaching a machine to behave morally is fine, but there are also strict standards of safety and transparency it has to fulfill. This is a political problem and can only be solved in a broader context, within the debate of how much AI should be allowed at all. For example, should AI judge your creditworthiness? If so, what false positive rate would be tolerate? I believe using AI for such purposes should be strictly prohibited. Others disagree - I'm sure AI is already used for that. Such issues can only be resolved politically.

I considered your post a bit naive, because you omitted these two crucial issues, the higher standards we expect from AI, and the political dimension. Other than that, I agree with much of what you've said.