| This sounds like a terrible rewrite. If I can think of places they fail, surely real life can come up with more. Essentially, this would lead to either: a) The tyranny of the majority, or b) The tyranny of those with strong preferences. Let's say 51% of a population wants to get rid of the other 49% and wishes they were dead. What would stop the machine making it happen? Let's say a super-person is born, who just wants stuff more than anyone else alive. Maybe he's had a bad childhood and been through terrible things, so now those things that he wants reach the level of desperation for him. Wouldn't this make the worth of his values more than other people to the machines? Finally: > Still, Russell feels optimistic. Although more algorithms and game theory research are needed, he said his gut feeling is that harmful preferences could be successfully down-weighted by programmers ... so we're back to square one. We've created a learning machine that can come up with its own morality. But... we're going to have to down-weight certain behaviours just to be sure. Doesn't that simply recurse to: a) Down-weighting removes the learning, and b) We have to trust ourselves to correctly program and define the down-weighting. |
I’ve blogged about this recently.
Morality, thy discount is hyperbolic: https://kitsunesoftware.wordpress.com/2020/01/08/morality-th...
Normalised, n-dimensional, utility monster: https://kitsunesoftware.wordpress.com/2018/01/21/normalised-...
Disclaimer: although I do have a formal qualification in philosophy, I did not get a very good grade.