[pdonis]

Even "two functions, in tension" still assumes that you can capture values as functions at all. But the reason ethics and morality are hard in the first place is that there are no such functions. We humans have multiple incommensurable, and sometimes incompatible, values that we can't capture with numbers. That means it's not even a matter of not being able to compute the "right" answer; it's that the very concept of there being a single "right" answer doesn't seem to work.

[at_a_remove]

I think that's what it will approach in the limit, yes, if you are talking about humans. For AIs, I think it will be somewhat less so, and that it would be preferable for the sake of predictability.