[derefr]

But that's a question for neuropsychology, isn't it? The question of what is good for an individual person won't be solved by conversation and introspection, because people are, on average, very bad at understanding themselves and the reasons for doing the things they do (they indeed have reasons to give, but these reasons rarely stand up as falsifiable hypotheses for predicting future actions, so they're mere rationalizations and should be discarded.)

Instead, the question of what an individual cares about will be solved by coming up with a technique to look at a person's brain and tell them, definitively, what values they care about at that moment in time. Anything said about individual ethical belief until then is just sophistry.

[ntoshev]

The answer may come from other places than neuropsychology. We as carriers of values live and evolve in an ecosystem where the values of those around us and how they fit with ours matters. Maybe we can derive some evolutionary stable strategies representing moral laws from a swarm of agents constantly playing Prisoner's dillema and games with other rewards against each other.

[derefr]

But the mechanisms by which we play those games will be evident in the brain, and the games themselves will be evident in our expectation-program memory. (I'm assuming here that sufficiently-advanced neuropsychology will be able to analyze the "software" running on each brain, not just the firmware, but even if that field ends up with a different name, that's what I'm talking about here.)

We may be able to model some sort of "objectively-good" cooperative-evolutionary game-players using mathematics, but those models would not necessarily represent us; there's nothing that says we're even evolutionarily stable as a species over the long term ;)