[cduan]

It's never clear to me why people answer thought experiments like this in this manner. The point of the question "would you kill someone for a trillion dollars" is to tease out an answer to an underlying philosophical question, such as, "is the act of killing as we know it a moral absolute wrong, or is it merely an act with great negative utility that can be compensated by sufficient positive utility?"

Yes, there are always creative ways to avoid really answering the question (such as doing a brain upload, as suggested), but these answers do nothing to answer the underlying question, which is what the thought experiment is really trying to get at.

[derefr]

And I've always ignored that sort of question—there's no such thing as morals or utility, really, only what animals are programmed to think in their animal brains. Robotic arms don't care when other robotic arms die—and neither do we. Paperclips don't care about increasing in number, even if someone is trying to make it so. What we have to ask is not "what does the universe say about right and wrong?" (because the answer is "nothing") but rather "what do our minds say about right and wrong?"—that is, "what is the human utility function?"

The problem with that, of course, is that even within our species, we have many different (and mutually-exclusive!) utility functions; sociopaths, for example, calculate theirs noticeably differently. So, it still ends up turned into a problem of cultural meta-ethics. That is, it's no longer a matter of "who do we shun and revile?" but "how do we get along?" or perhaps "do we want to get along?" (Which brings me to this: http://lesswrong.com/lw/y4/three_worlds_collide_08/)

[cduan]

That's not really a reason to avoid answering the question either. Regardless of the mechanism by which your mind arrives at the answer (be it animal instincts, brain chemistry, whatever), you are indeed capable of coming up with an answer, and even if you believe the answer says nothing about the universe in general, it still says something about you and your own mind. And I'm fairly interested in knowing what your mind--and the minds of other people--have to say about these sorts of questions.

To be a little more clear, your answer (yes or no) is of little value to me. What I really want to know is why you answer yes or no--is it because of some general principle you're applying, because of a gut instinct, because someone told you to say that, or something else?

EDIT: Based on the reply below, I'm not being clear. There are lots of reasons I'd like to know your reasoning process, beyond trying to generalize it to people in general. Among other things, I'd like to know whether I should be worried about going to dinner with you (particularly if your answer is "Sure, I'd kill for even just a dollar").

[derefr]

But that's a question for neuropsychology, isn't it? The question of what is good for an individual person won't be solved by conversation and introspection, because people are, on average, very bad at understanding themselves and the reasons for doing the things they do (they indeed have reasons to give, but these reasons rarely stand up as falsifiable hypotheses for predicting future actions, so they're mere rationalizations and should be discarded.)

Instead, the question of what an individual cares about will be solved by coming up with a technique to look at a person's brain and tell them, definitively, what values they care about at that moment in time. Anything said about individual ethical belief until then is just sophistry.

[ntoshev]

The answer may come from other places than neuropsychology. We as carriers of values live and evolve in an ecosystem where the values of those around us and how they fit with ours matters. Maybe we can derive some evolutionary stable strategies representing moral laws from a swarm of agents constantly playing Prisoner's dillema and games with other rewards against each other.

[derefr]

But the mechanisms by which we play those games will be evident in the brain, and the games themselves will be evident in our expectation-program memory. (I'm assuming here that sufficiently-advanced neuropsychology will be able to analyze the "software" running on each brain, not just the firmware, but even if that field ends up with a different name, that's what I'm talking about here.)

We may be able to model some sort of "objectively-good" cooperative-evolutionary game-players using mathematics, but those models would not necessarily represent us; there's nothing that says we're even evolutionarily stable as a species over the long term ;)

[Dunearhp]

Why does someone need a reason not to answer a silly question.

[henrikschroder]

there's no such thing as morals or utility, really

Just to counterpoint the frenzy of moral nihilism and relativism I suspect will appear, I respectfully disagree, I'm a moral universalist, i.e. morals do exist, moral judgements can be true or false.

But I also realize that this is a question of philosophical faith, and arguing about that is usually pretty futile. :)

[derefr]

A few questions:

1. Given the laws of physics, could you derive your morality? 2. If we didn't exist, and in our place were an alien species that, say, ate their babies[1], would that make the universe contain less utility as a whole?

[1] The same story I linked to above: http://lesswrong.com/lw/y4/three_worlds_collide_08/. Who is in the right? Do we have the "moral imperative" to destroy the baby-eaters? Do the third species have a moral imperative to destroy us?

[henrikschroder]

1) No, the only way to get my morality is by asking me. As for whether or not a specific moral judgement is true or not, no, you can't derive that from the laws of physics either. What an incredibly boring universe that would be if that was the case.

2) You're asking me to sum up and compare the values of two complete civilizations? That's a pretty tall order. Also, in one of the alternatives I wouldn't exist, that's a lot less utility right there.

[derefr]

You just said you were a moral universalist—that goes against the concept of there being a "your" morality. Either morals are universal (in which case, you don't need a specific living being to ask in order to figure them out), or they're, at least in some part, subjective. Further, you seem to be explicitly arguing against yourself with the "in one of the alternatives I wouldn't exist" statement—what system of universal morals would privilege your existence over the existence of someone who is better in every way, except for not being you?

What I said above was that "morals or utility" are subjective to an individual—which agrees with your point. I said "and thus don't really exist" because the definitions for epistemic morality or universal utilitarianism require them to be universal—and they're not, so they don't exist as defined. (And a system that accepts subjective morality is usually just called "ethics", by the way.)

[btmorex]

For an individual you can make those proclamations, but individuals don't operate in a vacuum. There is such a thing as morals and utility in a society. There is value in participating in a society. Therefor, there is value is having individual morals and calculating the utility of actions at least on a societal level.

[derefr]

Right—that's, effectively, the difference between "cultural meta-ethics" and "moral relativism." What I'm saying is that, on an individual level, ethics is a problem that will be completely solved by neuropsychology: once we find out our particular utility function, we just have obey it optimally. Thus, Ethics as a field of endeavor should drop that kind of individual-level moral quandary, and focus on inter-societal quandaries, since that's what we'll really need to figure out—how to handle, and interact with, societies that have different utility functions than our own (including the ones we end up creating ourselves using genetic engineering, AI, and so forth.)

[billswift]

From the most recent post on LW http://lesswrong.com/lw/2aa/virtue_ethics_for_consequentiali...

Humans are not inherently expected utility maximizers, they're bounded agents with little capacity for reflection. Utility functions are great and all, but in the words of Zack M. Davis, "Humans don't have utility functions."

[jimbokun]

"once we find out our particular utility function, we just have obey it optimally."

Is this function computable?

[nzmsv]

But the morals of a society are not constant. They are a function of the morals of the individuals living in a society. What was perfectly acceptable in the 12th century is barbaric now. In a century or two we will seem barbarians to our descendants. (And they will still be saying that their generation lost all morals and humans will soon die out.)

[bruceboughton]

I wonder for what reasons our society will be judged barbaric by our descendants.

[Shorel]

Irak, the SUV vehicles, the bullying in schools, people parasiting the welfare system, the predatory banking practices, all those things seem barbaric to me.

[bruceboughton]

Iraq: seems short-sighted to me. We don't tend to judge past civilisations based on one war.

SUVs: maybe, but I would say our continued reliance on oil in general rather than SUVs in particular.

Bullying: could be.

Parasites on the welfare system: not convinced. The fact we have a welfare system will be judged favourably I expect.

Predatory banking practices: pass, not sure.

It's hard to imagine what things it will be--their values may be very different from ours. I'd say oil reliance, over population, the gulf between the rich and the impoverished, but these things are all too obvious. I suspect we'd be surprised by what we're judged badly for.

[olliesaunders]

I came to the same conclusion a while ago. I like to phrase it as: morality is a function of evolution.

[JoeAltmaier]

My old minister said it well: "We have all the morals we can afford"

[xtho]

The example in the article is a good example how such thought experiments are sometimes turned into jokes for bored misanthrope/misogynist/latent homosexual men. At first, the situation could be interpreted as an exchange of a years of happy life for sleeping with somebody she considers attractive and would probably sleep with anyway. This problem then is turned into what would be her price if she slept with men for a living.

If this is your understanding of philosophy, I'd say you're misusing philosophy for dealing with personal problems.