[kybernetikos]

There's a difference between the utility of a world where one person has a mote of dust in their eye and the utility of a world where one person has a mote of dust in their eye because it saves someone else.

>At some point, you need to accept a huge jump in the number of people getting hurt in return for a tiny decrease in the amount of hurt for each one, where the decrease can be pretty much arbitrarily small and the jump can be arbitrarily large.

Well, the boundary is fuzzy, so I believe that different people will draw the line in different places, but yes, that is correct. There is a point at which I accept a huge jump in the number of people getting hurt in return for a tiny decrease in the amount of hurt for each one, and that point is the point at which I determine that the hurt falls under the threshold I would expect every person to be prepared to sacrifice for any other person.

I think the debatable range is actually quite large, so the tiny decrease part might not be fair (there are a lot of degrees of pain I would not demand someone to suffer to save others), but at that point of expected sacrifice, the number of people jumps to infinite.

[zyxley]

> and that point is the point at which I determine that the hurt falls under the threshold I would expect every person to be prepared to sacrifice for any other person.

That seems like a good way to put it.

The number of people doesn't really matter, when it the amount of hurt per person is so small that you can say with all confidence something like "I believe that literally any sane person would agree to get one speck in their eye to save a stranger from fifty years of torture".

[ikeboy]

So first of all, your preferences are still circular, unless you bite another bullet somewhere. But even your argument isn't quite accurate. It's not this one person who needs to get a speck, it's a literally unimaginable amount of people.

As it happens to be, a number of people in our tiny world have said they would choose torture, so your argument fails just considering them.

[kybernetikos]

> But even your argument isn't quite accurate. It's not this one person who needs to get a speck, it's a literally unimaginable amount of people.

Nearly all of whom prefer to receive the speck than to allow the individual to be tortured.

> As it happens to be, a number of people in our tiny world have said they would choose torture, so your argument fails just considering them.

Yes, I feel somewhat uncomfortable ignoring the agency of torturers and murderers in this scenario, and that reluctance would play into a very conservative estimate of where that threshold should be, but I would be reluctant to choose a lowest common denominator measure to establish morality.

[ikeboy]

So your intuition says that there's some point where everyone is suffering pain X, where X is very large, and they would each agree to cause the X-(1 mote speck) to a trillion times as many people, in return for reducing their own suffering by 1 mote speck? That strikes me as beyond regular selfishness, and non intuitive.

[kybernetikos]

Can you explain more clearly why what I said implies that, because I don't think I mean to say that.

I believe that there is an amount of suffering that it is reasonable to expect anyone to accept in order to help another person. That amount depends on the amount to be suffered, and the amount benefited by the recipient. Once the suffering falls under that threshold, I do not believe the number of people required to make the sacrifice comes into consideration, as each of them if reasonable would say "I prefer to belong to this world, where as part of a huge group I accept this small ill in order that someone else benefits". Therefore, the implied sacrifice results in greater utility for that choice.

Let me try a different tack.

Let's say that you observe a universe with some large number of people suffering dust specks in their eye. That sounds bad. But what if every single one of those people actually suffering thinks that this universe is better than the alternatives. You don't suffer from a dust spec, but are you going to ignore all those people in their estimation of the utility of the universe? If you switched to a universe where all of those people didn't suffer from dust specs, but someone else was suffering, they would tell you that that was a worse universe.

It's pretty obvious to me that even if that isn't the exact case, it's close to being the case in reality - that's why people find the dust speck argument to be unintuitive, not the large numbers thing. It's because some measure of sacrifice for other people is part of what we expect from everyone, and most people know instinctively that if everyone asked to make a sacrifice agrees that it's right to make that sacrifice, then the world is better because of it.

[ikeboy]

This is not really a good format for continuing this discussion, because lack of notifications and such. Would you consider opening an account on lesswrong and posting in open thread? Or you could PM my account there at http://lesswrong.com/user/ike/overview/. That said, here's my reply.

>Can you explain more clearly why what I said implies that, because I don't think I mean to say that.

It's basically a reformulation of the circularity argument.

I assume there's some level of pain that you would prefer to the specks; say a single second of torture, equivalent to a smack or such. (If you think we should prefer 3|||3 specks to one smack, I could go further, so let me know.)

So counting up from that one second at a time (i.e. 2 seconds of pain, 3 second, etc), eventually we reach a point where you no longer think it's better than the specks. Call this X.

So X and X-epsilon are qualitatively different; the lower amount is not bad enough to outweigh specks, but the higher amount is. You need to prefer giving X-epsilon to a large number of people rather than X to a single one, if the qualitative difference is to be upheld.

(I may not be phrasing this so well. Maybe try working through the circularity argument above, or the other phrasings I used in this thread.)

Now to respond to your line of reasoning: this proves too much.

Imagine instead of dust specks, we want everyone to donate a dollar to save the person from torture. Are you really going to say that we should be spending unbounded amounts of money (3|||3) to save anyone from torture? Have you donated all the money you could get to prevent torture? (and yes, I'm sure there are charities that are at least partially effective.)

Why doesn't your argument work for the case I just outlined as well?

[kybernetikos]

> I assume there's some level of pain that you would prefer to the specks; say a single second of torture, equivalent to a smack or such.

Quite possibly I prefer a world with one person tortured for a lifetime compared to 3|||3 specks if they are evaluated out of context. It's hard to say, because pain doesn't easily sum, there are different qualities of pain, and we are talking about situations where we may be losing an entire persons contribution to humanity. I just don't think any of this is relevant to the conversation, or to the reason that so many people find your conclusion unpalatable - and it's nothing to do with not understanding large numbers.

In the case that they are evaluated in the context of a choice between one of those worlds or the other world, I would take into account what I believe to be the value that the individuals involved would place on the worlds were they to know the details and have minimal moral standards like mine.

Let me phrase it another way:

What would you say the utility of a world where there are 3|||3 people with specks who chose it gladly and voluntarily to save someone from torture is?

Let's say you tell those 3|||3 who wanted to save someone from torture by accepting a speck in their eye that they cannot, and someone must be tortured instead. You've massively increased the unhappiness in the world - not only is an individual getting tortured, but 3|||3 have ended up with a situation that's worse than they wanted. Are you going to claim that it's still got a higher utility? Now that you notice that you're making those 3|||3 unhappy by the choice, you can see that the disutility scales with the number of people - that's why the number of people becomes irrelevant.

> Imagine instead of dust specks, we want everyone to donate a dollar to save the person from torture. Are you really going to say that we should be spending unbounded amounts of money (3|||3) to save anyone from torture?

Your phrasing is unnecessarily emotive here. You seem to be saying that 3|||3 dollars is an awful lot of dollars without giving me any context about what fraction of the dollars belonging to those 3|||3 people those 3|||3 dollars are or what else they could/should be spending it on. If it's a negligible fraction that scales, and I could plausibly think that any sane person should donate that fraction of their money to save a person from torture then yes. You'll notice that when it's phrased like that it does not require that I donate all the money I could get to prevent torture. I in fact do donate a small amount of money regularly to prevent torture, but I would demand much less of the 3|||3.

[ikeboy]

>Let's say you tell those 3|||3 who wanted to save someone from torture by accepting a speck in their eye that they cannot, and someone must be tortured instead. You've massively increased the unhappiness in the world - not only is an individual getting tortured, but 3|||3 have ended up with a situation that's worse than they wanted. Are you going to claim that it's still got a higher utility? Now that you notice that you're making those 3|||3 unhappy by the choice, you can see that the disutility scales with the number of people - that's why the number of people becomes irrelevant

If the people are told of the choice, that's a whole new problem, but that's kind of avoiding the point of the original question. To use a hacking analogy, you're using a side-channel to cheat.

Nobody is told about any of this. If they were, that would itself need to be factored in, and quite possibly lead me to prefer specks.

>If it's a negligible fraction that scales, and I could plausibly think that any sane person should donate that fraction of their money to save a person from torture then yes.

Each person isn't donating to save someone from torture, they're donating to save 1/3|||3 of torture.

Let's rephrase the original question to zero in on that last point. There are 3|||3 people. You choose the number of people who donate one dollar, which can be any number between 0 and 3|||3. After you make a decision, one of those people is chosen at random, and they are tortured iff they did not donate.

If you think about it, this results in the exact same outcomes in either choice, except you have more options than all or nothing. You basically choose the probability of torture.

To be consistent with your previous view, you'd need to not pick zero donaters. So for at least some people, it should be worth it for them to pay 1 dollar to avoid a 1/3|||3 probability of torture.

This dissolves, as you may have noticed, into Pascal's Wager. (Or Pascal's Mugging, more precisely, which was coined by Yudkowsky.) What if I tell you that unless you give me a dollar, I'm going to torture you for 50 years? The probability of that being true is more than 1/3|||3 (and if you disagree, then you are way too overconfident for life. 3|||3 is a huge number, and there's no conclusions I can think of in which I'd place that much confidence. We don't even have anywhere near the kind of raw data to draw any conclusion that confident.) So are you willing to give up a dollar to avoid the >1/3|||3 chance that I'm telling the truth?

If not, and all or most of those people in the problem would answer the same, then your previous rationale falls apart.

(Oh, and this is not the real Pascal's Mugging; that's much harder to deal with. But let's stick with the easy stuff for now, shall we?)

Also, what about the argument in http://lesswrong.com/lw/kn/torture_vs_dust_specks/ueo

(And if we're going to continue this, could you please move it over to another forum? It's getting harder to keep track of comments here.)

[kybernetikos]

> Nobody is told about any of this. If they were, that would itself need to be factored in, and quite possibly lead me to prefer specks.

Exactly. My whole contention is that the reason this question is considered unintuitive by so many people is that they're really considering a different question to the one you think you're asking.