[pyrois]

He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true.

To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort

And I don't agree that you can. The difference between dust-mote and torture is not one of degree, but one of _kind_. It's a discontinuous function (in my opinion). I don't know where the discontinuity is, but it's there.

[ikeboy]

>It's a discontinuous function (in my opinion).

There are a finite number of possible brain states, far less than the 3||3 number. All possible "feelings" is likewise less than that.

You need to be dividing all possible feelings of pain into two groups, and asserting that the absolute worst of the first group (containing dust specks) is incommensurate with the absolute best of the second group (containing torture).

At some point you need to say that you'd pick specks even over 1 minute of torture, even over 1 second, on which I think most people's intuitions would stop saying that. Or you need to find some point between 50 years and 1 second where it become commensurate.

[pyrois]

Sure, they are commensurate at some point. I'd pick one nanosecond of torture over the dust-motes, for instance. But I'm not certain that I would ever choose the 50 years of torture over the dust motes for any number of dust-moted people. That's because it's not continuous, so you can't do the sort of epsilon-delta proofs that Yudkowsky's argument depends on.

At least, in my opinion.

[ikeboy]

That means that for some amount of time, say X seconds, you would prefer X-1 seconds of torture for one person over specks, but prefer specks over X seconds of torture for one person.

But let's double each side; presumably you would make the same decision if asked again, right? So now you prefer X-1 seconds of torture done to each of two people, over twice as much specks.

Now, unless X is very low, you should prefer X for a single person over X-1 for two people. (If you disagree with this, please give a plausible value for X that makes it false.)

So you prefer X on a single person over double!specks, but prefer single!specks over X. This seems extremely unlikely. Or even if true, we should be able to make you pick torture for 50 years just by multiplying specks another couple of orders of magnitude.

Does this make his argument any clearer?

[kybernetikos]

We're social animals. There's an amount of discomfort (maybe not large) I would be prepared to undergo to help someone else, and I find myself thinking that others "should" also be prepared to undergo such an amount of discomfort to assist others. Given the choice to accept a dust mote temporarily in my eye as part of a huge crowd in order to save another person from torture, I would gladly accept that, and I think all reasonable people would too, therefore there is no number of people such that I believe the utility of dustmote vs torture turns out in favour of torture.

[philh]

> I would gladly accept that, and I think all reasonable people would too

Hello, apparently I'm unreasonable. And so is everyone else who has said that they choose torture over dust specks. If you select 3^^^3 people, you're going to find an awful lot of us. (And also some sociopaths who literally don't care if someone else gets tortured.)

Your argument seems to boil down to "specks is the correct answer, so anyone who gets it wrong doesn't count; and because we all agree that specks is the correct answer, it's okay to do specks".

On the other hand, I would totally accept the torture for myself, if it would prevent the specks. (At least I hope I would, and to the extent that I can model how I would act in that situation, it does seem plausible that I would.)

[ikeboy]

You aren't quite grokking the difference between "huge crowd" and 3||3. How do you deal with the argument from circularity? Are your preferences circular, and if not, which part do you reject?

[kybernetikos]

The size of the numbers is irrelevant. I personally would consider a world in which I had no mote of dust and someone else was tortured as a world with less utility than a world in which I had a mote of dust in my eye. I expect every single one of those 3||3 individuals to feel the same way. Therefore, there is no number of individuals that I would choose not to suffer the dust mote in preference to torturing someone.

The answer to the argument from circularity is obvious - there is a discontinuity. I can see the discontinuity in my own thinking, and you probably can too. I would accept a dust mote to save an individual from years of torture (and I think almost all reasonable people would), but I would not accept a dust mote to save two people from a dust mote. That may be unethical of me, since I would of course prefer the world that has fewer people with dust motes, but for me to expect the greater number of people to make the sacrifice, I must be prepared to make the sacrifice myself and it must be a sacrifice that I think all people should make. Exactly how much I think people should sacrifice for others is difficult to say, but there is a clear step change at some point.

[ikeboy]

I don't see how that's relevant to answering the circularity argument. How are you avoiding the claim that your preferences are inconsistent? At some point, you need to accept a huge jump in the number of people getting hurt in return for a tiny decrease in the amount of hurt for each one, where the decrease can be pretty much arbitrarily small and the jump can be arbitrarily large.

(Also, if you click on the comment you can reply).

[kybernetikos]

There's a difference between the utility of a world where one person has a mote of dust in their eye and the utility of a world where one person has a mote of dust in their eye because it saves someone else.

>At some point, you need to accept a huge jump in the number of people getting hurt in return for a tiny decrease in the amount of hurt for each one, where the decrease can be pretty much arbitrarily small and the jump can be arbitrarily large.

Well, the boundary is fuzzy, so I believe that different people will draw the line in different places, but yes, that is correct. There is a point at which I accept a huge jump in the number of people getting hurt in return for a tiny decrease in the amount of hurt for each one, and that point is the point at which I determine that the hurt falls under the threshold I would expect every person to be prepared to sacrifice for any other person.

I think the debatable range is actually quite large, so the tiny decrease part might not be fair (there are a lot of degrees of pain I would not demand someone to suffer to save others), but at that point of expected sacrifice, the number of people jumps to infinite.

[zyxley]

> and that point is the point at which I determine that the hurt falls under the threshold I would expect every person to be prepared to sacrifice for any other person.

That seems like a good way to put it.

The number of people doesn't really matter, when it the amount of hurt per person is so small that you can say with all confidence something like "I believe that literally any sane person would agree to get one speck in their eye to save a stranger from fifty years of torture".

[ikeboy]

So your intuition says that there's some point where everyone is suffering pain X, where X is very large, and they would each agree to cause the X-(1 mote speck) to a trillion times as many people, in return for reducing their own suffering by 1 mote speck? That strikes me as beyond regular selfishness, and non intuitive.

[ikeboy]

We're not talking about you accepting a dust mite in any of the scenarios. It's always other people.

So saying that you are selfish doesn't get you out of this, as long as you do prefer the world with less other people having specks.

Where is there a discontinuity when talking about other people being tortured or specked?

[kybernetikos]

I know that that is the formulation of the question, But since you're asking me to make the choice, you're asking me to inflict suffering on many extra people who would not have suffered otherwise. If what is gained by their suffering is large enough compared to that suffering that I believe they all (or nearly all) would have chosen voluntarily to accept the suffering for the benefit, then I am happy with the choice, and my best model for that is what suffering I would be prepared to undergo for what benefit. I think the confusion is between evaluating the utility between two possible alternatives (where obviously the world with fewer people suffering is better) vs evaluating the utility between two possible alternatives where those were the only two alternatives. If it's just two of many alternatives, the utility placed on a meaningful sacrifice doesn't come into it, but if it's a choice, the utility placed on a meaningful sacrifice does.

[ikeboy]

It doesn't need to be linear, just not slowing super-exponentially. At almost any slowdown rate, specks will still end up worse than torture.

[zyxley]

But that's assuming that unpleasantness per individual is a continuous function always distinguishable from normal human experience, rather than something like a line eventually rising from the discontinuous murky soup that contains all the constant minor annoyances of being human (itchy nose, wedgies, the feeling of the back of your tongue against the roof of your mouth, one shoe being a little tighter than the other, etc) that the brain is well-optimized to tune out and rapidly forget about after the fact.

[ikeboy]

Even if it's discontinuous, it still needs to grow incredibly slowly. The ratio of pain neurons that fire for a speck versus for torture over 50 years is nowhere near 3||3. You need a function that grows so slow that outweighs 3||3, which is damn near impossible for any plausible function.

[zyxley]

I care about impact on people, not neuronal activity, and the point I'm making is that there's a minimum below which the real impact on a given person, considered in aggregate with the impact of everything else that's a part of being human, is effectively nil.

[ikeboy]

So at some point you need to say that "person being hurt X" is fine, but "person being hurt X+ one speck" is not.