[Taek]

Are we sure that consciousness is "very scarce"? To define it as scarce, wouldn't we need to start with a definition? There are theories of consciousness that say rocks are more conscious than humans. Whether or not you want to take those theories seriously, they do highlight critical gaps in how we define consciousness.

[jvanderbot]

Then you're probably reinforcing my original point - we can't care so much or we'll enter some pretty alarming priority inversions.

[bondarchuk]

You are arguing from consequences, it's not a valid argument. "If we consider AIs to be conscious it would be a alarming, therefore they are not conscious".

[jvanderbot]

You made that up. My full argument is above. I'll try again.

"We must not consider consciousness as all too important because what matters is human flourishing and human rights."

And some of

"Even if we suddenly all agree an LLM is conscious it wouldn't and shouldn't influence us very much"

While acknowledging that some people will change their lives, the way some people like myself won't eat octopus or apes because it is probably more like murdering a sentient creature.

And oh by the way you cannot murder a copyable individual anyway. Did you read Enders Game?

[bondarchuk]

Thanks for the clarification, though "you made that up" is pretty strong. I still don't see how "we can't care so much or we'll enter some pretty alarming priority inversions" is anything but an argument from consequences.

>And oh by the way you cannot murder a copyable individual anyway.

I think for many people the concern is not so much about violating the desire to live that a conscious software entity might (or might not!) have, but about the subjective experience of suffering it might undergo.