[arethuza]

"You wouldn't mind being a slave for a few decades or hundreds of years if you were immortal?"

Being immortal implies you could, in the worse case, be kept as a slave for ever - which is a sufficient risk that I'd be appalled at the idea. I'm equally horrified at the idea of conscious machine slaves just as much as human slaves - but maybe I've been reading too many Culture novels...

[veridies]

Slightly off topic, but I'm horrified by the idea of ever creating consciousness in machines. Imagine we built a piece of software that could feel, and built controls for its emotions. I can't imagine it would take long before some bored teenager or sociopath, who in earlier years would torture individual squirrels or insects, created an infinite suffering machine. You could run thousands of instances of your suffering machine and simulate a holocaust on your desktop. That's not a power I would trust the world with.

[JoeAltmaier]

You can do that now. Write a console app, where if you type 'pain' it prints 'oh! How I'm suffering!'.

I don't see how that software is essentially different from the proposed software.

[aninhumer]

The parent explicitly hypothesised a conscious program.

You can argue that such a thing is impossible, but assuming that it is, arguing that such a program would be comparable to a shell script is facile.

[JoeAltmaier]

Its a mechanistic argument, that says no machinery is capable of consciousness. How do you decide if software is conscious? For instance, it could print "I'm conscious! Oh! I'm suffering!"

[aninhumer]

>How do you decide if software is conscious?

How do you decide if a human is conscious?

I don't think we know enough about the nature of consciousness to state with certainty that a "machine" is incapable of it.

[JoeAltmaier]

I'm currently residing inside a human right now. I 'know' what consciousness is, at least for me.