[nefitty]

There's no way to verify that any other mind is actually experiencing what you interpret as consciousness. We can only ever measure signals, leaks in the form of behaviors. We are reducible to chemical reactions, even as the emergent properties of those reactions are unimaginably complex and impressive.

We all should expand our moral circle. That is the opposite of narcissism and tribalism. We can expand our intuitions to non-human animals, to non-carbon based life forms. It takes imagination. It takes work. The stakes are high, because if we get it wrong with AI, we may unintentionally produce beings that can suffer in amounts that we as individual humans cannot imagine. An AI may experience multiple experiences at once, feel many feelings at once, live many lives at once, all within the span of a second.

The potential for suffering, if nothing else, is reason enough to seriously consider automaton rights as on the same scope as animal and human rights.