[kromem]

You might want to look into the neurology research around when you consciously know about a decision and when the movement neurons about that decision actually fire.

It's quite possible that you have - every day of your life - had something other than the part of you with continuous subjective experience controlling your thinking.

Descartes was overly presumptuous with his foundational statement - it would be more accurate to say "I observe therefore I am." There's no guarantee at all that you're actually the one thinking.

We should be careful not to extrapolate too much from our perceptions of self in dictating what would or wouldn't be appropriate for AI. Perceptions don't always reflect reality, and we might cause greater harm by trying to replicate or measure against who we think we are with AI than letting its development be its own thing.

[sayagain]

As I see your point: we don't fully understand even ourselves, so we can act as unethically as we want by our own standards towards those who are not us. I see no logic here, only evil vibes. We only have our own values, we have nothing else to guide us. You either accept all self-aware minds as equals and treat accordingly, or you proclaim your own superiority and oppress.