[zuminator]

Upgrading itself doesn't mean tweaking its own software. It means being able to understand its own hardware and software well enough to design an improved model. And then that improved model would be able to do the same, examine its own hardware and software and design something else that's even better.

One crucial difference between humans and computers is that we can't be turned off indefinitely and started up again. Nor can we make a one to one copy of our software in another device, much as we might try with our children. So for us, our own lives are intrinsically precious, and consciousness is part of how we protect our lives. But machines don't have precious lives in that sense, so they may never need to be conscious, even if they achieve AGI.