AGI will make many things possible that we’re not possible before. It will invent new things that wildly alter the reality of the world. The point is that you cannot predict what will happen; a million things could happen. But the good outcomes can be counted on one hand. The probability that we get a good outcome is one in millions, probably less. Obviously that is an incredibly stupid thing to do!
If you can’t draw a conclusion from that then you simply don’t get it.
AGI will, as I said, make many things possible. It will be possible to keep a human being alive forever and suspend them in a state of pure pain for eternity. It will be possible to extract memories and information from the human brain. It will be possible to control human behavior with invasive brain surgery. Imagine every grotesque thing that you can do to a human being — it’s on the table for AGI enabled governments in the early days if that’s how it plays out. And that’s just one slice in the infinite pie of grotesque possibilities. You can’t just opt out of these things if they provide a material advantage to your country/meta organism.
I've got an imagination, but you're letting yours run wild. AGI doesn't mean immediate sci-fi level medical technology, and it certainly doesn't mean human brain copying and transfer of consciousness to some artificial storage device.
Biological brains are incredibly complex and if that level of complexity is needed to host intelligence I think we're a long ways off from building any true AGI, even if we pumped every bit of engineering skill our society has at it. And even if we _can_ build AGI I doubt we'd have the resources to host more than a handful of them for years to decades after their creation or that their intelligence would be anywhere near small mammal level.
Anything with much greater capabilities than us that would find our resources useful and that doesn't explicitly have our best values at heart would be a danger to us. European settlers were not good news for the Native Americans. Early humans were not good news for many great apes.
(It might be tempting to imagine this from a higher-level perspective and assert that each of these successors were somehow more interesting or had more fulfilling lives than what they replaced, and so therefore these successions might be ultimately good in the long run, but that's not meant to be part of this analogy. We won't have some genetic lineage with AGI; it will probably be more alien to our mind's design than actual aliens produced by evolution like us would be. The AGI might not build a world replacing ours where many of its kind have lives we would find fulfilling or interesting. It might care just about making the world completely predictable and safe, empty of outside threats and life, and hibernating in a place of safety. Plenty of animals live solitarily and might do this if they were naively uplifted into superintelligence. Humans and social animals instinctually believe in the goodness of friendship and society because evolution saw we were good at surviving in groups and molded our instincts to encourage us to do that. AGIs won't necessarily have that if we don't design that into them (or set up the process that designs them to do that).)