[Jach]

I can see how talk of a paperclip maximizer, or any sort of AGI that can actually deliver on e.g. Drexler's nanotech or otherwise act in very powerful ways like, to quote, "colonizing the galaxies", pattern-matches to something roughly omnipotent by most of our standards. I can see how it roughly pattern-matches to something like "immortal", but this seems like the least of feats, much of our complex machinery with maintenance is effectively immortal already. Overall I don't see how this connects to what you originally wrote: "There is a common narrative bias to look at AGI as the Abrahamic God, if not explicitly, then just by saying that it is omniscient, omnipotent, immortal - and will judge us for our deeds." It is not omniscient, it only knows enough to develop the tech to enable paperclip maximization. It is not judging, either: "The AI does not hate you, nor does it love you, and you are made of atoms it can use for something else."

I should also note that the paperclip maximizer is not something the LW crowd believes should exist. Its primary function as an idea is to illustrate the orthogonality thesis: that goals and intelligence aren't dependent on each other. Its secondary function is to illustrate instrumental convergence.