[polygamous_bat]

Can I tell you something secret? The "AGI" of the form you are describing is already here! Not only that, it's been here since 1988! Crazy right? Just check out this link if you don't believe me: https://en.wikipedia.org/wiki/Morris_worm

(Snark aside, I've realized this type of interactions have always resulted in a very low perception of EA for me. I don't come out of these conversations thinking "what a clueless thing to say by this person who also happened to support EA", I think "wow, EA is full with looney bins who don't know what they are talking about." If you are really into giving what you can and aren't sure why EA gets a bad rep, look no further.)

[lurker616]

I've heard of some of the famous viruses which are obviously more complicated than what I mentioned. But that's the difference - they are hardcoded to perform these particular steps. I'm not saying AGI is the code that does this. AGI is one level higher - a more meta objective - given the objective of "accumulating power", it will learn lower level objectives like "replicating and spreading", which leads to the final output like this code.

Basically, AGI could unleash a thousand "morris worms" - and that would only be for the objective of replicating and spreading. There could be other objectives as well.