[jprete]

I am sympathetic to the point. I was somewhat concerned about nuclear war with Russia's Ukraine invasion, and I've become more concerned about climate change over time. I'm hoping either solar continues to expand - woot! - or we get over our fear of nuclear, or fusion finally becomes viable.

My point still stands - an AI that is reproducing (and/or mutating) by any method is going to be subject to selection pressure, just as any biological process is. All ML systems are effectively under selection pressure from the training process, and so they learn unexpected and mostly undesirable optimizations like "use a game-engine glitch to gain infinite flight and go to the goal without solving the maze" or "read text off the scanned X-ray sheet to get additional clues as to the correct diagnosis".