[khalic]

I’m going to shamelessly reuse the Rainman that needs a handler analogy

More seriously, the epistemic doubt relating to the evolution of these machines is quite something… what do we do if “intelligence” doesn’t have a ceiling, and we end up a bunch of (comparatively) dumb monkeys with AI caretakers/handlers?

[LogicFailsMe]

Absolutely, wouldn't be the first phrase I've pushed into meme space ;-)...

What happens if the AIs get smarter than us at doing things? Well, I always hired smarter people than myself at the things I needed to get done. But if you're worried about them realizing they can get smarter doing the things at which you are the expert, the long-term is likely BCI and even more blurring of the definitions of sentience and consciousness IMO. And with 20-30 years left on my lifeclock, I'm not sure I will live to see that day, but I absolutely do think I will be around long enough to see a few miracles like the end of cancer and Alzheimer's.

[khalic]

Oh no nothing that scifi, just not sure of my place in that

[etdznots]

Thankfully this isn’t the case, but given that true believers actually think this and go on trying to build it, it seems they may not belong in human society or at least they deserve a bit of a spanking for trying to genocide mankind

[LogicFailsMe]

I'm not an accelerationist out to build the ASI at all costs no matter what ASAP, but if I take the long view in combination with the Dark Forest and Fermi's Paradox, it seems like if we don't ultimately follow this path to its end, someone else who did genocides us instead. I don't see why it has to end badly for us, but I get why letting the current crop of power drunk mean girl billionaires crash the collective car into a tree in pursuit of it does.

What makes you think there is a ceiling to intelligence beyond energy (of which there's a lot more to harvest yet if we just pulled our heads out of our fossil fueled asses)?