[FloorEgg]

Our views definitely intersect.

On a long enough time scale my opinion is "of course humanity will go extinct", but the interesting and very speculative answers are in exactly how and when. It's highly plausible to me that humanity goes extinct via evolution into something else, and on a long enough time scale that you and I won't have any clue within our lifetimes.

Where I think we still differ is the "outperformed in our niche". Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit. I believe biological brains are around 3-4 orders of magnitude less energy efficient than the theoretical physical limit, but the ~99.9% of energy that's not being used for straight computation is allocated to redundancy and resiliency.

So overall my point is if you zoom out far enough yes humanity may get erased by way of evolutionary pressures, but on the timescale of our lifetimes we don't need to worry about that, and on the timescale of our careers we don't need to worry about an AI driven unemployment apocalypse.

What we do need to worry about now is AI being used in media to manipulate people into doing what's not in their best interest.

[breuleux]

> Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit.

I do tend to agree with you on that, but with lesser confidence. It doesn't necessarily matter whether they outperform us at "what we do best" if they perform well at doing things that we didn't evolve to deal with. For example, we drove many animal species to extinction, not because we were better than them at anything they were good at, but because we were a novel threat outside their adaptive range. AI could very well do to us what we did to these animals, by acting aggressively enough in a direction that challenges our capacity for adaptation. Basically we have to both perform in our niche, and maintain the relevance of the niche itself in the face of whatever completely unforeseen BS these new technologies may bring about.