[FloorEgg]

> Which would only be the case insofar that there exists categories of work we do better than AI at that juncture.

I'm pretty sure this is incomplete.

It's more like whether people find the work rewarding enough to be worth doing.

In some cases it can be rewarding for reasons other than money. Even when the primary reward is money, there could be a lot of demand for human work that is worse than AI when the AI is significantly more expensive. Some customers may just prefer the human do it for any number of reasons.

It's very possible we can have a rich prosperous economy and culture with lots of AI and people working together. It's just not clear how we get there, and its not popular to take the idea seriously right now. Fear propagates faster and easier than inspiration, at least in this cultural climate.

We're less likely to get what we want if we don't aim for it.

[breuleux]

The point is more that you can well imagine a future where AI is both better and cheaper than humans at basically all types of work, leading to a situation where we would be entirely unable to fight back in the eventuality that machine owners, or the machines themselves, were to repurpose the resources used for our sustenance to other ends. And this would still fit in the universe-old pattern you've observed.

To put it bluntly, if the economic value of human labour drops to zero, or below the value of human sustenance, it is plausible that the consequence of that, from the cold perspective of cosmic logic, would be the extinction of humans. That's not to say there isn't any way to keep the genie in the bottle and create a utopia for ourselves (I very much doubt it), but that would be against the grain of nature. Call me a pessimist, but if we ever get outperformed in our own niche, our days are numbered.

[FloorEgg]

Our views definitely intersect.

On a long enough time scale my opinion is "of course humanity will go extinct", but the interesting and very speculative answers are in exactly how and when. It's highly plausible to me that humanity goes extinct via evolution into something else, and on a long enough time scale that you and I won't have any clue within our lifetimes.

Where I think we still differ is the "outperformed in our niche". Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit. I believe biological brains are around 3-4 orders of magnitude less energy efficient than the theoretical physical limit, but the ~99.9% of energy that's not being used for straight computation is allocated to redundancy and resiliency.

So overall my point is if you zoom out far enough yes humanity may get erased by way of evolutionary pressures, but on the timescale of our lifetimes we don't need to worry about that, and on the timescale of our careers we don't need to worry about an AI driven unemployment apocalypse.

What we do need to worry about now is AI being used in media to manipulate people into doing what's not in their best interest.

[breuleux]

> Our biology is ridiculously optimized. Like 6-7 orders of magnitude more energy efficient than current day AI/computation stack. It's plausible that AI can never outperform us at what we do best because we are already at the limit.

I do tend to agree with you on that, but with lesser confidence. It doesn't necessarily matter whether they outperform us at "what we do best" if they perform well at doing things that we didn't evolve to deal with. For example, we drove many animal species to extinction, not because we were better than them at anything they were good at, but because we were a novel threat outside their adaptive range. AI could very well do to us what we did to these animals, by acting aggressively enough in a direction that challenges our capacity for adaptation. Basically we have to both perform in our niche, and maintain the relevance of the niche itself in the face of whatever completely unforeseen BS these new technologies may bring about.