[jmoggr]

I think it's credible to believe that that a system which is able to meaningfully improve it's own intelligence will lead to a far weirder world than the one imagined by the article.

The parent comment could be interpreted as frustration from a lack of imagination about how weird the future could be.

Perhaps we're not at the recursive self improvement yet, but it seems increasingly naive to believe that it isn't possible.

[encomiast]

Of course you might be right. But if we look at our past ability to predict the future by extrapolating from the present and recent past, you might also say it is naive to think that _this time_ is somehow different, that this time the future is clear.

Also, I think you've set up a straw man. I haven't seen anyone arguing that this future isn't possible. What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past.

[bilater]

I agree with you that there is no certainty, and in fact the future will almost certainly not look like anything we imagine. I think that's what makes it exciting.

We think about starships and humans mining Mars, and it'll probably be nothing like that. It might just be humanoids or AI drones that actually end up colonizing the solar system, with the humans never really leaving earth except as mind uploads...

But at the end of the day, things are changing. Things are accelerating. And our mental models of what works or doesn't work will almost certainly break. I don't fear the future, nor do I project any confidence on what that's going to look like, but I know I want to be part of it.

[jmoggr]

> you've set up a straw man

I think you're right, my bad.

> What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past.

I'm arguing that if recursive self improvement happens, that the trend line will be reasonably predictable. (and that we are close enough to RSI that we should take this possibility seriously).