[kamilner]

Why isn't it possible (or likely, even) that the difficulty of constructing capacity X+1 grows faster than the +1 capacity? Self-improvement would slow exponentially when it takes three times the resources/computation/whatever to construct something that's twice as good at self-improving, for example.

[smallnamespace]

You're arguing that it's not an exponential that doesn't continue to the right indefinitely.

But what if it follows a sigmoid instead, but the plateau is much higher than the current level?

This is what punctuated equilibria look like -- even if the 'new thing' isn't actually a singularity, it may be enormously disruptive and completely displace whatever came before.

[tome]

> This is what punctuated equilibria look like -- even if the 'new thing' isn't actually a singularity, it may be enormously disruptive and completely displace whatever came before.

Right, and this is a much more plausible claim than one of "singularity".

[mannykannot]

>You're arguing that it's not an exponential that doesn't continue to the right indefinitely.

Kalminer's post neither assumes that nor claims it to be so.

The argument is not that an exponential growth is impossible, it is that it cannot be assumed from what we currently know, because we don't know how the problem of bootstrapping intelligence scales. In fact, the idea that exponential growth would occur over any period of time, let alone indefinitely, is speculative - which is not a claim that it is impossible.