[aetherson]

Something to note about the "we're on an exponential (or at least more-than-linear) increase in technology" arguments: They generally need to reach back quite a long time to sound convincing.

Yudkowsky makes two arguments about how fast technology or society is evolving: in one he chooses 500 years ago, 1517. In another, he talks about "the last 10,000 years."

In contrast, Chollet compares 1900-1950 with 1950-2000.

I agree, we've changed the world in big ways compared to 1517 or 8000 BC. But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia.

In fact, if you consider a y = log(x) or y = sqrt(x) function, those more closely fit a narrative of "If you look back a long time, things seem almost crazily changed, but if you look into recent history, it looks slower" much better than a y = x^2 or y = e^x function.

[mannykannot]

This should be seen in the context of the article Yudkowsky is replying to. Collet's position is that runaway AGI is impossible, and in support of that, he claims that it is impossible for AI technology to grow super-linearly. In his refutation, Yudkowski has merely to show that it is not beyond the bounds of possibility.

Of course, insofar as he is also putting forward the view that the risk is significant, he is also putting forward the opinion that it is quite plausible - at least plausible enough to take the scenario seriously. That argument can survive occasional stalls in the rate of technological advance, if that is indeed what we are seeing.

[goatlover]

So the real question is about plausibility, not possibility?

It's possible that predatory aliens will show up on our doorstep sometime in the next few decades. But it's not very likely, and not something we need to prepare for.

[mannykannot]

By claiming the impossibility of an AI apocalypse, Chollet attempts to avoid discussing any other possibility / plausibility / likelihood than zero, but claiming impossibility brings with it a burden of proof that his arguments cannot carry.

As to whether an AI apocalypse is more or less likely than an invasion by predatory aliens, I would guess that it might be the more likely one, but I wouldn't put much effort into defending that position.

[benbreen]

Thank you for pointing this out. As an historian, the appeal to a deep historical argument in these situations drive me nuts. We find ultra-rationalist, data-driven people like Yudkowsky relying on a hugely flawed and imperfect historical record to make confident pronouncements.

In my opinion, the historical record from, say, 1517 (or even 1717) is so full of gaps and inconsistencies that it is impossible to make even an order of magnitude level estimate regarding something like rate of technological innovation (or even world GDP, arguably).

Economists and social scientists are often guilty of this as well - for instance, I really liked the parts of Pinker's Better Angels of Our Nature that dealt with relatively reliable quantitative data from the 20th and 19th centuries. But once he starts talking about things like the An Lushan rebellion or the fall of the Roman Empire, it's a complete mess from an empirical point of view, doing things like taking the numbers of fatalities cited by contemporary participants in an historical conflict at face value. No self-respecting historian would make such big assumptions from such faulty data.

Anyway, I realize this is incidental to the larger argument here but insofar as questions about exponential growth of technology draw on historical arguments, I wanted to throw it out there.

[naasking]

> But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia.

We are. Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime.

Look at the timeline from the first computer program to play chess, to beating chess world champions, to the recent announcement of DeepMind that beat all existing computer programs after teaching itself the game within the span of a few hours.

Look at the timeframe of computer programs that take dictation to programs that automatically translate between nearly all commonly used languages on Earth.

The same could be said for computer vision, computer music, computers driving, and so on.

I think the people skeptical of the intelligence explosion are missing the forest for the trees. Our progress in the last century alone is mind boggling. Certainly we can debate the values of the parameters in the intelligence explosion we're in the midst of, but denying it entirely is silly.

[aetherson]

Or it just seems like that to you because these are the changes that you've viscerally experienced instead of just reading about.

Someone born in 1900, looking back at their life in 1975, would be like "When I was born, heavier-than-air flight was impossible. Now people routinely fly across oceans at 600 mph, you can travel faster than the speed of sound for admittedly a lot of money, and we've gone to the motherfuckin' moon. Vast swathes of work has been automated, to the point where we essentially ended an entire industry (personal servants). Automobiles went from being curiosities to something that even poor people have and use every day. We split the atom, we brought women into the workforce, we invented electronic computers, we invented radar, we turned radio from a science project to TVs that every family have. We invented antibiotics and childhood mortality fell by some enormous percentage."

"You're very impressed that computers went from 'pretty good at playing chess' to 'extremely good at playing chess' in just 20 years. Maybe you're the one who's missing the forest for the trees."

[naasking]

Like the other poster, these are also examples of super-linear progress, so I'm not sure what you think this proves.

For my point, the super-linear progress in information tech is all that's needed to argue in favour of the intelligence explosion.

[aetherson]

You seem to think that a short period of fast technology growth in some areas corresponds to the claim that technology as a whole is improving more-than-linearly.

Compare the y values of the functions y = x vs. y = sqrt(x) over the x values in the interval [0..1]. Or the slopes of the lines.

It's been 42 years since 1975. If technological advance was faster from 1900-1975 than it has been from 1976-2017, or "only as fast," then that's important to understand, and probably more relevant to our immediate future than whether technological growth from 8000 BC to 1900 AD was either by some standard very impressive or slower than growth from 1900-2000.

[naasking]

Firstly, growth rates of non-information tech largely isn't relevant to AI. That said, overall growth in knowledge is at least directly proportional to population growth. Even pessimistically considering humans as dumb, an exponential population growth means an exponential discovery rate just from sheer trial and error. So overall progress is undeniably exponential, even if it's a low exponent.

Secondly, we know definitively that information density has been growing exponentially given Moore's law. The much decried end of Moore's law is for a particular incarnation of information tech, but there's still plenty of room to grow in other directions.

Even with our current tech base, we can continue to scale exponentially in horizontal directions with more parallelism (see the rise of core counts, GPU and distributed computing). We're nowhere near the end of that scaling in that direction, let alone longer term innovations like optical and quantum computing.

So really, what possible reasons do we really have for thinking that exponential growth will not continue well past human intelligence? Note, I didn't say infinitely, just well past our intelligence.

[aetherson]

Overall growth in knowledge isn't at least directly proportional to population growth, if the proportion of knowledge already known and shared grows with population growth.

But even if it did, we don't have another doubling of human population ahead of us, so you better hope we're already there.

As you point out, Moore's Law doesn't have a ton more power available to it either.

Lots of problems don't parallelize well, quantum computing has never demonstrated more power than classical computing, and who knows where optical computing will go, but more to the point, hardware growth doesn't in fact guarantee an intelligence explosion.

What possible reasons do we have for thinking that exponential growth has ever happened in terms of actual progress, rather than things like "transistor density"?

Look, every futurist in the world in 1975 thought that by 2017, we'd all be routinely traveling faster than sound, that we'd have colonies on the moon if not mars, and that probably we'd have AGI or something pretty close to it by 2017. The reasons we don't have supersonic travel and common space travel aren't simplistic things like "it's physically impossible to pack energy this densely" or "you can't go this fast."

[evgen]

> Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime.

Look at the time frame from from the invention of controlled heavier-than-air flight to landing a man on the moon; within a human lifetime and before you were born.

Look at the time frame from the point when the vast majority of the human population lived their entire life within a 50 mile radius of where they were born and when fast transportation and global travel exponentially increased the genetic mixing of humanity; within a human lifetime and before our grandparents were born.

Look at the time frame from the point when information could travel no faster than the speed of a good horse to the time when information could travel across the ocean in the time it took you to saddle a horse; within a human lifetime and almost two centuries ago.

Our progress in the last century is significant, but you over-estimate its importance because you are surrounded by it and have little understanding of the history of technology. Things that may seem trivial or even primitive to you were far more important and world-changing inventions, while a lot of what we currently consider significant advances are only important because, for example, we lived in a time when people played finite games better than machines and were around to see that era end.

[naasking]

Those are all examples of super-linear progress, so I'm not sure what point you think you've made.

> Things that may seem trivial or even primitive to you were far more important and world-changing inventions

Which has zero bearing on the point I was making, which is that super-linear progress in information technology is all around us. Information tech is all that matters to the question of general AI. Like I said, you're missing the forest for the trees.

[goatlover]

That's only true if the information technology is leading up to AGI, and not say, augmenting human intelligence instead. One could argue that the intelligence explosion has been happening for centuries, but it's human intelligence not machine, that is being amplified.

[naasking]

> One could argue that the intelligence explosion has been happening for centuries, but it's human intelligence not machine, that is being amplified.

I agree, humans have been amplifying their own abilities with tech. It's been our biggest competitive advantage. However, at some point information tech will become sophisticated enough to match the capabilities of human brains. At that point, humans will be left behind.

The best outcome in this scenario is humans merging with their machines, and that would be a continuation of that same trend. But, it's not the only plausible outcome, and that's what's troubling.

[renox]

> Look at the time frame from the point when information could travel no faster than the speed of a good horse

Incorrect example: semaphore relays existed already, but yes transoceanic communications happened very fast indeed.

[LolWolf]

Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.

[llamaz]

Which is what differential geometry is based on - you take a surface (for example - the one dimensional functions you mentioned work too, but those are too trivial). Then you associate to each point a plane (re-centered at to the origin). The collection of all vectors that fit into the plane is a vector space called a tangent space, and the collection of all tangent spaces is a tangent bundle. And now you've set up differential geometry and can study it.

[aetherson]

Only on infinitely small subintervals of x.

[LolWolf]

This approximation has vanishing error from a linear function (as the interval decreases) and (most importantly!) we only have finite noisy samples; so it’s essentially indistinguishable (in a statistical way) for a small enough interval given some variance.

[dekhn]

sin(x) = x for very small x, this is a useful tool for several proofs.

[cdancette]

for all functions, f(x) = f(0) + f'(0) * x for x very close to 0

[btilly]

The approximation is particularly good for sin(x) because the next term in the Taylor series, f''(0) * x^2 / 2, happens to be 0. So the error is O(x^3) rather than the more common O(x^2).

[Smaug123]

This is a specific instance of what LolWolf just said in the grandparent.

[dekhn]

yeah I didn't know that part, just the more specific one.

[jjaredsimpson]

It's the first term of the Maclaurin series

sin(x) = sin(0) + cos(1)x = 0 + x

[btilly]

In The Innovator's Dilemma over a dozen technologies were documented that were on exponential improvement curves for a period of decades or more. They generally last until the technology either hits a wall in physical possibility, or the thing being delivered is no longer the primary measure of the technology.

Here are some examples off of the top of my head that are happening right now:

Operations per second of a CPU (The famous Moore's law) bits stored per dollar of RAM Energy density of batteries Energy produced per dollar of solar panels

Here are some examples from the past.

Distance a steam ship could travel without refueling Maximum power of a gasoline engine Volume of dirt a hydraulic scoop can pick up

The history of technological progress is dominated by exponential curves. Saying that it is logically impossible for the future to be likewise dominated by exponential curves is just silly.