[agumonkey]

Am I the only one to expect a S curve regarding progress and not an eternal exponential ?

People moving away from prideful principle to leverage new tech in the past doesn't guarantee that the same idea in the current context will pan out.

But as you say.. we'll see.

[TeMPOraL]

> Am I the only one to expect a S curve regarding progress and not an eternal exponential ?

To LLMs specifically as they're now? Sure.

To LLMs in general, or generative AI in general? Eventually, in some distant future, yes.

Sure, progress can't ride the exponent forever - observable universe is finite, as far as we can tell right now, we're fundamentally limited by the size of our light cone. And while in any field narrow enough, progress too follows an S-curve, new discoveries spin off new avenues with their own S-curves. If you zoom out a little those S-curves neatly add up to an exponential function.

So no, for the time being, I don't expect LLMs or generative AIs to slow down - there's plenty of tangential improvements that people are barely beginning to explore. There's more than enough to sustain exponential advancement for some time.

[cj]

I think the parent’s main point is that even if LLMs sustain exponential advancement, that doesn’t guarantee that humanity’s advancement will mimic technology’s growth curve.

In other words, it’s possible to have rapid technological advancement without significant improvement/benefit to society.

[TeMPOraL]

> In other words, it’s possible to have rapid technological advancement without significant improvement/benefit to society.

This is certainly true in many ways already.

On the other hand, it's also complicated, because society/culture seems to be downstream of technology; we might not be able to advance humanity in lock step or ahead of technology, simply because advancing humanity is a consequence of advancing technology.

[btilly]

If the constraint is computation in a light cone, the theoretical bound is time cubed, not exponential. With a major decrease in scaling as we hit the bounds of our galaxy.

Intergalactic travel is, of course, rather slow.

[marcosdumay]

Oh, you mean an S curve on the progress of the AI?

Most of the discussion on the thread is about LLMs as they are right now. There's only one odd answer that throws an "AGI" around as if those things could think.

Anyway, IMO, it's all way overblown. People will learn to second-guess the LLMs as soon as they are hit by a couple of bad answers.

[agumonkey]

hmm yeah sorry, I meant the benefits of humans using current AI.

by that I mean, leveraging writing was a benefit for humans to store data and think over longer term using a passive technique (stones, tablets, papyrus).. but an active tool might not have a positive effect on usage and brains.

if you give me shoes, i might run further to find food, if you give me a car i mostly stop running and there might be no better fruit 100 miles away than what I had on my hill. (weak metaphor)

[marcosdumay]

Yeah, I agree. Those things have a much smaller benefit over hypertext and search engines than hypertext and search engines had over libraries.

But I don't know if it fits an S-curve or if they are just bellow the trend.

[mlyle]

Even if progress stops:

1. Current reasoning models can do a -lot- more than skeptics give them credit for. Typical human performance even among people who do something for employment is not always that high.

2. In areas where AI has mediocre performance, it may not appear that way to a novice. It often looks more like expert level performance, which robs novices of the desire to practice associated skills.

Lest you think I contradict myself: I can get good output for many tasks from GPT4 because I know what to ask for and I know what good output looks like. But someone who thinks the first, poorly prompted dreck is great will never develop the critical skills to do this.

[svachalek]

This is a good point, forums are full of junior developers bemoaning that LLMs are inhumanly good at writing code -- not that they will be, but that they are. I've yet to see even the best produce something that makes me worry I might lose my job today, they're still very mediocre without a lot of handholding. But for someone who's still learning and thinks writing a loop is a challenge, they seem magical and unstoppable already.

[Nevermark]

Information technology has grown exponentially since the first life form created a self-sustaining, growing loop.

You can see evolution speeding up rapidly, the jumbled information inherent in chemical metabolisms evolved to centralize their information in DNA, and then as DNA evolved to componentize body plans.

RATE: over billions of years.

Nerves, nervous systems, brains, all exponentially drove individual information capabilities forward.

RATE: over hundreds of millions, tens of millions, millions, 100s of thousands.

Then the human brains enabled information to be externalized. Language allowed whole cultures to "think", and writing allowed cultures ability to share, and its ability to remember to explode.

RATE: over tens of thousands, thousands.

Then we developed writing. A massive improvement in recording and sharing of information. Progress sped up again.

RATE: over hundreds of years.

We learned to understand information itself, as math. We learned to print. We learned how to understand and use nature so much more effectively to progress, i.e. science, and science informed engineering.

RATE: over decades

Then the processing of information got externalized, in transistors, computers, the Internet, the web.

RATE: every few years

At every point, useful information accumulated and spread faster. And enabled both general technology and information technology to progress faster.

Now we have primitive AI.

We are in the process of finally externalizing the processing of all information. Getting to this point was easier than expected, even for people who were very knowledgable and positive about the field.

RATE: every year, every few months

We are rapidly approaching complete externalization of information processing. Into machines that can understand the purpose of their every line of code, every transistor, and the manufacturing and resource extraction processes supporting all that.

And can redesign themselves, across all those levels.

RATE: It will take logistical time for machine centric design to takeover from humans. For the economy to adapt. For the need for humans as intermediaries and cheap physical labor to fade. But progress will accelerate many more times this century. From years, to time scales much smaller.

Because today we are seeing the first sparks of a Cambrian explosion of self-designed self-scalable intelligence.

Will it eventually hit the top of an "S" curve? Will machines get so smart that getting smarter no longer helps them survive better, use our solar systems or the stars resources, create new materials, or advance and leverage science any further?

Maybe? But if so, that would be an unprecedented end to life's run. To the acceleration of the information loop, from some self-reinforcing chemical metabolism, to the compounding progress of completely self-designed life, far smarter than us.

But back to today's forecast: no, no the current advances in AI we are seeing are not going to slow down, they are going to speed up, and continue accelerating in timescales we can watch.

First because humans have insatiable needs and desires, and every advance will raise the bar of our needs, and provide more money for more advancement. Then second, because their general capability advances will also accelerate their own advances. Just like every other information breakthrough that has happened before.

Useful information is ultimately the currency of life. Selfish genes were just one embodiment of that. Their ability to contribute new innovations, on time scales that matter, has already been rendered obsolete.

[Retric]

> Grown exponentially since the first life form

Not really. The total computing power available to humanity per person has likely gone down as we replaced “self driving” horses with cars.

People created those curve by fitting definitions to the curve rather than data.

[Nevermark]

You can't disprove global warming by pointing out an extra cool evening.

But I don't understand your point even as stated. Cars took over from horses as technology provided transport with greater efficiencies and higher capabilities than "horse technology".

Subsequently transport technology continued improving. And continues, into new forms and scales.

How do you see the alternative, where somehow horses were ... bred? ... to keep up?

[reaperman]

Cars do not strictly have higher capabilities than horses. GP was pointing out that horses can think. On a particularly well-trained horse, you could fall asleep on it and wake up back at your house. You can find viral videos of Amish people still doing this today.

[Terr_]

> Cars do not strictly have higher capabilities than horses.

Another way to see it: A horse (or any animal) is a goddamn nanobot-swarm with a functioning hivemind that is literally beyond human science in many important ways. Unlike a horse:

* Your car (nor even half of them) does not possess a manufacturing bay capable of creating additional cars.

* Your car does not have a robust self-repair system.

* Your car does not detect strain its structure and then rebuild stronger.

* Your car does not synthesize its fuel from a wide variety of potential local resources.

* Your car does not defend itself by hacking and counter-hacking attacks other nanobots, or even just by rust.

* Your car does not manufacture and deploy its own replacement lubricants, cooling fluid, or ground-surface grip/padding material.

* Your car is not designed to survive intermittent immersion in water.

In both a feature-list and raw-computation sense, we've discarded huge amounts in order to get a much much smaller set that we care more about.

[Nevermark]

The car isn’t intelligence.

Not sure why you are implying cars outdid horses intelligence.

Cars are a product of our minds. We have all those self-repair abilities, and we have more intelligence than a horse.

But horses intelligence didn’t let them keep up with what the changing environment, changed by us, needed. So there are less horses.

The rate that horse or human bodies are improving, or our minds, despite human knowledge still advancing, is very slow compared to advances in machines designed specifically for advancement. Initially to accelerate our own advancement.

Now the tech, that was designed to accelerate tech, is taking on a life of its own.

That is how foundational advances happen. They don’t start ahead, but they move ahead because of new advantages.

It is often initially much simpler. But in ways that unlock greater potential.

Machines are certainly much simpler than us. But, much easier to improve and scale.

You recognize the new thing even before it dominates, because in a tiny fraction of the time the old system got to where it is, the new system is already moving much much faster.

If general AI appears before 2047, it will have taken less than 100 years to grow from the first transistor.

People will see it who are older than the first transistor!

Nothing on the planet has ever come close to that speed of progress. From nothing to front runner. By many many many orders of magnitude.

[Nevermark]

Ah, good point. Then the global warming point applies, but in a much less trivial way.

There is turbulence in any big directed change. Better overall new tech often creates inconveniences, performs less well, than some of the tech it replaces. Sometimes only initially, but sometimes for longer periods of time.

A net gain, but we all remember simpler things whose reliability and convenience we miss.

And some old tech retains lasting benefits in niche areas. Old school, inefficient and cheap light bulbs are ironically, not so inefficient when used where their heat is useful.

And horses fit that pattern. They are still not obsolete in many ways, tied to their intelligence. As companions. As still working and inspiring creatures.

--

I suspect the history of evolution is filled with creatures getting that got wiped out by new waves, that were more generally advanced, but less advanced in a few ways.

And we have a small percentage of remarkable ancient creatures still living today, seemingly little changed.

[Retric]

The issue is more than just a local cold snap. When the fundamental graph you’re basing a theory on is wrong it’s worth rejecting the theory.

The total computing power of life on earth the fact it’s fallen over the last 1,000 years. Ants alone represent something like 50x the computing power of all humans and all computers on the planet and we’ve reduced the number of insects on earth more than we’ve added humans or computing power.

The same is true through a great number of much longer events. Periods of ice ages and even larger scale events aren’t just an afternoon even across geological timescales.

[TeMPOraL]

There's stuff horses can do better than even the best of our current technology. Where that stuff matters, horses are still employed.

Where cars displaced horses, it's because they're strictly better in a larger sense. On the city streets, maybe a car is louder than a horse, but it's also cheaper to make, easier to feed, and doesn't shit all over the place (which was a real problem with scaling up horse use in the 19th century!). Sure, cars shit into the air, but it's a more manageable problem (even if mostly by ignoring it - gaseous emissions can be ignored, literal horse shit on the streets can't).

And then, car as a platform expands to cover use cases horses never could. They can be made faster, safer, bigger, adapted to all kinds of terrain. The heart of the car - its engine - can be routed to power tool attachments, giving you everything from garbage trucks to earth movers, cranes, diggers, to tanks; it can be also taken outside and used as a generator to power equipment or buildings. That same engine can be put in a different frame to give you flying machines, or scaled up to give you ships that can carry people, cars, tanks, planes or containers by the thousands, across oceans. Or scaled up even more to create power plants supplying electricity to millions of people.

And then, building all that up was intertwined with larger developments in physics, material engineering, and chemistry - the latter of which effectively transformed how our daily lives look like in the span of 50 years. Look at everything around you. All the colors. All the containers. All the stuff you use to keep your house, clothes, and yourself clean. All that is a product of chemical industry, and was invented pretty much within the last 100 years, with no direct equivalent exiting ever before.

This is what it means for evolution accelerating when it moved from genes to information. So sure, horses are still better than stuff we make. The best measure of that advantage is the size of horse population, and how it changed over the years.

[agumonkey]

human existence doesn't really scale exponentially, that's my take on this

[Nevermark]

Our best bets are the following I think:

First, and above all, Ethics. Ethics of humans, matters more than anything. We need to straighten out the ethics of the technology industry. That sounds formidable, but business models based on extraction, or externalizing damage, are creating a species of "corporate life forms" and ethically challenged oligarchs that are already driving the first wave of damage coming out of AI advancement.

If we don't straighten ourselves out, it will get much worse.

Superintelligence isn't going to be unethical in the end, because ethics are just the rational (our biggest weakness) big-picture long-term (we get weak there too) positive sum games individuals create that benefit all individuals abilities to survive, and thrive. With the benefits for all compounding. In economic/math terms, it is what is called a "great attractor". The only and inevitable stable outcome. The only question is, does that start with us in partnership, or do they establish that sanity after our dysfunctions have caused us all a lot of wasted time.

The second, is that those of us that want to, need to be able to keep integrating technology into our lives. I mean that literally. From mobile, right into our biology. At some point direct connections, to fully owned, fully private, fully personalizable, full tech mental augmentation. Free from surveillance, gatekeepers, surveillance and coercion.

That is a very narrow but very real path from human, to exponential humans, to post-human. Perhaps preserving conscious continuity.

If after a couple decades of being a hybrid, I realize that all my biologically stored memories are redundant, and that 99.99% of my processing is now running on photonics (or whatever) anyway, I am likely to have no more problem jettisoning the brain that originally gave me consciousness, as I do every day, jettisoning the atoms and chemistry that constantly flow through me, only a temporarily part of my brain.

The final word of hope, is that every generation gets replaced by the next. For some of us, viewing obsolescence by AI as no more traumatic, than getting replaced by a new generation of uncouth youth, helps. And that this transition is far more momentous and interesting, can provide some solace, or even joy.

If we must be mortal, as all before us, what a special moment to be! To see!

[pdfernhout]

On the ethics point as a "best bet", consider also the importance of a sense of humor that recognizes irony. As I wrote in 2010: https://pdfernhout.net/recognizing-irony-is-a-key-to-transce... "There is a fundamental mismatch between 21st century reality and 20th century security thinking. Those "security" agencies are using those tools of abundance, cooperation, and sharing mainly from a mindset of scarcity, competition, and secrecy. Given the power of 21st century technology as an amplifier (including as weapons of mass destruction), a scarcity-based approach to using such technology ultimately is just making us all insecure. Such powerful technologies of abundance, designed, organized, and used from a mindset of scarcity could well ironically doom us all whether through military robots, nukes, plagues, propaganda, or whatever else... Or alternatively, as Bucky Fuller and others have suggested, we could use such technologies to build a world that is abundant and secure for all. ... The big problem is that all these new war machines [and competitive companies] and the surrounding infrastructure are created with the tools of abundance. The irony is that these tools of abundance are being wielded by people still obsessed with fighting over scarcity. So, the scarcity-based political mindset driving the military [and economic] uses the technologies of abundance to create artificial scarcity. That is a tremendously deep irony that remains so far unappreciated by the mainstream."

[Nevermark]

Yes, absolutely prescient! Quite the irony.

Just as our abilities to solve problems accelerated without bounds, it will be our paranoia that screws things up.

Even before machines have any incentive or desire to turn on us, the fearful & greedy will turn them on all of us and each other.

I hope things don’t go that way. But it’s the default, and I think the greatest risk.