[tossandthrow]

> As we advance it becomes more difficult to advance

I don't think this follows.

We have advanced tremendously over the past 200 years, and we are likely going into a time with rapid advancement again.

With advancement, we also develop tools (eg. Llms) that assist advancing.

[Windchaser]

To put it another way, the cost per advancement increases. That’s “as we advance it becomes more difficult to advance”. However, because of the prior advances, you also have more resources to throw at future advancements.

So, then, the question is whether the “profits” on the last advance are enough to pay for the next one. We can define a new term, “affordability”, as what % “profit” you can expect from each advance relative to its cost, telling us whether it becomes relatively easier or harder to continue to advance.

[tossandthrow]

You could argue that there is a capital built up - but even that is difficult as a lot of the knowledge for, eg., building LLMs can not demand rent. Everybody can build their own transformer networks.

> To put it another way, the cost per advancement increases.

This is not true when you don't use capital that demands rent. On the contrary, the cost per advancement is actually decreasing as we develop more knowledge.

LLMs are a cognitive technology (as contrasted to a communication technology). And it will help us tremendously manage knowledge such that you can utilize it better decreasing the cost of advancement.

[RandomLensman]

Whether or not it decreases the costs of advancedment would depend on what is causing advancement to have costs. Some things might be addressable by cognitive technologies, others might not.

If capital doesn't create a return evenutally, why would it invest? (Or did you mean "rent" in the narrow economics definition? But then not sure how that applies here).

[breuleux]

> We have advanced tremendously over the past 200 years

Would most of that have happened if we hadn't found oil, though?

It's difficult to precisely identify how much of progress is owed to intelligence, and how much is simply energy availability. Energy is intrinsically transformative -- the dumbest of organisms can take over the world in a blitz if they can process available energy better than the others, whereas the smartest of entities is going to be ineffective without fuel. Intelligence can, of course, unlock the capability to exploit new energy sources, but there's still an intrinsic physical distribution of it that's outside of anyone's control.

[tossandthrow]

Reducing progress to increasing entropy makes it even more clear that making progress will likely become easier, not harder.

Yes, oil was important back then. Now we can augment with wind, solar, nuclear, etc.

And more technologies will come.

[breuleux]

Maybe, maybe not. The point is that you can't simply will energy into existence. Its availability, including the ease of exploitation, is a physical given. Which is true of resources in general. Arbitrarily advanced technology can't simply make exploitation arbitrarily easy, so for example there can always be situations where the very best of all possible technologies will be less productive than setting fire to petrol would have been, if there had been any. It's complicated. It's not just what we do, it's not just what we have, it's both.

Probably the best test of how the future might trend long term is to see how well we can extract value from all the stuff and waste we are currently producing, because these will inevitably become the primary resources from which we have to build everything new. Unfortunately that shift is likely to happen extremely quickly and catastrophically, because typically you only start processing bad materials once you're done with the good.

[windexh8er]

Then use batteries as an example. Going on 200+ years now most of the major changes have been an easy swap: chemistry. Lead acid --> nickel cadium --> nickel metal hydride --> lithium ion. Improvements in batteries are hard and the surrounding ecosystem has gotten better. Better BMS and better motors have also made gains to increase the efficiency of the battery. Compounding gains within the ecosystem doesn't truly improve the battery directly, but improves them for implementation and use cases.

All the remaining problems with batteries are a hard solve because it's tradeoff after tradeoff. Lithium sulfur, for example, could be amazing but die after a couple hundred cycles at most [0]. So... If LLMs are so novel, as many continue to claim, why hasn't this been solved? Probably because LLMs are an interpretation of our current understanding of everything. An LLM currently only does what any human can do, albeit in a compressed timeframe, mostly.

I think it follows. LLMs have made some considerable improvements, but use "reasoning" (really just a mechanical, autoregressive, loop: generate tokens --> append tokens to context --> feed the expanded context back into the model --> generate the next token --> repeat) as an example. You can do this by hand as well in a traditional chat volley, but it's slower. So we improved LLMs by putting them in an automatic loop (oversimplification - but at the end of the day holds water).

Until LLMs showcase true external breakthroughs that aren't driven by human guidance (not happening anytime soon) we are in this loop of hacks being used to improve the 80% (the LLM itself). Notable jumps? "Reasoning" and with Mythos-like models we now have "Advanced Reasoning" (by leveraging a more capable looping framework such as an agentic harness).

[0] https://advanced.onlinelibrary.wiley.com/doi/10.1002/adfm.20...

[skinfaxi]

It seems the rate of advancement has increased over time though. Like from ancient times to 1800s the rate of progress was slow, then basically a relative explosion of progress compressed in 200 years.

[godelski]

  > It seems the rate of advancement has increased over time though.

Effort and expenditures are not fixed. You cannot hand wave away these variables.

Here's an example let's compare two investment strategies:

  - Alice puts $1 into VOO every day. 
  - Bob puts in $10000 into VOO every day.

You don't get to say "wow, Bob is a much better investor than Alice. He's so smart, look how fast his money grows!" You get to say "Bob is investing more money than Alice."

Now pretend Bob is investing into a slower growing account than Alice. His account will still grow faster as long as his account is not losing >99.9% per day. He could have a worse strategy and still grow faster because he's throwing more money at it

[bobson381]

Because of access to a giant store of energy. Our advancement has also created a future need to consume more and more energy to fuel increased complexity.

[godelski]

Which has created all sorts of other problems even if we exclude climate change.

I'm not arguing we shouldn't live in a modern world, but rather that complexity increases