[Arnt]

Nothing forces the rebound effect to dominate. Computers grow cheaper, we rebound by buying ones with higher capacity, but the overall price still shrinks. I bet the computer you used to post today cost much less than Colossus.

Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead. You can stream films via 5G that you might not have done via 4G, but you might've streamed via WLAN or perhaps cat5 cable instead. The rebound effect doesn't force 5G to use more power than WLAN/GBE. Or more power than driving to a cinema, if you want to compare really widely. The film you stream makes it comparable, not?

[everdrive]

>Nothing forces the rebound effect to dominate.

Human nature does. We're like a gas, and we fill to expand the space we're in. If technology uses less power, in general, we'll just use more of it until we hit whatever natural limits are present. (usually cost, or availability) I'm not sure I'm a proponent of usage taxes, but they definitely have the right idea; people will just keep doing more things until it becomes too expensive or they are otherwise restricted. The problem you run into is how the public reacts when "they" are trying to force a bunch of limitations on you that you didn't previously need to live with. It's politically impossible, even in a case where it's the right choice.

[Arnt]

I don't understand why "we're like a gas, and expand to fill the space we're in". What makes the simile apply to e.g. AI or 5G when it doesn't apply to others, e.g. computer prices?

[everdrive]

I think the Apple ARM chips are a good example. They're fantastically more efficient and fantastically powerful. We _could_ take this incredible platform and say "we can probably do personal computing on 3-5 watts if we really focus on efficiency." But we don't do that. With more powerful chips, websites and apps and operating systems will get less efficient, bigger, more bloated. If there's any slack in the system we'll just take it up with crap. The chips will be faster next year so why bother making things more efficient? Repeat this process forever, and we eat up all of our efficiency gains.

[vel0city]

A lot of households did make the change from having a few hundred watt PC with a hundred watt monitor to a couple of phones and maybe a tablet that don't use anywhere near as much energy. They use those devices all day, but overall they use less power than a few hours a week of an older desktop.

[Arnt]

My Apple ARM laptop has 24+ hours battery lifetime in practice, three times as much as the best laptop I had in the 2000-2015 period, and it's lighter than most of those laptops too. (Can't remember the battery lifetime of my 2015-2019 laptop.) Clearly not all efficiency gains have been eaten up.

[everdrive]

Agreed, it's not a perfect linear line, but if we how 2000-2015 computing requirements _and_ the Apple ARM efficiency we could be somewhere very special.

[bilekas]

> Similarly, nothing forces AI or 5G to use more power than whatever you would have done instead

Am I missing something or has the need to vast GPU horsepower been solved ? Those requirements were not in DC's before and they're only going up. Whatever way you look at it, there's got to be an increase in power consumption somewhere no ?

[Arnt]

Not necessarily, no.

You can pick and choose your comparisons, and make an incease appear or not.

Take weather forecasts as an example. Weather forecasting uses massively powerful computers today. If you compare that forecasting with the lack of forecasts two hundred years ago there obviously is an increase in power usage (no electricity was used then) or there obviously isn't (today's result is something we didn't have then, so it would be an apples-to-nothing comparison).

If you say "the GPUs are using power now that they weren't using before" you're implicitly doing the former kind of comparison. Which is obviously correct or obviously wrong ;)

[timschmidt]

GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long. There's nothing computationally special about AI compared to other workloads, and in fact it seems to lend itself to multiplexing quite efficiently - it's possible to process thousands of prompts for a negligable memory bandwidth increase over a single prompt.

AI is still very near the beginning of the optimization process. We're still using (relatively) general purpose processors to run it. Dedicated accelerators are beginning to appear. Many software optimizations will be found. FPGAs and ASICs will be designed and fabbed. Process nodes will continue to shrink. Moore will continue to exponentially decrease costs over time as with all other workloads.

[philipwhiuk]

> Moore will continue to exponentially decrease costs over time as with all other workloads.

There's absolutely no guarantee of this. The continuation of Moore's law is far from certain (NVIDIA think it's dead already).

[timschmidt]

> NVIDIA think it's dead already

Perhaps that's what Jensen says publicly, but Nvidia's next generation chip contains more transistors than the last. And the one after that will too.

Let me know when they align their $Trillions behind smaller less complex designs, then I'll believe that they think Moore's law is out of juice.

Until then, they can sit with the group of people who've been vocally wrong about moore's law's end for the last 50 years.

Our chips are still overwhelmingly 2D in design, just a few dozen layers thick but billions of transistors wide. We have quite a ways to go based on a first principles analysis alone. And indeed, that's what chip engineers like Jim Keller say: https://www.youtube.com/watch?v=c01BlUDIlK4

So ask yourself how it benefits Jensen to convince you otherwise.

[adgjlsfhk1]

progress continues, but at far slower rates than they used to. nvidia has gained ~6x density in the past 9 years (1080 to 5090), while a doubling every 2 years would be >20x density in 9 years. the past 6 years (3090) are even worse with only a 3x of density

[timschmidt]

Moore's law says nothing about density.

"The complexity for minimum component costs has increased at a rate of roughly a factor of two per year."

Density is one way in which industry has met this observation over decades. New processes (NMOS, CMOS, etc) is another. New packaging techniques (flip chip, BGA, etc). New substrates. There's no limit to process innovation.

Nvidia's also optimizing their designs for things other than minimum component cost. I.e. higher clock speeds, lower temperatures, lower power consumption, etc. It may seem like I'm picking a nit here, but such compromises are fundamental to the cost efficiency Moore was referencing.

All data I've seen, once fully considered, indicates that Moore's law is healthy and thriving.

[bilekas]

> GPU compute in datacenters has been a thing for at least 20 years. Many of the top500 have included significant GPU clusters for that long.

Of course they've been a thing, but for specialised situations, maybe rendering farms or backroom mining centers but it's disingenuous to claim that there's not an exponential growth in gpu useage.

[timschmidt]

Of course they've been a thing, but for specialized situations, maybe calculating trajectories or breaking codes but it's disingenuous to claim that there's not an exponential growth in digital computer usage.

Jest aside, the use of digital computation has exploded exponentially, for sure. But alongside that explosion, fueled by it and fueling it reciprocally, the cost (in energy and dollars) of each computation has plummeted exponentially.

[bilekas]

I really would like more of your data to show that, I think it would put this discussion to rest actually because I keep seeing articles that dispute it. At least older ones that ring bells, specifically https://epoch.ai/blog/trends-in-the-dollar-training-cost-of-...

[timschmidt]

You can find plenty of jumping off points for research here: https://en.wikipedia.org/wiki/Performance_per_watt

Along with this lovely graph captioned: "Exponential growth of supercomputer performance per watt based on data from the Green500 list." (note the log scale): https://en.wikipedia.org/wiki/Performance_per_watt#/media/Fi...

From the section about GPU performance per watt, I'll quote:

"With modern GPUs, energy usage is an important constraint on the maximum computational capabilities that can be achieved. GPU designs are usually highly scalable, allowing the manufacturer to put multiple chips on the same video card, or to use multiple video cards that work in parallel. Peak performance of any system is essentially limited by the amount of power it can draw and the amount of heat it can dissipate. Consequently, performance per watt of a GPU design translates directly into peak performance of a system that uses that design."

[palata]

> Nothing forces the rebound effect to dominate.

Not sure what to say to that. Yeah, it would be great if we didn't put so much resources into destroying our own world. I agree.

The fact is that rebound effect very much dominates everything we do. I'm not saying it should, I'm saying it does. It's an observation.

[Arnt]

I think you're telling me that graphs like these always point upwards, right?

https://ourworldindata.org/grapher/per-capita-energy-use?tab...

My own impression is that sometimes the aggregate total grows and sometimes it doesn't. And when it grows, sometimes that's because the rebound effect dominates.

[palata]

Read the part in your link that says "Here, energy refers to primary energy".

Primary energy in the US ignores the primary energy used in China for goods that end up being imported in the US.

[Arnt]

I googled briefly now, and found https://tradingeconomics.com/united-states/imports/china. Looking at that (note that the X axis is not zero) it's clear that Chinese imports cannot account for any significant part of the changes to the the USA graph on https://ourworldindata.org/grapher/per-capita-energy-use?tab...

[Analemma_]

There is some limit to the rebound effect because people only have so many hours in the day, but we’re nowhere near the ceiling of how much AI compute people could use.

Note how many people pay for the $200/month plans from Anthropic, OAI etc. and still hit limits because they constantly spend $8000 worth of tokens letting the agents burn and churn. It’s pretty obvious that as compute gets cheaper via hardware improvements and power buildout, usage is going to climb exponentially as people go “eh, let the agent just run on autopilot, who cares if it takes 2MM tokens to do [simple task]”.

I think for the foreseeable future we should consider the rebound effect in this sector to be in full force and not expect any decreases in power usage for a long time.