[cynusx]

The comparison doesn't really hold.

He is comparing energy spend during inference in humans with energy spend during training in LLM's.

Humans spend their lifetimes training their brain so one would have to sum up the total training time if you are going to compare it to the training time of LLM's.

At age 30 the total energy use of the brain sums up to about 5000 Wh, which is 1440 times more efficient.

But at age 30 we didn't learn good representations for most of the stuff on the internet so one could argue that given the knowledge learned, LLMs outperform the brain on energy consumption.

That said, LLM's have it easier as they are already learning from an abstract layer (language) that already has a lot of good representations while humans have to first learn to parse this through imagery.

Half the human brain is dedicated to processing imagery, so one could argue the human brain only spend 2500 Wh on equivalent tasks which makes it 3000x more efficient.

Liked the article though, didn't know about HNSW's.

Edit: made some quick comparisons for inference

Assuming a human spends 20 minutes answering in a well-thought out fashion.

Human watt-hours: 0.00646

GPT-4 watt-hours (openAI data): 0.833

That makes our brains still 128x more energy efficient but people spend a lot more time to generate the answer.

Edit: numbers are off by 1000 as I used calories instead of kilocalories to calculate brain energy expense.

Corrected:

human brains are 1.44x more efficient during training and 0.128x (or 8x less efficient) during inference.

[wanderingmind]

Not just that the brain of a newborn comes pretrained with billions of years of evolution. There is an energy cost associated with that which must be taken into account

[eloeffler]

Then you must also take that cost into account when calculating the cost of training LLMs, as well as the cost humans operating the devices and their respective individual brain development.

LLMs are always an additional cost, never more efficient because they add to the calculation, if you look at it that way.

[Closi]

Only if we are counting the cost to generate all the inputs to training, and not just the training itself - it just depends on the scope of the analysis.

(i.e. taken to the extreme, as humans learn from their environment, do we have to count all energy that has gone into creating the world as we know it?)

[daflip]

If taken to the extreme I can't help but quote Carl Sagan :-)

"If you wish to make an apple pie from scratch you must first invent the universe"

[coldtea]

Well, LLMs also pressupose humans and evolution, since they needed us to create them, so their tally is even higher by definition...

[madhatter999]

Also take into consideration the speed of evolution. LLM training might be much faster because a lot of competition power is used for its training. Maybe if it was the same speed as evolution then it would take billions of years, too?

[eru]

Also our brains and our language are co-optimised to be compatible.

ChatGPT has to deal with the languages we already created, it doesn't get to co-adapt.

[thfuran]

Brains are only about half a billion years old.

[AbstractH24]

Those are sunk costs

[bamboozled]

Humans spend their lifetimes training their brain

I don't think this is true personally, ideally as children, we spend out time having fun and learning about the world is a side effect. This borg like thinking applied to intelligence because we have LLMs is unusual to me.

I learned surfing through play and enjoyment, not through training like a robot.

We can train for something with intention, but I think that is mostly a waste of energy, albeit necessary on occasion.

[Jensson]

> we spend out time having fun and learning about the world is a side effect

What do you think "play" is? Animals play to learn about themselves and the world, you see most intelligent animals play as kids with the play being a simplification of what they do as adults. Human kids similarly play fight, play build things, play cook food, play take care of babies etc, it is all to make you ready for an adult life.

Playing is fun since playing helps us learn, otherwise we wouldn't evolve to play, we would evolve to be like ants that just work all day long if that was more efficient. So the humans who played around beat those who worked their ass off, otherwise we would all be hard workers.

[bamboozled]

But I just play because it's fun, I roll dice for fun, are you trying to tell me all this is a secret front for "training" ?

[Jensson]

Fun is your brain rewarding you for something it thinks is practice on a useful skill. You get bored once you mastered it enough.

Some people continue playing a game even when it stops being fun, they are addicted to the reward mechanism in the game, and now the brain thinks that playing the game is a good way to work and provide for itself. I don't call that "play", its work, just not productive work.

Why is dice fun? Because your brain wants to map the pattern of the dice, trying to figure out how to get good rolls. You see that in most dice players, they develop a lot of superstition about what is good and bad dice, or how they always roll bad in critical moments etc. I'd assume that is from nature where you try to figure out what is a good nut to crack or where to find prey etc, basically a way to figure out useful patterns from random events.

[bamboozled]

So when I look at a sunset and enjoy that, I think it's fun to chase sunsets, my "brain" is telling me it's fun because I'm learning a new skill and receiving a dopamine reward for looking at the sunset and feeling good about it?

[Jensson]

Looking at a sunset isn't a game you play, kids don't go and play "look at the sunset", I feel like you are grasping at straws.

Why would you feel calm and comfortable from a sunset? Probably to get you sleepy so you go find a place to sleep since there isn't much useful to do at night. That would be unrelated to play.

Anyway, most of our feelings comes from nature, we didn't evolve to be faulty, we evolved to do things efficiently, play is a part of efficiency. If it isn't for learning you would have to explain what it is more likely to be for. When kittens play and chase things or play fight with each other, do you think they are just wasting energy for no reason? No, they sharpen their senses and learn to hunt and fight.

[gnramires]

It's probably not something you secretly wish when playing, and perhaps for the best (that you don't have fun and enjoy things always with an ulterior motive, except enjoying the experiences). I guess he's saying in the sense of the natural function of play, and we playing is mostly a consequence of the natural function. It's also very much true that we learn a lot, perhaps a significant chunk of what we learn is through play[1], so it's also undeniable that we do learn from play, even if humans have this great gift -- we are able to understand the nature of things (such as play) and choose to do them just for the sake of experiences, fun, joy, happiness, etc..

Which we should (finally :) ) recognize to be the source of all meaning.

We still should learn (and do practical stuff in general) because it supports our inner lives, including building technology, producing things (buildings, infrastructure) that support us and indeed enables our (inner) lives.

[1] Also of note humans, unlike LLMs, can learn all the time, we don't have a hard "training phase". It's true brain plasticity decays, and it becomes harder to learn as we age, but we can still learn more or less quickly at any age. This is why dedicating childhood to learning (as well as play) is natural.

[bamboozled]

I'm conscious I have to types of play though, I'm fully conscious sometimes play can be about learning and training, it's why I ski more difficult terrain than I'm comfortable with, but then I might go eat a massive bowl of pasta and have a glass or three of wine. On an intellectual level, I know there are healthy more rewarding things to eat. I know wine isn't great for me at those quantities. But I consciously make the choice to do it because it's fun.

[Jensson]

I don't think that many calls indulging in food or bodily needs "play". Those are just core rewards, play is an active activity that is fun without being directly related to your survival, like eating is.

[gnramires]

Yes, although I say, if you can make your play healthy as well as fun, might as well :)

[james-bcn]

Yes, evolution makes play fun, but it's really learning, in the same that that evolution has made sweet and fatty things extra tasty, because they are full of energy.

[bamboozled]

This is “common sense” but knowing all we know about food and calories, we still eat the donut…

[nkrisc]

Jokes on you. Every time you played ball you were secretly learning about ballistic trajectories and estimating velocities using visual cues such as apparent angular size and parallax.

[grugagag]

The brain uses heuristics for that

[Jensson]

Heuristics that you practice and finetune via play, for example by throwing and catching balls.

[nkrisc]

Being a father to two young kids, I can confidently say we aren’t born with those heuristics already tuned.

[glenstein]

>we spend out time having fun and learning about the world is a side effect

I think the part of this that resonates as most true to me is how this reframes learning in a way that tracks truth more closely. It's not all the time, 100% of the time, it's in fits and starts, its opportunistic, and there are long intervals that are not active learning.

But the big part where I would phrase things differently is in the insistence that play in and of itself is not a form of learning. It certainly is, or certainly can be, and while you're right that it's something other than Borg-like accumulation I think there's still learning happening there.

[bamboozled]

Sorry I didn't mean to imply that learning isn't part of play, I just don't believe the end goal of life is to "train". I think if you're an AI researcher it would make sense that life is about training. However I think it's just fashion.

I always think if we could built an AGI, it would probably enjoy some form of play too. It would need to invent some level of excitement, else it would just be a machine with no ambition, no inspiration.

[pessimizer]

That's like saying that you eat because it tastes good.

[Closi]

I think you would probably have to take into account the full functioning power of a human too.

We don't know how to fully operate a human brain when it's fully disconnected from eyes, a mouth, limbs, ears and a human heart.

[londons_explore]

> At age 30 the total energy use of the brain sums up to about 5000 Wh,

That doesn't sound right... 30 years * 20 Watts = 1.9E10 Joules = 5300 kWh.

[cynusx]

Where did you get the 20 Watt from?

My number is based on calorie usage

[cynusx]

oh ok, I used 400 calories/day and not 400 kcal/day.

Yea, then the numbers are off by 1000

[1992spacemovie]

I respect that you replied to the comment and owned your math error :) The rest of your comment is an interesting observation. Never thought about it starting out at the cal level.

[CuriouslyC]

You're doing apples and oranges.

Humans who spend a long time doing inference have not fully learned the thing being inferred - unlike LLMs, when we are undertrained, rather than a huge spike in error rate, we go slower.

When humans are well trained, human inference absolutely destroys LLMs.

[cheema33]

> When humans are well trained, human inference absolutely destroys LLMs.

This isn't an apt comparison. You are comparing a human trained in a specific field to an LLM trained on everything. When an LLM is trained with a narrow focus as well, human brain cannot compete. See Garry Kasparov vs Deep Blue. And Deep Blue is very old tech.

[karmakaze]

Also DeepBlue isn't an ML it's an "expert system, relying upon rules and variables defined and fine-tuned by chess masters and computer scientists" from Wikipedia. AlphaGo (or AlphaGo Zero) would be a better example.

[eru]

> AlphaGo (or AlphaGo Zero) would be a better example.

Yes, they are better examples, but still not great examples: neither of them are LLMs.

In general, I have very high hopes for AI, but I would be surprised if LLMs are the one universal hammer for every nail. (We already have lots of other network architectures.)

[CuriouslyC]

1. Deep blue isn't a LLM. I don't care how well you train a LLM, it's not going to be more efficient than an optimally trained human, not even close. It's actually arrogant as hell to assume that we can achieve a higher level of energy efficiency than billions of years of evolution, particularly so early in the game. 2. Chess is a closed form system with a finite and relatively small number of position compared with the real world.

[eru]

> It's actually arrogant as hell to assume that we can achieve a higher level of energy efficiency than billions of years of evolution, particularly so early in the game.

You are right that LLMs are still far off from the performance of the human brain. Both in absolute terms, and also relative to the power used.

However, I don't see anything arrogant here. We have lots of machines that can do many tasks more energy efficient (and better) than humans. Both mechanical and intellectual tasks.

[CuriouslyC]

It's not arrogance to think you can create a tool that does one thing the brain does better than the brain for less power. It's arrogance to think that you can do everything the brain does for less power. Living organisms have been relentlessly honed for the ability to efficiently solve varied problems across ~10^40 experiments over the age of the earth. If some marginally intelligent monkeys think they can build an error corrected, digital system that encompasses all of that functionality while using less power, I'd say that's obviously arrogance, particularly if it hasn't been the subject of a civilizational drive for a few millennia already.

[eru]

> Living organisms have been relentlessly honed for the ability to efficiently solve varied problems across ~10^40 experiments over the age of the earth.

Evolution has been optimising them for creating descendants, not general problem solving with minimum energy expenditure.

No one expects that LLMs can solve all problems: they can't. They can only predict text, nothing else. They can't fight off a virus infection or evade a lion. Specifically, LLMs can't reproduce at all either, yet alone efficiently. Reproduction is what evolution is all about.

[cynusx]

Depends on the person I guess, but yes. Humans are more accurate for now.

[greenthrow]

The article is a bit of a stretch but this is even more of a stretch. Humans can do way more than an LLM, humans are never in only learning mode, our brains are always at least running our bodies as well, etc.

[glenstein]

Exactly right - we are obviously not persistently in all-out training mode over the course of our lifetimes.

I suppose they intended that as a back-of-the-envelope starting point rather than a strict claim however. But even so, gotta be accountable to your starting assumptions, and I think a lot changes when this one is reconsidered.

[bufferoverflow]

Also, human brains come pre-trained by billions of years of evolution. It doesn't start as a randomly-connected structure. It already knows how to breathe, how to swallow, how to lean new things.

[bognition]

If we’re going to exclude the cortical areas associated with vision, you also need to exclude areas involved in motor control and planning. Those also account for a huge percent of the total brain volume.

We probably need to exclude the cerebellum as well (which is 50% of the neurons in the brain) as it’s used for error correction in movement.

Realistically you probably just need a few parts of the lambic system. Hippocampus, amygdala, and a few of the deep brain dopamine centers.

[philipov]

A lot of our cognition is mapped to areas that are used for something else, so excluding areas simply because they are used for something else is not valid. They can still be used for higher-level cognition. For example, we use the same area of the brain to process the taste of disgusting food as we do for moral disgust.

[pama]

Thanks. So after your corrected energy estimate and more reasonable assumptions it appeaars that the clickbaity title of the article is off by more than 7 orders of magnitude. With the upcoming NVidia inference chips later this year it will be off by another log unit. It is hard for biomatter to compete with electrons in silicon and copper.

[mirekrusin]

Also you can't cp human brain.

[vasco]

We can clone humans at current level of technology, otherwise there wouldn't be agreements about not doing it due to the ethical implications. Of course its just reproducing the initial hardware and not the memory contents or the changes in connections that happen at runtime.

[mirekrusin]

Well, we know how to make kids, but then cp takes 20 years and rarely works.

[marginalia_nu]

You kinda can do a sort of LoRA though. Reading the right book can not only change what you hold true, but how you think.

[Rinzler89]

The plot of The Matrix would beg to differ.

[phantompeace]

Not yet, anyway.

[freehorse]

> representations for most of the stuff on the internet

Yes we have learnt far more complex stuff, ffs.

[jryan49]

How about the fact that llm's don't work unless humans generate all that data in the first place. I'd say the llm's energy usage is the amount it takes to train plus the amount to generate all that data. Humans are more efficient at learning with less data.

[Closi]

Humans also learn from other humans (we stand on the shoulders of giants), so we would need to account for all the energy that has gone into generating all of human knowledge in the 'human' scenario too.

i.e. not many humans invent calculus or relativity from scratch.

I think OP's point stands - these comparisons end up being overly hand-wavey and very dependent on your assumptions and view.

[jryan49]

Yes I agree. The whole concept of trying to compare energy usage is incredibly complicated.

[dist-epoch]

For every calorie a human consumes, hundreds or thousands more are used by external support systems.

So yeah, you do use 2000 calories a day, but unless you live in an isolated jungle tribe, vast amounts of energy are consumed on delivering you food, climate control, electricity, water, education, protection, entertainment and so on.

[b112]

By that metric, the electricity is only part of it. The cost of building the harsware, the cost of building the roof and walls for the datacentre, the cost of clearing the land, cost of humans maintaining the hardware, the cost of all the labour making the linux kernel, libc6, etc, etc. Lots of additionals here too.

[bamboozled]

It's almost like...nothing exists in a vacuum.

[greenthrow]

Are you going to include all the externalities to build and power the datacenters behind LLMs then? Because i guarantee those far outweigh what it takes to feed one human.

[unyttigfjelltol]

Including support from ChatGPT. It really is a comparison of calories without ChatGPT and calories with, and that gets to the real issue of whether ChatGPT justifies its energy intensity or not. History suggests we won't know until the technology exits the startup phase.