[edmundsauto]

I don’t think this is an apt comparison, but I do think the amount of energy it takes to grow a human into brain maturity in adulthood is an interesting one. Brains + bodies over a 20 year development cycle is still probably much less than training even a low quality Llm.

[4death4]

Let’s say a human needs an average of 2000 calories a day. A calorie is roughly equivalent to 1 Watt hour, so over 20 years, it takes about 15 MWh to sustain a human.

Let’s say a single A100 has a peak power draw of 250W, and you need 100 to train an LLM. So each hour of training consumes 25,000 Wh of energy. 15 MWh / 25,000 W = 600 hours, or 25 days, which is probably pretty close to the true training time.

So the numbers are actually pretty close. But a human brain doesn’t start out as a set of random weights like an LLM. The human brain has predefined structure that’s the result of an extremely long evolutionary process.

[edmundsauto]

It's probably taking the analogy too far, but perhaps the brain's predefined structure is akin to the original LLM training and our "life" is the fine tuning.

I wonder how many MWh the entirety of evolution represents.