Hacker News new | ask | show | jobs
by nn3 2011 days ago
Also birds (and insects/bats/pterosaurs) flight is a lot more energy efficient than any plane. Today's deep learning is essentially brute force, burning thousands of watts for anything more complicated which a single human brain can often do in ~15Watts.

The advanced models like GPT-3 are burning millions of watts in the cloud but they're not that much better than what a brain can do (and in many ways worse, as in often requiring supervised learning)

That's the key point. The algorithms need to become more energy efficient to make significant leaps, thus become more like brains.

3 comments

Also, birds produce themselves out of an egg, with only food, water and air as production input. They also can produce more of themselves with minimal input. They are also self-repairing/maintaining, something planes cant do.
> single human brain can often do in ~15Watts

There were similar arguments when AlphaGo showed up and beat master Go player Lee Sedol, but is power(in Watt) the right measurement? I always feel like it should be the total energy(in J or Cal) required to transform a computing device like biological brain or electrical computer from knowing nothing to being capable of a skill like Go game. In such sense, deep learning is still more energy efficient than human.

Lee Sedol's lifetime energy consumption for all biological process is around 50MWh, Alpha Go consumed more than 100kW and was likely to have been trained more than 500h, so even counting all the energy Sedol spent eating, dancing or having sex ends up being less than the amount of energy spent to train the machine used to beat him.
No, it's not. What's your comparison? Are there birds that can carry 80,000 lb of passenger + cargo weight? Condors fly like fixed-wing aircraft for 99% of their flight, hummingbirds fly more like insects. There isn't one type of bird flight.

This whole HN discussion of bird flight is a trainwreck and reflects massive gaps in understanding of aerodynamics. This is '00s "computer virus news report" level competence in this subject.

We understand the aerodynamics of bird flight, and used it to make fixed-wing planes optimized for carrying lots of cargo. Once we understand the principles behind intelligence, we can make very efficient AI optimized for our usage. But we're still at the point where we don't understand intelligence as well as we understood aerodynamics when building the first planes, so we still have a lot to learn from "birds" - animal brains.
> But we're still at the point where we don't understand intelligence as well as we understood aerodynamics when building the first planes

Actually, I'd say that our understanding of intelligence is right about at the level of aerodynamics at the dawn of heavier than air flight:

https://youtu.be/Sp7MHZY2ADI

https://youtu.be/gN-ZktmjIfE

I mean, we could quibble about exactly where we are pre- or post-Wright Flyer, but given the amount of AI research that amounts to brute-force flailing about in search of incremental improvements, disagreements on the importance of "biological plausibility" and so on, it's pretty clear that, roughly speaking, AI is currently somewhere in the equivalent of the Lilienthal-Langley-Wright-Curtis continuum (ie. 1890-1910-ish) and still prior to the most important theoretical breakthroughs. IOW, AI has not in my opinion yet achieved an equivalent to aerodynamics' Prandtl lifting-line theory: https://en.m.wikipedia.org/wiki/Lifting-line_theory

I believe AI will start as a basic principle or idea that can be applied to any sufficiently big state machine that controls e.g. an RC airplane or traffic lights. That idea will be obvious in a hindsight. I'd even make a guess that it will be like a "stateful" state machine that accumulates state in a particular manner and uses that to control the underlying state machine. We still will be nowhere near understanding intelligence, but that clever trick will be enough in most cases.