[runarberg]

> Are you confident that will be the case in 5-10 years?

Yes, I am. I am very confident that general purpose digital computers will never be more efficient then human minds in generating moderately complex code.

Why am I so confident... Well, it has been over 10 years since AlphaGo beat top go player Lee Sedol. AlphaGo was able to beat the a world class go player by doing several thousands orders of magnitude more computations then Lee Sedol, and it did so by spending several orders of magnitude more energy then the top human go player. Today, over 10 years later, the top go machines are able to beat world class go players much easier, but still do so using the exact same strategy of outcomputing the humans with thousands of orders of magnitude more computations, and spending orders of magnitudes more energy.

Things did not change in the past 10 years, I see no reason why it should change 10 years from now.

[supern0va]

>Things did not change in the past 10 years, I see no reason why it should change 10 years from now.

Has it not? Why do you say that?

Also, do we still require a Deep Blue sized supercomputer for chess? :)

[runarberg]

What has not change is the strategy of throwing a gargantious amount of computations at the problem. If anything we throw more computations at more problems now than in 2016 (and in 1997 for that matter). The underlying technology is pretty much the same, just more parameters, more calculations, etc. Yes every individual calculations takes less power now then in 2016, but we make up for that by making millions of millions of more calculations, even for simpler tasks.

[supern0va]

Sure, but there will be an upper bound after which we will be close to human level performance on the vast majority of tasks, and then at that point the focus becomes efficiency (or a continuing road to superintelligence for some tasks).

But regardless, compute will get to a point where human level intelligence close to as efficient as we are. You could argue it already is today, when you factor in the resources that the average person in the west already uses in terms of their overall impact on the planet.

[runarberg]

You are describing a science fiction. There is nothing in the measured reality which indicate your predictions will come close to materialize.

I can just as well describe the future evolution of the internal combustion engine and claim it will get more and more efficient and eventually we will be able to burn oil so efficiently that our personal vehicles can fly through the atmosphere at twice the speed of sound.

There is limitations to digital computers just as there are limitations to internal combustion engines. Our brains are not digital computers. When we use our brains we don’t just do a bunch of linear algebra.

[supern0va]

>I can just as well describe the future evolution of the internal combustion engine and claim it will get more and more efficient and eventually we will be able to burn oil so efficiently that our personal vehicles can fly through the atmosphere at twice the speed of sound.

This is a silly comparison. There is a certain quantity of energy stored in oil, so we know what peak efficiency looks like. We don't actually know what amount of energy is required to solve certain problems. We quite literally have models with quite a bit of capability that can run locally on a phone today, right alongside Stockfish, for example.

And this is to say nothing of work happening now on new hardware approaches, such as Normal Computing's work on thermodynamic matrix math: https://www.normalcomputing.com/blog/a-first-demonstration-o...

That said, this feels like a strange tangent: I'm not sure it's that important that the models be as energy efficient as a human brain. We don't avoid cars because they're less energy efficient than our legs. ;)

[runarberg]

Point is that both are science fiction narratives and neither reflect reality in any way what-so-ever. How fast a car can drive and how much a LLMs can compute are bounded quantities, limited by the physical reality. In both cases we can imagine a world where this limit does not exist, but that is not the reality we live in.

This matters because unlike cars LLMs are only doing stuff we can already do using our brains, just several orders of magnitudes less efficiently. Cars can at least take us distances we would never be able to using our muscles. In comparison, if I need to compile CPython into a WASM binary I can simply download a library that does it, or copy paste code in a few seconds, for a million billionth of the energy it takes an LLM to do the same. Except when I download the library or copy-paste the code I (hopefully) attribute the original author and give them credit for their work.