I feel like it's incredibly likely that we find an algorithm 100x as performant for less than 10 trillion dollars. I say that mostly because we built an algorithm playing around with math and plugging in different functions and it seems to work in a way that we fundamentally don't understand.
As evidence - if I learn a new game, like chess, my rate of mastery per game is going to be orders of magnitude higher than an AI (which will take millions or billions of training sessions). Granted my brain has more synapses than GPT4, but it's unclear how many of those I use playing chess (I wouldn't be surprised if it's less). And regardless I don't think that anybody's arguing that adding more layers accelerates training but rather increases the peak.
So perhaps a place to start is studying cases where AIs learn much slower than humans and trying to understand better algorithms of training/reinforcement that may come closer to what the human brain does.
Yeah, your brain has been pre-trained over an incredibly long evolutionary period to be able to learn stuff efficiently.
We may eventually build machines that are able to do that, but it may well take us enormous amounts of "brute force" training in order to produce something that then no longer uses brute force to learn further.
As an analogy, look at the history of CPU design. The first CPUs have been designed manually, even "drawn" manually. Each generation of CPUs empowered engineers of the future generation to build more complex tasks because they could tap the computation power to assist them in the design and realization of the circuits.
Development is incremental.
Yes, sometimes you can come up with some new groundbreaking idea that will invalidate what's currently being worked on, but usually these ground breaking ideas come up later when the ground is fertile because you already live in the next generation.
True understanding will always topple these attempts at brute forcing our way through, but here we are.
I really think we should focus on the science first, as you've hinted at, and these desperate pursuits for compute are making that ever more clear.
We already know that neural networks don't require the high precision of classical CPUs/Gpus. Photonics are many orders of magnitude more efficient at these imprecise matrix multiplications. And thanks to Hintons forward forward architecture there could be a direct implementation without too many photon electron bridges/clutches. If you know any company working on this please dm me, I'm willing to quit my job and to bet my life savings that it will indeed become a trillion dollar company in no time. When OpenAI is the new microsoft this will be the new Intel.
As evidence - if I learn a new game, like chess, my rate of mastery per game is going to be orders of magnitude higher than an AI (which will take millions or billions of training sessions). Granted my brain has more synapses than GPT4, but it's unclear how many of those I use playing chess (I wouldn't be surprised if it's less). And regardless I don't think that anybody's arguing that adding more layers accelerates training but rather increases the peak.
So perhaps a place to start is studying cases where AIs learn much slower than humans and trying to understand better algorithms of training/reinforcement that may come closer to what the human brain does.