|
|
|
|
|
|
by zug_zug
865 days ago
|
|
|
I feel like it's incredibly likely that we find an algorithm 100x as performant for less than 10 trillion dollars. I say that mostly because we built an algorithm playing around with math and plugging in different functions and it seems to work in a way that we fundamentally don't understand. As evidence - if I learn a new game, like chess, my rate of mastery per game is going to be orders of magnitude higher than an AI (which will take millions or billions of training sessions). Granted my brain has more synapses than GPT4, but it's unclear how many of those I use playing chess (I wouldn't be surprised if it's less). And regardless I don't think that anybody's arguing that adding more layers accelerates training but rather increases the peak. So perhaps a place to start is studying cases where AIs learn much slower than humans and trying to understand better algorithms of training/reinforcement that may come closer to what the human brain does. |
|
|
We may eventually build machines that are able to do that, but it may well take us enormous amounts of "brute force" training in order to produce something that then no longer uses brute force to learn further.
As an analogy, look at the history of CPU design. The first CPUs have been designed manually, even "drawn" manually. Each generation of CPUs empowered engineers of the future generation to build more complex tasks because they could tap the computation power to assist them in the design and realization of the circuits.
Development is incremental.
Yes, sometimes you can come up with some new groundbreaking idea that will invalidate what's currently being worked on, but usually these ground breaking ideas come up later when the ground is fertile because you already live in the next generation.