My reaction too. They've reinvented genetic/evolutionary programming. They should probably read some of the decades of work that have already been done on it.
The paper [1] cites Koza among a total of 102 citations.
"An early example of a symbolically discovered optimizer
is that of Bengio et al. [8], who represent F as a tree: the
leaves are the possible inputs to the optimizer (i.e. the xi
above) and the nodes are one of {+, −, ×, ÷}. F is then
evolved, making this an example of genetic programming
[36]. Our search method is similar to genetic programming
but we choose to represent the program as a sequence of
instructions—like a programmer would type it—rather than
a tree. "
"[36]" is "Koza, J. R. and Koza, J. R. Genetic programming: on
the programming of computers by means of natural
selection. MIT press, 1992."
Same here. When I studied genetic programming, I was hoping that's where problem solving evolve from as it was flawless. But recent events prove otherwise which made me believe we are using the wrong tool for the wrong problem. Here is why.
When AI gets to 100% accuracy, the equation to find the answer becomes 100% accurate. We no longer have to run the AI with heavy resources and equation can be converted to an executable program. This modal of AI will save computing power, and uses resources smartly.
Example.
AI tries to find right equation to add two numbers.
You might not be familiar with how neural networks work. When training they do use a lot of computing power. But when running they don't. Yes, they still require some external boilerplate code to multiply the matrices, but you already have it and it's not heavy. So yes there is some convenience in program synthesis in a human programming language, but it is a small convenience, not a game changer.
If the solution program is sufficiently complex (as one would imagine it to be in non-trivial cases where we use AI, e.g. computer vision, speech synthesis, etc.), what makes you think the solution program is going to be more lightweight than running inference on an "AI model"? Futhermore, what guarantee do you have that the discovered solution is going to be efficient w.r.t. computation at all?
"An early example of a symbolically discovered optimizer is that of Bengio et al. [8], who represent F as a tree: the leaves are the possible inputs to the optimizer (i.e. the xi above) and the nodes are one of {+, −, ×, ÷}. F is then evolved, making this an example of genetic programming [36]. Our search method is similar to genetic programming but we choose to represent the program as a sequence of instructions—like a programmer would type it—rather than a tree. "
"[36]" is "Koza, J. R. and Koza, J. R. Genetic programming: on the programming of computers by means of natural selection. MIT press, 1992."
[1] https://arxiv.org/pdf/2003.03384.pdf