[wreath]

> Then what will AI be trained on in let's say 20-30 years? Old code? It's own AI developed code for vibe coded projects?

I’ve seen variation of this question since first few weeks /months after the release of ChatGPT and I havent seen an answer to this from leading figures in the AI coding space, whats the general answer or point of view on this?

[righthand]

The general answer is what they’re already doing: ignoring the facts and riding the wave.

[lowbloodsugar]

Is it hard to imagine that things will just stay the same for 20-30 years or longer? Here is an example of the B programming language from 1969, over 50 years ago:

  printn(n,b) {
   extrn putchar;
   auto a;

   if(a=n/b) /* assignment, not test for equality */
      printn(a, b); /* recursive */
   putchar(n%b + '0');
  }

You'd think we'd have a much better way of expressing the details of software, 50 years later? But here we are, still using ASCII text, separated by curly braces.

[SoftTalker]

I observed this myself at least 10 years ago. I was reflecting on what I had done in the approximately 30 years I had been programming at that time, and how little had fundamentally changed. We still programmed by sitting at a keyboard, entering text on a screen, running a compiler, etc. Some languages and methodologies had their moments in the sun and then faded, the internet made sharing code and accessing documentation and examples much easier, but the experience of programming had changed little since the 1980s.

[sockaddr]

I suspect a more general and much more clever learning algorithm will emerge by then and will require less training data to get to a competent problem solving state faster even with dirty data. Something able to discriminate between novel information and junk. Until then I think there will be a quality decline after a few more years.

[c22]

How will it emerge? In the past we've been told that the a(g)i will write itself, rapidly iterating itself into a super intelligence that handily solves all our current and future problems, but it's beginning to look like a chicken or the egg scenario.

Living systems were able to brute force their way to human brain, but it took billions of years and access to parallel processes that make the entire collective history of human computation seem like a mote to a star.

What novel spark do you see accelerating this process to such a hyperbolic extreme?

[plopz]

I would imagine a trajectory similar to AlphaGo, it starts out trying to replicate humans and then at a certain point pivots to entirely self-play. I think the main hurdle with llms, is that there isn't a strong reward target to go after. It seems like the current target is to simply replicate humans, but to go beyond that they will need a different target.

[c22]

I agree in general, but defining an appropriate target seems intractable at the moment. Perhaps it is something the AIs will have to define for themselves.

I think real intelligences are working with myriad such targets, but an adversarial environment seems essential for developing intelligence along this axis.

I do think if there's a path to AGI from current efforts it will be through game play, but that could just be the impressionable kid who watched Wargames in the 80s speaking through me.

[Jean-Papoulos]

It took a billion years to get to the tool-making state, and then less than a 1000th of that time to making CPUs. Then a 1000th of that time to make LLMs. We are in a parabolic extreme

[c22]

This is begging the question. What evidence is there that this is all the same "stuff" driving towards some future apex? What does it mean to "get to" the tool making state outside of a Civ-style video game?

[Eisenstein]

I don't think that more code makes LLMs better at writing code past a certain point.