[callamdelaney]

AI trains AI already, agents are happy to spin up real training pipelines for deep learning or regression models or whatever you want right? I guess the advantage to your project is that it provides a framework to allow the agent to access extra compute?

[Danau5tin]

Yes I'd heard the labs (Anthropic mostly) speaking about LLMs training LLMs, so I wanted to make things a little more concrete and test it out myself! Essentially you are correct though, my framework allows the agent access to compute, but also the agent itself is being trained to become better at training models with that compute.

[vorticalbox]

i remeber reading in one of the release blog posts that that version was the "first that codex helped train"

[Danau5tin]

Yes exactly, and also there is auto-research by Karpathy, Fable-5 on parameter golf etc which point to the direction of highly-assisted / autonomous AI training as the future. Wild.

[amelius]

I guess more important than LLMs training LLMs (which is mostly the same code over and over) is LLMs cleaning and curating training data.

[Danau5tin]

Very good point. Sifting through noisy data and creating curated datasets could be of great value. Perhaps worth a project by itself!

[throwa356262]

Not sure how much of it was marketing but M2.7 supposedly trained itself

https://www.minimax.io/blog/minimax-m27

[Danau5tin]

Very cool! Thank you