[eachro]

That's very impressive. Hats off to them! I dont think this is too out of the ordinary either though. I'd guess they started off with a LLM from hugging face, set up some pipeline to ingest code from replit repos to finetune the LLM. The ML aspect of this is not terribly hard given that they probably dont need to train a LLM from scratch. Figuring out how store and serve from replit repos (or publicly available code bases) is not too difficult. From there it's a matter of productionalizing: how to serve the model in real time, figuring out they want the product to look/feel like and I suppose this part of it might take a while. I'd estimate you'd need 1-2 ML engineers, 2 data engineers, 2-3 swes, 1 PM for the team for a minimal viable product.

[giansegato]

yep, true! however, the devil is in the details. from what i've been told, the big challenge was latency: they worked a lot to bring the latency down to acceptable levels - essentially to be usable in a cloud IDE

iirc the team managed to bring it to a lever an order of magnitude lower than off-the-shelf models

[nerdponx]

8 weeks is impressive for something like that, and it goes to show just how powerful our off-the-shelf tools have become.

I think it's also a bit scary, because 8 weeks is very little time for testing, tuning, and validation of something as opaque as a machine learning model. If it worked right the first time, that's great. But there is still a lot of inherent uncertainty in ML projects. Decision makers need to take that uncertainty into account when planning.

That, or, the 8 weeks only covers the final training runs and the implementation/deployment, and doesn't include time spent developing and tuning proof-of-concept prototype models.

[mradek]

In 2022 you test live in production lol

[conductr]

It’s not life or death though. It’s just giving some useful boilerplate that you’ll have to touch to make useful. They can learn as they go.