[janee]

Yes but you need to setup quite a bit of tooling to provide feedback loops.

It's one thing to get an llm to do something unattended for long durations, it's a other to give it the means of verification.

For example I'm busy upgrading a 500k LoC rails 1 codebase to rails 8 and built several DSLs that give it proper authorised sessions in a headless browser with basic html parsing tooling so it can "see" what affect it's fixes have. Then you somehow need to also give it a reliable way to keep track of the past and it's own learnings, which sound simple but I have yet to see any tool or model solve it on this scale...will give sonnet 4.5 a try this weekend, but yeah none of the models I tried are able to produce meaningful results over long periods on this upgrade task without good tooling and strong feedback loops

Btw I have upgraded the app and taking it to alpha testing now so it is possible

[majortennis]

I've tried asking it to log every request and response to a project_log.md but it routinely ignores that.

I've also tried using playwright for testing in a headless browser and taking screenshots for a blog that can effectively act as a log , it just seems like too tall an order for it.

It sounds like you're streets ahead of where I am could you give me some pointers on getting started with a feed back loop please

[grncdr]

> rails 1 codebase to rails 8

A bit off topic, but Rails *1* ? I hope this was an internal app and not on the public internet somewhere …

[janee]

haha no it's an old (15years old) abandoned enterprise app running on-prem that hasn't seen updates in more than a decade.

[sarchertech]

Wow Rails 3 came out 15 years ago, so that thing started life out of date.

[lsaferite]

> enterprise app

> started life out of date

That tracks my experiences.