[JyB]

Assuming you're using GitHub or similar, make pushing branches and creating PRs part of their prompts, and review on the GitHub app or equivalent? Seem like an orthogonal problem to those LLMs tools.

[stavros]

I can review the code that way, but not the outputs. I could have it write tests and run them, but that's usually fiddly for web apps.

[johntash]

If it's a webapp, have the CI pipeline create a temporary env and deploy the branch to it?

[stavros]

Yeah, that's basically the only solution I can think of, but requires quite a bit of infra work. I guess that's what LLMs are for, huh...

[_1tem]

Checkout the PRs to your local machine and test it there?

[stavros]

That kind of defeats the purpose of running the agents in the cloud, no?