[latentsea]

And you can't think of a way to bring an engineers mindset to solve that?

If you leave it up to raw LLM outputs you're not going to get the results you're after. If you put on your engineering hat and think about your problem statement and what tools you can bring to bear on it you'll find you can likely craft a PR authoring skill and a set of evals for that skill that satisfies your requirements with a high enough degree of reliability.

[danpalmer]

I'm not coming at this cold, I've worked on the skill for PR (CL) creation in the Google monorepo. I'm not talking about raw LLM outputs here. It's a hard problem that is fairly fundamental to the training material being used for LLMs today.

I'm sure we can improve things, I'm sure it will get better over time, but I'd say we're a small fraction of the way towards where we need to be, and trending in the wrong direction with long trajectory development creating bigger changes on average.