[ivorius]

> - Negative: Submitters just add stylistic markers to make their accounts and output seem human-generated. This is like syntactic sugar: the core content and the size of contributions stay the same, but the style gets quirkier.

From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.

> Positive: Submitters actually provide to-the-point, no-bullshit commits and comments

That would be a dream.

[QuantumNomad_]

> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.

True. At least with a policy about it, the project maintainers can unilaterally close such PRs without further internal or external discussion on any case-by-case basis.

[maybewhenthesun]

Dingdingding, we have a winner. The main use of such a policy is to be able to just close those giant wall-of-text PRs and have something to point to when people start to scream it's not fair.

[mcphage]

> when people start to scream it's not fair

Or LLMs, as we have seen.

[chrisjj]

Or, who knows, the "AI" might gain sufficient intelligence to read the policy...

[julianlam]

Then we'll amend the policy to instruct LLMs to author PRs as Fred Flintstone, yabba dabba doo!

[tayo42]

Why is a policy necessary. you were never entitled to have your pr merged in the first place? If pr wasn't reviewable pre AI I'd expect it to be closed or ignored too

[the_hoser]

The policy isn't necessary to close the PR. The policy just helps to shut down the ensuing discussion after closing the PR. It helps in quickly dealing with well-meaning onlookers asking for clarification when you block PRs from the account.

[tayo42]

Still, your not even entitled to a discussion though.

[bluGill]

Before AI a large pull request was enough effort to make that you could assume good faith work on the part of everyone. Likely even if it is bad for architectural reasons it is solving an itch other users have and it is reasonable for them to want an explanation why you refused someone who made this much effort. And since the effort required meant it didn't happen often it wasn't a big deal to provide that.

These days large PRs are easy to create and so humans need to shut them down.

[LordDragonfang]

Whether or not someone is entitled to something has very little bearing on whether someone believes they are entitled to something (and are willing to waste everyone's time to make a stink about it). Having clear rules to point to, even after the fact, is surprisingly effective in mitigating that.

Or, put more bluntly, your belief in what people ought to feel entitled to has no bearing on what they do believe, and policy needs to address the latter, not the former.

[jononor]

Prior to AI making a PR involved considerable effort from a human. So the default position for many open source projects was that it deserved some level of attention for the effort. Even if many projects in practice would struggle to review every PR. But with AI tools this dynamic has shifted dramatically - many PRs have basically zero effort been put into it. Additionally there are many more of them, and often way bigger also.

[whateverboat]

But now with AI, this should be "easier" for some definition of easy. In the sense that in the past, this might have taken 15 minutes to write, now with AI, this can take 5 minutes to write by first getting AI to produce a summary and then using human judgement to make it better. So, it's a good idea now to actually demand the dream.

[ivorius]

If people knew how to get AI to write terse, focused summaries, sure, that might help. I haven't seen many that do (well, ignoring the toupee fallacy).

Though the most important aspect is that we need to know the motivation and thought process, and all AI can do is fabricate a 'plausible' one.

[adalacelove]

Reading AI PRs reminds me of Monty Python's holy grenade:

"And the Lord spake, saying, ''First shalt thou take out the Holy Pin. Then shalt thou count to three, no more, no less. Three shall be the number thou shalt count, and the number of the counting shall be three. Four shalt thou not count, neither count thou two, excepting that thou then proceed to three. Five is right out. Once the number three, being the third number, be reached, then lobbest thou thy Holy Hand Grenade of Antioch towards thy foe, who, being naughty in My sight, shall snuff it.'

[dsign]

I wouldn't mind reading that and having a good chuckle while processing an MR, as long as the comment had been crafted by a person. But now I think writing in grunts is gonna become the thing. "pete? you good boy? we won't hire you/ paper you gave HR girl with older gigs? remember it pete my boy? too many words/ words were too long/ dots you used dots/ you scratched long dash with knife, but baby saw scratch/we don't ai pete/ask if they have job in next cave/good luck."

[lokar]

I have spent many years reviewing and editing design and product documentation from more Jr engineers and product managers.

It was a constant struggle to get them to be concise, one I mostly lost.

[chrisjj]

> If people knew how to get AI to write terse, focused summaries...

... then flooded maintainers would be doing it.

[ray_kay777]

I've found that the instructions "be extremely concise" gets me much closer to output that's actually sensible/helpful rather than another wall of text.

[watwut]

> From my experience reviewing, most contributors never read the policies, especially those making a "quick AI PR". I don't expect the new policy to change this much.

The policy allows the reviewer to reject it on the "AI" grounds.

[dspillett]

> allows the reviewer to reject it on the "AI" grounds

… but still unfortunately leaves reviewers having to spend time checking submissions and rejecting them.

[jon-wood]

At least half the people firing off LLM generated PRs will have left the "Coauthored-by: Claude" line on it allowing automated rejections.

[ivorius]

Unfortunately, only a single PR like this comes to mind. Most AI authors we've seen were identifiable mainly by overly verbose PR descriptions, meaningless code changes and copy-pasting more AI output when questioned.

[fendy3002]

oh but it'll be very2 helpful and the time spent will be short. It's easy to verify:

* new contributor?

* more than 10 files affected (higher count are more valid)?

* wall of text on description without screenshots, etc?

just close the PR as AI, and then the contributor can challenge it if they feel it should not.

[HelloNurse]

A contributor in good faith is going to accept criticism and resubmit an improved change: less files modified, more explanation, more focus, references to actual tickets and discussion with actual developers.

[snarfy]

They could allow AI PRs, but then have another AI PR reviewer reject them if they do not match the definitions for `to-the-point` `no-bullshit` commits.

[skeeter2020]

Please provide 3 examples where layering on MORE of the offending technology has solved the problem. Spam? Malware & Viruses? Customer Service? Hiring & Recruitment?

[mort96]

And who pays for the (likely significant, and controllable by everyone) tokens such a system would use?

[julianlam]

A small 4-9B model would be able to run cheaply for this sort of work.

[krainboltgreene]

What question do you think you're answering?

[Cpoll]

The Godot Foundation pays for it if it furthers their mission.

[mort96]

Does it?

[stronglikedan]

> That would be a dream.

We just recently started that policy so we'll see how it goes. If anything, having it stated as policy lets us filter out these requests without spending brain tokens on them.

[DonsDiscountGas]

I've always instructed Claude to check the policies first, frankly I'm surprised it's not smart enough to do that already. Would be easy to add to a system prompt. But usually it doesn't matter because many projects have no policies, or maybe they exist but only in hidden forum posts issues or something.

[paulddraper]

> That would be a dream.

“Mission. Fucking. Accomplished.”

https://xkcd.com/810/