[mijoharas]

One thing to note.

They were quite conservative in their approach, so the only things that were rejected were from people who had agreed not to use an LLM and almost definitely did use an LLM (since they fed hidden watermarked instructions to the llm's).

This means the true number of people that used LLM's in their review (even in group A that had agreed not to) is likely higher.

Also worth noting, 10% of these authors used them in more than half of their reviews.

[grey-area]

Yes for those in group B I'd suspect many were doing exactly what these cheaters in group A were doing - submitting the unaltered output of an LLM as their review.

[kombookcha]

The rejection is based on the dishonesty of explicitly committing to standard A and then knowingly violating it, not on LLM use as such. I think that's pretty fair, considering that everyone could have just chosen B if they wanted to.

[grey-area]

Sure, I'm just pointing out that the 2% headline figure is very conservative if not misleading as a far greater unknown number in group B will have done exactly the same (which I doubt ICML or those submitting papers actually want). This is probably a first step in clamping down on anyone doing this.

[kombookcha]

Oh, I misread your post - that's fair!