[thedevilslawyer]

Assuming that even works from a researcher's perspective, it's working back from a specific goal. There's 0 actual instances (and I've been looking) where verbatim code has been spat out.

It's a convenient criticism of LLMs, but a wrong one. We need to do better.

[latexr]

> There's 0 actual instances (and I've been looking) where verbatim code has been spat out.

That’s not true. I’ve seen it happen and remember reports where it was obvious it happened (and trivial to verify) because the LLM reproduced the comments with source information.

Either way, plagiarism doesn’t require one to copy 100% verbatim (otherwise every plagiarist would easily be off the hook). It still counts as plagiarism if you move a space or rename a variable.

https://xcancel.com/DocSparse/status/1581461734665367554

https://xcancel.com/mitsuhiko/status/1410886329924194309

> We need to do better.

I agree. We have to start by not dismissing valid criticisms by appealing to irrelevant technicalities which don’t excuse anything.

[thedevilslawyer]

Ok you win.

You should take your findings to the large media organizations including NYT who've been trying to prove this for years now. Your discovery is probably going to win them their case.

[dayjaby]

Why so cynic? This is a serious issue. And media coverage has nothing to do with the immoral state of the art of ignoring copyrights.

[thechao]

I don't know code examples, but this tracks, for me. Anytime I have an agent write something "obvious" and crazy hard -- say a new compiler for a new language? Golden. I ask it to write a fairly simple stack invariant version of an old algorithm using a novel representation (topology) using a novel construction (free module) ... zip. It's 200loc, and after 20+ attempts, I've given up.