[Wowfunhappy]

…All I know is that sometimes I will read e.g. a HN submission, and it becomes pretty obvious partway through that the article was AI generated.

If I can do it, an algorithm should be able to do it. Maybe in the future the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.

[joebo]

I've noticed there seems to be a default style that is easier to detect. I've noticed it harder to detect when asking an LLM to use a different style (more conversational, avoid sounding like an AI, don't use emdashes, etc). I wonder if that's what you're picking up too - the instances where people make no effort to change the style of the output.

[mr_mitm]

How do you estimate your false negative rate?

[Wowfunhappy]

No idea, I'm not convinced it matters that much? Like, if people are using AI and I legitimately can't tell at all (and I'm not their teacher or something)... okay, fair.

Edit: But I'm also super conflicted about this, because I really want to read what humans think, not what an AI thinks, regardless of the writing quality.

[mr_mitm]

My point is that you cannot know that AI generated text is obviously AI generated. It's like the old "hair pieces look awful, I can tell immediately!" - no, you can only tell when they're awful. Maybe you're not as good as you think at detecting AI.

[Wowfunhappy]

Well, but I think my false positive rate is very low, it’s just the false negative rate that could potentially be quite high. This bothers me much less than the reverse.

[pishpash]

What's likely to happen though, is human idiomatic writing will degrade to AI level and the two will converge. Just like nowadays it is harder to tell if some (human) non-native English speaker wrote some English text online whereas 20 years ago it was very obvious.