[docheinestages]

I think figuring out if a text is AI-made is a losing battle. What could work is gauging how much effort went into writing the text, regardless of who the author might be. What's easy today is generating mountains of text that are extremely hard to read. What requires effort is knowing how to engage the reader, how to keep out extraneous information, and how to keep the text as short as possible without losing details. That needs effort, with or without AI.

[calebh]

The easiest way is to keep track of the text's edit history, keeping a block of edits over time and having them signed by a timestamp authority. The final edit history can then be inspected by some external authority, then signed if the edit history looks human. I have a blog post from 2023 on this topic: https://helbl.ing/Written-Proof-of-Work/

For Google Doc users, you can already inspect the edit history over time to verify that text is written by a human.

[visarga]

That human might have used AI. You can never know. Hand fixed AI output, human just polished the corners? Light rewording of a full text written by hand, because the author is not confident in their writing? Actual human text, but after researching with AI?

[theoreticalmal]

Exactly. Detecting AI writing is an arms race that can only end with detection coming in second place.

[warkdarrior]

I am working on a browser extension to help with that. Basically it interposes on any text field and canvas and if user pastes a large amount of text (copied form example from a chat bot), the extension will "replay" that text at normal, human-editing pace, and introduce typos that are fixed through later edits.

[bulder]

Any specific reason as to why you'd want to make that, outside of intentionally enabling fraud?

[warkdarrior]

People should be allowed to use any tools they find useful, and their writing should be judged on quality.

[echoangle]

In all contexts? If there is a paper to hand in and the instructions are that you’re not allowed to use LLMs, do you think it’s fine to circumvent that? If you have a math exam, would you say you can still use a hidden calculator even if calculators are forbidden?

[Footprint0521]

As a verbal processor, I hate this. I use whisper v3 large all the time from voice memo dumps that are AI grammar corrected and this would flag when I copy paste it in :/

I wish there was a solution

[duskdozer]

Use it to transcribe directly without correcting grammar or formatting.

[throwaway219450]

Nevertheless this PoW is used by some conference venues now, in the case of a dispute against human authorship. NeurIPS does allow LLM use for "minor" polish like spelling, grammar, correcting non-native speakers. But you can't have the AI write the entire paper:

See https://blog.neurips.cc/2026/06/02/ai-generated-papers-in-th...

> Unfortunately, given the volume of submissions that appear non-compliant, relying on author declarations is insufficient.

...

> Authors whose submissions show significant AI involvement must provide an audit trail that clearly demonstrates that they complied with the policy. We expect that in future years this kind of audit trail will become a default.

The method uses Pangram, but that's somewhat arbitrary. The authors must provide the draft that was used to prompt the AI and show how the changes were made, which should be possible with a chat history. Where it doesn't work is if the authors claim they did not use AI, but otherwise it's quite hard to spoof. Edit history on its own isn't good enough because you could trivially have a program emulate keystrokes to type at a human speed into a document with tracked changes; the proof here accepts that AI was probably used and aims to determine to what extent.

I do think this goes back to if the text is high quality enough that humans don't notice, is that OK? Probably yes. What we don't want is poor quality writing from any source.

[Wowfunhappy]

…All I know is that sometimes I will read e.g. a HN submission, and it becomes pretty obvious partway through that the article was AI generated.

If I can do it, an algorithm should be able to do it. Maybe in the future the models will get so good that it is literally impossible to differentiate human vs computer authorship, but that’s obviously not the case today.

[joebo]

I've noticed there seems to be a default style that is easier to detect. I've noticed it harder to detect when asking an LLM to use a different style (more conversational, avoid sounding like an AI, don't use emdashes, etc). I wonder if that's what you're picking up too - the instances where people make no effort to change the style of the output.

[mr_mitm]

How do you estimate your false negative rate?

[Wowfunhappy]

No idea, I'm not convinced it matters that much? Like, if people are using AI and I legitimately can't tell at all (and I'm not their teacher or something)... okay, fair.

Edit: But I'm also super conflicted about this, because I really want to read what humans think, not what an AI thinks, regardless of the writing quality.

[mr_mitm]

My point is that you cannot know that AI generated text is obviously AI generated. It's like the old "hair pieces look awful, I can tell immediately!" - no, you can only tell when they're awful. Maybe you're not as good as you think at detecting AI.

[Wowfunhappy]

Well, but I think my false positive rate is very low, it’s just the false negative rate that could potentially be quite high. This bothers me much less than the reverse.

[pishpash]

What's likely to happen though, is human idiomatic writing will degrade to AI level and the two will converge. Just like nowadays it is harder to tell if some (human) non-native English speaker wrote some English text online whereas 20 years ago it was very obvious.

[thatjoeoverthr]

Sufficiently advanced AI use is probably fine. The slop everyone complains about has certain tells specifically due to some combination of the following:

- The author is conducting some kind of hustle.

- The author doesn't bother editing.

- The author lacks the taste and awareness enough to see it looks.

- The author thinks you, the reader, lack taste and awareness.

- The author is using it as a kind of smoke bomb to get rid of you.

In such cases, nothing is done about the LLM's distinctive "voice". It dominates the text and it's easy to detect. It stands as a signifier of the above, even if it's otherwise not intrinsically a problem to use AI.

[HDThoreaun]

There cant be a way / except of course if you pay / mind to my syllables