Hacker News new | ask | show | jobs
by chs20 1142 days ago
I'm pretty sure you can't reliably distinguish LLM text from real person text.
2 comments

I probably can't reliably distinguish LLM output from average writing from a real person -- the kind of "content" I find online. I believe I can distinguish LLM output from what a good writer produces. Anyone can write a Buzzfeed listicle or "review" of Madrid as pointless as what ChatGPT produces. But ChatGPT won't write Crime and Punishment, 1984, Lolita, Blood Meridian, or even On The Road or In Cold Blood.

People losing their jobs to ChatGPT don't even call themselves writers, at least not seriously. They produce content, most of it regurgitated the same way ChatGPT does it. If I produced content for online click farms I would worry about so-called AI. I don't think Murakami or Cormac McCarthy have to worry, but James Patterson might get replaced by an LLM (if that didn't happen already -- hard to tell).

In my own profession, some programmers already express fear for their future because Github Copilot and ChatGPT can write code just as good as they can. That should tell them to level up their skills, because if whatever they do can just get automated away by a glorified auto-complete they weren't adding enough value in the first place.

ChatGPT is pretty easy to spot (in some circumstances). Just look for text that has absolutely no voice and reads like a high school essay where what should be a 20-word answer is stretched out to meet a 750 word count. Example: https://www.quora.com/What-is-the-maximum-data-capacity-for-...
That's just the default style. Try a request "...in the style of marvin the paranoid android" or "talkie toaster from red dwarf" or"Ernest Hemingway" or "Donald Trump" for a bit of variety.
How well does it do with anything that's not just a pastiche? It's easy to throw in the tics of somebody famous, but can it actually do what a real writer does: focus on what's interesting, build a flowing narrative, target the material to the audience?

Exaggerated voices might give you variety, but I don't actually want to read a piece on the style of any of those voices. I would, however, be intrigued if it could write like me, or like any of the famous nonfiction writers who I've consciously patterned my style after (Isaac Asimov, Mary Roach, Cecil Adams, Michael Pollan)?