Hacker News new | ask | show | jobs
by dvt 19 days ago
Pangram doesn't work, and I wish people would stop treating it as gospel (but the AI/anti-AI grift is real). Here's a fun paradox: I can literally tell ChatGPT: "Say X" and it will say "X"—so that's a case where content is both AI generated and not. What if it changes a few words? Moves some sentences around? Where does something go from human- to AI-generated? (This is the classic Sorites paradox.)

Pangram tries to look for common patterns (rule of three, em dashes, etc.) but these are heuristic methods and not to be taken as gospel. There is no provable method to make a distinction between AI and human-generated other than the fact that AI-generated text tends to reek of pseudo-intellectual undergrad with a thesaurus.
1 comments
wgd 19 days ago
Pangram does work, in the specific sense that when it says something was AI authored it is vanishingly unlikely that it was written by a human (who was not deliberately trying to write like an AI), and IMO getting people to recognize that we actually do have a decent solution in this space now is pretty important if we want the Internet to remain a place for humans and not just bot swarms.

> rule of three, em dashes, etc

You appear to be misinformed about how Pangram specifically works, it is not based on pattern detection of that sort. I recommend reading their whitepaper, it's a pretty understandable explanation of exactly how they trained their classifier.

link
timpera 19 days ago
I just tried it, created an account, and wrote a few sentences about how my day went. These sentences got classified as AI assisted, so clearly their classifier doesn't work that well.
link
dvt 19 days ago
> Pangram does work

It's trivial to see how many people think Pangram is absolute trash[1] (because it is).

> You appear to be misinformed about how Pangram specifically works, it is not based on pattern detection of that sort. I recommend reading their whitepaper, it's a pretty understandable explanation of exactly how they trained their classifier.

I did read their paper (which is, by the way, very scant on details), and they trained their classifier in the laziest way possible: here's a chunk of "human-written" text and here's a chunk of "AI-written" text, put them in the right bucket, and do this a zillion times. Literally zero sophistication. Also: what do you think "pattern recognition" is, if not a "classifier"?

[1] https://www.reddit.com/r/academia/comments/1rm11rs/pangram_c...

link