Hacker News new | ask | show | jobs
by CGamesPlay 1289 days ago
Hmm, the HuggingFace detector seems to identify notes from my personal notebook as being about 80% fake. Passages from Wikipedia, between 72% and 96% fake. I think this is actually detecting "grammatically correct well-punctuated prose" rather than "from GPT-2".
1 comments

Yes. The point is that it is intended to detect comments obviously written completely by GPT, not detecting that a human a wrote it. You have to discern between the two, as the scale is not really utilized here. What I've found over the last few days of playing with this is that most AI generated text will be 99.97% accurately identified, while human text ranges somewhere between 70-98. So I could move the threshold for my script quite a lot. I just wanted to see how it works with these thresholds for a while.