Hacker News new | ask | show | jobs
by salahadawi 28 days ago
I've actually been tracking the front page of HN for AI posts for a while now: https://www.salahadawi.com/hacker-news-ai-detector

This post evaluates to 99% AI generated.

2 comments

Nice! Have you considered doing a Show HN for that?

That's valuable in at least three different ways: public education, showing that most of the articles are still human-written which can be easy to forget about sometimes, and as an easy way to cross-validate my intuition when flagging something as AI-generated without having to manually run Pangram.

I despair a little bit about how many HN voters either seem to want to read slop or don't understand when they're reading it. This post is obviously AI generated from the first paragraph on, and still has 480 votes.

I've thought about it, will probably fix a few things and then do it
Seems someone at HN didn't like my post, tried posting in Show HN but was immediately shadowbanned
Pangram claims to be super accurate. Do you know how? I understood detection is a hard problem.
Their technical report (arxiv.org/abs/2402.14873) goes into it.

They train a classifier on human text paired with AI-generated versions of the same documents. Dataset scale is comparable to LLM pretraining sets.