Hacker News new | ask | show | jobs
by Seattle3503 27 days ago
You are talking about false negatives, but this thread started with a discussion around the potential for false positives causing writers anxiety

> Even if 2/10000 is true that’s nowhere near accurate enough to make aggressive accusations that create anxiety at levels people need to medicate with potentially fatal consequences.

False positives and false negatives are different problems that have different impacts.

Does your single example prove that Pangram doesn't work, or that it doesn't work on short snippets of text? Try to get a detection error on a longer run of text. I'd be curious to see the results, particularly if you can get a false positive.

1 comments

Well, false positives must anyway be zero for a product which is built around detecting AI. let me explain - the value proposition of this product is that we detect AI. Its not we don't call human writing AI generated. The latter is implied and must not happen ever. The value is that pangram detects AI generated content which it did not in the above example. if this explanation doesn't make sense then think of pangram as a pregnancy test where presence of AI == presence of baby. If a pregnancy test keeps saying you are not pregnant when you actually are then that's a problem, right? - you can't argue that at least its not saying you are pregnant when you are not.

But here's a bigger problem - Imagine a teacher grading 50 students - 40 of them use AI to write their answers and 10 write honestly. All 40 of them use the hack that I used and get 100% human from pangram and the other 10 also get 100 human from pangram. what is the product really adding to the workflow? Nothing, zero or zilch!!! - but then you come along and say hey at least those 10 honest students also got 100% from pangram. This whole argument of false positives being very very low works if your false negatives are tight which they are not.