Yeah it's weird, there was one case where I thought it was AI but wasn't sure. Several other comments pointed it out, too. Author claimed he wrote it manually. (Which is honestly even more concerning!)
Maybe there can be a dedicated 'flag botspam' button?
Then again it's a nuanced issue. I see AI used in a large percentage of writing now, so would this rule apply to the article as well?
> Yeah it's weird, there was one case where I thought it was AI but wasn't sure. Several other comments pointed it out, too. Author claimed he wrote it manually. (Which is honestly even more concerning!)
I find the above comment concerning, so I ask: to what degree is the above commenter calibrated to ground truth? How would they know? How would we know?
He was using a dozen obvious ChatGPT-isms. So either he was lying about writing it manually (the comforting option), or he actually writes like that, which is what I meant being concerning.
But yeah, there isn't a way to prove it one way or the over, even when it's "obvious".
I saw in some schools they're using systems where you have to type the essay in a web app, and the web app analyzes your keystrokes to determine if you're human.
Thanks for sharing. / Speaking of keystroke analysis, have you read "Fall" by Neal Stephenson? A fun read; there is a generalization of this idea therein.
Abusing the flag button by reporting LLM generated posts and comments (which are not breaking any current guidelines) seems like a good way to get your flags ignored.
"Not belonging on HN" is an open invitation to flag anything someone disagrees with. Many posts are flagged simply because they express an unpopular opinion.
Community moderation won't fix this problem. It can only be mitigated if the site owners invest significant resources in addressing it. And judging by how little YC actually invests in HN, I wouldn't hold my breath. This website will succumb to this problem just like most others.
Something we need to remember that AI was trained on every public internet comment, the vast majority of which are legit terrible. The biggest tell that someone is using AI is having multiple paragraphs saying the same point over and over again. Even trolls are more succinct.
Huh, this is what specifically drove me to complain about LLM-generated tickets at work - multiple paragraphs rewording and emphasizing the same point, all of which was topically relevant, but not necessary.
(i.e. it was obvious in the first place, think along the lines of a ticket about a screen loading slowly, and then multiple paragraphs explaining the benefits of faster-loading screens.)
I would argue that those cases are really the ones that cause an LLM-specific harm, i.e., which make people feel like they aren't exclusively among fellow humans.
If someone posts something that doesn't clearly read LLM-ish, but is otherwise terrible, it's not really different from if the same terrible thing had been written by hand.
I don't think anyone who objects to LLM comments is really demanding a super-low false negative rate. Just get rid of the zero-effort stuff. For example, recently I've seen a lot of comments from new accounts that are just sycophantic towards TFA and try to highlight / summarize a specific idea or two, but don't really demonstrate any original thought (just, like, basic reading comprehension and an ability to express agreement). And they'll take a paragraph to do so, where a human with the same level of interest in the material might just say "good post" (granted, there's an argument to be made for excluding that, too).
Sorry, updated my original comment—I meant to qualify it to only those cases where it's blatantly obvious. Obviously a lot of ambiguous comments will slip through as a result, but I agree with you that false negatives are better than false positives.
Your comments use em dashes. Many would claim those are vastly overrepresented in AI language and thus an account overly using them are blatantly AI.
I don't think your account is AI just by these few comments, but I would like to point out that most rubrics one might use to determine what is obviously AI might end up including the way you talk.
If there was a truly accurate tell, some algorithm you could feed a few sentences in and it could tell you "yep, this is 100% AI", then yeah sure use that. I don't know you could realistically build that machine, especially when it comes to the generation of text.
For what it's worth, there are modern LLM detectors with extremely low false-positive rates. The tech has advanced quite a bit since the ZeroGPT days. Personally I've gotten very good results from Pangram Labs. Still can't directly ban people though because false positives are always possible.
Are they great at detecting normal prompts that don't try to make the LLM speak non-LLM-ishly? If you make the LLM not use em dashes, "it's not; it's" phrases and similar things, and if you make it make a few mistakes here and there, would it still be detected? My point is that if people aren't trying to hide their LLM use, it might work, otherwise it probably wouldn't. How would a detector tool work against output where the prompt tells the LLM to alter the way it writes? Or if the LLM output is being modified by another LLM specifically designed to mimic certain styles?
Like, why would my comment (or yours, or any other comment) pass or fail the LLM check the I/you/someone else used specific prompts or another LLM to edit the output? It seems like these tools would work on 99.9% of the outputs, but those outputs likely weren't created in an adversarial way.
I will never, ever forgive these techbros for ruining emdashes. I will also never stop using them -- they are a permanent part of my writing style -- no matter the personal consequences.
Oof. Some of those seemed reasonable at first. Ex: CloakHQ's comment on Compaq/DEC...
....until you start scrolling down the page and it becomes screamingly obvious that everything it says comes from the same template.
Maybe the problem isn't just that AI produces gobs of useless crap. Maybe what's worse is that it can produce even more mediocre crap that crowds out the good?
All oatmeal, no steak, leads to "starvation" by poor nutrition.
Maybe there can be a dedicated 'flag botspam' button?
Then again it's a nuanced issue. I see AI used in a large percentage of writing now, so would this rule apply to the article as well?