Hacker News new | ask | show | jobs
by ggorlen 1116 days ago
Having flagged hundreds of such answers before SE suppressed LLM moderation, I can vouch that they stick out like a sore thumb:

- they're all written in the same style ("It looks like you're trying to do convert a string to an integer...") - they have perfect grammar - they're often/usually wrong - they tend to be tone-deaf to the question or are answering a different question than asked - their formatting is often poor by virtue of copy-paste from ChatGPT and the author's lack of familiarity with markdown - they're usually posted by new or new-ish users that have few reputation points (ostensibly, their goal is to farm reputation rather than help curate a quality resource) - they're posted in large quantities (a dozen or more in an hour) that'd be virtually impossible for a human to churn out (especially a brand new user!) - they're posted in random tags that show no clear connection from one to the next--most answerers are subject matter enthusiasts or experts and stick to a narrow range of tags - they tend to contain hallucinations obvious to SMEs, like invoking methods that don't exist - the code is plain wrong when executed - the code style is "boring"/"vanilla" and tends to steer clear of idiomatic language features that a seasoned programmer would employ and has few formatting quirks that a human might use - the code is often heavily commented in a predictable and artificial manner - the explanation after the code and overall layout/flow of the post is often the same - they tend to be dramatically different than the user's normal answers, which have typos.

As you'd expect, it's difficult to _prove_ that a particular answer was generated by an LLM (the fact that LLMs can't reliably detect themselves is part of the problem--they're inaccurate!). However, the possibility of occasional false positives (SE has provided no actual evidence of this being an issue), seems a necessary price to pay, and could be solved in a more balanced manner than prohibiting all LLM moderation. SO would be unusable if it became a stale cache of mostly-incorrect LLM spew, which is what SE's new moderator guidelines seem to be OK with.

If I want an LLM answer, I'll ask an LLM. I can then prompt engineer and iterate. If I want a human answer, I want to be able to ask on SO so I can be guaranteed I'm "speaking with a human".

Disclosure/context: I'm a power-user on strike.