[amarraja]

We had a small bespoke customer support system we wrote back in 2000 or so. The system would allow an operator to type the message body, and would automatically generate the mail text e.g. "Dear {customer name}, {body}, Regards {operator name}". Cutting edge stuff back then!

One day, we get a call that an operator couldn't send an email since the third word was classed as "offensive", however the message body was fine. After firing up the debugger, it became obvious... the customer's surname was "Dick".

We never got to the bottom of how to solve it, so hacked something in. I wonder how many times Mr. Dick has issues with false positives in profanity filters.

[atomwaffel]

Probably often enough to be used to it, although it's still bad design and needs to be fixed. (I'm not sure why a non-public-facing system needed a profanity filter in the first place.) https://en.wikipedia.org/wiki/Scunthorpe_problem

I find it interesting that this Twitter bot seems to have the same problem in reverse: it can't reliably filter out things that aren't offensive.

Edit: Thinking about it, it's really still the same problem, i.e. false positives when trying to automatically determine whether or not a given string contains swearwords.