| It's a very hard problem to solve. I've been working in social networks for many years at this point and there is no silver bullets when opening a nonexclusive one to many relationship. Things that seem to prevent abuse like this is linking the user to their comments publicly. Removing the level of anonymity makes them accountable and far less likely to tarnish their own reputation for the sake of slandering another persons, the more removal of anonymity the more this becomes true. People are far more likely to make a harsh comment on a personally unidentifiable reddit username than say facebook or a verified twitter account (e.g. throwaways are popular in subs with less popular social stances... gonewild, cringe, etc). Searching for negative words is much harder than you're assuming. Even using a phonetic library (like soundex) will produce tons of false positives. On top of that there is no library for assessing context in conversation or the underlying relationship of the user. "I love how you always handle Dick at family reunions, it means a lot to all of us." That sentence is VERY wtf out of context. So is that an insult insinuating the user gives hand jobs to their family members? Is it a personal attack because they're sexually promiscuous? Is Dick the rowdy family Drunk who this user often keeps in check when they're about cross the line after a few drinks? To further that idea the underlying relationship between User A and User B doesn't translate without previous knowledge of their past. I for example am generally a nice person and friendly, but I've friends from many years and walks of life at this point. For people I kinda sorta know, I'm polite and kind, I know the limits of our friendship and expected behavior. For very close friends and such communication is usually peppered with light jabs and inside jokes... think "tough love". "10/10 looks great half naked and covered in mud, could fuck up a mothers love." Seems mean, to the person getting that message it likely relates back to a past shared experience that will bring a smile to their face upon reading it and in the context of the relationship is par for the course. Coming from a co-worker in reference to a botched project in regards to installing hardware in a remote location... not so much. There is even more to this problem that I won't go into, but by now you should get the drift, and you can't account for or detect these subtleties in human to human relationships (if you find a way you'll be a very rich person), especially coming in blind on an semi-anonymous platform. This is why places like /b/ are simultaneously some of the best and worst places on the net. |