I hope the take-away that comes out of the leaked documents is this: community moderation is very, very hard. It's not just a simple cut-and-dry "free speech vs censorship" issue. There's an enormous amount of nuance involved to sustain a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there.
In my experience working with startups, many of them don't treat their content moderation team well, or it tends to be an afterthought. In reality, this can be the hardest and most important job at a company. PTSD is a real problem with these teams, and pay is often sub-par. And when they make mistakes (or when they don't but people still get mad), they tend to get the bulk of the criticism from it. And then at the end, they usually aren't allowed to talk publicly about any of the work they do.
Please take your content moderation department seriously. If you have one, read through the crap Facebook's team has to deal with on a daily basis, and then go over and hug them.
>There's an enormous amount of nuance involved to sustain a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there.
Is it? I always admired the much-scoffed YouTube as the free-est place to talk on the internet. Where a vegan gay liberal from NY can discuss on the same video with a neo-con Mongolian and a 15-year old hipster from Sweden, an Israeli Jew, an Arab, and a black South African truck driver, and even agree and share experiences, and at worse you just get a few insults thrown and that's it.
I agree with you, but this isn't just about morons lobbing dumb insults at each other in all caps with bad spelling. I'm not even comfortable describing in a comment some of the horrible things that content moderators have had to deal with. Dig through the documents (particularly the child abuse doc) and you'll get an idea. Almost all of these policies were likely created from real-life examples, because people do things that are disturbing beyond imagination. They certainly didn't pre-plan it when they were hacking together Facebook in a dorm.
I can only imagine how much worse it is for the team over at Youtube. The first thing I would do as CEO over there is hug them, give them a raise, double their vacation time, and make sure they have very good access to mental health services in case they need it. I'm not saying they're treated poorly right now, but again, it would be one of the first things I looked at.
I find Reddit a lot better for that kind of conversation (excepting certain subs like /r/the_donald, /r/resist, /r/worldnews, /r/politics and certain others).
/r/AskReddit is one of the best places to have such free and open conversations and the best thing is that you get to set the topic for the conversation instead of some arbitrary video remotely related to what you want to talk about.
>and the best thing is that you get to set the topic for the conversation instead of some arbitrary video remotely related to what you want to talk about.
That's a good point (allowing for deeper discussion), but in YouTube I also like how the image/video transcends borders and politics etc in a way that targeted discussion doesn't.
(You can be anything, black, KKK, hipster barrista, 40yo soccer mon, Jew, Arab, donald-fan, etc, but still like/dislike the same cat clip, or old bluegrass song, or belly dancing or whatever).
That's probably a good example of why "freedom" isn't the right way to measure a community's value. Because the entirety of all youtube comments' value can only be measured in what worse things they authors were prevented from doing while commenting there.
Freedom of speech is great, and important. That doesn't mean that there's any value in just anybody exploring the boundaries of it.
> Because the entirety of all youtube comments' value can only be measured in what worse things they authors were prevented from doing while commenting there.
This says more about your sampling bias than youtube itself.
And the fact you are posting this says more about the magnitude of your political bias (its exact direction being yet tbd) than anything you read. I'm not sure if you realize how many nuts really say what they believe and believe what they say.
>This says more about your sampling bias than youtube itself. And the fact you are posting this says more about the magnitude of your political bias (its exact direction being yet tbd) than anything you read.
And both of the above say more about your willingness to characterize the parent than about the issue under discussion.
>I'm not sure if you realize how many nuts really say what they believe and believe what they say.
Doesn't seem to matter, as long as they don't also ACT on what they believe/say. Which is the case for most people (both positive, e.g. they are racist but they wont act on it, and negative, e.g. they say their are this or that good thing, but they're all talk about it).
To rephrase a flagged comment: 4chan is a pretty good counterexample. It's not necessarily true that a community destroys itself if you have no moderation. This community would; 4chan hasn't. Why?
Yeah, I flubbed the question. Let's try again: Why is it possible for 4chan to have the line far afield of what we'd normally think of as acceptable behavior? It's not only survived, but prospered.
A naive explanation is that only awful people go there, and there are a large number of those types. But that'd be mistaken.
I don't know, but it seems like an interesting question.
I guess I'd take some issue with the claim that it has prospered; it's survived, but according to every report I've ever seen, it's never actually made money. As for why it's stayed alive, given the anonymization, it's hard to draw significant conclusions about the community there without having access to 4chan's own analytics, but my suspicion is that the number of people who are long-time contributors to the site are actually relatively few. I'm biased, but based on the people I know who were or are 4chan users and my own personal experience with the site, I suspect most users start using it young, probably in their teens, see their highest level of involvement there over the next few years after they join, then drift away from it. That was the pattern I experienced, and the pattern I saw repeat in a lot of other people. There's a constant influx, but it's not what you would think of as a contiguous community (unless you want to get into some Ship of Theseus questions about what defines a community). 4chan has some of the hallmarks of other web communities, but it diverges from somewhere like HN, Metafilter, or even Reddit by virtue of that lack of continuity. You can survive for a long time on suburban teenagers who want somewhere to play-act as nihilists, but that's never going to be a demographic that produces much by way of value.
> I guess I'd take some issue with the claim that it has prospered; it's survived, but according to every report I've ever seen, it's never actually made money.
I don't think it was ever meant to make money. Lots of great (for various aspects of great) things exist for non-monetary reasons.
4chan is a very niche community. There are some who don't care what kinds of shit gets posted, but there's a very good reason their user base is so small compared to other sites like reddit.
It's true that off-topic posts are removed, but I'm mainly curious why this kind of "say whatever you want" attitude is so pervasive on 4chan yet doesn't kill the site. On Reddit, for example, comments like that would be moderated via downvoting, which physically hides the comment unless the user clicks on it. That's still a form of moderation.
> I'm mainly curious why this kind of "say whatever you want" attitude is so pervasive on 4chan yet doesn't kill the site. On Reddit, for example, comments like that would be moderated via downvoting, which physically hides the comment unless the user clicks on it. That's still a form of moderation.
I don't hang out on chans but sometimes end up reading them for various reasons.
I think the constant noise and random insults actually are moderation: they keep away people prone to butthurt, who due to their stubbornness and obnoxity are absolutely the worst fun killers and threat to any discussion. Normally people call each other retards as a matter of course and don't give much fuck, but if you try to argue emotionally then you are flamed to the death.
To me it was quite eye opening to read a bit of Encyclopedia Dramatica once. Many times I saw it throw shit all over some people or idea just to click on the "see also: some ostensibly opposite idea" link at the bottom and see them throw shit all over that one too. That's a very simple and elegant solution to tribalism if you ask me, HN totally could take inspiration from them because the mods keep complaining about tribalism here :)
But people like that material; is it against the rules? If not, what's the problem? You may think it's trashy, but apparently it's rather popular and doesn't seem to be trolling, at least not to me.
Eh, /a/ is pretty trashy too. The thing about 4chan is that the users of the trashier boards have grown used to it. Reddit users feel entitled to a more moderated space. A lot of 4chan posts are also highly satirical, there's very few serious posts, and the posts thare are serioues are mocked for being serious. This mostly applies to the trashy boards, though. The majority of 4chan's boards are not like the subset you may be familiar with.
4chan has no economy of scale in the UI (i.e., reddit gets more interesting with more posters but imageboards get less interesting), and has no signup barriers to stop new users. The veneer of rudeness is intentional and acts to maintain quality, not reduce it.
Also there are many active topical boards like /an/ /cgl/ /fit/ /u/ you're not reading!
I thought 4chan was the largest forum on the Internet, and in fact I remember when 4chan displayed the total amount of live content on their servers as being 100GB. Now it's more than 1300GB. 4chan thus seems to have become more popular.
People really do hang out on 4chan, though it's lost some share recently to 8chan, but 4chan is certainly a very large forum, the 200k+ current users, as displayed on their homepage too, confirms this.
For a place where you don't have to worry about drive-by downvoters, for all its faults, 4chan is excellent for getting your opinion across.
I meant compared to the really large scale social networks.
I don't know how accurate this is, but to put some numbers on it, Reddit seems to be #9 and Twitter #11 according to Alexa [1] and 4chan is rated much lower [2].
Although, Hacker News ranks higher than I expected [3], which seems suspicious. Perhaps there's a better way to measure it?
There's a chance HN could be artificially compressing the number of upvotes site-wide in order to maintain the illusion of smallness. Reddit did this same thing for nearly a decade.
E.g. Obama's AMA only appeared to have 14k upvotes even though in reality hundreds of thousands of people voted on it.
It'd be harder to get away with on HN, but maintaining the illusion of a tight-knit community for as long as possible is pretty crucial to the site's success.
4chan makes almost $0, for all the time moot poured into it and all its supposed users. Moderation or lack thereof is probably why the ad space is so worthless. Its silly to compare it to Facebook.
Actually I'd say it has hurt it. Perceived excessive moderation on various boards is why people left to go to 8chan, where you can make and moderate your own boards, so long as all the content is legal.
I'd rather have no moderation than lax moderation, which is why I'm in favor of decentralizing imageboards; the nntpchan software, which is an imageboard over nntp, is very good for this. Perhaps some system using bittorrent technology, too.
To reply to astrange (I am unable to reply using the usual method because the HN mods don't like it when I post quickly, so they've stopped me from doing that, ironically in this discussion about moderation):
I wasn't trying to say that 8chan is unmoderated, I recognize that it is moderated, because of the "only legal content" rule, rather I was trying to say that an unmoderated imageboard could exist in theory, but an imageboard which insists on only legal content is of course moderated.
4chan's moderators are in an unenviable position. For example, a large number of users left because the moderators put their foot down and decided that the site would not be the base for a widespread Internet harassment campaign.
Are the users haemorrhaged from that to the detriment, or the benefit of the site?
That's largely a matter of opinion, and you must factor in what counts as a detriment or benefit, which varies with 4chan's goals. Arguably 4chan doesn't really have a goal, though some (particularly the alt-right type people) would say it's a detriment, and people like me or some users of non-political boards think it's a benefit, as one of the large concerns is that /pol/ is infecting other boards, despite the fact that it was made as a containment board.
Others boards such as /mlp/ however do not show such growth into other boards, perhaps because the rule against ponies is stronger than the rule against racism.
You can't have "no moderation" and "only legal content", and I don't think you've thought through the consequences of picking the first one. Law enforcement actually really does exist, you know.
> a community that won't just devolve into a toxic waste dump that's poison for 99% of the people there
Have you seen Facebook?
That's not what it's optimizing for. Profit and eyeballs. Don't delude yourself. Toxic? Poison? Sure thing, as long as they can sell your attention span, Facebook don't care.
Honestly. You know what it's doing. Doing to everybody.
In as far as this team is building or sustaining a community, it's a tiny sandcastle on a beach facing the automated tidal wave of corporate exploitation of our dopamine/addiction response.
Sustaining a community of junkies is very, very hard. It's not just a simple cut-and-dry "freedom to do whatever to your body" vs "personal health". There's an enormous amount of nuance involved to sustain a crack house that won't just devolve into a toxic waste dump that's poison for 99% of the people there.
In my experience working at crack houses, many of them don't treat their bouncers well, or it tends to be an afterthought. In reality, this can be the hardest and most important job. PTSD is a real problem with these teams, and pay is often sub-par. And when they make mistakes (or when they don't but people still get mad), they tend to get the bulk of the criticism from it. And then at the end, they usually aren't allowed to talk publicly about any of the work they do.
(the analogy holds about as far as until you try to hug them--which is when they'll punch your lights out. but they do mean well, or something)
Yeah, that's not even close. They might get stressed, but it's not like seeing your best friend(s) getting blown to pieces every time you close your eyes.
Listen. I used the phone since 40 years. With the phone I had conversations with people that could have, well, annoyed others to say the least. I've said super bad things with my phone. Was I moderated ? No. Was it a problem ? No. Moderating is just here to make sure that FaceBook can still operate. That's just a problem for FaceBook to protect its image.
Let's imagine that there is no moderation. What will happen ? OK, some people will have argument, there will be insult, blood, etc. And, well, depending on a tipping point, we may realize that oooooops, letting people talk in the open is not so good. FB would be blamed and hen disappear. So be it. Now, another thing to do is : prosecute those who say unacceptable things. With prosecution, you have the justice system that will handle the case. That'll be slower but there will be discussions and, hopefully better laws.
FaceBook is becoming the Judge Dredd of free speech. It decides what's criminal and what's not and it applies his own power to handle the case directly.
This also happens in sports. Someone says something in an interview, in a social network post, in a private conversation which gets published... and what happens? Depending on offense the team may, ignore, fine, suspend or fire the person in question, outside the legal framework. That is, someone may say something insensitive or can be accused (not yet proven) to have done something unbecoming... teams often feel pressure to act extrajudicially. It's just how it is where money and the public sphere of opinion intersect.
You're right. But the ratio between number of people employed by FaceBook versus number of customer managed by FB is much greater, so I guess it makes me a tad more nervous :-)
Why do you think getting the police involved all the time would be better? They only have limited capacity; do you think they really want to look at every flagged post?
Moderators do refer the most serious cases to the police. But there's a lot of crap that falls short of that, so milder penalties work better.
>>> do you think they really want to look at every flagged post?
You actually understood me very well. That's my secret point. I don't see why FB must do better than police. If someone insults me, then, well, I fight back, go to the police, prosecute, etc. That works well because there is the work of justice.
By displacing the problem out of FB into the realm of justice, I want to make two points :
1/ The social norm of what is acceptable or not acceptable speech is governed by the people, not by a single entity.
2/ If there are too many cases to handle for the judge, then, let's add more judges (in my country, justice department is starving for budgets).
FB has way too much power, that's its (totally fair) success but now, it's too big to be let alone.
People would be flagging less if they knew this means involving the police. Which would even be considered a good thing by many people, but then I have no illusion that there always are others who can't stand anything even mildly offensive.
My first thought on reading this makes me deeply empathize with the people that have to deal with these questions. It must be extremely draining and emotionally difficult work, probably traumatic. Just having to write a policy about what constitutes acceptable nudity in the context of the Holocaust by itself is pretty terrible. And yet it's important for people to see those images. Just like the controversy over the image of the naked Vietnamese girl running down a road with napalm burns. That too is an important image. But to allow that image, someone has to look at these kinds of images and decide what's ok and what's not. I can't even imagine all the disturbing shit one has to look at to come up with those rules (which will be imperfect).
I can't imagine that there are many other single entities that have to deal with the ugliness of humanity at the scale of FB.
The sheer volume of content is unlike anything else humans have ever seen before. By comparison news outlets like the Guardian have the luxury to struggle with these kinds of questions on a 1-off basis.
Given all that I have to wonder how government regulations would make this better?
> Given all that I have to wonder how government regulations would make this better?
No no, that would impact free speech.
You want all large user content-handling companies to be able to tell users wanting to talk about X to go somewhere else, and it then has to be legal for those users to make that somewhere else themselves. There's enough idiots screaming about free speech on private platforms already, we don't want more of them.
> Anyone with more than 100,000 followers on a social media platform is designated as a public figure – which denies them the full protections given to private individuals.
100k followers is a curiously high threshold to be labeled as a public figure. I'd wager many people who are notable enough to have Wikipedia pages would not be able to hit that threshold even on Twitter.
Pedantry: you can keep a Wikipedia page with virtually no followers or following of any kind. The criteria for Wikipedia pages is simply:
(a) That there be some recognizable claim of notability --- this is an extremely low bar.
(b) That the notability claim, and any other material in the article, be backed entirely by reliable secondary sources.
(c) That after the article is stripped down to facts that can be verified in reliable secondary sources, there's still an encyclopedia article's worth of content left. This, too, is a very low bar.
Where pages tend to run afoul of WP's notability requirement is item (b).
But it's easy to see --- and to provide examples of --- people that with no public influence at all still having a Wikipedia page.
It's a sufficient, not a necessary condition for being considered a public figure.
(Although I'd disagree with your implication that having a wikipedia page should constitute proof of the publics' interest in someone's life, and all the consequences being a "public figure" entails in some jurisdictions)
The right to free speech does not imply a right to be heard (read, viewed, etc), by groups or individuals. If you spout things onto Social Netowrk, Inc's platform from which Social Network, Inc has decided to protect me, a private individual, they have not infringed your right to freely spout those things.
Notwithstanding the fact that it's Social Network, Inc's platform (and thus property) and you have no free speech rights on someone else's platform. Your right to free speech means the government cannot unduly curtail your speech.
Why is there so much emphasis on the credibility of the threat? Why not just blanket remove all explicit threats? Establishing credibility is impossibly hard.
Where I live "I'll kill you" is a common way to express dissatisfaction with someone's actions and while explicit, it's neither serious nor credible. I'm sure facebook is full of such posts.
This is the type of thing that can really benefit from public disclosure and discourse. What's "acceptable" varies so much by person and over time that having the censoring done in a black box will never be the answer.
The guidelines in the article seem overall reasonable given the range of possibilities versus the number of people working on it. I hope this stays out of the reach of government as long as possible.
There was a recent documentary I saw shared by a couple blogs about the moderators from Facebook,YouTube, etc and how they have to deal with depression now because of the job.
> Leaderless Palestinian youth, inspired by instructional videos and photos on social media encouraging people to "Stab a Jew," are thought to be behind a new wave of violence in Israel and the West Bank. Uncoordinated and spontaneous attacks by individual young Palestinians, mostly under the age of 25, started to occur almost daily from October 2015, with assailants often using a household weapon — a knife, axe, meat cleaver, screwdriver — before being fired upon by nearby Israeli security forces. So far, the bloodshed has claimed the lives of at least 28 Israelis and 189 Palestinians, 128 of whom Israel says were assailants.
What does it even mean? What are the red and green marks?
What's even weirder to me is that they don't touch upon more examples. It's just so incredibly arbitrary that it can be used to easily cover up anything with zero consequences.
If you refer to "#stab and be the fear" etc., I guess it's not because it's about Zionism, but rather because it's an incitement to commit terrorist acts (totally credible, seen the situatuon). So I guess it makes sense. (And I say this as a total sympathizer with the plight of Palestinians and their struggle against the Israeli occupation.)
I doubt documents like that would ever be published willingly.
If those documents were public, that (in my opinion) would just enable bullies etc., as they would know exactly what they could get away with. That would then trigger tightening the rules, which would then trigger much complaining about Facebook suppressing free speech etc..
There are also the motivations of whomever wants this out there. Some people may only want the debate, but some want it out there so as to enable the bullies. And others just want the fame and fortune of being the one who published it.
As I have said many times, violence is considered more tolerable than sex. Guess what you're gonna get more of. And yet there is no mechanism to decide who has input into FB's community standards.
I havent heard of the right to be heard, but most people know of free speech.
So where are you getting the idea of the right to be heard ?
Is it a legal idea, your personal idea, a philosophical idea ?
Are you trying to derail, or are you seriously not understanding that there is no right to be heard, it's a thing GP made up on the spot to illustrate what free speech is not?
That's because there's no "right to be heard." Just because we have a right to speak freely doesn't mean we have a right to be listened to. You might be free to say whatever you like, but everyone else is free to ignore you; owners of publishing platforms are free to not publish your recorded words.
As I said, freedom of speech does not automatically transfer into a right to be heard.
The cult around ML here is pretty amusing. As another poster said, AI is currently staggeringly bad at this task, and I'm not sure it's going to be good enough in the near future either.
I think ML could be used to flag certain content as potentially problematic (I'm hoping to experiment with this in the future), but yeah, it's definitely not going to be doing the final verdicts anytime soon.
Machine learning is incredibly bad at this kind of stuff. There are far too many false positives and special cases to leave to ML. And the stakes are fairly high for Facebook: if they block content that shouldn't be blocked, it's newsworthy.
In my experience working with startups, many of them don't treat their content moderation team well, or it tends to be an afterthought. In reality, this can be the hardest and most important job at a company. PTSD is a real problem with these teams, and pay is often sub-par. And when they make mistakes (or when they don't but people still get mad), they tend to get the bulk of the criticism from it. And then at the end, they usually aren't allowed to talk publicly about any of the work they do.
Please take your content moderation department seriously. If you have one, read through the crap Facebook's team has to deal with on a daily basis, and then go over and hug them.