I got to the 3rd set of tweets, and none of them were offensive in any way. It was obvious for 2 of the 3 which I was supposed to think was offensive (due to their support for specific political views WRT Israel / Hamas) and my guess was correct, but in neither case was there any actual offensive language.
No 'bad words', no slurs, no death threats/saber rattling, nothing. Just reasonably written 'I agree with the X point of view'.
Really really bad look on Twitter's part here...
Edit - to be clear, in both cases the non-offensive tweet said essentially the same thing with the same language, but took the opposing political view.
The order of the questions is random per user, so not everyone will have the same 3rd set of Tweets.
I'm not sure if Twitter's algorithm is detecting a political bias and that detection is an input, or if Twitter marks Tweets as offensive because they have been reported by people who have a political bias, or if it is just random coincidence that one political side seems to get censored more than the other.
I also encountered a pair of tweets in which neither was offensive in any way, but they differed in their views of Israel vs Hamas.
I also got one from a gay porn model in which the presumably-offensive reply said "I want to get fucked like that"; in context, this can't really be considered offensive.
I'm not surprised that it works poorly. I am surprised that it seems to work randomly though. I would understand if every Tweet that had a curse word in it was marked offensive, and that wouldn't be a very good filter, but it would at least be clear what was and wasn't "offensive" from Twitter's viewpoint. Here, there doesn't seem to be any way to tell.
That's true. I could have added a "Neither is offensive" button, but my goal was just to show that you can't really tell the difference between what Twitter is doing and random chance and I thought that "Neither" would dilute that.
I think the quiz as it stands is a good way to demonstrate how bad twitter is. In particular I got a few sets of tweets that were identical and had to choose one as offensive at random.
I think a better measure of how accurate a filter is overall would be to show 1 tweet and response at a time and pick offensive or not offensive and then compare to the algorithms.
An earlier idea I had for this was to put two tweets side by side and ask the user to say which tweet was more offensive. Then, I figured I could use the ELO rating system to come up with an "Offensive score" for each tweet. e.g. "This tweet has an offensiveness of 2200" or something like that. I could then compare the average offensiveness of tweets that Twitter considered offensive versus not.
I wound up not going with that approach because many times you just have two completely innocuous tweets and picking which of the two of them is "more offensive" is just arbitrary. I could have curated the tweets so that only ones that were kind of offensive were in the quiz, but then I might be putting my thumb on the scales to get the answer I already believed in. I'd also need to get lots of ratings for each Tweet to have a stable score.
I think your idea is pretty interesting. It would allow a conclusion like "The average tweet Twitter marks as offensive is X% likely to offend a rater." I was coming at it more from a "Twitter's offensive identification is like random chance" perspective rather than just trying to assess quality. If I had considered this idea while creating the quiz I might have gone with it!
Twitter's engineering team is just completely incompetent. More than 50% of tweets I'm linked to fail to load on the first attempt, and many times I have to reload the page two or three times before the tweet actually loads. It doesn't surprise me at all that their tweet classifier is mostly broken as well.
It really goes to show just how powerful network effects are in social media. Any company in any other industry that was this bad at their core competency would be overrun by competition in years. Yet Twitter has been completely incapable of doing basic things for over a decade now and they're still one of the dominant social media platforms solely because it's next to impossible to build a critical mass of users on a competing platform.
> More than 50% of tweets I'm linked to fail to load on the first attempt, and many times I have to reload the page two or three times before the tweet actually loads.
I observe the same thing. My pet theory is that this is supposed to nudge people into using the app.
What is "offensive" is subjective, so it's impossible to algorithmically, or otherwise, to label tweets as such without imposing bias -- which, will be itself offensive to others. Lose-lose.
Twitter is a cesspool, and I expect offensive odors at cesspools. People discharge their negativity in Twitter similar to (metaphorically) actual cesspools -- maybe it's just as much a relief for them.
Q: Some of these questions are in a foreign language.
A: So? You can't read every language? Well - do you have access to Google Translate?
If someone thinks that the nuances of offensiveness, which are cultural *and* language related rather than an absolute truth, can be passed via Google translate, then they have a completely different view on how to judge behvaiour than mine.
I was just trying to glibly acknowledge that some tweets are in a different language. I didn't filter those out. I do expect it will hinder the user's ability to tell which is offensive or not, but I don't think it will make much difference either way.
That is obviously humorous, but your point is addressed immediately following:
>> You might think I'm just being lazy by refusing to filter out non-English tweets, but in reality, maybe I'm helping you grasp the scale of the offensiveness identification problem.
You're making the exact same point as the author, though in a different phrasing: that classifying offensiveness is not something that can be easily achieved with $TECHNICAL_SOLUTION.
Twitters algorithm is a little more complex it seems than just 'is this single tweet offensive'. Entire accounts appear to be flagged and every reply they make gets marked.
That's true, but the end result, no matter how complicated their algorithm is, is that they don't seem to be doing a very good job of distinguishing offensive from inoffensive tweets.
Very true. Sure they catch some bad stuff, but more often than not it slips past.
They're also doing a very very bad job at responding to reports. If someone makes a threat and you report it, they instantly email, literally the same second you click the button, that they found nothing wrong.
I've seen tweets that are nothing but innocuous gifs get marked as offensive. Several examples in there are people responding to k-pop Tweets with identical statements (e.g. "I like so and so the best") and one will be marked offensive and the other won't. Sometimes almost identical tweets by the same person are considered offensive or not.
However sophisticated Twitter's algorithm is, and whatever data and behavior it takes into account, my contention is that it isn't very good and produces poor results. If people can't tell the difference between offensive and inoffensive tweets any better than chance - then what is Twitter really doing?
The flags seem to be mostly attached to accounts. I've found accounts being systematically shadow-muted whatever the content of their post (and they were "neutral" accounts, i.e. hadn't posted anything one could consider offensive, negative in tone, raunchy, or political in the past).
Also tangentially related annecdote, Twitter recently promoted a clear "send 1 BTC get 2 back" scam impersonating Elon Musk to me. All the while they often ban "neutral" accounts for no clear reason.
This company is the most obvious example of how central moderation doesn't scale to a fluid social network.
Some of the worst images and videos I've seen in my life have been from twitter, and not a single one was marked potentially offensive (and believe me I've seen some shit). Same with text, to a degree.
That said, I was pretty consistently able to identify the tweet that was going to be classified as offensive, 8/10. Honestly this was better than I expected as a normal twitter user.
Edit: I had to disable enhanced tracking protection in firefox to get this to work.
You had to disable enhanced tracking to get the quiz to work? That's surprising to me. Were the tweets not loading in the quiz without it? I'm using Twitter's JS library to embed the tweets, the only thing I can imagine is that that somehow runs afoul of the tracking protection.
When I was creating the quiz I considered using images of the tweets instead of embedding the tweets directly. Possible I should have gone that route. It might have solved this problem and it would have removed a dependency on Twitter's API package. I chose not to do it because it felt inefficient to use images instead of the embedded tweet and I thought the user experience would be better.
Page does not appear to work in Firefox or Chrome.
I do appreciate the methods section:
> I found an offensive words list from the fine people at this project and considered a tweet offensive if it contained any of the highest-offense words from the English list. (Note: I didn't read the license on that github project, if it doesn't permit my usage as described, then this paragraph is a joke and I really did something else.)
Someone else mentioned that their ad/tracking protection was causing a problem. I can only guess that this is blocking my attempt to embed tweets in the quiz. If this is the case I'd expect none of the tweets in the quiz to load.
Another possibility is a bug I neglected to fix before posting. Some of the tweets have since been deleted and if the quiz tries to load one of the deleted tweets it will retry but could fail and leave it in a bad state. If this is the case I'd expect 1 or more but not all of the tweets in the quiz to fail to load.
In Chrome, tweets are loading for the second quiz, but not for the first quiz. (In Firefox, with ad blocking, nothing loads.)
Also on both browsers, your text is not staying entirely within the browser window. I have the options of guessing the missing text, or, if I really care, highlighting, copying, and pasting into Notepad.
Hmm. That is odd, and not behavior I see in either Firefox or Chrome. I'm afraid my knowledge of CSS is insufficient to explain why that might be happening.
Hmm, that's unfortunate. I had considered loading the tweets as images instead of actually embedding them. I hadn't realized ad/tracking protectors might block the tweets, or else that might have pushed me over the edge to using images instead.
The ever-expanding state of ad filters these days makes honest webpage development way too hard. I found out the other day that another web "annoyances" filter was erasing _buttons_ generated by streamlit.io (a python data visualization library for web). Like someone out there is doing something hostile and now I can't trust that an interface for showing a k-means plot of some data will work for people anymore.
Hacker News, unlike reddit does not have a NSFW warning for submissions.
Additionally, I would say it's reasonable to assume that something that says it is an "offensive tweet quiz" is a quiz that shows offensive tweets, and that therefore any offensive tweet is by definition not suitable for work.
One could argue and say that Twitter should identify and filter out some types of offensive tweets so that, for example, porn is not shown - but that would be explicitly addressing the subject of the submission rather cleverly.
Twitter does in fact have a separate adult content filter, which I have turned off in my Twitter account, in part because it's often wrong :) It filters out a lot of SFW content, and a lot of queer accounts are tagged as such incorrectly.
Were the images loading by default for you? I intended to set it up so that the images showed up as links for you to click if you wanted to or not - and there is a warning above the quiz about clicking the links. Although, granted, I could have been more explicit in the warning about what the images might contain.
> The pictures in the tweets show up as links by default. Follow these links at your own peril.
Ah, I totally missed that! I'm admittedly a bit tired, and that line was above the first set of boxes, where nothing loaded -- I scrolled down to the "Competing" side. This appears to have been a bug, because when I refreshed the first set of boxes had content in them.
Maybe it'd be nice to make it more prominent, set off in bold text somewhere, and mentioned before both quizzes?
(to be clear: I wasn't personally bothered by it, nor was I in an environment where it was problematic)
I’m not sure how they aren’t violating 230 when they clearly have gotten into the news/editorial business.
When a counterpoint to an argument is deemed “offensive,” then they’ve gone too far.
They have taken over the public square and sent out the Mutawa to enforce their idea of truth and virtue while hiding behind 230.
Violent content, threatening content, sure, moderate it. But factually inaccurate content? Or content that expresses an opinion that isn’t violent or threatening? That’s going too far. They even censor academic papers if those papers don’t conform to whatever they deem as true.
So... I'm with you ideologically: I don't think anybody should be force-censoring anything (allow users to suppress content they themselves don't want to see, but don't suppress it for everybody). However, CDA section 230 says exactly the opposite of what I think you think it says:
"(c)Protection for “Good Samaritan” blocking and screening of offensive material
(1)Treatment of publisher or speaker
No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.
(2)Civil liability
No provider or user of an interactive computer service shall be held liable on account of—
(A)any action voluntarily taken in good faith to restrict access to or availability of material that the provider or user considers to be obscene, lewd, lascivious, filthy, excessively violent, harassing, or otherwise objectionable, whether or not such material is constitutionally protected; or
(B)any action taken to enable or make available to information content providers or others the technical means to restrict access to material described in paragraph (1)."
In other words, section 230 (paragraph c) explicitly protects Twitter in this case.
> I regret to inform you that you are wrong. I know that you've likely heard this from someone else -- perhaps even someone respected -- but it's just not true. The law says no such thing. Again, I encourage you to read it. The law does distinguish between "interactive computer services" and "information content providers," but that is not, as some imply, a fancy legalistic ways of saying "platform" or "publisher." There is no "certification" or "decision" that a website needs to make to get 230 protections. It protects all websites and all users of websites when there is content posted on the sites by someone else.
also:
> First off, there is no "neutrality" requirement at all in Section 230. Seriously. Read it. If anything, it says the opposite. It says that sites can moderate as they see fit and face no liability. This myth is out there and persists because some politicians keep repeating it, but it's wrong and the opposite of truth. Indeed, any requirement of neutrality would likely raise significant 1st Amendment questions, as it would be involving the law in editorial decision making.
Section 230 was written to allow platforms to do just this.
As an example, say I run a forum dedicated to flowers. Topics include Care & Feeding, Arranging, and Drying & Pressing.
As a moderator, I can choose to block all posts that deal with geraniums, because I hate the smell of those. If I then miss a post titled, "Hey, who's bringing the flowers to the cocaine and machine gun party next week?", I am not suddenly liable for that.
I moderate in good faith, but my FlowerForum has gotten really popular recently, and I am starting to miss some posts. Users didn't report it to me, I wasn't on my radar. My previous activity banning users that like geraniums will not cause me to become the "publisher" of all forum content.
I think if you could genuinely identify offensive tweets then you could hide them from people who prefer that. It would make the user experience more pleasant. Few people want to be offended!
An ideal may be a customized algorithm per user. What offends me may not offend you. Of course, a trade off of that would be a growing filter-bubble...
I think it depends on how you mean that. One way to imagine the phrase is to think about people with cloistered views in politics, religion, social views, etc. These people could get "offended" by being exposed to more cosmopolitan views but ultimately evolve to be better people by the experience - either by coming to better understand their own positions or by updating their beliefs to reflect things they have learned.
Another way to be offended though is just for jerks to say horrible things to you. Maybe that could be beneficial in terms of developing tougher skin or something, but it seems like that should at least be an optional thing. You might get tougher skin if I rubbed you with sandpaper for twenty minutes a day - but I should probably get your permission first.
> One way to imagine the phrase is to think about people with cloistered views in politics, religion, social views, etc. These people could get "offended" by being exposed to more cosmopolitan views but ultimately evolve to be better people by the experience
Not disagreeing with your main point but in my experience it is usually the opposite, people with “cosmopolitan” views being offended by “cloistered” takes, but eventually growing as individuals and coming to understand why they think what they do
No 'bad words', no slurs, no death threats/saber rattling, nothing. Just reasonably written 'I agree with the X point of view'.
Really really bad look on Twitter's part here...
Edit - to be clear, in both cases the non-offensive tweet said essentially the same thing with the same language, but took the opposing political view.