There are massive bot farms on Twitter that are obvious to even casual observers and most of them don't have "egg" accounts anymore. Look at the auto-replies to any political tweet, major news story, or from a popular account, all automated garbage that follows the same template.
Why Twitter ignores the crap which pollutes their product is amazing to me. Maybe they don't want to touch them because it ups the engagement numbers and inflates active users?
What tools do people use to administer these? Since Twitter and other platforms seem to have no interest in fixing the problem I figure I might as well stop trying to abide by their ToS and leverage the technology for my own agenda.
What needs to happen is that they open up all the blocked account information for all users, and show when users that are blocked by X many users have deleted tweets.
The problem defined is that Twitter allows viral communication to occur in Humans, but which is being exploited in a very specific way by a small group of people who understand how to spread dissonant ideas by intentionally using polarized, meme-based arguments, which themselves are viral in nature. It's like a logic "jingle" you get stuck in your head. Or a virus.
It's on Twitter to fix the Pandora's Box they have opened with this type of infrastructure. (And I'm still sticking to my claim that Twitter is infrastructure given how much it can govern our behavior.)
OK, but any ideas on the tool front? I'm not being facetious, I've just decided that since it's a de facto free-for-all I might as well retaliate against bad actors rather than waiting for people to grow a conscience.
If you're seriously asking: for the most part the tools are in-house. There isn't a single widely used framework (to my knowledge...) that covers everything. Basically, you'd use a mix of something like redis, beautifulsoup, digitalocean, ansible, a fleet of shady proxies and various middleware for everything. Throw in postgres too.
If you're not using the API, all you're doing is programmatically signing up for accounts, storing the corresponding login credentials, writing a little library that assigns all outgoing bot activities to a user-agent and proxy, queues their activities and executes them. Naturally you have a little library that manages the random profile information creator - collect CSVs from data.gov such that you can create convincing random names and addresses, then use random profile pictures from Google images.
The secret sauce is not in the orchestration (someone could wrap all this up into a framework pretty easily), it's in the structuring of activities so that you don't get caught by e.g. having your bots follow each other incestuously. That's a rather less automated process and require active vigilance and tweaking.
What I have seen work in the past is partitioning the botnet such that blocks of them slowly establish signal to noisy credibility in specific niches before intermingling. Many botnet creators attempt to create a multiplier effect, recursively improving their aggregate signal score by having the bots in a massive echo chamber with each other. This is easily caught.
Bot block A should have a few thousand from around the country commenting on the election and sometimes posting memes from reddit. Bot block B should retweet thinkpieces from the tech industry and be the first submitter of various obscure but passable Medium articles. And so on, and so forth. Basically, have the bots act like humans who mostly talk about one category of thing on Twitter, but who still have enough nuance to not seem spammy.
Once your botnet hits a critical mass, you no longer need to do this as strictly in the future. You can spin up new bot blocks quickly and have the mature blocks retweet and interact with them to bring them up to the requisite signal score more quickly. At this point you can capitalize on trends and have a credible mass of followers influencing a conversation on Twitter within hours or days of the trend emerging. As it snowballs, you compound the other blocks to simulate rippling popularity throughout the system (i.e. trends become less about one niche and more about everyone, like Uber->Uber's Sexual Harassment Scandal->Sexual Harassment).
The other secret sauce is in successfully managing the network on a budget, because while all those proxies are what diversify your botnet's origins, individual proxies typically score negatively for websites actively looking to reduce bots.
"500.000 new Members last month!!1" (I just totally made that up)
I suspect its the same as facebooks "2 Billion Users" - its just good for PR to have huge numbers. If you look too closely you might even lose Members in a Month, and we all know "growth" is very important..
You know, I could see ol' Zuck sitting there in his Herman-Miller Aeron chair looking at the internal numbers of FB 'users' and seeing the number be ~15 billion accounts that their in-house 'bot filters still think are 'real' people. He says to himself: 'You know, maybe I could tell the UN that there really are 15 billion people out there and they are all on FB. Ha, I mean, a lot of people really might believe me.' He looks out the window, sighs, and puts out an email via Thunderbird to the marketing team leads that says to keep the number at 2 billion.
Facebook and I believe Twitter report Monthly Active Users. This isn't some smoke and mirrors number. Sure, bots may count in this number, but they aren't referencing total signups when they say "Users" like you imply.
"Why Twitter ignores the crap which pollutes their product", this is easy to answer. Investors.
Investors who don't care about nothing but vanity metrics, investors who don't make CEOs accountable. This is why, twitter is not the exception.
Sorry, but this explanation makes no sense. $TWTR has been a public company since 2013, with investors who emphatically do care about things other than vanity metrics.
A list of the largest Twitter shareholders[1] includes Blackrock, Vanguard, Fidelity, Morgan Stanley etc. These folks really don't care about "engagement metrics", except as proxies for revenue.
This argument can hold for earlier-stage companies with investors hoping to sell at inflated valuations to a greater fool later in line, but once you've been publicly held for several years, it's quite a stretch. Whatever's hurting Twitter's ability to fix its problems, it isn't "investor pressure".
Interested to see how penalizing accounts for 'repeatedly tweeting at non-followers' is going to work when most of what I see on the platform is people messaging celebs who don't follow them back
This is what I was going to post, seems to be about 90% of the activity on twitter. They seem to only care about the experience from the perspective of the haves, anyone else just isn't part of the conversation.
One of the main problems with Twitter is people sign up and don't know what to do with it - except message famous people. If Twitter prevents the plebs from shouting at the intelligentsia then it will make the platform even more irrelevant. The last time I used it, it was like talking into a void and that was with hundreds of real followers.
So much for the strategy of tweeting at a company to resolve consumer disputes. Normally, I would say that's a good thing, but sometimes these companies just don't respond to non-public channels. Google, for instance, is notorious for making it difficult to reach an actual human in customer service.
That got me banned a couple of weeks ago. I wrote an application that recommended an event to the user once he\she would tweet "I am bored". I made the mistake by giving providing a fall back once the location could not be determined. This lead to ~40 tweets with identical texts. Twitter quickly pulled out the banhammer.
The listener is a pretty much off the shelf Tweepy listener writing into a Redis queue. For the recommendation is comes via an API where I can't share the code.
>Except when Twitter makes that a heuristic for detecting bots
There are several dozen. Maybe not turned on in prod, but that's a whole another story.
In fact, within three months of joining, way back in 2012, one of my very first tasks was to write a standard datamining job that would compute the difference between the GPS location during office hours (9-5pm) and the GPS location during home hours (7pm-7am) of everyone who tweets. A histogram of those differences would tell you about the commute distance of the average American who tweets. You could then bucket by region and say interesting things like the average NY tweeter commutes 25 miles more than the average CA tweeter.
Looking at the results we got, it was clear there was a substantial percent of bots, because their location varied so widely, minute to minute hour to hour. Haversine of GPS diffs will be reasonably stable, because your IP maps to the GPS ( we used the standard Maxmind geoip2 API) , and those IPs are relatively stable....Except if you are a bot and switching IPs willy-nilly.
This was just one instance, but there were several such projects...usually interns and new employees would work on these to get their feet wet, and then move on to more substantial projects.
Provenance of data was not in scope, 'twas more of a standard datamining "see if you can dig up something interesting" project.
Like I said, there were scores of these - one of my colleagues wrote the famous soda vs pop thingy which once again put location stats to good use- http://blog.echen.me/2012/07/06/soda-vs-pop-with-twitter/
With quantitative analytics being used and abused by ever more businesses, the advantage will go to those that can apply qualitative checks to their assumptions - and scale it.
Are you suggesting to get a Gmail account, turn on the setting that makes Twitter email you notifications, mark tweets as spam in Gmail, and then use Google's spam detection as an existence proof of a solution if it's successful?
There are enough people talking to each other to make it interesting, at least for me. But the issue that most people are having is political nuts who just go around doing a search for "Donald Trump" or whoever and then abusing anyone who said anything remotely critical. Even worse are the actual white supremacists who do that or worse.
The bots are a problem as well, but they aren't what runs people off the platform.
That sounds like an informal description of a heuristic to flag accounts as potentially abusive rather than a specific rule with a specific punishment.
So because I see no reason to upload a photo my 7 year old twitter account is now considered second rate whereas the bot account created last week is not because it was updated to upload a random profile image.
Seems a bit like trying to claim they are doing something while nothing important gets done.
Today, the egg, tomorrow, unverified account names. Are they going full Facebook?
And filtering out keywords in messages... I suppose that's ok, and certainly their prerogative, but talk about creating more social bubbles. As much as I disagree with the random twits on the site, it reminds me that not everyone is a left-leaning political hobbyist. Hearing things you don't necessarily like is part of being an adult.
Having concerted bombs of right-wing harassment drop on one's head for having the temerity to do something like "be a woman with political opinions" is not part of being an adult. And that's the reality of Twitter right now.
This isn't nearly enough; actually showing the trackable legions of smurf-account harassers the door might be beyond Twitter's meager capabilities at the moment (after all, they only have a small army of developers, right?), but implementing BlockTogether as a first-party tool, including "automatically block accounts under X days old that @ me", is just part of being a decent host.
The question then becomes is the necessity of having a twitter account "part of being an adult"? I'm not saying the harassment is right, mind you, and certainly doxxing and real physical threats should be treated as the criminal acts that they are, but I don't think it's something that can ever really be remedied unless the victim removes themselves from the situation.
As I see it there is a sort of give and take relationship with inherently public social media like twitter. Your comments and thoughts are presented to a wide audience, but that inherently subjects you to possible dissenting opinions or harassment from that audience. The alternative is to, say, create a personal blog. You could write your opinions all day there and nobody will judge them or attack you for them, because without a lot of effort in SEO and marketing, nobody will ever see them.
Most people oppose bullying in school because kids "have" to go to school, so in a sense they're forced into the environment and should not be subject to attacks there. But who is forcing you to be on twitter?
> The question then becomes is the necessity of having a twitter account "part of being an adult"? I'm not saying the harassment is right, mind you, and certainly doxxing and real (as in tangible) physical threats should be treated as the criminal acts that they are, but I don't think it's something that can ever really be remedied unless the victim removes themselves from the situation.
Oh, then we should just give up and let literal white supremacists and anti-feminists and gay-bashers chase the weakest among us out of the social discourse. I'm sure that isn't a political tactic being employed intentionally against them or anything.
Or, you know, we can fucking not do that.
Stop normalizing evil. Doing so literally-not-figuratively arms those who would do harm to the people among us who need our support. Show them the door, not their victims.
The harassment from Twitter is everywhere on both sides of the political spectrum though. You're painting a very one sided picture. I'm not attacking you here I'm just saying there is a narrative about the "alt-right" harassing the innocent leftists and it's not the whole picture.
I'm a (democrat) mixed race woman, and I get more harassment and vitriol from "tolerant liberals" when I post anything remotely in support of the president than I ever do from "white supremacists or anti feminists". I have noticed if I don't go right down the party line on an issue, I'm attacked violently. The "N" word I get called frequently is Nazi, not the other.
I'm not trying to say your experiences are invalid, I believe you're experiencing it, but I think assigning it to one political party or movement is inaccurate. It's really just the culture of Twitter, and maybe society in general.
Yeah, the thing that's probably ultimately going to limit the life of Twitter's latest automated anti-abuse measures is that left-wingers who fire nasty, vitrolic, vulgarity-filled tweets and invitations to kill themselves at users on the right are also getting their accounts automatically put in time out or even suspended for it, and they're getting really pissed off about it. Sooner or later the press is going to get hold of a sufficiently sympathetic example and that'll be it.
If Twitter wants to show the alt-left tools the door too, I'm certainly not gonna cape up for them. But there is a difference of orders of magnitude in both size and cohesiveness to wrestle with here, and it's not towards the burned Bernie bros. That alt-left doesn't have members of state-blessed media outlets (well, until recently, when cheering on pedophilia became a little too much for them) on campus tours where he'd out trans people with a pack of jackals ready to pounce.
>Stop normalizing evil. Doing so literally-not-figuratively arms those who would do harm to the people among us who need our support. Show them the door, not their victims.
To be completely fair, they will remove people for blatantly racist/sexist/homophobic tweets, and often do, where "f* white males" talk is completely allowed there.
> I'm sure that isn't a political tactic being employed intentionally against them or anything.
> Or, you know, we can fucking not do that.
> Stop normalizing evil.
Your rhetoric is very emotional, and you want to use "abuse" a s an excuse to ban people who simply disagree with you ("anti-feminists", really?). I don't think people who share your views should be given the power to control discourse[0]. It's funny that you insult the "alt-left" ("bernie bros") when many would define the alt-left as the regressive left, in other words those who want to ban people simply because they disagree, like you're trying to do.
[0]: especially since you could argue that Twitter is the #1 platform for political discourse in the US, and is therefore vital for free speech (not as a legal obligation, but moral principle).
These people have always existed. They will continue to exist. They will continue to find ways to harass the people they see as easy targets because they find enjoyment in it. Nothing you can say or do, no policies twitter puts in place, will ever eradicate them. At best it will slow them down. There's always some work around and they have lives sad enough to dedicate to finding these work-arounds.
For all of human existence up until last decade, these types of people didn't have a wide social outlet for their thoughts, just like everyone else didn't. Now it's open to everyone. Yes, it's morally reprehensible. But this is not something that we or any social media company can ever solve. It will always be a cat and mouse game. But it's a game that you don't have to play.
You can still be a citizen of the 21st century world and not be on social media. It does not put you at any disadvantage to not have a twitter account. If you believe that it would, reconsider your priorities in life.
> You can still be a citizen of the 21st century world and not be on social media. It does not put you at any disadvantage to not have a twitter account. If you believe that it would, reconsider your priorities in life.
This is analogous to all those arguments that "If you have nothing to hide, you have ", or, "If you want privacy, you always have the option to become a hermit and live completely off the grid with no contact with friends or family" What if I want privacy and to participate in modern society? Why should I have to choose?
It's the same here: Twitter, for all its faults, is very useful. Why should I have to choose between not using it and enduring a bunch of abuse on it, if Twitter can fix that? To protect the "right" of some anonymous shitheads to have victims be forced to listen to their harassment? Please.
You keep replying in this thread but you keep making the same error because you're starting from the axiom that "blocking is bad" and deducing forward from there. I reject that axiom.
> Nothing you can say or do, no policies twitter puts in place, will ever eradicate them. At best it will slow them down. There's always some work around and they have lives sad enough to dedicate to finding these work-arounds.
First of all, this isn't true: plenty of platforms have "good enough" moderation that harassment is either eliminated or at least reduced to a tolerable level. But even if it were true, it would not be a reason for Twitter not to attempt anything. Again, you're starting from entirely the wrong premises here.
I agree with this 100%. These types of people have been around for centuries, the only difference is now they have a mouthpiece and encouragement. Use your block button, it's free.
This post reveals a pretty significant misunderstanding of what Twitter is for marginalized communities. And, as such, this conversation is fruitless.
But one last thing: if it's as abhorrent as you seem to actually think it is, stop caping for them as being something that can't be stopped. Because you help them by doing so.
If you were Twitter, why on Earth would you want the accepted answer to anybody's problems to be "don't use Twitter"? Your entire business model is built around getting people to use your service.
And that's the problem that twitter needs to come to terms with and attempt to resolve, not the end user. I personally don't have faith that it is a solvable problem, at least not while still remaining an open platform. As long as anyone can sign up, "undesirables" will be among them.
The end user has precisely two options -- use twitter or don't. If using it causes you too much mental anguish, and twitter is incapable of making the experience better for you, then don't use it.
> And that's the problem that twitter needs to come to terms with and attempt to resolve, not the end user. I personally don't have faith that it is a solvable problem
So your advice for Twitter is to throw up their hands and close down their business?
I mean, it's easy for those of us outside the company to say "your product is fundamentally unfixable." But if you're inside the company, and the product as-is has giant problems that are keeping potential new users out and driving existing ones away, you have to try something. Your job literally depends on it.
Maybe the things they're trying now will work, maybe they won't. But as long as Twitter has employees and enough cash on hand to pay them, "just tell people to not use Twitter" is not advice anybody there is going to consider useful.
The fact that you're not forced to be on Twitter means that victims can leave, but it also means that bullies can be banned. Why prefer the first option?
I'm not preferring either option. Of course they can be banned. But they will come back. They always do. Why subject yourself to that continuous cycle of stress?
The real question to ask is: why are some of these political nuts working so hard to push people off of Twitter? The answer, in my opinion, is because it allows people they don't approve of the same public voice as themselves, for the first time ever in many cases.
> including "automatically block accounts under X days old that @ me"
How can a new user (who isn't a harasser) engage with the Twitter service if they get auto-blocked when they try to engage with anyone on the platform?
Personally? I interact with people I know, not #brands. And I do that by following, not by @'ing out of the blue. Everybody I know does the same. If somebody with a name I recognize follows me, I'm gonna follow them back and @'ing is fine.
What you are describing is a tinder approach to twitter, which serve fundamentally different roles in the online social sphere. I understand that is how you and yours use twitter, but extrapolating that behavior across hundreds of millions of users is almost certainly hasty.
What if engage means "I want to blow the whistle on something, can you get me in touch with a journalist?" for 0.01% of blocked messages and "u suk vote yrump" for the rest?
> maybe Twitter isn't a good place for them
could have fooled me because if you built a platform purely for harassment it would resemble twitter closely
I believe what he was getting at is as an adult, you should be able to realize that a website on the internet is just that--and have the ability to take it with a grain of salt without being highly offended / outraged.
I don't think the intention was to say everyone on Twitter are adults--in fact it's the complete opposite of that. It's a platform where you can get both the best and the worst from all walks of life across the world. Sometimes there will be jerks and people who are the polar opposite ideologically to what you agree with.
And the point is that disagreement isn't what runs people off that platform. It's abusive behavior by large groups of people (and sometimes their bots) towards whoever they don't like.
Do you think Twitter should have a shared default blocklist?
That would kill Twitter almost immediately as everybody is added to the list (if nothing else, as a retaliation for somebody else adding them to the list) or do you mean that they should have the slightly more intelligent blocking features it has (including young accounts and/or accounts with few followers)? Because they are pretty easy to bypass by having a bunch of accounts follow each other and create them well ahead of time (and no Twitter can't prevent this: finding cliques in a graph is NP-Complete, and Twitters graph is crazy big).
Twitter could add many things, but they haven't even figured out that they need to accounts with female names and portraits and user names that ends in a number, even though they are always spammers.
I think shared blocklists are great. As defaults, probably not. I don't use any shared lists because nobody's going to take a serious run at me, I just don't want to spike my blood pressure seeing the egg/anime smurf accounts well-actually at me.
But I think network-effect block lists are probably a good idea. Nobody who follows Sargon of Akkad or other white-supremacist dorks is somebody I am ever going to want to hear from, so tube their tweets. That sort of thing.
Sure but harassment isn't. My understanding is that all of this is optional. As long as anonymous accounts are not abused there's no reason to filter them out. If anonymous accounts are abused people will filter them out. Thus the only people we have to blame are the abusers of anonymous accounts.
Please show me an intelligent left-wing and an intelligent-right wing person, both of consistently post well reasoned and well argued content from their perspectives.
It doesn't matter what side they are on, all that gets shared on twitter that would qualify as political is absolutely nauseating waving that sides flag and telling the world how stupid the other side is.
It would be better if it was blocked, preferably by default.
Good day there! How are you today? I am a girl who just moved to live in this city! I have done a search on Twitter to look for man in our area and yeah I found your Twitter. In case you don't mind we can make friend and chat chit! Do you have snapchat? Please add me on Snapchat nick: [redacted] so we will chat and I can show you my personal photos! I don't like to chitchat here:) It's boring! Lets Snapchat! Sometimes I also use another Snapchat nick: [redacted] on another cellphone, kindly add this nick too if you did not see me online on Snapchat nick: [redacted] :) Hope to chat with you really soon.
That's a lot of tweets(5+). Did you get them all at the same time, or did they have some kind of randomized delay? Also uh...what's the benefit of spamming you out of Twitter and into Snapchat? Do people get $$ for views of their snapchat images, or something?
No mention of how filtering might apply to Twitter's trending algorithms. Since much of the activities of bots are aimed at pushing a narrative to a wider audience (getting it trending), it seems like Twitter is currently neglecting one of the most serious harms that bots pose to the community.
I get that Twitter is focusing on harassment in order to make people feel safe when using their platform, but I hope the solution will also effect this other aspect of bot use/account abuse.
I see Twitter is getting harder and harder to use anonymously. As someone living under a totalitarian government that is tightening its grip everyday on free speech and the internet in general, this worries me.
Trying to signup and use Twitter using Tor is practically impossible.
I understand the spam problem on Twitter is out of control,but I wish there was a way to use it anonymously.
What I don't understand: Why doesn't twitter take moderation seriously? On reddit it's left up to the moderators of each sub. It does very. But having a very "up to interpretation" but yet not politically biased set of rules and enforcement is great.
We don't have a code of conduct or any of the nonsense. Someone violates it and we catch it, they're gone. (Also that's subjective to time based on how many times we caught them as well) End of.
I'd love them to take moderation seriously, but it couldn't scale and there's no natural "partitions" between communities of interest with different norms, so you can't delegate it to users. They've only just managed to deal with the most high profile example of "brigading".
Dammit. I thought I could now remove any tweet that had the word 'trump' in it[^2], which meant I could refollow a bunch of people and make twitter much more useful, but it seems that it only works for notifications in which you are mentioned[1], which means the feature is pretty pointless, at least for most people.
[1]:https://support.twitter.com/articles/20175032
[^2]: this is a complaint I have with almost any social media, even google plus. I want a persons insight on subject x, I don't want to hear their political ramblings, no matter what side they are on - politics are poison.
Faster please. Anything that accelerates Twitter's demise is ok in my books. Bonus points if wounds are self-inflicted.
I'm old enough to remember when you only had to deal with the crazies holding signs and hollering if you went to certain parts of town. If you didn't go and/or engage the crazies they didn't really exist. Sort of like if a tree falls but nobody is there...
Now everyone thinks their opinion is valuable and spouts nonsense all over Twitter. Time to take away the soapbox.
Why Twitter ignores the crap which pollutes their product is amazing to me. Maybe they don't want to touch them because it ups the engagement numbers and inflates active users?