Hacker News new | ask | show | jobs
by Tipewryter 2010 days ago
Thought experiment:

How hard would it be to write a centralized 1-to-many message pipe that replicates Twitters core functionality?

How hard would it be to run such a thing? The moral and legal needs to censor content would probably be a bigger task then the technical implementation?

8 comments

Mastodon (and other fediverse FLOSS) already have millions of active users across thousands of providers. https://the-federation.info/mastodon

It's easy to set up your presence and start small, e.g. with a web app which syncs your Mastodon and Twitter accounts. https://moa.party/

Is it really "millions of active users"? (I don't doubt it's possible, but the incentives are very much aligned with "let's count this bot activity as a real user." It doesn't even need to be malicious -- I fell into that trap myself.)

EDIT: Odd, I'd like to report a bug in that site. https://imgur.com/EsGyvnT It doesn't seem to load. Or rather, I saw it briefly flash the statistics, once, before it went back to loading. The 0.3 seconds I saw the stats looked impressive though. :)

The only thing in dev console is a warning that seems unrelated: "<ApolloProvider>.provide() is deprecated. Use the 'apolloProvider' option instead with the provider object directly."

Ah, the root cause is that the graphql fetch takes ~32sec: https://imgur.com/q769ozR So, never mind! Just a bit slow.

---

Anyway, it claims 400,000 active users in the last month. What counts as an active Mastodon user? It also claims 400 million Mastodon posts (total?) which I suppose are tweets.

Interesting... Twitter has about ~17M tweets per day, iirc. If this growth is legit – a big if – then Mastodon may be gaining more momentum than it seemed.

Actually, twitter gets 431M tweets/day: https://twitter.com/jasonbaumgartne/status/13320033501922713...

Hmm. I wonder how much exponential growth would be required for Mastodon to catch up to twitter...

Does it matter whether the aggregate Mastodon user count ever catches up to Twitter? Is Twitter today actually better for its users than the Twitter of yesteryear that only had ~17 million tweets per day?

I have met many people on the fediverse, and at least two friends of mine have moved thousands of miles due to relationships and opportunities that likely would not have occured if the fediverse (Mastodon, Pleroma, etc) and its variety of platforms and userbase did not congregate in this manner.

One friend even bummed a plane ride on a tiny general aviation plane by serendipitous happenstance through the fediverse to make the multi-state move out of an abusive living situation.

Unbounded growth is not always good, and many fediverse instances have limited or closed new account registrations to ensure their community doesn't become the next Voat, FreeSpeechExtremist (which I was banned from for fairly benign speech) or /r/TheDonald.

I agree in spirit, but a community that isn’t growing is probably dead.
Growth Hacking is a thing. Whether that counts as real growth or not I don’t know. But metrics can become suspect when they become a target.

There are quite a number of people using Mastodon and it is quite nice but it does feel a bit echoey all the same. At the same time there is a lot of garbage on twitter that you really couldn’t call “active users” and keep a straight face. As usual e truth is usually somewhere in the middle ...

In my experience bots are not that common, but it's very common for the same person to have multiple accounts.
> It's easy to set up your presence and start small, e.g. with a web app which syncs your Mastodon and Twitter accounts. https://moa.party/

True, but some of the more avidly pro-Mastodon and anti-Twitter regulars take it upon themselves to be gatekeepers (some nice, some nasty) if you try the dual-platform method. Just noting this for the sake of those who haven't yet done so. Feel free to give it a go, but don't be surprised if you get smacked in the face fairly early in the experience.

> moral and legal needs to censor content

I for one would love for there to be a decentralized system that by design makes it impossible to censor any content, and if the content would be stored in a decentralized, encrypted manner it would also be impossible to legally enforce it as it would not even easily be known where it would be stored at all.

Think of perhaps a network that retrieves its content viā an union-routing-esque mechanism where information is published that cannot feasibly be altered any more once put there whose origins and locations would be computationally infeasible to trace and as it's stored in an encrypted fashion the parts of the network that do store it do not even know what they are storing.

This exists, its called Freenet: https://freenetproject.org

Not a particularly new concept, Freenet has been around for over 20 years now. Tor and I2P have more traction in this segment.

Sounds like an interesting thing; let's see whether there's some interesting content on it amidst all the child porn.

And sure enough, I already found a website that says: “Please note that the Freenet network (much like Tor) attracts pedophiles and a large amount of sites contain child pornography. Some sites jokingly add a disclaimer saying This site does not contain child pornography. click here to continue.”

Mhmm, hence why the Freenet proposition is a hard sell. The average person doesn't want to devote half their computer's storage space to storing encrypted data they have no insight into, especially when that data could be considered illicit in some jurisdictions when decrypted.
I suppose the problem of such ideas is that it will generally be filled with content that is illegal in at least some jurisdictions and will only be used by those that need it to evade the law and the amount of content that isn't legal on it will be minimal.

So the end result is that all one finds on it is child pornography, controversial opinions, leaks of government data, and the lot.

I suppose my ideal wish was a platform such as twitter that intermixes legal and illegal content.

That is what stopped me, but it is also a check on society. If something else ends up on freenet that is not cp, it is an indicator of something having gone terribly wrong.

I didn't realize it then, but if Hunter Bidens data had been put there they would have been basically impossible to censor and it might have swayed the election.

Freenet is designed for data storage, while I2P (https://geti2p.net) provides any service on top of the anonymizing network.
While those do exist (and plenty of them), many people avoid them precisely because content moderation isn't possible or regularly done in those communities. From people who prefer not to sift through egregious amounts of bigotry/spam, to people who are at very real risk of having their physical safety or livelihoods damaged by online abuse (e.g. children), there are plenty of reasons to avoid those communities--reasons why those platforms are often ghost towns or toxic echo chambers.

Like, this isn't a "think of the children" argument that those communities shouldn't be allowed to exist. Of course they should. But don't be surprised when people prefer more centralized and "censored" (policed/moderated) communities. For the vast majority of people, that is a feature, not a bug--and they tend to realize that and boomerang back to Twitter etc. when they try out decentralized/anticensorship-oriented alternatives.

Usually when a man speaks out against censorship, he tends to still want to censor whatever he finds abhorrent himself.

A rather big difference is also the relatively lower level of activity on those alternatives.

In the recent past, I've seen Twitter harshly ban account I randomly make. Sometimes its weeks before they ask for a phone number and if you don't have a real number (iirc they check against VoIP) they won't unrestrict you. If you have the same IP as other account I think they also might auto restrict you.
They banned my account for unnamed terms of use violations after my first post saying “well, I finally made a Twitter account.” Their support replied with “queries will not be responded to.” It was for the best
When they locked my freshly created account and didn't let me use it unless I gave them my phone number I mailed them and told them I would not do so for privacy reasons. My account was unlocked the next day.
Several years ago I tried to make a Twitter account. It got banned before I posted anything. I hadn't even subscribed to any users. I was going through the settings, and then \bam\, banned for being a bot, or some such nonsense. Nothing of value was lost.
Twitter aggressively tries to get phone numbers from new users - providing it is the main way to get unbanned in cases like this.
Unless you're my mother, or maybe if you're part of my immediate family, you're not getting my phone number.

American giant corporation with questionable ethics: hard no.

Yeah, I had to get a separate number just to please twitter's automation.

Their approach is somewhat understandable - stopping bots is a hard problem, phone number in theory represents a material cost and a hurdle to someone who wants to rapidly register thousands of new accounts.

Don't you have a burner phone/sim for these types of requests?
This is brought up every time this subject is discussed. Is a burner phone a normal thing that regular people have?

But no, I don't have a burner phone. I'm not going to buy a phone and a SIM just as some kind of non-solution to tech companies having a hard-on for invading peoples privacy.

Many alternatives exist but none reach critical mass because twitter has buy in from the major media outlets and the journalists they employ. Unlike other social networks that grew spontaneously, Twitter was effectively crowned.

Every twitter clone that tries to compete without first having the approval of the major media outlets will either be publicly ignored, belittled, or outright attacked.

This is a very conspiratorial vision of things.

Really, it's just that social networks work by having a lot of people, and lots of people are on Twitter because other people are on Twitter (I think there's an audience thing as well).

Though, to your point, I think Twitter has outsized influence for the same reason Google Reader did: loud nerds and journalists were always using it. Though now the politicians using it probably count for a lot too

I don’t think it’s a conspiracy theory to acknowledge that the exposure twitter got from media outlets universally promoting their twitter handles next to their core brands had a significant impact on twitters early growth, as well as a reinforcing effect to this day. Not every small social network gets that type of treatment.

Facebook did not grow this way, even though their relationship with the media is nearly indistinguishable from twitter’s today.

> This is a very conspiratorial vision of things.

History is full of conspiracies.

> Really, it's just that social networks work by having a lot of people, and lots of people are on Twitter because other people are on Twitter (I think there's an audience thing as well).

The network effect is real, but it is not the only effect. Twitter and other tech giants now think that their role is to protect the world from information they deem wrong. As well, prominent people in media and politics are urging them to take on that role more expansively. Naturally, therefore, they will oppose competitors whose distinguishing feature is to have less or no censorship.

The media establishment does not look kindly upon those who seek to deconstruct it.

> The network effect is real, but it is not the only effect. Twitter and other tech giants now think that their role is to protect the world from information they deem wrong

Do you have any examples of anything "they" "deemed wrong" which wasn't factually wrong? And not "alternative facts" bullshit, regular, proven facts.

If you care about the truth, it's your responsibility to find it, not to demand that I feed it to you. You are on the Internet, after all, and since DNS has yet to be censored, you can still seek alternative sources.

Now, you place the burden of proof on me, assuming that Twitter is in the right by censoring information and demanding that I prove that their censorship is wrong. But why don't you place the burden of proof on Twitter? Why do you apologize for censorship?

In your other comments, you say that "Only China can stop China," and you say that the U.S. mustn't respond to cyber attacks by China, Russia, and Iran, or those nations might escalate, so we must look the other way when these nations infiltrate and attack us.

One wonders what you're after.

When it comes to publishing, it doesn't matter where you publish, only that you can put your words out there. If someone has a web browser they can read what you say. The only thing that suffers is engagement, which you'll have a hard time getting regardless what website you use to publish if you're just starting out. But engagement is either meaningful or noise, and meaningful engagement comes from thoughtful words, not the size of the website.

So don't worry about critical mass. Pick where you want to publish based on what is important to you.

I almost exclusively use FOSS, federated networks to publish my thoughts, the only exception being this site. I engage here pretty often because I like the quality of the discussion and the rule of only contributing if you have something constructive to contribute, but ideally this website wouldn't be a silo.

I made no judgments on the value or importance of critical mass.

Critical mass seems important for exposure and personally I don’t think exposure is important unless you are seeking to maximize something that comes from exposure, usually it’s profit.

My comment was mainly to bring attention to what I consider the primary causal factor behind twitter’s critical mass.

Twitter got popular because early adopters used it, then celebrities, then Oprah and then media journalists and then Arab Spring and then news journalists and then early adopters left.
Do you have more information and / or evidence of this? I was an early Twitter user (2008) and I don’t recall things playing out this way. I’m not suggesting you’re wrong, and it’s entirely possible I was (am?) being naive about how things played out and would love to learn more.
My recollection matches yours. Twitter didn’t have an incumbent to overcome (insofar it was, and largely continues to be, complimentary to the other social networks). The early days of Twitter were a wasteland as far as old media engagement is concerned. It was dominated by TechCrunch, Scoble, etc.

And then in 2008 it ramped up for a multitude of reasons. A politician nobody had heard of somehow managed to mobilise a lot of support via a tech platform many hadn’t heard of, and get to a point where it looked like he had a legitimate shot at becoming US President. By early 2009 it was popular enough that Facebook updated their timeline to try and combat it, everyone was talking about their growth, and Ashton Kutcher was goading CNN into a competition to be the first account on the platform to have 1M followers (he was the first btw).

Old media was late to the game here and had to play catch up to stay relevant. I suspect that was also the tipping point though which changed journalism forever into this clickbait and eyeball driven economy. If anything Twitter inadvertently controlled old media, not the other way.

Can you name some alternatives? I am not aware of any.

Please note that I explicitly said "centralized". Because decentralized solutions like Mastodon have not resulted in viable alternatives for the average user so far.

ActivityPub and the Fediverse of which Mastodon is part have absolutely resulted in viable alternatives. I use the network for probably 90% of my online social interaction. It is far superior to any alternative I've found so far, mainly because I can publish freely, cannot be banned from the network, nobody has to have an account to see what I say, and it is actually a network, not a website calling itself a network. I can think of no way to reach millions of people that is less restrictive than this network.
What is the difference? Centralization applies only to moderation policy.
Minds, gab, and Parler come to mind. There are certainly others.
Parler is a no go for me partly because I don't like ideological monoculture, but mainly because users have to have an account to even see what you say. IMO this makes it worse than Twitter, you shouldn't publish your thoughts somewhere with conditions on who can see what you say.

I've never used Minds and dabbled with Gab when it first came online only to find that it is as noisy and worthless a place to engage as Twitter is, possibly moreso.

> users have to have an account to even see what you say

fwiw, I thought the same until recently but discovered they are visible with the following format:

https://parler.com/profile/[username]/posts

Interesting.

Looking at Minds, I already see a big flaw: All posts are under the /newsfeed/ namespace.

So instead of

    /username/1234
your posts are here:

    /newsfeed/1234
I don't think users want to get "robbed" of their posts like this.

There also is no embed functionality. For Twitter type shoutouts it is absolutely vital that they can be embedded around the web.

Fun fact: the user part of a Twitter URL isn’t important. You can replace it with any other user.
Username in status url and “embedibility” are good features to have but i don’t think they are causal in determining whether or not an twitter clone reaches critical mass. Maybe they are though. You should build an alternative!
The latter two are dominated by people who've managed to be too fascist for Twitter, which is quite a feat.
> Every twitter clone that tries to compete without first having the approval of the major media outlets will either be publicly ignored, belittled, or outright attacked.

You proved that point

Is the person you're responding to wrong? You can't just slap a conspiracy label as if that somehow makes what they said wrong
How so?
Easy.

https://twtxt.readthedocs.io/en/latest/ is an interesting way to do it.

RSS with comments can work. Mastodon, Pleroma, misskey, gnusocial all do it. They have the added benefit that a user cannot be censored from the network entirely, only from servers by their admins, and blocked by users.

You can't even get people to write their stories on their blogs and then link to Twitter, instead somebody created a twitter bot that will do that, but it has to be summoned in the comments.
That is a software repo. Not a public centralized service that lets users communicate with each other.
One more click leads to https://joinmastodon.org/

> ... centralized ...

That was not in the GP’s requirements.

Edit: plus: GP explicitly asked for software.

Why do you want it to be centralised? Surely a federated system like email is just as fine, if it provides a good user experience?
If. But so far, no federated system has resulted in good user experience.
Twitter tried to build one of those back in 2007.

You can go back and read tech media from the time about all the engineering challenges they ran into and the regular downtimes from trying to scale the thing to a tiny fraction of its current volume.

Turns out to be hard.

That’s because they wrote it all in ruby, which don’t get me wrong is a fine language but did have scaling issues back in the day. As it happens this was a growth experience for ruby too.

However limited their technology choice might have been from a performance standpoint it did give them the adapatability and flexibility to provide the fun and entertaining product that got them off the ground in the first place.

It did inevitably lead to the growing pains we discuss but I doubt anybody expected it to get as big as it did.

WRT the original comment here ... the main limiting factor nowadays is just getting people to use it. The network effect. Mastodon is a thing, and people do use it but it’s a minnow relatively speaking.

The next big challenge I think will be some kind of decentralised twitter based on some kind of blockchain (by which I mean progressive content hashing and consensus) and I don’t mean federated, like mastodon. The mastodon name is ridiculous too .. I’m not going to go into why but if you know you know, and stupid as it is it does matter.