[spankalee]

This ultimately polynomial direct-connection approach is clearly fatally unscalable.

The fediverse needs to figure out a hubs-and-spokes or supernodes pattern so that service providers can scale up syncing, indexing etc.

ie, my personal instance should be able to offload most of the message passing to an supernode intermediary that lots of other instances use for federation so that my instance only needs one connection, and the supernodes only need to connect to each other and their local network.

[rglullis]

They do have it for some things already. There is this concept of "relays", which you can use as a feed of the data from the larger instances. But AFAIK it's used only as a content source and it's not something that you can set up now as a way to help with scalability.

I am also closely following https://github.com/nostr-protocol/nostr to see how they go along, because I am growing weary of the "tech elite" that is moving to Mastodon and is pushing for "moderation by committee". I've gotten myself with discussions already with people who actually want server operators that want only to open federation for those that abide by some "Covenant". This seems rooted in good intentions, but it reeks of something that might lead to a corporate copout of a network which is supposed to be open.

[masukomi]

> This ultimately polynomial direct-connection approach is clearly fatally unscalable.

Is it? I mean, let's assume you post something popular, and 17,000 servers request it. How many people do 17,000 servers cover? Like even if we assume only 10 people per server. That's 170,000 people. How many people have 17,000 followers, nevermind 170,000 ? And 10 per server seems implausibly low for an average.

17,000 hits is... not particularly notable from a server perspective, triply so when they're all requesting the same "just posted" item which is still cached.

Sure if like, someone with multiple millions of followers is on your server you're going to have issues, but seriously, twitter had issues with that too.

Also keep in mind that there's no algorithm pushing people towards the same "popular" posts. Things grow organically.

So, at what point are you suggesting that this becomes "fatal" and is that a point that anyone that isn't hosting literal superstars is going to encounter?

[bscphil]

> I mean, let's assume you post something popular,

The issue is that there's no federated "popularity" metric. Every user that has followers on 17,000 servers has every single one of their posts pushed to all 17,000. Automatically and immediately, not on demand. Posts by users with only a few followers will occasionally go viral, but an outsize portion of the load is due to "whales".