Hacker News new | ask | show | jobs
by lorean_victor 845 days ago
yes it is a federated system by design. doesn't mean it is not a limitation though.

to give you an example: in case of email, though it is not that difficult to host your own server (and even if you are a small startup you'll most probably do so without much effort), in the end basically Google decides on "who is an accepted participant in the network". If Google deems you spam, you are spam. If Google deems your authentication emails "promotion", for most intents and purposes, you are "promotion" and your users will miss your emails.

that, I feel, is an inherent limitation of any federated system, specifically one whose design is really inspired by email.

> I don't see how that follows.

as you've mentioned right after.

> work is better distributed temporally (since consumers hopefully poll the feed in a randomized way independent of new posts appearing).

1 comments

Still, what's the benefit of that? If peak load is a concern, a push-based system can stagger out individual post deliveries just as well, and push gives the producer much more control over load management. If that's not enough, several posts can be combined too.

In a pull system, you are at the mercy of your consumers' refresh rate setting, and for infrequent producers, you'll have lots of wasted cycles fetching nothing new on top of that.

I do agree that pull is simpler to implement (since subscription management is handled entirely on the consumer side, requiring no network protocol and server-side state for it), but in terms of network calls, it's strictly worse.

the system doesn't need to be completely pull-based though. I don't even think most modern RSS readers are fully pull-based, don't they support WebSub?

the main point is to separate publishing and distribution, making publishing far more accessible and decentralised. for that to happen, I guess, from a publisher's point of view, the system should be pull based. of course we can have hubs and relays to add pull-based mechanisms to ease the load of the system.

p.s. even in that case, strictly speaking, yes a pull-based system, or even a hybrid one, will always require more work than a fully push-based system.

> the main point is to separate publishing and distribution, making publishing far more accessible and decentralised. for that to happen, I guess, from a publisher's point of view, the system should be pull based.

Oh, I completely agree with that assertion: Static content hosting is much easier than stateful subscription management, knowing which aggregators to post to etc.

I just think that this does inherently put more work on the subscribers, and there's no real way to do it efficiently in a relatively flat architecture (with subscribers directly polling publishers). WebSub helps with pure distribution, but not with aggregation (e.g. to allow keyword/hashtag search), for example.

That doesn't mean it's not worth still designing a system like that (in my view, the benefits are significant!), but I wouldn't call it a performance win.

agreed. it won't be a performance gain at all.

but I suspect aggregating / indexing / etc. still is going to be the most resource consuming part of a push-based system if you want your content discovery to not be limited to two-way interactions, which means the gains of a push-based system, in terms of performance, shouldn't be that much (I suspect the main gains will be in realtimeyness instead).