[KirillPanov]

Same experience here.

The #1 lesson they failed to learn from Bittorrent and Tox: your DHT must be sessionless.

QUIC is not a solution to this particular problem. Although QUIC uses UDP, it still creates and manages sessions. This is incompatible with decentralized scaling. At the scale of Bittorrent and DNS you cannot manage this kind of session state without centralization.

[2color]

IPFS encompasses more than just a DHT: - representing content addressed data with hashes in CIDs - Discovering them (via the DHT) - Transfering (Bitswap, HTTP, Graphsync, sneakernet and more in development)

I'm not 100% certain what you mean by the DHT being sessionless.

> At the scale of Bittorrent and DNS you cannot manage this kind of session state without centralization.

You don't have to route content via the public DHT. Depending on what kind of data you're publishing, you can use a private DHT swarm, or delegated routing with HTTP.

> This is incompatible with decentralized scaling.

[anacrolix]

Absolutely. Too many people try to implement DHT on top of stateful connections, it doesn't scale. It must be as cheap as a single packet or two, with no context.

[jzm2k]

Could you elaborate on why it can't scale if it's stateful?

[KirillPanov]

That should probably read "doesn't scale without centralization".

The full answer to that question is way, way, way beyond the appropriate length for an HN comment.

DHTs need to accept requests from any random IP address, anywhere on the Internet. If you allocate state, you have to deal with resource exhaustion attacks. Dealing with them turns out to be way harder than anybody thinks when they first start down this path.

Cloudflare and Google can use stateful HTTP(S) because they are so massive. They can employ hordes of people to constantly tune DOS filters to block griefers before state allocation occurs, and lawyers to deal with any griefers who are as large as they are. If you're happy with that kind of centralization you shouldn't be wasting your time with DHTs; you don't need them.

Aside from resource exhaustion attacks, there are some pretty deep lessons connected to systems and architecture. The bittorrent bootstrap nodes serve some really insane number of requests/second on very modest hardware (I'm still trying to find the link that detailed this...). The Internet root zone DNS servers were, for a very long time (and perhaps still today in one or two cases) just a few good workstations at universities.

Sessionless protocols scale like crazy, on cheap hardware. But they are very hard to design correctly. If you have VC bucks, spend them on hardware and use sessionful protocols.

[survirtual]

You can restrict DHTs to accept well-formed requests and further restrict them using a web of trust. With a large enough network and an on onboarding path using nodes that have higher bandwidth, it can work to mitigate that vector I’d think.

It would be really hard to launch that attack with a DHT that implements appropriate pruning, drop, and block rules for bad behavior. All that can feed into a personal (& autonomous) web of trust. Any references for me to read more about this?

[KirillPanov]

> an on onboarding path using nodes that have higher bandwidth

i.e. Cloudflare. It works great. It is centralized.

> Any references for me to read more about this?

https://www.bittorrent.org/beps/bep_0000.html

https://zetok.github.io/tox-spec/

[survirtual]

No, not Cloudflare.

You can score member nodes and gauge bandwidth in the known network. Nodes can set their constraints accordingly and take more requests than others. Nodes that are open to edge & can handle a lot more connections can handle the bulk of onboarding requests.

This does not need to be a centralized process.