[kazinator]

Car-brutal: new congestion control algorithm!

Pass everyone fast in the left lane, get to within 50 yards of the target exit (where they are are all also going) and then hit your brakes to cut back in, sending a peristaltic wave of stoppage backward in the fast lane.

In networking, packets can disappear due to hardware (mainly on wireless) or due to being deliberately dropped due to congestion. You can't tell these two scenarios apart. It makes sense to try harder in the former scenario, but trying harder in the latter scenario makes the congestion worse for the entire network, while eking out a minor improvement for that connection. If every node does it, the entire network will be far worse off, so it is counterproductive.

TCP congestion control depends on cooperation: that every node complies with the RFC requirements to implement all the necessary algorithms. That unfortunately leaves room for idiots and dickheads, hence from time to time we may see work like this. "Hey, if I blatantly break the RFC, I get faster transfers. Holy shit, how come nobody knows about this?"

[twic]

> Pass everyone fast in the left lane, get to within 50 yards of the target exit (where they are are all also going) and then hit your brakes to cut back in, sending a peristaltic wave of stoppage backward in the fast lane.

Interesting. Looking into it.

[JohnBooty]

Reminds me of those "alternative" bittorrent clients that did neat things by, basically, behaving in selfish ways.

The main example behavior I remember was that these clients would sort of cheat by downloading the file chunks in sequential order (so they could start playing the media sooner) rather than in random order (which is a big part of what makes bittorrent a pretty effective protocol/algorithm)

[FeepingCreature]

If lots of people are watching the file sequentially, then sequential order is close to optimal. Consider a Bittorrent-based lifestream. Clients should focus on getting and distributing the chunks that everyone wants.

[JohnBooty]

Bittorrent does livestreams? Did not know that.

But for "normal" Bittorrent usage, oh hell naaaaah.

Where you are trying to actually obtain a complete file, that can be a disaster. What happens is the chunks at the end of the file will be much rarer than the chunks at the beginning. Getting the blocks sequentially offers a small convenience benefit (you can start viewing a video while the torrent is still downloading) but if significant numbers of people did that it would harm every other use case including your own ability to actually download the entire file.

If you have a surplus of seeders this isn't a problem, but for sparsely-seeded torrents that is not good.

[zamadatix]

This implies the reason most sparsely seeded torrents have chunk availability is because the transient leechers just happen to arrive consistently in time to hand off to one another. I think reality is sparse torrents tend to be kept alive by a few seeders slowly kicking out data (most likely because they are seeding a ton of other stuff too). Sequential or not, if you and another guy are planning on dumping a few seconds after finishing the torrent then it doesn't really matter if that is a chunk near the end or a random chunk - either way you're both trying to squeeze the last chunk out of the seeder.

Densely seeded torrents may be slightly different and play out more like you describe, but it doesn't really matter as much there as the first parts can be sourced from those watching sequentially and the end parts left to those that actually seed after getting a full file.

[JohnBooty]

    This implies the reason most sparsely seeded torrents 
    have chunk availability is because the transient leechers 
    just happen to arrive consistently in time to hand off to 
    one another.

No. I'm not sure where you got that idea.

This is a pathological and unrealistic example but please try to extrapolate to more realistic scenarios. Imagine a file with three pieces. 200 torrent users have zero pieces, 100 torrent users have the first piece, 50 have the first and second piece, and "Joe" is the sole seeder that has all three.

Obviously, assuming Joe doesn't have godlike bandwidth, there are 350 users who want that third piece and only a single Joe and there is going to be contention. In this pathological situation things will work itself out as soon as supply of the third piece increases, assuming people don't kill their clients as soon as they hit 100%, but things will be slower than they need to be at first.

What I think you're trying to point out is that there are situations where sequential torrent downloads aren't harmful to download speed or overall availability. Yes, that is true. If seeder bandwidth for a given piece is more than sufficient for leecher demand for that piece then... yes, no harm done.

As another user posted, http://bittorrent.org/bittorrentecon.pdf section 2.4.2 "Rarest first"

You're free to disagree with the design of bittorrent, obviously.

[FeepingCreature]

If people download the complete file eventually, it should average out. And if people view half the file and then drop out, then the earlier chunks being more available is optimal.

[JohnBooty]

    If people download the complete file eventually, it should average out

Yes. As supply of the rarer chunks increases, the problems go away.

Notice the implicit fact up there. The uneven distribution of chunks causes problems.

    then the earlier chunks being more available is optimal.

For most use cases, no, the first chunks of the file are not more valuable. Having the first 99 chunks of a Linux .ISO is not valuable at all if you don't have the 100th chunk. The chunks are equally valuable and the chunks are useless unless you have them all.

There are some (or were?) torrent clients that let you start watching incomplete downloads while they're still downloading. This is a dubious use case. I do not notice many people using these nonstandard clients.

Also even for some kind of livestreaming scenario, what are even the "first" chunks? The start of the stream? The stream at the current time? It's probably a moot point anyway because it seems like Bittorrent livestreaming hasn't exactly taken the world by storm but maybe there are uses of it with which I'm not familiar: https://www.google.com/search?client=firefox-b-1-d&q=bittorr...

[FeepingCreature]

> For most use cases, no, the first chunks of the file are not more valuable. Having the first 99 chunks of a Linux .ISO is not valuable at all if you don't have the 100th chunk.

Sure but we just established that people are dropping out in mid-watch. For most use cases, people aren't interested in downloading the first chunks first; those cases where they are, and where it doesn't eventually average out, are exactly the cases where the earlier chunks are more important.

[nix0n]

If multiple users download in sequential order, wouldn't that actually make it pretty likely that the previous downloader would have the chunk that the next downloader is looking for?

Or is there a specific advantage to random order?

[TonyTrapp]

With clients constantly appearing and disappearing, the probability that you will be able to finish the download will be much higher if everyone has a random collection of file parts. Imagine the original seeder goes away after one day. Everyone is stuck at 99% because they downloaded in sequential order. Nobody has the last missing part. This would be much less likely if the clients had downloaded random parts instead, because every part would be roughly equally likely to be available in the end.

[the8472]

see http://bittorrent.org/bittorrentecon.pdf section 2.4.2

[nix0n]

Thanks! That answers my question.

For anyone who doesn't feel like clicking on a PDF, the section title is "Rarest First".

[fmajid]

That's pretty much what Google is doing with QUIC and BBR, aka HTTP/3. Tragedy of the Commons in the making.

[progbits]

I don't think you even get faster transfers, at least beyond some short term gain. You will overload some router on the path and just increase packetloss for everyone without improving your goodput.

Maybe my intuition is wrong but there are no graphs and real world measurements.

[ta1243]

In some cases you could. I had an issue last week with my 1G leased circuit with a 160ms rtt

That circuit had a 1% packet loss on it due to a dodgy SFP. This devastated cubic, with peak TCP transfer dropping from 150mbit (we police it to about that) to less than 1mbit. BBR was better but still down a fair bit.

Using this algorithm I presume it would have continued at about 140-150mbit.

I don't really do TCP so haven't looked too closely at different algorithms in different latency/loss (burst or constant) conditions, but I can see where this type of algorithm could be useful.

[xiconfjs]

You could check if you have SACKs [1] enabled - these can help by a lot on lossy uplinks using tcp.

[1] https://packetlife.net/blog/2010/jun/17/tcp-selective-acknow...

[progbits]

OK I suppose that is what GP meant with the first category - if there is no congestion but your physical layer is dodgy this can indeed help.

But if many people start using it this will fail on congestion (and even if there is little congestion to begin with, this will amplify it so there is sure to be some).

[kazinator]

If many people are driving aggressively and cutting in, everyone is made late; but those who engage in the behavior arrive earlier than those who refrain.

A router that is dropping packets due to being congested is still forwarding some packets. If you wallop that router with your own packets, that's your best chance of getting more of your packets to be forwarded, at the expense of someone else's being dropped. (Assuming the router has no countermeasure against that behavior.)

If we take a simple leaky bucket model: the router drops the packet because it has no resources in that moment. A moment later, space opens up because of transmitted/cleared data, so when a packet is received in that later moment, is not dropped. There is likely a chain of leaky buckets: multiple points in the router traversal where packets can be dropped due to resource issues: dropping could be on the receive side or transmit. Wherever there is a queue whose length is capped or a memory allocation operation.

If you wallop that machine with your packets, you increase the chances that when those moments come when resources are available that allow a packet to be retained rather than leaked onto the floor, that opportunity goes to one of your packets rather than someone else's.

[the8472]

That's only true if the router drops randomly rather than hashing into buckets by host

[zamadatix]

Yeah, you'd probably get a better rate in this case. In this kind of non-congestion related path loss you'd be even better off just doing something like FEC instead though.