[_flux]

A long time ago I had a bad cable internet connection (high packet loss), but I also had good shell access to Uni's computers, so what I did was that I downloaded large files there and then I had encountered a tool that would be given three parameters:

- Destination IP and port

- Bytes per second

- Files to transfer

On the receiving end there was the counterpart.

The tool would send the data from the beginning to the end with given BPS and then start again from the beginning, skipping frames that it had received acknowledgements for, until all frames were acked.

Worked great! I was able to just select a bitrate that worked well and let it churn.

I don't remember the name of the tool, but I doubt there would be many use cases for it today—nor would it be very difficult to reimplement, given its brutal nature.

[rft]

About 2 years ago I had a similarly bad cable connection. I could push nearly 500mbit/s UDP through it, as advertised, but HTTPS downloads and TCP streams would only reach about 80mbit/s each. I could run multiple of them in parallel to max out the downstream on TCP alone. I also tracked it down to a low packet loss on the connection that caused TCP to reduce its rate.

I wondered whether you could add a VPN-like layer that retransmits and potentially reorders TCP packets itself without letting the actual endpoint TCP stack handle it. That way it should transparently remove the packet loss at the cost of additional complexity and higher latency.

I wish I knew about Hysteria, but I never made the connection to censored networks. It seems like it could be (ab)used for this use case.

[don-code]

Norton Ghost could do something similar, but with UDP multicast. You'd boot a room full of machines to Ghost, have them join a multicast group, multicast an OS image to them, and then have the host machine just send the image out once (from its perspective), saving a ton of bandwidth in the process. The individual machines would then rerequest any chunks that they missed.

[ale42]

Except that Ghost was using (or still uses? I switched to udpcast since ages) ACKs from the destination hosts to adapt the bandwidth dynamically. So you didn't have to give/impose the bandwidth. The funny part was when a machine in a room with 45 PCs was misbehaving (read: failing HD), basically bringing the transfer rate close to zero for the whole deployment... and good luck finding out which machine it was.

[EvanAnderson]

Exactly my experience w/ Ghost, too. I moved over to udpcast as well. (The last time I did this kind of work I simply ended up blasting the output of "ntfsclone" via udpcast. I was very happy w/ how it worked.)

[ale42]

Am almost doing the same, basically udpcasting a file that was created with partclone.ntfs -c, and piping that to partclone.ntfs -r :)

[phyzome]

Sounds very torrent-y.

[wbl]

Fountain codes would be another choice for that application