[littlecranky67]

This and so much this. I also worked with HTTP/2 (when it was still a SPDY draft) and came to the same conclusions: TCP peculiarities (most notably congestion control and its dynamic window scaling) mostly eradicate the benefits of HTTP/2. HTTP/1.1 was using 5-6 different TCP connections (which re-use OS level window scaling caches), while HTTP/2 you had to transfer all resources through a single TCP connection.

People often claim HTTP/2 to be superior due to its multiplexing capabilities, but always reason with the mental model of having true stream-based flows and leaving TCP completely out of the argument loop. But guess what, that model is just a model, indeed you have packet-based TCP flows that are abstracted away to mimic a continuous stream as a socket - and those matter.

[im3w1l]

Would it be possible to use a patched TCP stack to start up quicker?

[littlecranky67]

sure, this is basically increasing the initcwnd (the initial number of TCP segments allowed to be sent out before you received an ACK from the other end). Linux supports this through the `ip` command (look for initcwnd). This, however, requires root rights and affects the whole network device. Google submitted a patch to the linux kernel that would have allowed tweaking the initcwn from userspace for individual connections; however it got rejected and never made it upstream (for good reasons).

See, the problem is, if the initcwnd is set higher on OSes by default, you risk introducing more packet loss, and at the same time reducing everybodys network throughput - if you send more data than the network path to your destination can handle (i.e. full router buffers along the way, more data than client can receive etc.). TCPs congestion control is pretty smart about avoiding congestion or increasing load in already congested networks. So yes, you can set a higher initcwnd, but if everybody was about to increase it by a significant factor, chances are networks become slower for everyone, without anyone benefiting.

Long story short, the limitations of TCP for HTTP are known and the major motivation for QUIC (over UDP), that will allow Browser vendors (Google) and server operators (Google, but also other large players like Facebook etc.) to just "give a shit" about others and put as much pressure on the network as they want in order to bring latency down. Once Chrome rolls out with HTTP/3 support, Google couldn't care less what the linux kernel devs or IEEE network engineers have to say.

[im3w1l]

If I use wired internet, shouldn't it be the ISP's responsibility to throttle / apply backpressure? Like if I'm paying for a certain speed why shouldn't I try my best to make use of it from the start?

(I get that mobile internet requires everyone to cooperate to avoid everyone stepping on each others' toes)

[littlecranky67]

Well if a server sends more data to a requesting client that the path to that client can handle (considering current state of congestion along that path) the result is packet loss. That is, data that got send is lost, and the packets need to be re-transmitted. So if in that situation you set your initcwnd too high in an effort to improve speed, you would actually achieve the opposite effect: The data that you send gets lost and needs to be retransmitted - resulting in even slower throughput than if you had started more easy with a smaller initcwnd.

If a big player like Google or Netflix would decide to, say, double or triple its initcwnd from one day to the other, the result would probably be more congestion (more overloaded nodes in the whole network), higher packet loss, increase of traffic at peering points and the overall throughput to their users would be less than before - at the same time putting more load on the network so that even users not using Netflix or Google would be affected and suffer from lower throughputs.

You can view the whole situation also under aspects of game theory: If in a movie theater at the end of the film everybody got up immediately and ran to the exit door at the same time, the effect would be a clogged door and people would get out of the theater at a slower rate than in the current situation, where people look at the congestion rate and say "naah to many people at the exit, i sit for a while longer". TCP has built-in backoff and there is a wide range of different algorithms that solve this "try to get to the point so I can send as much as possible as quickly as possible, but if packet loss occurs go slower in order not to put even more pressure on the network". Search for TCP Reno, Tahoe, Vegas, Cubic, BIC etc. or look at the Wikipedia page of what is available: https://en.wikipedia.org/wiki/TCP_congestion_control

EDIT: regarding ISP applying backpressure: They do, in fact every router (even the ones in your home) does this automatically: You have a buffer you put packets in scheduled for delivery. If the buffer-file rate exceeds the buffer drain rate (=input data rate is higher than outgoing) the buffer runs full and packets get dropped. But this is not mitigating the problem: Every dropped packet gets retransmitted and will come again a few ms later. So if the sending side wouldn't back off (=throttle down transmission rate) and just retransmit at the same rate as before, the buffer would permanently run full, packet drops would occur again, and your connection will basically fail.