[toast0]

I wanted to like it, and Apple included it in iOS, but supporting it on real servers was going to be too hard...

When I was deployed on FreeBSD with no load balancers, there weren't recent patches. And even if there were, I'd need to do some serious work to avoid advertising the private network ips as alternates...

When I was on Linux behind a load balancer, it's too complex to get the streams to the right place. And the load balancer doesn't want to do it anyway.

Processing two streams together involves a lot of complexity in a high throughput code path. It's a lot of risk, and you've got to reboot for changes.

And then you do all that work and it only benefits iOS users, who tend to be on better networks anyway.

[bbss]

Apple also contributed[1] MPTCP support to Envoy Proxy.

[1]https://github.com/envoyproxy/envoy/pull/18780

[DEADMINCE]

> iOS users, who tend to be on better networks anyway.

I don't think there is any basis to claim that.

[wil421]

> A U.S. analysis of Wi-Fi and mobile Internet usage across unique smartphones on the iOS and Android platforms reveals that 71 percent of all unique iPhones used both mobile and Wi-Fi networks to connect to the Internet, while only 32 percent of unique Android mobile phones used both types of connections. A further analysis of this pattern of behavior in the U.K. shows consistent results, as 87 percent of unique iPhones used both mobile and Wi-Fi networks for web access compared to a lower 57 percent of Android phones.

https://www.comscore.com/lat/Prensa-y-Eventos/Infographics/i...

[DEADMINCE]

Since wi-fi networks tend to be higher quality than cell networks, what you provided works against the point I responded to.

[Longhanks]

Someone paying for a premium phone is probably also inclined to pay for a premium mobile network.

[DEADMINCE]

Not all iPhones are 'premium' phones, and there are not really 'premium' cell phone networks in the US. Or anywhere.

[Longhanks]

Lol, have you ever been to Europe? iPhones are definitely considered premium and there definitely are networks that are more expensive but offer better reception. In Germany, that would be Telekom, in Switzerland, it's Swisscom.

[DEADMINCE]

Yes used to live in Germany. I was talking mainly about the US though.

iPhone isn't always 'premium', since they have their version of cheap phones as well. Point is cell network service quality is independent from phone quality.

[proneb1rd]

A lot of this is bought in instalments isn’t it?

[RulerOf]

It sounds like this would have taken off if it were added to various managed cloud load balancers based on what you're saying.

The only question I have is if it opens up a different can of worms even if you've got a magic box terminating layer 7 for you or not. Never dug deep enough into mptcp myself to know.

[toast0]

I think it's a no brainer if it's no effort or small effort (set a socket option on the client, somehow)... but it's a big effort to support it in a large load balancing situation.

If you balance your load balancers with ECMP, I don't know if you can get two client streams to the same mptcp terminating place.

If you've optimized the heck out of your tcp flows, this throws a wrench in there, because the second stream is likely to get hashed into a different nic queue, and then you have communication between cpus to move forward on the logical stream.

It would have been really handy though, and solve real issues with real users.

Edit to add: it could also solve some issues on private networking / interserver networking I saw... although the contention would be a much bigger problem on higher bandwidth streams. On networks with link aggregation, while there are many paths from one host to another, usually path selection is by hashing the connection 5-tuple {src ip, dst ip, protocol, src port, dst port} so a long running tcp connection remains on the same path for the duration, if a path segment has high loss/corruption or is congested, MPTCP could help if you had an extra connection that hit a different path. Otherwise, you need to find the segment and get network operations to fix it; it's not easy to figure that out (i had to write a tool to sample and find port combinations with trouble and then a patch for mtr to run a trace with fixed ports) and then you still need to reconnect your affected tcp sockets unless you can get a quick response from net ops (sometimes they can check error stats once the right devices are pointed out to them, and then replacing a cable/fiber often helps, or disconnecting it during investigation can help the traffic flow across the redundant links)

[vitus]

> If you balance your load balancers with ECMP, I don't know if you can get two client streams to the same mptcp terminating place.

At Google, we do something similar with QUIC and connection migration. Our mechanism for ensuring these hit the same backend is Maglev [0], where we use the QUIC connection ID for hashing purposes in software. (Our routers still mostly use ECMP based on the 5-tuple, so being able to consistently hash to the same backend across multiple LB instances is crucial.)

> if a path segment has high loss/corruption or is congested, MPTCP could help if you had an extra connection that hit a different path.

Incidentally, we also have a family of internal mechanisms that do this, although we don't rely on MPTCP. (We instead twiddle some other bits in the packet that we make sure our routers use for hashing, at least for RPCs between prod machines.) This inspired some of the connection migration work in our QUIC implementation [1], wherein we can migrate to a different ephemeral port if we detect issues with the current path. This works shockingly often for routing around network problems.

[0] https://research.google/pubs/maglev-a-fast-and-reliable-soft...

[1] https://github.com/google/quiche/blob/main/quiche/quic/core/...