[pkkm]

You don't need to make up your own for this experiment. There's already a pretty old protocol that's far superior to TCP, but failed to get adoption because of network hardware dropping everything other than TCP and UDP. It's called SCTP.

[jeroenhd]

SCTP is fascinating because it's one of the backbone technologies that makes communication possible for most people on the planet (as the mobile network stack pretty much relies on it), yet it's effectively unsupported on almost every consumer device. If you want to use it, you're probably going to have to ship a userland implementation that needs privileges to open a raw network socket, because kernel implementations are rare (and often slow).

We could've had it as a QUIC replacement if it weren't for terrible middleboxes and IPv4 NAT screwing everyone over once again. Hell, NAT wouldn't even have been an issue had SCTP support been widespread before consumer NAT devices started being implemented.

It's a prime example of how new network protocols now have to lie and deceive to work over the internet. TLS needs to pretend to be TLS 1.2, QUIC needs to pretend to be an application on top of UDP while reimplementing SCTP and TCP, and even things like secure DNS are now piped over HTTPS just in case a shitty middlebox can't deal with raw TLS.

[tsimionescu]

While the gist of your post is spot on, I do feel it should be noted that DoH is preferred over DoT not to protect from middleboxes that don't work properly, but from middleboxes that are actively trying to outright censor encrypted DNS, but can't afford to snoop on/prevent all HTTPS traffic. It's an anti-censorship measure, not a compatibility measure.

[capitainenemo]

It's also about user privacy with ISPs as well as anti-censorship. https://www.cloudflare.com/learning/dns/dns-over-tls/

[tsimionescu]

DoT (DNS over TLS) would have been enough for privacy from your ISP, using a dedicated port. It's only when you want to protect from censorship that you need to hide the (encrypted) DNS traffic among other traffic that can't be easily blocked.

[nickburns]

Security through obscurity is not the same as actual concealment. That DoH is specced to operate over port TCP/443 makes it no more or less efficacious than DoT over TCP/853 with regard to avoiding censorship. I.e., they're both encrypted.

Many LAN operators conclude that the pragmatic impossiblility of blocking DoH is a net-negative for both network security and censorship avoidance.

[tsimionescu]

> That DoH is specced to operate over port TCP/443 makes it no more or less efficacious than DoT over TCP/853 with regard to avoiding censorship. I.e., they're both encrypted.

Of course there is. Blocking all traffic with destination port 443 is virtually impossible. Conversely, blocking port 853 is trivial, and it forces all clients to either not resolve DNS, or downgrade to un-encrypted DNS.

Of course, if DoH had not been encrypted, it wouldn't have mattered that it uses port 443. But being encrypted yet easily identifiable would have also defeated half the point.

[bewo001]

Actually, every browser supporting webrtc datachannel supports SCTP over UDP.

[tepmoc]

Diameter in mobile network is heavy user of SCTP, but from what I've read they moving away from diameter into HTTP calls for 6G.

[bennettnate5]

For communication between carriers and some communications within a given carrier's network components, yes. Base stations all still communicate with the core network over SCTP, though.

[api]

It’s too bad the original IP could not have included some kind of stronger header integrity lock to block middle boxes.

It would have forced us to adopt V6 and… my god… the network would be so much cleaner and superior in every way. Things would just have addresses, and protocol innovation at L3 would be possible.

[hylaride]

I'd like to agree with you, but IPv6 had such a classic case of suffocation by committee during its birth that I doubt it would have been pushed by such a situation. IPSec was mandated in the original IPv6 RFC, for god's sake. That alone delayed a lot of work in implementing it as crypto code needed to be integrated into kernels, which was not common in those days. That's to say nothing about the fact that IPSec is loosely defined enough that setting it up between different vendors is always an adventure - adding support to an IP stack was a big headache (I followed OpenBSD at the time they were integrating IPv6 in the early 2000s and there was a lot of hard problems around compatibility).

Header integrity was so far off of consideration during IPv4's implementation because the internet was a dozen universities and DoD sites that it was overkill (and possibly a waste of limited CPU cycles at the time).

What's far more likely to have happened is that we'd see more proxies instead of NATs (SOCKS, etc). I don't think that'd be better than NAT.

[graealex]

It's funny that you mention IPSec, since that would have made most of the application-level encryption we see today obsolete. They did have good intentions, and if it was widely accepted, it would have meant that barely any applications would have had to deal with the details of encryption, including the ever-looming possibility of doing it wrong (doing encryption right is hard!).

Now we have a slew of protocols that either implement TLS, or roll their own custom thing, or have X-over-HTTPS protocols, including SSTP and DoH.

[hylaride]

IPSec was far too complicated, loosely defined, and over-engineered to have ever been widely accepted. Any host verification would need to involve application level verification anyways to make sure the other end is who you expect. So your browser would need to verify the encrypted tunnel is in face connected to google, or whoever. There’s a reason SSL/TLS is done at the application level.

[api]

There's some history I'm not aware of here. I didn't know just how bad the V6 second system committee shitshow was.

Today V6 has been stripped down almost to what it should have been: bigger addresses, then stop. All that's left to get there is to deprecate SLAAC or make it optional.

[kstrauser]

SLAAC’s awesome, though. It’s one of the good parts we definitely want to keep.

[hylaride]

SLAAC is awesome, but DNS support in it didn't show up until much later. The result is a mish-mash of DHCPv6/SLAAC support (and android famously doesn't support DHCPv6 at all and windows only supported DHCPv6 until windows 10).

[api]

Let's say your ISP gives you a /64. Now you have to use V6 NAT... or assign a /96 internally. SLAAC won't let you do that.

That, among other things, is a problem. SLAAC is too limited.

You can use DHCPv6 but then you can't use Android because Android, and I think they're alone here, stubbornly and dogmatically refuses to implement it. I guess you could go around and statically assign V6 IPs to Android devices, or run NATv6 with SLAAC for those and DHCPv6 for everything else, but that's annoying as hell.

[nly]

It gets worse in practice. Congestional control algorithms on the internet need to play nice with TCP Cubic regardless of how good they are.

[nesarkvechnep]

As with most of the network stack, *BSD implementation is the reference implementation.

[immibis]

What we can learn from this is that there are networks other than The Internet, and even The Internet can be subdivided into parts that don't really work together.

[valorzard]

SCTP is really cool, I first found out about it because it’s the basis for WebRTC data channels. It’s basically reliable UDP, but you can turn off the reliability if you want. Makes me wonder why QUIC exists when SCTP does…

[15155]

> why QUIC exists when SCTP does

Because QUIC uses UDP, which is supported by most/all intermediate routing equipment.

[nly]

The whole point of UDP is to allow alternative protocols to be implemented on top.

SCTPs mistake was it wasn't implemented as a userland library on top of UDP to begin with.

[noam_k]

Is this a real issue? SCTP runs over IP, so unless your talking about firewalls and such, the support should be there.

Edit: a quick search showed that NAT traversal is an issue (of course!)

[Karrot_Kream]

Yes this is called protocol ossification [1] or ossification for short. Other transport layer protocol rollouts have been stymied by ossification such as MPTCP. QUIC specifically went with UDP to prevent ossification yet if you hang out in networking forums you'll still find netops who want to block QUIC if they can.

[1]: https://en.m.wikipedia.org/wiki/Protocol_ossification

[crims0n]

Because from an enterprise security perspective, it breaks a lot of tools. You can’t decrypt, IDS/IPS signatures don’t work, and you lose visibility to what is going on in your network.

[Karrot_Kream]

Yes I know why netops want to block QUIC but that just shows the tension between the folks who want to build new functionality and the folks who are in charge of enterprise security. I get it, I've held SRE-like roles in the past myself. When you're in charge of security and maintenance, you have no positive incentive to allow innovation. New functionality gives you nothing. You never get called into a meeting and congratulated for new functionality you help unlock. You only get called in if something goes wrong, and so you have every incentive to monitor, lock down, and steer traffic as best as you can so things don't go wrong on your watch.

IMO it's a structural problem that blocks a lot of innovation. The same thing happens when a popular open source project that's author led switches to an external maintainer. When the incentives to block innovation are stronger than the incentives to allow it, you get ossification.

[Dylan16807]

Wrapping everything in UDP breaks the same tools but it's more obnoxious for everyone involved.

[wahern]

> Other transport layer protocol rollouts have been stymied by ossification such as MPTCP

AFAIU, Apple has flexed their muscle to improve MPTCP support on networks. I've never seen numbers, though, regarding success and usage rates. Google has published alot of data for QUIC. It would be nice to be able compare QUIC and MPTCP. (Maybe the data is out there?) I wouldn't presume MPTCP is less well supported by networks than QUIC. For one thing, it mostly looks like vanilla TCP to routers, including wrt NAT. And while I'd assume SCTP is definitely more problematic, it might not be as bad as we think, at least relative to QUIC and MPTCP.

I suspect the real thing holding back MPTCP is kernel support. QUIC is, for now, handled purely in user land, whereas MPTCP requires kernel support if you don't want to break application process security models (i.e. grant raw socket access). Mature MPTCP support in the Linux kernel has only been around for a few years, and I don't know if Windows even supports it, yet.

[Karrot_Kream]

It would be nice to generate some sort of report card here. Maybe I should try.

[immibis]

Every home user is behind a NAT. While you can send any protocol between datacenter servers, IPv4 home users are stuck with TCP or UDP.

[evrimoztamur]

Hole punching is perhaps why UDP is de facto?

[saurik]

WebRTC runs SCTP over DTLS over UDP.

[lttlrck]

That's not the reason, SCTP over UDP was already standardized

[Imustaskforhelp]

I am now genuinely wondering

Maybe its me being stupid but why don't we use quic always instead of tcp?

I think it has to do with something that I read that tcp can do upto 1000 connections simultaneously no worries and they won't interfere with each other's bandwidth / impact each other , but udp does make it possible for one service being very high to impact other.

There was this latest test by anton putra with udp vs tcp and the difference was IIRC negligible. Someone said that he should probably use udp in kernel mode to essentially get insane performance I amnot sure

[WorldMaker]

> Maybe its me being stupid but why don't we use quic always instead of tcp?

A big reason is because QUIC is a lot younger than TCP and it will take a while for all the use cases of TCP to decide (if they are actively maintained and looking at possible upgrades) if QUIC is a good option worth testing.

QUIC's rollout so far hasn't been entirely without bugs/controversies/quirks/obstacles/challenges. You still see a lot more HTTP/2 than HTTP/3 connections in the current wild and that doesn't seem to be changing near as fast as major providers upgraded HTTP/1.x to HTTP/2. There's still a bunch of languages and contemporary OSes without strong QUIC support. (Just the other day on HN was a binding for Erlang to msquic, IIRC, for a first pass at QUIC support in that language.)

Some point soon QUIC might start feeling as rock solid as TCP, but today TCP is (decades of) rock solid and QUIC is still a lot new and a little quirky.

[walth]

Safari on IOS still has a ton lingering HTTP/3 / QUIC bugs.

I think it is to the point that if your user base doesn't warrant it, (i.e. you are targeting well connected devices with minimal latency/packetloss) it's not even worth turning HTTP/3 on

[Imustaskforhelp]

so quic just lacks the decades of experience but is a better protocol than tcp overall ?

That is kind of nice to know actually. The support will come considering its built on top of UDP. You just need people pushing and google is already pushing it hard .

The main problem is quic's support in languages. But support will come.So after reading this comment of yours , I am pretty optimistic about quic overall

[WorldMaker]

Not necessarily a "better protocol overall", it still seems too early to tell. I think we're still in the "Find Out" stages because of the rollout issues and the lack of language support and lack of diversity of implementations.

(On the diversity of implementations front: So far we've got Google's somewhat proprietary implementation, Apple's kind of broken entirely proprietary implementation, and Microsoft's surprisingly robust and [also surprisingly to some] entirely open source C implementation. General language support would be even worse without msquic and the number of languages binding to it. Microsoft seems to be doing a lot more for faster/stronger/better QUIC adoption than Google today, which I know just writing that sentence will surprise a lot of people.)

There will be trade-offs to be found with TCP. For instance, a lot of discussion elsewhere in these threads is on the overbearing/complicated/nuanced congestion control of TCP, but that's as much a feature as a bug and when TCP congestion control works well it quietly does the internet a wealth of good. QUIC congestion control is much more a binary: dropped packets or not. That's a good thing as an application author, especially if you are expecting the default case to be "not" on dropped packets, but it doesn't give the infrastructure a lot of options and when pressure happens and those "allow UDP packet" switches are turned off and most of your packets as an application developer are dropped how do you expect to route around that? At least for now most of the webservers built to support HTTP/3 still fallback to HTTP/2 on request, go back to the known working congestion control of TCP that most of the internet and especially the web was built on top of.

I'm not a pessimist on QUIC, I think it has great potential. I also am not an optimist about it 100% replacing TCP in our near future, and maybe not even in our lifetime. As an application developer, it will be a great tool to have in your toolbelt as a "third" compromise option between TCP and UDP, but deciding between TCP and QUIC is probably going to be an application-by-application pros/cons list debate, at least in the short term and I think probably in the long term too.

[signa11]

> It’s basically reliable UDP, ...

more importantly though, it transmits multiple independent streams of message chunks in parallel.

similarity with UDP ends at message oriented nature of the protocol. closest equivalent for TCP would be MPTCP I suppose ?

[0x457]

Because pure SCTP can't survive outside your LAN, thanks to everything in-between you and your destination. Why not use SCTP on top of UDP? Well, because one of the main benefits of QUIC is TLS being at its core.

SCTP you're talking about runs on top of DTLS on top of UDP. DTLS has issues on its own, but even if it didn't it wouldn't beat QUIC in TTFB.

[bdd8f1df777b]

Others have mentioned protocol ossification which is indeed the primary reason. A secondary reason is that QUIC fuses TLS so its latency is further reduced by one RTT. For high latency networks, the difference is palpable.

[ianburrell]

SCTP can run over UDP.

QUIC is supposed to be faster than SCTP by combining layers and eliminate round trips. Also, QUIC is a stream protocol like TCP. SCTP makes messages explicit. Both have multiplexing which is why seem different.

[tepmoc]

https://news.ycombinator.com/item?id=41488416

[immibis]

It's actually universal within a certain niche. I think phone networks are doing just about everything over SCTP internally. When SS7 links get replaced they get replaced with something that uses SCTP. Not sure of the details because I don't work there.

Related: there's a parallel Internet with a different root of number allocation called GRX/IPX (GPRS Roaming Exchange/Internetwork Packet Exchange)

[AnotherGoodName]

IPX is another that was very common just 20years ago. Many old games only support ipx networking and you need to run an ipx over tcp emulator to play them multiplayer nowadays.

[dakra137]

TCP would be fine if it had the concept of message, in addition to stream. The sender sends a message. It flows to the recipient. The recipient program can specify that a receive receives the entire message, no more and no less, as long as the application's target input buffer is large enough.

SCTP does this.

A shim on top of TCP socket receive could also do this also, as long as there is a convention to prefix each message with a length field, say 16 bits, with the MSB indicating that the message is incomplete and is continued in the next length delimited segment.

[nubinetwork]

I thought the point was that they wanted to use something that didn't exist yet (either in RL use or in RFC form)...