[Veserv]

That is exactly the type of poor design that I was saying should be rectified.

You can already configure your initial congestion window, and if you are connecting to a system expecting the use of PQ encryption, you should set your initial congestion window to be large enough for the certificate; doing otherwise is height of incompetence and should be fixed.

You could also use better protocols like QUIC which has a independently flow controlled crypto stream and you can avoid amplification attacks by pre-sending adequate amounts of data to stop amplification prevention from activating.

And I fail to see how going from 4 KB of certificate chain to 160 KB of certificate chain poses a serious storage or transmission problem. You can fit literal millions into RAM on reasonable servers. You can fit literal billions into storage on reasonable servers. Sure, if you exactly right-sized your CT servers you might need to upgrade them, but the absolute amount of resources you need for this is miniscule.

[nickf]

Your failure to see the problem doesn’t mean it doesn’t exist. 40x the size might not really be an issue for the hypothetical server you’ve suggested - but that isn’t the reality for the world. Many devices do HTTPS and TLS. Not to mention the issue is more with the clients. CT logs would get a lot harder to run (and they’re already not so easy).

[raggi]

> You can already configure your initial congestion window, and if you are connecting to a system expecting the use of PQ encryption, you should set your initial congestion window to be large enough for the certificate; doing otherwise is height of incompetence and should be fixed.

The aggressive tone is no defense against practical problems such as the poor scalability of such a solution.

> You could also use better protocols like QUIC which has a independently flow controlled crypto stream and you can avoid amplification attacks by pre-sending adequate amounts of data to stop amplification prevention from activating.

Not before key exchange it doesn't. There's no magic bullet here.

A refresher on the state of TFO and QUIC PMTU might be worthwhile here before jumping this far ahead.

[Veserv]

You have asserted without evidence that the increased certificate chain size is the primary scaling bottleneck. I assert that the bottleneck is most likely due to accidental complexity elsewhere on the argument that claimed problems look to be far in excess of the essential complexity.

> Not before key exchange it doesn't. There's no magic bullet here.

I was incorrect. Rereading the QUIC standard I see that they do not flow control the CRYPTO packet number space/stream. I thought they did because it is so easy to do that I did it as a afterthought. Truly another example of fundamental design errors introducing accidental complexity that should be fixed instead of papered over.

[ekr____]

Can you elaborate a bit more about what you think the unnecessary complexity here?

A basic source of concern here is whether it's safe for the server to use an initial congestion window large enough to handle the entire PQ certificate chain without having an unacceptable risk of congestion collapse or other negative consequences. This is a fairly complicated question of network dynamics and the interaction of a bunch of different potentially machines sharing the same network resources, and is largely independent of the network protocol in use (QUIC versus TCP). It's possible that IW20 (or whatever) is fine, but it may well may not be.

There are two secondary issues: 1. Whether the certificate chain is consuming an unacceptable fraction of total bandwidth. I agree that this is less likely for many network flows, but as noted above, there are some flows where it is a large fraction of the total.

2. Potential additional latency introduced by packet loss and the necessary round trip. Every additional packet increases the chance of one of them being lost and you need the entire certificate chain.

It seems you disagree about the importance of these issues, which is an understandable position, but where you're losing me is that you seem to be attributing this to the design of the protocols we're using. Can you explain further how you think (for instance) QUIC could be different that would ameliorate these issues?

[Veserv]

For point 1, as I noted here [1], total bandwidth and resources are dominated by large flows. Endpoints are powerful enough to handle these large flows. The primary problems would lie with poor intervening networks and setup overhead.

For point 2, that is a valid concern of any case where you have just plain old more data. This dovetails into my actual point.

The problem of going from a 4 KB certificate chain to a 16 KB certificate chain, 160 KB certificate chain, or any arbitrary sized certificate chain should be equivalent to the problem of "server sends N byte response like normal". To simplify the problem a little it is just: the client sends a R-byte request message, the server responds with the Q-byte response message (which happens to be a certificate chain), the client sends the P-byte actual request, the server responds with a K-byte response message. So, at the risk of over-simplification, the problem should only be marginally harder than any generic "time to Q + K bytes".

Of course, if you previously had a 4 KB actual response and a 4 KB certificate chain and now it is a 160 KB certificate chain, you are going from "time to 8 KB" to "time to 164 KB". That is the essential complexity to the problem. But as I noted in my response to your point 1, the amount of server and client resources actually being expended on "small" requests is small with only poor networks where you are now consuming significantly increased bandwidth being a problem.

This then leads into the question of why "time to 8 KB" versus "time to 164 KB" is viewed as such a dramatic difference. This is a artifact of poor protocol design.

From a network perspective, the things that mostly matter are end-to-end bandwidth, end-to-end latency, endpoint receive buffer size, and per-hop bandwidth/buffering. You have a transport channel with unknown, dynamic bandwidth and unknown latency and your protocol attempts to discover the true transport channel parameters. Furthermore, excessive usage degrades overall network performance, so you want to avoid over-saturating the network during your discovery. In a ideal world, you would infer the transport parameters of every hop along your path to determine your holistic end-to-end transport channel parameters. This is problematic due to paths shifting or just plain dynamic throttling, so you will probably only limit yourself to "client to common bottleneck (e.g. your router) path" and "common bottleneck to server path". The "client to common bottleneck path" is likely client controlled and can be safely divided and allocated by the client. The "common bottleneck to server path" is not efficiently controllable by the client so requires safe discovery/inference.

The "initial congestion window" is a initial bandwidth-delay product to avoid over-saturating the network. This does not directly map to the transport parameters that matter. What you actually want is a initial safe "end-to-end bandwidth" which you refine via the discovery process. The latency of your roundtrip then only matters if the endpoint receive buffer size is too small and only effects how quickly you can refine/increase the computed safe "end-to-end" bandwidth.

Under the assumption that a 16 KB "initial congestion window" is fine and we assume the default RTT is ~100 ms (a somewhat reasonable assumption for geographically distributed servers who want to minimize latency) then that is actually a initial safe "end-to-end bandwidth" assumption of (16 KB / 0.1 s * 8 B/b) = ~1.3 Mb/s. Assuming the client advertises a receive buffer large enough for the entire certificate chain (which it absolutely should) and there are no packet losses, the client would get the entire certificate chain in ~(1 s + RTT) in the worst case. Note how that has only a minor dependency on the end-to-end latency. Of course it could get the data sooner if the bandwidth gets refined to a higher number, and a lower RTT gives more opportunities to get refined to a higher number, but that bounds our worst case (assuming no packet loss) to something that is not really that bad especially for the poor network throughput that we are assuming.

This then makes it obvious how to improve this scheme by choosing better initial estimates of "end-to-end" bandwidth or actively communicating that information back and forth. The "client to common bottleneck path" can be "controlled" by the client, so it can allocate bandwidth amongst all of its connections and it can set aside bandwidth on that leg for receiving. This allows higher initial "end-to-end" bandwidth assumptions that can be safely clipped when the client realizes it is in bad network conditions such as plane wifi. If the server determines "I have set aside N b/s to the 'internet' for this client" and the client determines "I have set aside M b/s from the 'internet' for this server" then your only problem is if there is a bottleneck in the broader backbone connections between the server and client. You would almost certainly be able to support better initial bandwidth assumptions or at least faster convergence after first RTT if you communicated that information both ways. This is just a example of what and how things could be improved with fairly minimal changes.

And this all assumes that we are even trying to tackle this fairly fundamental root issue rather than what are probably heaps of other forms of accidental complexity like middleboxes just giving up if the certificates are too large or whatever else nonsense there is which is what I am pretty sure is the real impetus by why they want the networking equivalent of wanting the 737-MAX to handle the same as a 737.

[1] https://news.ycombinator.com/item?id=47210252

[ekr____]

A few points of technical clarification might help here.

1. The reason for a relatively small initial congestion window (cwnd) is to avoid situations where a lot of connections start up and collectively exceed the capacity of the network, causing congestion collapse. Instead, you start slow and then gradually ramp up, as you learn the available capacity. Slow start started in TCP but it's in QUIC too. Initial windows actually used to be a lot smaller and TCP only moved up to its current 10 packet initial window (IW10) after a bunch of experimentation that determined it was safe.

2. The congestion window is actually a property of the sender, not the receiver. The receiver advertises the size of their flow control window, but that's about the buffer, not the sending rate (see section 7 of RFC 9002 for the discussion of slow start in QUIC). So in this case, the server controls cwnd, no matter what the client advertises (though the server isn't allowed to exceed the client's advertised flow control window).

3. QUIC and TCP behave fairly similarly in terms of the broad strokes of rate control. As I noted above, QUIC also uses Slow Start. The amplification limit you mention is a separate limit from initial cwnd, which is intended to avoid blind amplification attacks, because, unlike TCP, QUIC servers can start sending data immediately upon receiving the first packet, so you don't know that the IP address wasn't forged. However, even if the peer's IP is authenticated, that doesn't mean it's safe to use an arbitarily large initial cwnd.