[1vuio0pswjnm7]

"By making sites request this information rather than simply sending it like the User-Agent header currently does..."

This is also true with respect to SNI which leaks the domain name in clear text on the wire. The popular browsers send it even when it is not required.

The forward proxy configuration I wrote distinguishes the sites (CDNs) that actually need SNI and the proxy only sends it when required. The majority of websites submitted to HN do not need it. I also require TLSv1.3 and strip out unecessary headers. It all works flawlessly with very few exceptions.

We could argue that sending so much unecessary information as popular browsers do when technically it is not necessary for the user is user hostile. It is one-sided. "Tech" companies and others interested in online advertising have been using this data to their advantage for decades.

[billyhoffman]

How would this work?

SNI is sent by the client in the initial part of the TLS handshake. If you don't send it, the server sends the wrong/bad cert. The client could retry the handshake using SNI to get the correct cert but:

- This adds an extra RTT, on the critical path of getting the base HTML, hurting performance.

- A MITM could send back an invalid cert, causing the browser to retry with SNI, leaking it anyway (since we aren't talking about TLS 1.3 and an encrypted SNI).

I suppose the client could maintain a list of sites that don't need SNI, like the HSTS preload list, but that seems like a ton of overhead to avoid sending unneeded SNI, especially when most DNS is unencrypted and would leak the hostname just like SNI anyways.

[1vuio0pswjnm7]

"I suppose the client could maintain a list of sites that don't need SNI."

That list would be much larger than the list of sites that do require SNI.

Generally, I can determine whether SNI is required by IP address, i.e., whether it belongs to a CDN that requires SNI. Popular CDNs like AWS publish lists of their public IPs. I use TLSv1.3 plus ESNI with Cloudflare but they are currently the only CDN that supports it. Experimental but works great, IME.

The proxy maintains the list not the browser. The proxy is designed for this and can easily hold lists of 10s of 1000s of domains in memory. That's more domains than I visit in one day, week, month or year.

Is it not a question of whether this is possible. "How would this work". I have already implemented it. It works. It is not difficult to set up.

Why this works for me and would unlikely work for others.

I am not a heavy user of popular browsers, I "live on the command line". Installing a custom root certificate with appropriate SANs to suppress browser warnings is a nusiance that would likely dissuade others since they are heavy users of those programs. However I generally do not use those browsers to retrieve content from the web.

[billyhoffman]

Ahhh. I see, you are default-no-SNI, and whitelist those that do.

If your threat model is such that you absolutely positively cannot leak the signal of what domain names you want to make HTTPS connections to, then I suppose this is an approach that can be used. But if you believe that is your threat model, I imagine you have bigger issues to protect against. As you say, it's unlikely to work for others.

[1vuio0pswjnm7]

No "threat model" here, just a dissatisfaction with so-called "modern" browsers and TLS extensions that disproportionally benefit hosting companies over users (privacy in this case). Plus I genuinely prefer commandline TCP clients and text-only browser to read HTML for most web use. I like the speed, reliability and more uniform presentation I get across all web sites. I like text. Big browsers that do everything under the sun written by people working for "tech" companies funded by advertising are not interesting to me. In fact, I find them annoying.

Some folks write "browser extensions" to control graphical browsers to their liking. I generally do not use graphical javascript-enabled browsers; I prefer to use a different program, a proxy, to control the browser. It works with both graphical browsers and text-only ones.

[marcosdumay]

I don't think you can ever determine that a site doesn't need SNI using HTTP alone. All you can have is that it doesn't or you don't know.

[1vuio0pswjnm7]

I do not use "HTTP alone", I use DNS, more specifically IP address. I generate lists. The lists are largely based on the hosting provider and created automatically, but I also edit them manually when necessary, which is the exception not the rule. Most sites requiring SNI that are submitted to HN all use the same CDNs: AWS and Cloudflare. The SNI list is dominated by sites hosted on AWS. The ESNI list is all sites hosted on Cloudlfare.

When I first started developing this workaround I thought I would be manually editing the SNI list constantly for "all those random sites that use SNI". This has not been the case. For the sites submitted to HN, use of SNI is mostly a CDN phenomena.

The important point here is that I do not send SNI by default. The default is privacy-by-design: no SNI. If I encounter a site that fails because it needs SNI, I add it to the list. The failure is caught by the proxy (the proxy verifies certificates, I do not rely on the browser), the SSL error is visible in the logs, and the error page the browser receives is a custom one I created myself that tells me where in the configuration the failure occured. I can test whether a site requires SNI very quickly.

Popular browsers cannot do this, we know that. If they could, I would not be coming up with workarounds. They routinely send more data than is needed, including SNI. That is the point of the original comment.

[1vuio0pswjnm7]

s/phenomena/phenomenon/