On the other hand, it's possible to reach https://coffeeshopwifi.com and (with a cloudfront certificate error) https://neverssl.com which makes me wonder if something in whatever I'm using is trying to upgrade to HTTPS when I fail to reach them.
It's actually a moderate pain to set up something that doesn't answer on port 443 unless you want to host it yourself. I tried with http://wifi.help, but it's at Cloudflare and I don't know of a way to block port 443 without paying for their WAF.
I think you're misunderstanding the GP. The HTTPS connection will not be redirected when you are behind a captive portal, you'll receive an invalid certificate error (when the captive portal tries to serve itself at that address) or the website will simply not respond. Only if the HTTPS response is cached in your browser would the redirection work behind a captive portal.