The method I described doesn't require you to use a S3 static website. We don't have that option enabled, so the bucket name isn't mysite.com, it's "mysite-html", and the S3 website hosting is not enabled.
CloudFront has its own IAM user that is permitted to access the contents of the bucket, which means that it has to use the API. I don't think it's even possible to access the S3 API without using HTTPS. Therefore I think it is highly unlikely that the connection is unencrypted.
Awesome! I'm new to AWS, and when I tried to setup a static website using cloudfront + s3, I was only aware of the static website option on s3. This is what the docs uses, and it led me to the wrong path it seems! Thanks