If anyone's interested, I wrote a guide a while ago on adding these headers via Cloudflare Workers, which can be helpful if you're hosting a static site on S3, GitHub Pages, etc. where you can't add these headers directly:
https://jarv.is/notes/security-headers-cloudflare-workers/