I use ProtonMail + Thunderbird, it does require a client to handle the decryption/encryption to the servers. It was open sourced a while ago which is nice, although admittedly it'd be nice to have it just work.
That's cool, I wasn't aware Protonmail had this feature. I use Tutanota and their client is required to access their service, which does work great as is.