The end-to-end encryption is only between protonmail addresses, in practice when you email people with gmail/hotmail/yahoo etc. it doesn't matter if protonmail can't read the e-mail, the other party can. (Their solution for that is to send an e-mail that contains a password-protected link with the actual message [0], I find this procedure inconvenient.)
Gmail could be as secure as Protonmail by using PGP yourself [1]. And then you can keep your desktop mail client.
I wanted to touch this part: "And then you can keep your desktop mail client." . Actually you can use your desktop mail client with ProtonMail too. All you need to do, is install ProtonMail Bridge. It's in closed Beta right now, but it works pretty well (i'm using with Thunderbird).