The hard part about e2e is that it only gets you anything if none of the parties involved are being surveilled after decryption. This is easily within the capabilities of any adversary worth worrying about.
e2e means you don't have to worry about transparent 3rd parties keeping secrets for you. It prevents casual abuses of your privacy and sets the bar for violating it (deserved or not)