E2EE strictly means that it is encrypted between endpoints. It makes no promises about how the keys are handled, how the data is protected at rest, or how secure either endpoint is. TLS is a version of this, but also demonstrates how useless such a version of encryption is if either endpoint is malicious.
This is one of the reasons homomorphic encryption is interesting. It should enable this kinda of "total user control" over your data, but it's highly unlikely consumer systems will implement it.