Matrix doesn’t protect metadata on the server currently - so you have to trust the admins that run the servers you are participating in. In the longer term we want to fix this (https://matrix.org/~matthew/2016-12-22%20Matrix%20Balancing%...) but we haven’t started on it yet.