[acqq]

But the "observer" can still know which mobile phone is yours and who communicates with whom? Especially if the "observer" has the info from the Signal servers.

Edit (as i can't post you reply to your answer):

And based on the NSA principle of the "thee levels of distance" everybody is reachable as long as some common numbers are in our contact lists which we happily upload.

[haffenloher]

The question wasn't what metadata Signal gets, but what metadata Google gets on Signal messages. Yes, Signal servers know who communicates with whom.