For signal - phone numbers as IDs allow them to retain no other info on their servers. They don’t have your phone book (though if someone gets your device and forced you to unlock it that’s different, but then they’d have your contacts anyway). There’s a reasonable tradeoff here imo to retain as little as possible server side. Signal is also working on a non phone ID option iirc.