[Reelin]

But you already can't opt out of them sharing their entire address book with the latest sketchy app they downloaded. What's the difference?

They either have your phone number and other contact details in their phone or they don't. They either make good decisions or they don't. You choose how much to trust them and what with. Federation and third party implementations of identity servers for one particular app changes absolutely none of that.