Yes, signal would manage that. This is still more privacy preserving than a phone number because the the public key isn't attached to your identity in the way a phone number is. You'd still need to getAllContactsPubKey()->encr(PersonAKey)->SendToAllContacts
So signal knows you requested all your contact pubkeys, that you sent a duplicate broadcast to all contacts, obscuring who it was intended for... that could be 0-n of m persons.
So signal knows you requested all your contact pubkeys, that you sent a duplicate broadcast to all contacts, obscuring who it was intended for... that could be 0-n of m persons.