You have no other good mechanism to verify they are who they say they are, unless you initiate the communication.