[jeltz]

Well, this is obviously a different use case for what Whatsapp uses the contact list for. I really do not like Whatsapp's harvesting, but I can easily see why Google can make different calls on your app and Whatsapp. Especially since what you do could be construed as blackmail.

[heroic]

Google isn't citing the use case as issue:

This is what they told us as the reason:

Your app is uploading users' [Contact] information to [URL] without a prominent disclosure. Prior to the collection and transmission, it must prominently highlight how the user data will be used, describe the type of data being collected and have the user provide affirmative consent for such use.