[jsnell]

That's a bit misleading. You can unpublish the app[0], which hides it in the Play Store and prevents new installs. It just doesn't remove the app from the people who already have the app installed, and I think they'll be able to install it on new phones.

I don't know why the OP is choosing not to unpublish the app.

[0] https://support.google.com/googleplay/android-developer/answ...

[multimoon]

Unpublishing the app doesn’t remove you from having to comply with address requirement.

[jsnell]

Yes, but it would mitigate the issue almost completely since the app (and thus the address) would not be discoverable by new users.