[noxer]

I think you misunderstood that. This setting isn't about privacy of your friends number its about who can find you with your number. For example by brute-forcing numbers. Whether you upload your address book or not is another story and it does ask if you want to do that. Obvious if you decline and then set to above setting to "my contacts" then no one will be able to find you by your number (which is exactly what I personally want)

There is absolutely no need to upload your address book to make you number unsearchable.