Curryfinger seems to work by querying Shodan and other scanners. Those scanners seem to work by just connecting to an IP address's port 443 and look at the certificate. If you always require correct SNI (the domain you host) then that scanning stops working (you literally disappear from Shodan for example). The fix (to scanners) would be to try and resolve every domain name you know of or scan every IP with every domain name you know, that's unfeasible. Only replying to correct SNI is not a defense mechanism by itself, but it does make it more difficult for attackers.
It turns out that if you have a targeted domain you have a good chance of finding it in one of the popular cloud hosting ranges. Masscan + curryfinger work well together. Alexatop + masscan + curryfinger makes an interesting dataset.