Hacker News new | ask | show | jobs
by Cenk 2326 days ago
Basically what the other commenters said, they scan the entire web to find out who links to whom and how many backlinks each site has.

One thing you can do is stop them from scanning your site with robots.txt – some SEO people recommend this to prevent your competitors from finding out where you got your backlinks from, but I haven’t tried it.