Hacker News new | ask | show | jobs
by dx034 3076 days ago
But probably not Google. The google crawler is very careful and stops as soon as they encounter higher error rates. Bing appears to do the same.