|
|
|
|
|
by blueskin_
4117 days ago
|
|
>Your site has been assigned special crawl rate settings. You will not be able to change the crawl rate. That's the messed up part. I guess the question is, does robots.txt override that or not? If it does, fine. If not, all you need to do is make a few "google ignores robots.txt" posts and the problem solves itself. |
|
It's not really ignoring robots.txt either, as crawl delay is not an 'official' setting