I'm asking because I've been running a web crawler for years now, and in the past week, I have noticed that the crawler is being rejected in more websites then usual.