Hacker News new | ask | show | jobs
by bryanrasmussen 4117 days ago
I guess if I gave myself the restrictions you've laid out, that I wouldn't necessarily do, I would have a check at the beginning of the request - if it was the google bot then give it back a 503 or something else. Then you can use this function based on other parameters, for example what time of day it is being used or maybe open up for parts of things you want indexed at times as others have suggested.
1 comments

I'd be concerned as to adverse effects on my indexing. But it wouldn't fix my problem as it would still be a request that would require a GAE instance to handle it.

(edit) Yes, the bot is still hitting the GAE site atm even though it's returning a quota error.

But would the bot do the same number of requests if it got an error?