Hacker News new | ask | show | jobs
by jfim 1498 days ago
It might be a better idea to return low quality results than nothing at all. The idea is that it's pretty obvious when the bot is banned when it receives no results at all. Having to look at the results manually to determine whether one is banned is a much more time consuming endeavor.
1 comments
MicahKV 1498 days ago
Well what I'm suggesting isn't about blocking the bots, it's about removing the incentive. So in this case, I think the more obvious it is the better. I would want them to realize as soon as possible that they are 100% wasting their time.

If anything, it might be best to return a page that explicitly states "Sorry, this search engine no longer supports SEO footprint search queries."

*edit for typo & wording

link
bornfreddy 1498 days ago
On the other hand, making content difficult to parse is easy to do and a very strong weapon. Make them waste dev time... It is much easier to make variants of HTML than it is to parse it. You can even automate it to some degree.
link