Hacker News new | ask | show | jobs
by AznHisoka 3270 days ago
It would help if Yelp allowed new, emerging search engines (potential google competitors) to crawl their site. But their robots.txt only allows Google, Bing and a few others to crawl them (https://yelp.com/robots.txt)

allow all crawlers and I would have at least a little sympathy for Yelp.

2 comments

Presumably they want a legal defense against scrapers.
There are probably food delivery and restaurant reservation startups complaining that Yelp has a monopoly position on restaurant search.