Hacker News new | ask | show | jobs
by seanmcdirmid 681 days ago
I see a lot of Robots.txt files that use non-wildcards for Allow but what would be the use case for using a non-widlcard for Disallow?