Hacker News new | ask | show | jobs
by bill-nordwall 5707 days ago
Be careful with these robots.txt suggestions.

Disallowing your css/js files in your robots.txt is probably not a good idea - Matt Cutts said as much himself: http://www.youtube.com/watch?v=PNEipHjsEPU

If you're running Wordpress, disallowing your /uploads/ directory will nuke your Google Image Search prospects, as Googlebot won't be able to crawl any of your images to begin with.

Also, submitting to a paid directory such as Best of the Web or the Yahoo! Directory would be a much better use of your time. DMOZ is still a valuable directory (for a lot of reasons), but the likelihood is small that they will review, let alone add your site to the directory in a timely manner (if ever).

A few other things worth doing: - Create a Twitter account for your site. - Create a Facebook page for your site.

1 comments

Indeed. Blocking JS and CSS from all robots will also cause your site to render poorly in most web archives, like the Internet Archive Wayback Machine.