Hacker News new | ask | show | jobs
by tectec 1093 days ago
You'd think that reddit would be smart enough to show web crawlers a cleaner version than real users, or is that not allowed? Maybe google needs whitelist some of the biggest sites on the web so they can feed searchable data into google in a more efficient way.