Hacker News new | ask | show | jobs
by tkk23 1331 days ago
How could the crawler discover the pay-walled content? If sites explicitly allow the crawler to enter, what would be their argument to prevent that the search engine shows a cached version, especially if that's the search engine's policy from the start?

That said, search engine for pay-walled sites with integrated micro payment isn't a bad outcome. What else could create an alternative to the ad-based business model?