Hacker News new | ask | show | jobs
by JetSpiegel 2615 days ago
> How can one automatically determine whether a page is unique or static?

They crawled it for years and it never changed? It is a blog post.

1 comments

The person you are replying to said "unique immutable pages", by definition you would be able to crawl these for years and they would never change. [1] is a site that contains all possible 3200 page books with the ability to consistently index content as an example.

[1] http://libraryofbabel.info/About.html