Hacker News new | ask | show | jobs
by schwanksta 5148 days ago
Woah, the /indexes/ url just blew my mind; so it just gives you the last update on that date?
1 comments

Yes.

NYT homepage editor has flexibility on when they "roll" the date depending on news events (and content can be ranked ahead of time too). But yeah, basically midnight.

Internally I know editorial preserves them on an hourly basis. You might ask current NYT-ers if you could get access to that?

I know I did a scrape every 10 (or 2?) minutes of the Homepage HTML for a couple of years (2007 to 2011?). If I can find that data and its still meaningful I'll get it to you. It was quite a few GB.