Y
Hacker News
new
|
ask
|
show
|
jobs
by
standardUser
907 days ago
How do you deal with dynamically/contextually generated content? And how about paywalls and login-required content?
1 comments
jeffreyw128
907 days ago
Do our best at getting the right content.
For paywalls/login - we play pretty straight, always obey robots.txt, etc.
link
For paywalls/login - we play pretty straight, always obey robots.txt, etc.