Hacker News new | ask | show | jobs
by cratermoon 433 days ago
If they are well-coded, they don't constantly crawl. They use and pay attention to headers like ETag, If-Modified-Since and/or If-None-Match and support conditional requests.

Badly behaving RSS readers on the other hand....

https://rachelbythebay.com/w/2024/05/27/feed/