[ashryan]

Love all of these RSS resources. Thanks for sharing!

Last week, I spent a couple of hours at a local hack event putting an RSS aggregator[1] together for our community. Just something fun to do.

One thing I realized when I deployed is that Substack gives a 403 if you try to read their RSS feeds from a GitHub Action. The only obvious workaround to me is to pull the content on local periodically, commit it, and then deploy. But I'd much rather have this site updating itself via GitHub Action and cron.

Have you run into this situation before?

[1]: https://github.com/astoria-tech/subcurrent-astro/

[ChrisHardie]

The usual case I run in to is that a site will block requests with User-Agent header strings that don't at least try to look like a regular browser, or that appear on some list of known bots/automation tools. (If they are using Cloudflare, this is a very easy state for a site to get in to.) I'm not sure if GH actions lets you customize the user agent in the spot you're hitting the issue, but that's where I'd start.