[A4ET8a8uTh0]

<< there should be an app like an 'auto-RAG' that scrapes RSS feeds and URLs,

I am not aware if that exists yet, but the challenge I see with it is rather simple: you get overwhelmed with information really quickly. In other words, you would still need human somewhere in that process to review those scrapes and the quality of that varies widely. For example, even on HN it is not a given a link will be pure gold ( you still want to check if it fits your use case ).

That said, as ideas goes, it sounds like a fun weekend project.

[be_erik]

I do exactly this with hoarder. I passively build tagged knowledge bases with the archived pages and then feed it to a RAG setup.

[swyx]

https://github.com/hoarder-app/hoarder for the mention

[A4ET8a8uTh0]

Thank you. And I just found out, there is a docker image ready for Unraid. Lets see if it will work for me.

[fallinditch]

Cool. Hoarder looks interesting, thanks for the tip. How is it working out for you? Are you using the feature for auto hoarding RSS feeds?

[be_erik]

I am! It works great and it’s reasonably easy to snapshot sites without RSS on a cron.