[abound]

I've been building something similar for my own purposes. Basically, I use Calibre's "Fetch News" functionality to get epubs of all the news sources I trust, and then once a month (interval is configurable), I run a pipeline over all of it (some LLM magic, partly local partly API) to dedupe and rank stories, then compile it into an actual honest-to-goodness newspaper (using Typst for layout and stuff), then read that at my leisure.

[jonathanlb]

I ran a similar personal project where weekly I scraped a country's news stories across curated sources using Brave News API and used an LLM to dedupe and cluster stories. This got expensive fast, so I'm wondering what your approach is to keep costs down across what I imagine is a lot of volume, i.e., one months' worth of stories across several sources.

[sheesdev]

This sounds really interesting and something I'd love to try. Do you have more details on how someone else can utilise this?

[abound]

Not yet, but I'll see if I can publish the basics for it this week.

[kristerv]

The what now. But not all sources have epubs, right? Is there any one that you're missing now?

[advisedwang]

Calibre has a feature that will fetch a rss feed or (IIRC) scrape a site and make an epub from it.

[abound]

Nope! Every source I care about either has an RSS feed, or I've been able to make a custom recipe [1] to handle things like auth for paid sites and whatnot.

[1] https://manual.calibre-ebook.com/glossary.html#term-recipe

[imhoguy]

epub is ZIP with HTML+CSS+images+some metadata.