[alexdoesstuff]

Feels surprising that there isn't a modern best-in-class non-LLM alternative for this task. Even in the post, they described that they used a hodgepodge of headless Chrome, readability, lots of regex to create content-only HTML.

Best I can tell, everyone is doing something similar, only differing in the amount of custom situation regex being used.

[monacobolid]

How could it possibly be (a better solution) when there are X different ways to do any single thing in html(/css/js)? If you have a website that uses a canvas to showcase the content (think presentation or something like that), where would you even start? People are still discussing whether the semantic web is important; not every page is utf8 encoded, etc. IMHO small LLMS (trained specifically for this) combined with some other (more predictable) techniques are the best solution we are going to get.

[alexdoesstuff]

Fully agree on the premise: there are X different ways to do anything on the web. But - prior to this - the solution seemed to be: everyone starts from scratch with some ad-hoc Regex, and plays a game of whackamole to cover the first n of the x different ways to do things.

Best of my knowledge there isn't anything more modern than Mozilla's readability and that's essentially a tool from the early 2010s.