[dagurp]

These days I just let chagpt generate a script that scrapes a site and spits out an rss file. Then I run it with cron.

[notadev]

I’m guessing they paste a portion of the website’s source then tell ChatGPT to generate a script that can generate an RSS feed from that site.

[dagurp]

Yeah I just copy the html that's relevant. There's some manual work involved but it doesn't take a lot of time.

[dopidopHN]

Are you not limited by the cut off date of the content the model is trained off ?

[pinkcan]

1. the script is generated by the llm

2. the user runs the script that does the scraping

these are temporally separate actions

[dopidopHN]

Fine, but it’s subject to html selectors brittleness no? Oh, you subject the raw html when you need it maybe?

[dagurp]

Here's how I do it.

1. Tell chatGPT to create a python script that scrapes example.com and generate an rss file.

2. Paste a snippet of the html and tell it to modify the script to use that.

3. I do some minor tweaks myself to fix the date format.