[marcell]

I'm working on a Chrome extension to do web scraping using OpenAI, and I've been impressed by what ChatGPT can do. It can scrape complicated text/html, and usually returns the correct results.

It's very early still but check it out at https://FetchFoxAI.com

One of the cool things is that you can scrape non-uniform pages easily. For example I helped someone scrape auto dealer leads from different websites: https://youtu.be/QlWX83uHgHs . This would be a lot harder with a "traditional" scraper.

[hydrogenpolo]

Cool, would this work on something like instagram? Scraping pages?

[marcell]

Yes! I actually just had someone else ask about Instagram. Try it out :)

I got these results just now: https://fetchfoxai.com/s/UOqL5HtuNe

If you want to do the same scrape, here is the prompt I used: https://imgur.com/XhguCk4

[dghlsakjg]

Instagram really doesn’t want you scraping. There are almost certainly terms against it in the user agreement

[bangaladore]

Companies like Instagram (Facebook/Meta/Garbage) abuse their users' data day in and day out. Who cares what their TOS says. Let them spend millions of dollars trying to block you, its a drop in the bucket.

[houseplant]

instead, don't do it because it's disrespectful to people. A lot of people weren't made aware- or didn't have the option- to object to that TOS change. Saying "well, THOSE guys do it! why can't I!" isn't a mature stance. Don't take their images because it's the right thing to do