[joshdavham]

> can you write a web scraper in your browser? The answer is: YES, you can! So why is nobody doing it?

Completely agree with this sentiment.

I just spent the last couple of months developing a chrome extension, but recently also did an unrleated web scraping project where I looked into all the common tools like beautiful soup, selenium, playwright, pupeteer, etc, etc.

All of these tools were needlessly complicated and I was having a ton of trouble with sites that required authentication. I then realized it would be way easier to write some javascript and paste it in my browser to do the scraping. Worked like a charm!

[moritzwarhier]

Is Playwright really that complicated?

I feel that when it has been set up, it's very straightforward to use.

Maybe in contrast to other solutions you posted? Not sure about that though; having only brief experiences with both, Playwright seems like an improved Cypress to me.

[metadat]

You might like Tamper monkey. You can add a button to kick it off or whatever your heart desires.

Tampermoney also works around CORs issues with relative ease.

[crtasm]

Userscripts for the win. Consider Violentmonkey over Tampermonkey though.

[metadat]

> Consider Violentmonkey over Tampermonkey though.

Why?

[MC995]

Probably because it looks to be closed source these days.

https://github.com/Tampermonkey/tampermonkey

> This repository contains the source of the Tampermonkey extension up to version 2.9. All newer versions are distributed under a proprietary license.

[panphora]

Open source

[joshdavham]

I love Tamper monkey! I've never made my own scripts though - just used other peoples'.