I’ve got several years of experience of webscraping, mainly in python.
Scrapy is the first choice for “basic websites” while playwright is used then things get difficult.
I’m collecting my experience in using these tools in this “web scraping open knowledge project” on github (https://github.com/reanalytics-databoutique/webscraping-open...) and on my substack (http://thewebscraping.club/) for longer free content