Do not use Selenium for web scraping

Y	Hacker News new \| ask \| show \| jobs

	Do not use Selenium for web scraping (buklijas.info)
	5 points by sasa_buklijas 2749 days ago

1 comments

snaky 2749 days ago

> The only exception that I could see for using Selenium as web scraping tool is if a website that you are scraping is using JavaScript to get/display data that you need to scrape.

Like 95% of web sites?

link

sasa_buklijas 2748 days ago

Scrapy does have the solution for JavaScript with Splash, but I have never used it, so far I always found some workaround.

link