Hacker News new | ask | show | jobs
by aswadalime 423 days ago
Awesome! I agree, recently I’ve seen a huge spike in scraping demand lately, especially from teams building AI products. Everything from training data to competitive monitoring.

LLMs are great for speeding up spider development, but maintenance is still tough with dynamic content and aggressive bot protection.

Scraping in 2025 isn’t just about data collection, it’ll be about reliability, scale, and staying compliant. Feels like we’re entering the enterprise era of scraping.