Hacker News new | ask | show | jobs
by tikazyq 2499 days ago
Hi,

Thanks for the upvotes.

Crawlab is a golang-based distributed web crawler management platform, supporting various languages including Python, NodeJS, Go, Java, PHP and various web crawler frameworks including Scrapy, Puppeteer, Selenium. Technically you can run any spider on it. It has both English and Chinese language support.

Github Repo: https://github.com/tikazyq/crawlab Demo: http://crawlab.cn/demo

Since its launch in March, Crawlab has received a lot of positive feedbacks, especially about the flexibility and appealing Web UI. And Crawlab is evolving fast, we have developed many features through continuous iterations.