[janreges]

Hi andrei-akopian. I am the main author of SiteOne Crawler and thank you for this post!

For about 20 years, I have been leading the development and infrastructure of the Czech webdev company SiteOne, so I wouldn't say that I am not a professional ;) I have around 50,000 hours of practice. But I definitely know a lot of better programmers than me and many of them are at SiteOne :)

However, after the extremely premature birth of my son, I had to change my role at SiteOne, but all the more I wanted to help my colleagues with a useful tool that we generally lacked and at the same time I wanted to give myself joy in difficult times.

I want to implement a number of other useful improvements into the crawler (some of them are also described in the documentation and roadmap) and work on its promoting in parallel. The more people will use it, the more it will help to optimize the website, the more it will help developers and testers with various needs, the more joy it will give me :)

By the way, the GUI application https://github.com/janreges/siteone-crawler-gui is just a visual wrapper over the command-line version of SiteOne Crawler and integrates it into itself.