[nyolfen]

https://gist.github.com/yoki/fd2a3642b059703529d158de8233bbc...

[cpeffer]

It crawls webpages (finds subdirectories), handles JS blocking with fallbacks to headless browsers, and does this all concurrently.

If only that script worked for every website. But, alas, it does not.