[mschuster91]

To anyone experimenting with such stuff, take care and don't make your services publically available. Especially the dark web is full with highly illegal content such as child pornography and in some jurisdictions even "involuntary possession" such as in browser caches may be enough to convict you.

[creekorful]

Do you think I should add a license in Github to mention that? To protect me and the users who will use the crawler?

[weatherlight]

yes.