Hacker News new | ask | show | jobs
by rmbyrro 600 days ago
Genuinely interested in your thinking: superficially looking, your anti-bot ideas are a bit contradictory to your Stealth browser, which enables bots. Why did you choose to make your browser useful for bot activity?

[1] https://github.com/tholian-network/stealth
1 comments
cookiengineer 600 days ago
Just because web browsers like Firefox, Safari, and Chrome are trackable by default and generally care not about the anonymity of their users - doesn't mean that a web browser should behave that way.

Being able to use the web anonymously is a value that should be held up against moral values. Malicious scraping of websites should be, too. Both ideas can simultaneously true, they don't have to be mutually exclusive. I also support initiatives like the web archive which I consider "good behavior" of web scraping.

If a person asks me for the dataset of my website and they don't have a competitive business that they run, I'm even happy to open source the dataset generation part. Contributors get more rights, abusers get less rights. That's how it should be in my opinion.

I don't see them as contradictory.

link
rmbyrro 600 days ago
Sure, that's why I said "superficially looking", because the back of my head said you obviously have a conscious intention consistent overall.

Would you mind elaborating more on where you draw the line between "good" and "bad" behavior when it comes to scraping?

Is it obeying robots.txt, the monetary purposes (profit / non-profit)?..

I personally have a hard time accepting that a website can allow Google to scrape all their content, but prevent others. In my view, this is anti-competitive behavior favoring a large and evil corporation. Argumenting that Google delivers value back in the form of traffic is unacceptable to me, because it goes against basic principles of net neutrality, which I support.

I'm intetested in reading your thoughts on this.

link
cookiengineer 600 days ago
I wanted to reply here, and actually posted the comment but then deleted it because some 4chan folks were already starting to comment here again.

I wrote you an email, would love to discuss this further.

link
rmbyrro 599 days ago
Sure thing, I see weird things here sometimes.. Haven't got your email. You can send to my username at gmail.
link