Hacker News new | ask | show | jobs
by eli 2461 days ago
Not sure what "legitimate" means here? Legal? Running aggressive web crawlers is in many instances against the rules for consumer cloud servers. For example, AWS requires that you obey robots.txt if you run a crawler there. https://aws.amazon.com/premiumsupport/knowledge-center/repor...

In my experience a lot of bots seem to be running on hacked servers or through hacked/insecure proxies. I'd imagine tracking down the owner or someone upstream of those boxes could be effective in taking them offline.

1 comments

What does that have to with my point? Bots used to purchase inventory (and that aren't otherwise commiting fraud by using stolen credit cards or something) are not malicious.
> Bots used to purchase inventory are not malicious.

There's no way you're in this conversation without being aware that scalping is a controversial practice at best.

https://theconversation.com/the-economics-of-ticket-scalping...

https://en.wikipedia.org/wiki/Ticket_scalping

I'm well aware that many economically illiterate people like to scaremonger about scalping.

That doesn't make them right.

Or perhaps they are interested in optimizing the process for something different than what you are optimizing for.
Maybe I'd believe that if they stopped saying things like scalping harms the producer, and acknowledged that they want a less efficient market.
if a single bot controller can buy up an entire stock of limited items legitimately, that is malicious as that company is not longer able to meet the needs of their consumers. That's bad for the company.
It is illegal if the website’s TOS for making a purchase prohibits the use of automated software.

It doesn't matter if it's legal, it matters if the website owner doesn't want x doing y on their site. A bot consistently not abiding by owners' intent is inheritly malicious.

What law does using a bot to make a purchase against ToS violate?

The LinkedIn case shows that it's not illegal in some cases - would you classify the behavior of the defendant in that case as malicious?

Are they following the sneaker website's robots.txt while doing that? If not, they are probably violating the AWS terms regardless of whether you believe that activity is "malicious."
if they're running on AWS, which most crawlers are not

When I've run scraping software in the past I used DigitalOcean, which doesn't contain a requirement to abide by robots.txt. As far as I can tell it's both legal and consistent with their ToS to run a program that makes purchases on a website.