[gerpsh]

Dude, you're making ScraperAPI requests (with your API key in plain text as a parameter) directly from the client app...this thing is a nice idea but it'd probably be a good idea to clean it up and repost.

[criddell]

I thought these multi-region searches were against Craigslist's terms of service.

[smt88]

Craigslist is famously aggressive about scraping, too. While helping with some university research, I found that their anti-scraping defenses were really difficult to get around.

They've also been litigious about this in the past, so this project would undoubtedly get a cease-and-desist if it ever became a blip on their radar.

[MegaButts]

I'm guessing that's because of AirBnB and how they got their start by essentially stealing posts from Craigslist.

[massivecali]

I thought Padmapper got their start that way, by essentially providing a better apartment hunt interface than CL had at the time.

[CalRobert]

Which was a shame, because padmapper stuck it on a map and linked to Craigslist, instead of just taking the content. If Google Maps did it nobody would think twice.

[giancarlostoro]

CraigsList might make them think twice. I don't think they want to overly exhaust their own resources, what they have works for them.

[rhizome]

And they got the banhammer, too.

Not too hard to see where the fulcrum of this "settlement" is: https://www.eff.org/deeplinks/2015/06/padmapper-and-3taps-se...

[Reedx]

> I found that their anti-scraping defenses were really difficult to get around

What sort of defenses are they using?

[smt88]

They had fantastic detection of headless Chrome and curl requests, a thorough IP blacklist, aggressive rate limiting, and possibly some JS stuff.

[stevenwoo]

I was a contractor at a company where one of the divisions did Craiglist scraping as part of the business model and they had a closet full of laptops doing the scraping part of the job - thirdhand what I heard was separate PC's were necessary for the anti-scraping workaround they were using.

[rhizome]

All scrapey automated stuff is. I doubt this will be alive next week, or tomorrow.

[throwaway190102]

already appears broken.

[eagsalazar2]

What a shame to have invested so much in an app that will immediately be shut down by CL.

[nydel]

i'm not seeing this, has it already been cleaned up, or am i missing it?

[guessmyname]

I clicked on the “search” button.

Then it sent 503 XHR requests:

https://i.imgur.com/R9D23iU.png

[jolmg]

I see it.

[chovy]

I know. It’s just a poc

[smt88]

Even a PoC needs basic security before being released publicly. It wouldn't even delay the release by a single day.

[everdev]

Proof of Concept is a vague term and doesn't give you any sort of protection or defense. If it's against Craigslist's terms you're taking on some risk by making and public and promoting it.