Hacker News new | ask | show | jobs
by throwup238 71 days ago
FWIW GovDeals does not care as long your scraping load is reasonable, at least they didn’t years ago when I asked them. They prefer personal scrapers (i.e. buyers looking for deals) stick to precise searches but they were okay with properly throttled site wide scraping if its a public site.

They make money not by optimizing per item profit or by exploiting information asymmetry, but by getting as many eyeballs on their site as possible to drive demand (and thus drive auction price up). They’re happy to be scraped as long as scrapers don’t bring them down because their core competency is giving municipal and state governments an aggregated platform and making the process easier from a bureaucratic point of view.

If you do the work of marketing for them (especially for free!) that’s a plus in their eyes. You’re not a competitor because they do the work of actually dealing with government departments like handling payments and paperwork.