Hacker News new | ask | show | jobs
by jacquesm 6188 days ago
I'd hate to see your power & network bill :)

But I think that once you have enough customers the cost of 'crawling' goes down for every new customer you sign up because you only need to crawl a page once and you can sell the crawled result to many customers. Or do I misread your model and is every page crawled over and over again for every user ?