Hacker News new | ask | show | jobs
by thwarted 6429 days ago
See if the sites in question are part of an affiliate network, like Commission Junction or Link Share. They often provide plain-text feeds to affiliates through these programs, and many of their terms of service enable you to set up these kinds of services (although some have restrictions on mixing their data with data from their competitors). However, I've found that even this data isn't all that great, cleanliness wise (sometimes you can't trust the name of the product, the price, the link, or the SKU to even match the website) and isn't updated very often (like product availability). But it's a hell of a lot easier than writing a custom parser for each site's HTML (although when I was working on project like this, I had to write a custom parser for each feed in order to put them in a more consistent format).