Hacker News new | ask | show | jobs
by dewey 409 days ago
If you have a lot of product pages (millions) it can make sense to not have all of them indexed by a search engine. If you have pages that are more profitable and might hit more keywords than some very specific product SKU it makes sense to index these primarily.