|
|
|
|
|
|
by jarofgreen
317 days ago
|
|
|
> SEO is built in since search engines can crawl every state combination. This isn't always a plus - bots can find a very large number of pages to crawl and swamp your server with traffic. Maybe they would get stuck on all the combinations of listing page filters and miss the important pages. Not saying the conclusion is wrong - just something to consider. |
|
|
A public-facing page at $DAYJOB has 44 boolean filters for a bunch of different hardware components (think screws and fasteners rather than boards and drives), meaning 2^44 different combinations ~= 17 trillion different pages to crawl (from the disrespectful AI crawler's perspective).
That said, it wasn't really an issue except we had a misconfiguration in the in-memory caching database we were using so it didn't delete old stuff from memory and started writing to disk, which meant that we ran out of our (millions of) file inodes on the production server. Just saying to point out that keeping the state in the URL wasn't the problem in this case, and it is generally not a problem; it can be a problem but, when it is, it will be obvious why.