[LinuxBender]

Would it not cause a performance issue for your OS to have that many routes to lookup?

Not for a blog. If this were a high performance computing node that needed extremely low latency CPU instruction performance then I would find another way but route enumeration at least in Linux and BSD are incredibly fast and efficient. I can't speak for other operating systems.

One method to test this would be to just start null routing more of those netsets not in a startup script and run load testing tools against the blog to see where it starts getting slower and to keep an eye on memory usage.

The bigger files to test with are:

    2475418 firehol_proxies.netset
    2480723 firehol_anonymous.netset

[lovehashbrowns]

For millions of IPs I put them all in redis as a key, and wrote some Lua code for nginx. The lua code would do a key look up in redis, and store the result of that lookup in cache. If the IP was in redis as a key, nginx would let the request time out. No error of any kind.

I also had to update that IP blacklist daily so that’s also why I chose redis.

Probably better ways to do it nowadays but that’s what I did like 8 years ago and it’s still one of my favorite solutions ever.

Also part of why nginx and redis will forever be two of my favorite technologies alongside Linux.