Hacker News new | ask | show | jobs
by AndrewCopeland 1377 days ago
Thank you very much for your input. I will look into having the privacy be more verbose around the system logs and for how long I store them.

> I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it.

What do you mean by this? Like having the ability to create your own deny/black list of ip addresses and then you can validate against it?
1 comments
toast0 1377 days ago
>> I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it.

> What do you mean by this? Like having the ability to create your own deny/black list of ip addresses and then you can validate against it?

They want to download your database and run queries on it locally, rather than call your API. This way they don't share data with you and don't need to worry about your data practices.

link
AndrewCopeland 1377 days ago
Ahhh makes sense. I could definitely offer that it's pretty large being 250 million IP addresses. I wonder what the best format would be? Csv? SQLite?
link
stevenicr 1375 days ago
Currently to make things a little smaller I am putting many ip's in with cidr notation..

whichever format you would offer I could likely find ways to convert if needed.

I am still researching what is the fastest / lightest on server resources for blocking ips (mainly from data centers lately been the main problems) -

ip-block-file that is ready via cpanel's WHM is what I am using on one system, and iptables/uffirewall / fail2ban on another server..

I'm using wordpress firewall plugin to block some on another site - but I think it's lighter on server resources to block before the requests hit wordpress.. I wonder if it'd be lighter to pull the file from a separate server and return yay or nay - or if it's fine loading up 250 million ips via one of the firewall options already in place.

How large of a file it 250 mill ips? (non-compressed) - I suppose available RAM could also play into these things.

link
AndrewCopeland 1374 days ago
All of the IP are stored in memory. This takes up about 3.5 GB. I have a text file that is 445K lines that contains all of the specific IP addresses and the IP prefixes (CIDR) that make up that 250 million ips.

Uncompressed I do not know. Are you interested in the ranges and ip addresses or a list of all the ip addresses that make up all the ranges as well?

Let me know what you need and I could definitely try and accommodate for you.

link