| This weekend a VIP forgot their VPN password, tried to log in 50+ times, and got their home IP permanently blocked by the anti bruteforce setting on their firewall. Their helpdesk got confused and escalated, then their network team, who knew their internet wasnt down, escalated it as a massive emergency internet down. PRTG was clear on both sides that it wasnt down. To make it even more confusing, a totally separate unrelated major peering link actually went down at the same time for the ISP, so ISP helpdesk assumed it was related. It wasnt. When that peering came back, the problem was still occurring, but it escalates up the chain for the major internet outage that both sides of the networking teams agreed wasnt happening. The root cause, besides forgotten password and failing >50 times was just that our security policy doesn't auto-expire the IP block. Something the firewall doesnt let you configure. About a month prior to this, they had hackers hitting the remote vpn trying to brute force in, resulting in AD locking out many accounts, since the firewall didnt block. I recommended to implement brute forcing protection but to set the # high so as to avoid real users from getting blocked. But this 'major internet outage' that only affected 1 person on their home network is my fault. So yesterday I started building a flask app that can manage the block list. |