I was writing an netfilter/iptables rate limiter and a nginx rate limiter (both able to handle quite a lot of requests) but all the system admins I talked to shamed me for trying to rate limit. They raised all kinds of points about blocking access to large companies or small countries and it not being effective for DDoS anyway. Any thoughts on this?
A bandwidth rate limiter is indeed of dubious utility. There's a lot of ways that can go wrong, and there's a lot of ways it can fail to do what the user expected anyhow, and there's a lot of ways it can do both.
But there's a lot of things that can be sensibly rate limited, such as logins attempts to a given account. Now that has its own considerations, too, if pushed to the limit... you'd prefer that an attacker can't lock down your service just by spuriously trying to log in to all your accounts 5 times every 5 minutes or something. But loud downtime (which you can then react to) may be preferable to getting your users silently hacked.
I definitely agree that they are less useful than they look at first glance, a great deal more complicated than you'd like, and more subtle than you'd think. But they can still be a useful tool.
The only effective counter against a proper DDoS is to bring more bandwidth than your attacker has.
A small site is unlikely to attract a proper DDoS, but will get hit by the usual misbehaving web scrapers, broken clients in a silly loop (I'm looking at you, iTunes) and keyboards with super sensitive F5 key. In those cases, serving an occasional 503 might work better than the alternatives.
Looks to be the case based on the config. I'm suprised storage wasn't abstracted to accommodate redis or the like which is essential in a load balanced scenario.
The API doesn't seem quite idiomatic either, I'd expect to create a struct containing options and a function that closes over the http.Handler interface e.g. func(l *Limiter) Limit(next http.Handler) http.Handler or a function that takes options and a next http.Handler that creates a struct implementing http.Handler.
This is neat. What's the best way to do rate limiting in from of Apache for wordpress/drupal sites these days? everytime I go looking for something new I'm dissapointed.
For general rate limiting, yes HAProxy or Nginx make more sense.
Having different rate limits for different endpoints becomes unmanageable in HAProxy/Nginx; and having different rate limits say across different plans or users is all but impossible.
The best reason I can imagine is so that it can all be wrapped in your portable Go binary. Because of the portability aspect, there are a lot of use cases for Go similar to Java where there is an incentive to do everything "in Go".
(I thought rate limiters were very simple timers)