[dansingerman]

The code examples are written in ruby and lua - I take it that in your live environment you use the lua script compiled into nginx?

Does this mean every rate limited request is hitting redis?

I had this problem years ago, and I used varnish to offload the rate limited traffic, which scaled very well then. Here is a blog post I wrote about it: http://blog.dansingerman.com/post/4604532761/how-to-block-ra...

(it was written directly in response to a 'One of your users has a misbehaving script which is accidentally sending you a lot of requests' issue)

[ptarjan]

We actually use the LUA scripts on Redis to guarantee the atomicity that the token bucket algorithm needs. Then the Ruby middleware does the HTTP 429 response logic. The code samples I provided are pretty close to what we use in production.

So yes, our Redis traffic is strictly higher than our whole API requests per second. This is actually pretty easy since Redis scales pretty well horizontally (the keys shard well) in addition to being able to handle many orders of magnitude more traffic than a normal web stack per machine.

Thanks for linking to your blog post. I like the simplicity of not having to enter the web stack at all. I'm not very familiar with varnish, how do you end up doing the actual rate limiting algorithm? I see you making a hash but not dripping tokens into a bucket.

[dansingerman]

The rate limiting was done by the application server, but was completely separate from the varnish layer. (I think it was something fairly naive to do with memcached, but that's not terribly important). What was important was that the application server could serve the 400* with an expiry, and varnish would deal with that load until the expiry.

*This was 2011. The 429 response code wasn't defined until 2012.

[kbenson]

That's an incredibly simple and elegant solution, and it's obvious how simple it is to scale it. Thanks for sharing!