[fasteo]

>>> It makes perfect sense to me

Yes. Other than some very basic scripts, I have never worked with Ruby, so I was not aware that it is so slow. Thanks for pointing that out.

As for my recommendation, it is pretty standard worker architecture:

Their system seems to be an ingesting-only system, that is, the clients are getting an empty HTTP 200 OK response. Given this, I would put openresty (nginx) in the front, with some trivial Lua code[1] to en-queue payloads to beanstalkd. Then, you can either have your workers inside openresty (using Openresty timers) or have them as separate processes and written in the language of choice. We have been using this for a couple of years now and it is working really well for our use case, also an ingesting-only system.

[1] https://github.com/smallfish/lua-resty-beanstalkd/blob/maste...

[sinzone]

we use the same technology with KONG [1]

[1] https://github.com/mashape/kong