Oh wow, some interesting stuff in here. Looks like they use MySQL as a queue for scheduling the ORC Peons? Would have loved to hear more about why they did that.
We use it because it works well for us. We've put a lot of work into making MySQL scale for us to the point where it's a very well supported system and one of the main choices for a lot of storage decisions.
We even use MySQL as a queue for Facebook Messenger. More details about this:
Because it works perfectly fine? I tend to write anyone off who scoffs at simple database-as-queue designs without understanding what the scaling requirements are. You can use a database as a job queue for 10s of thousands of jobs per day without any sweat.
Please forgive me, but I don't understand your hostility.
I understand that there are perfectly legitimate reasons for using a database as a queue. If you frequently need to look at and rearrange your jobs while they're in flight, chances are a pure FIFO structure probably doesn't work that well for you anyways. If the enqueue is contingent on a transaction committing, probably makes a lot of sense for the job to be in the same database. You don't need to tell me -- I've seen more than a few in production.
But an actual message queue also "works perfectly fine" based on the information provided, and I would imagine that a company like Facebook probably already has a few of those lying around. It would have been a conscious choice to use MySQL as a queue.
We use it because it works well for us. We've put a lot of work into making MySQL scale for us to the point where it's a very well supported system and one of the main choices for a lot of storage decisions.
We even use MySQL as a queue for Facebook Messenger. More details about this:
https://www.youtube.com/watch?v=eADBCKKf8PA
https://code.facebook.com/posts/820258981365363/building-mob...