[Dirlewanger]

Excuse my poor understanding of Ruby's GIL, but is the need for an Elixir worker because Ruby is by default single-threaded, and therefore actual default concurrent Sidekiq workers are not possible? Or this a problem scaled up to the degree that we're talking ~thousands of jobs per second that Ruby's limitations start to show?

[phamilton]

Since Ruby's GIL means you can't do true parallel processing, it's common to kick off multiple worker processes and have them each take one job at a time.

If the job involves an expensive API call it will tie up a worker for the duration of the call.

So it's not so much ~thousands of jobs per second as it is a limitation of only 10 ruby processes per 1GB of memory. (A ruby process with the Rails env loaded is about 100MB.)

Elixir gives you the ability to make a few orders of magnitude more calls in a tiny amount of memory. (100,000 Elixir "workers" can be had in less than 1GB).

[mperham]

Sidekiq runs 25 worker threads by default. MRI will context switch upon I/O so typical server-side applications will get plenty of concurrency, even with the GIL, because they spend lots of time talking to the DB, memcached, redis, 3rd party APIs, etc.

tl;dr - Sidekiq can do a LOT of work, even on MRI with the GIL.