| > ie, requiring very specific ergonomics. If you have to change what you're doing, it's a new domain to learn. Resque & Sidekiq build this in by converting job arguments to JSON. There's nothing extra to learn. > I didn't mention ETS or Mnesia? The OP was talking specifically about using job queues to get concurrency/parallelism, in which case you absolutely don't need job queues. If you need a job queue, you need a job queue. Sorry, I thought you were talking about building a background job system in Erlang using out of the box OTP but it sounds like you're actually talking about trying to get parallelism in Ruby by doing RPC over Sidekiq? That's always a bad idea! > Redis queues have millisecond latency, Ruby using Redis queues does not. That's the part that polls when there's nothing else going on. If you're never running out of jobs to do then your latency is fast, but you're also not accomplishing things as fast as possible(since it's waiting on whatever is in front of it). Ahhh! When Mike Perham says "Sidekiq Pro cannot reliably handle multiple queues without polling" what this really means is a Redis client can only block on and immediately process from the highest priority queue. The lower priority queues are only checked when blocking timeout expires. There's no "check all queues and sleep" polling loop which adds artificial latency. > And are incredibly difficult to use and pass information back and forth(hence why Elixir exists at all- Jose Valim was the person implementing this on the Rails core team). Jose Valim didn't join Rails core until a couple years after Josh Peek (now working for GitHub) made Rails thread-safe. > And are incredibly difficult to use and pass information back and forth(hence why Elixir exists at all- Jose Valim was the person implementing this on the Rails core team). It's really not that hard anymore! results = ['url1','url2'].parallel_map{|url| HTTParty.get(url) } 2012-2013 onwards Ruby got great libraries like concurrent-ruby and parallel that make things a lot easier. > Which Ruby has a lot of trouble maintaining performantly. When my original team went to use Rails5 sockets, we found we could barely support 50 sockets per machine. ActionCable is designed for convenience not performance. https://github.com/websocket-rails/websocket-rails will handle thousands of connections per process. |