[joelbluminator]

"Java does not typically use a background job set up, because it is a first-class multithreaded runtime " : Is there really no equivalent of Sidekiq in java world, or that's just rarely used? I would assume a separate process would need to run over the jobs (so can be scaled easily) so then you need to write the jobs to some shared memory like redis. Is this rarely used? https://docs.spring.io/spring-integration/reference/html/red...

[lfn716]

The advantage of Java and Erlang the author was referring to is that you don't need extra technology. With sidekiq, you must setup Redis and often separate instances to process the job queues. With Node and websockets in Ruby/Python, you typically must move any CPU task to jobs/task queues.

Sure, you may want to move your Java/Erlang jobs to Redis/SQS/etc for different reasons, but it is not your starting point. And given you were explicitly talking about CRUD apps, being able to have everything you need with only your app + database, without a need for Redis, workers, or tasks queues, leads to operational simplicity.

[joelbluminator]

But part of the idea behind a queue job is not to overwhelm the server with heavy jobs that aren't urgent. Even with a thread pool, in java, you may still overwhelm the server with heavy jobs like sending emails, if you do not clog the threads you may still spike the memory - do you really want your server and your background jobs to share the same memory and resources?. So the separate instance make sense to me - they are a feature in fact. I really want to know from someone from the java ecosystem more about how they handle heavy background jobs. I get that Elixir may have this part done in a nicer way, but still ,Sidekiq is a VERY polished tool.

[cpursley]

> do you really want your server and your background jobs to share the same memory and resources

Erlang/Elixir processes don't share memory (Actor model) like Java threads do, giving it a distinct advantage (see Akka for Actor model in Java).

But yeah, for heavy stuff even in Elixir, you might want to outsource that to a Job server (which there are many libraries in Elixir for this).

[joelbluminator]

I'm really asking because I don't know - in java world background queues are actually rare?

[jonesetc]

probably a bit of loose wording there. It's probably not what either of us would call objectively rare, but it is comparatively rare compared to many other runtimes. There's quite amount of this that can be avoided with some vertical or horizontal scaling depending on what you are doing with these events.

For instance you linked the spring application events integration with redis, while that is definitely an option it's not one you'd immediately reach for. Instead you'd more likely set up and use application events and/or domain events (if using spring data) handled locally with thread pool. The benefit here is that of course way less set up and complexity, but you can also handle these events sync or async.