[krishvs]

Looking for some advice.

We have written several microservices primarily for websockets in Elixir. They are great with literal zero maintenance costs..but how do Elixir developers handle the following when going all in:

1. Long running workflows - there do not seem to be popular frameworks like camunda, jbpm, temporal or cadence for elixir

2. Integration libraries - similar to apache camel

3. Inbuilt scripting engines to run user scripts like nashorn, graaljs or groovy

We really enjoy working with rails and would like to go all in into elixir. But the ecosystem of available frameworks seems to always come in the way and makes us choose spring boot or rails.

[lostcolony]

Pre-emptive caveat: I do not know much about the three things you mention. THAT SAID, I'm familiar with Erlang, and moderately familiar with Elixir, so wanted to still give you some areas to explore.

For #3, you probably would need to NIF out to something. You can also execute an uncompiled script of Erlang using escript, but that, obviously, is not something you'd expect most users to learn (and not sure you can execute it in a running Erlang context, rather than from an external shell). You can also evaluate a string, and/or compile and load new Erlang code from a running Erlang program, but these are suuuuuper dangerous if it's user supplied.

For #1, I've never seen anything, but if you just need something that allows you to change out or customize logic on the fly you can do so in a pretty straightforward manner. Because it's a functional language, you can do stuff like have a stateful process template that you can swap out on the fly, i.e., submit a list of function identifiers a la [read_data/1, transform/1, write_data/1], and now whenever you call run(data), it spins up a new actor that effectively calls write(transform(read_data(data))). Technically you can even provide new code snippets (using one of the mechanisms from #1, or, if opening up an Erlang shell to the running instance, supplying it directly as a lambda), but you'll need to be mindful about persistence. I am not that well versed with workflow engines, but that partly comes from the fact that I haven't really seen the point given how easy many languages make it to create and customize workflows without having to learn special semantics.

I am not at all familiar with apache camel, but I think your concerns here aren't easily addressed; certainly, the library support for Erlang/Elixir isn't anywhere close to the JVM. But I -will- mention that sometimes writing the integration(s) you need to enable using a different technology are worth it.

[di4na]

For #3 in general the advice is a NIF or a Port to an engine (polar is a good example) or to use Luerl

For #2 i know nothing about this kind of stuff

For #1 it depends a lot. Something like Oban can be enough, or Ulf Wiger's Job or Broadway. These tends to be more ad hoc and extracted to independent services. It depends a lot of what kind of constraints you have for these workflows. There are lot of options but they are all more specific to use case and constraints than these broad projects that try to do it all.

[elcritch]

The other comments make good points, but here's a few more:

> 1. Long running workflows - there do not seem to be popular frameworks like camunda, jbpm, temporal or cadence for elixir

Oban seems good for job processing. Otherwise Elixir builds on Erlang's OTP. That means you can also use ets, mnesia, or even Rick's distributed dynamo application runners.

> 2. Integration libraries - similar to apache camel

Nothing quite like this, but you can make use of Flow or GenStage for data pipelines.

> 3. Inbuilt scripting engines to run user scripts like nashorn, graaljs or groovy

There's a nice Lua implementation 'Luerl' IIRC. Also as others mentioned you can do a NIF. In particular Rust & Rustler would provide lots of scripting language runtime.