[dr_kiszonka]

[I have no experience with Elixir.] Would that be possible to have elixir intelligently manage multiprocessing of python scripts? I would be especially interested in being able to have the scripts talk to each other, but have no idea how it could work, besides perhaps having them communicate by writing to / reading from the same files, which seems risky.

[chasers]

There is actually a case study on elixir-lang.org of exactly this!

https://elixir-lang.org/blog/2021/01/13/orchestrating-comput...

[dr_kiszonka]

Very informative - thanks!

[jononomo]

I think you can have the scripts talk to each other through Elixir via the BEAM/OTP approach: for each script you will have a GenServer module that manages it in a separate process. Call this module MyApp.ScriptServer and put "use GenServer" at the top of the file. This GenServer process will have a client API, which you can write, which will be how you interact with your script. In the BEAM programming model, processes communicate by sending messages to each other's mailbox, so you set up some logic for this and maybe have another module called MyApp.ScriptCoordinationServer that manages some global state.

This is just my amateur guess -- I'd be happy to hear an Elixir/BEAM/OTP expert chime in.

[dr_kiszonka]

Thanks for the explanation!

[jononomo]

This is exactly the solution I was thinking of as well.