[_pdp_]

5 years ago I built my own workflow programming environment to automate various security processes. What I've learned form this experience is that workflow tools kind of suck. They are effectively really bad scripting languages - fine for doing high-level composition but not remotely useful when it comes to putting simple algorithms. But I think this is just my own perspective coming from experience with various programming environments. There are plenty of users who find this style of programming essential to get something done. For me though, I rather write the code.

I think these types of workflow tools will be ultimately replaced by AI agents.

[fock]

For reasons I recently got acquainted with enterprise scheduling solutions. IBM/HCL workload automation specifically - the company is actively looking for a replacement but nothing except for the chinese sponsored open source comes close it seems to me (and that is a step too far for management...).

The thing you miss with scripting is that a good workflow engine essentially provides a simple and well-defined layer allowing you to couple many concurrent processes and providing ressource management. Curiously out of the box all the opensource workflow tools do so too - like you said people (Airflow has pools, but thrn ...) essentially built very weird DSLs to do high-level composition and there you can indeed rather easily code it yourself. I assume the companies originating these have integrations for the other bits too while everyone else does fake elastic scaling with some halfbaked workarounds. And from time to time someone comes up with a new DSL and some GUI - ignoring all the hard parts.

[mrdoops]

Main limitation of these kinds of products is that the graph/workflow itself is not very expressive of dataflow options. Eventually you need map, reduce, and state machines and likewise concurrency and durable execution. It's not easy to have a high level declarative tool but also expose powerful programming primitives; at some point technical need-to-knows cross the point where you might as well have the user write code.

[dexwiz]

I think the last point is key. These tools are used by those who can’t program, too scared by the learning barrier, or trapped in a SaaS/Security ecosystem that bans user scripts. All three of these can be solved by an LLM putting together scripts in a sandbox environment.

[shivsarthak34]

Maybe a vision for this product could be to be that "Sandbox Environment" with the LLM. So you just connect your backend and triggers and just create flows telling what needs to be done in natural language (one time setup) and it creates the flow.