[ismaelct]

Good observation about stack traces and abstractions. Re. your question, the pattern itself is no different than, say, Rack middleware, so you'd see similar cost and benefits. In essence you're running one callable object after the other.

A pipeline is essentially this

steps = [ ->(r) { r }, ->(r) { r }, ->(r) { r }, ]

Wrap initial data in a common Value object

initial = Result.new(some_data)

Run the Result through the steps, in order

result = steps.reduce(initial) { |r, step| step.call(r) }

That's the pattern, really. A reduce operation.

Re. stack trace, it can add noise because you're iterating over steps instead of calling them procedurally one by one, and you may want to decorate steps (put steps inside steps) for encapsulation, caching, etc. but again no different than Rack.

[codesnik]

yes, it is similar to rack. But what rack has this requirement of being pluggable from multiple decoupled codebases (rails or other framework, your app, some gems you're adding to your app). And that also warranted things like "insert_before" etc.

Even with rack, decoupling of code definition (where you've added your middleware) and code execution could be pretty hard to debug, but at least with enough exposure you know where to look, and you usually have only one rack stack in your app.

If all those steps in your pipeline are defined in the same codebase, rack approach becomes much less useful.

[ismaelct]

> If all those steps in your pipeline are defined in the same codebase

I find that rule a bit arbitrary. For example, I have data-import pipelines where some steps are unique to each task, but some others are shared across tasks. Why does it matter whether the steps are defined in the same codebase or not?