[scottlf]

No disagreement from me.

It's definitely not easy to keep queues shallow end-to-end. In my experiences with Erlang systems, queue management is an area where the BEAM VM's runtime is definitely not actively helpful enough. One anecdote: within the last few months, the weak scheme that the BEAM uses for runtime back-pressure was removed because it isn't effective enough to justify the complexity of its code.

When a system does have the ability to keep queues shallow end-to-end, then I think it's a good base to build on, adding additional features to allow deeper queues where and when we want them.

Regarding Wallaroo: Today, you can use Kafka as a data source, allowing Kafka to be your deep-as-you-wish buffer upstream and/or downstream of Wallaroo. Tomorrow (a.k.a. vapor, though we've discussed the feature internally) it's certainly feasible to add an option to (for example) `TCPSource` that would add a large buffer at the entry to a Wallaroo cluster ... with flexibility to queue in RAM, disk, or elsewhere. It would also make failure recovery more complex, and it's one reason why we've deferred implementing it.