[jeffail]

Hey, Benthos doesn't yet provide any general tooling for exactly-once processing like Wallaroo does, that's possibly a goal for the future.

My main focus has been providing general purpose stateless processors. So you can build a your own stream processor focusing on what makes it unique, and then the moment it's compiled and packaged it can read and write to anything, and convert any kind of payload to anything else just through configuration.

[spooneybarger]

Hi Wallaroo developer here,

I'm curious, does Benthos manage state for the user/application like we do in Wallaroo or is it purely stateless computations at this point?

[jeffail]

Processor implementations are able to carry their own state, or share state across worker threads or deployments using their own mechanisms, but Benthos doesn't provide any tooling for that. None of the processors you get out of the box need computational state. You currently get an ALO stream (provided you use ALO protocols), vertical & horizontal scaling as per your config, and any glue you need between services.

It would be a nice stretch goal to have standard tooling within Benthos to share distributed state, perhaps with some ability to do exactly-once processing, but that's not the focus of the project right now.

[guywhocodes]

That is a very interesting prospect actually, any plans on making python integration simple? Some parts of my pipelines rely on very large python modules I don't see being rebuilt in anything else in the next few years.