Hacker News new | ask | show | jobs
by techno_modus 2865 days ago
After reading the section about processors it is not clear whether it can do stateful computations, for example, running sum or moving average. Can I define sliding windows? Can I join two or more streams?
2 comments

All Benthos processors are stateless, except for a few such as batch, combine, etc, which do basic tasks such as building up batches from consecutive messages. It's possible to create sliding window processors on top of Benthos, but there aren't any general tools for doing that yet.
For stateful processing you need something like:

* https://kafka.apache.org/documentation/streams/ - Kafka Streams (and its KTable)

* https://flink.apache.org/ - Flink

* https://spark.apache.org/streaming/ - Spark Streaming

* https://github.com/asavinov/bistro - Bistro Streams

The focus of Benthod seems to be performance and resilience (implemented without persistence).

Or to toot the horn of the project I work on:

* https://github.com/wallaroolabs/wallaroo - Wallaroo