Integration and webhooks are very well suited for streams. Having your core product emitting messages as event have many benefits:
- You can have a team working on the core product sending messages and another working on integrations triggering actions.
- If your integrations fails, your core product is not impacted. You can also replay old messages once you've fixed the stream consumer.
- Having streams allows you to do all kind of experiments too, you can connect a new project to a stream and go through a week of data almost instantly and see the result... rince and repeat as much as necessary
Event-based architectures, for example: https://martinfowler.com/eaaDev/EventSourcing.html For some types of applications moving to a model with a stream of events as the source of truth can solve a ton of hairy distributed computing problems.
I was recently thinking about this: let’s say you built a chrome extension and wants to collect some basic usage analytics (with explicit user consent and knowledge, while preserving privacy) - you could batch-send activities at intervals to a REST-style API, but would be nicer to handle as a stream (eg to respond in real-time somehow).
Something lightweight like MQTT is also well suited for this. It was originally designed for telemetry messages in IoT situations but it also supports websockets so it can be used in web applications or browser extensions.
- You can have a team working on the core product sending messages and another working on integrations triggering actions.
- If your integrations fails, your core product is not impacted. You can also replay old messages once you've fixed the stream consumer.
- Having streams allows you to do all kind of experiments too, you can connect a new project to a stream and go through a week of data almost instantly and see the result... rince and repeat as much as necessary