Hacker News new | ask | show | jobs
by thingification 1657 days ago
One way to deal with this is to divide up your event streams into small streams - say one per order. Those small streams then may be aggregated into a larger stream so that you can just process events for all the orders together, for example.

If you hit a poison message, block just that smaller stream, not the aggregated larger stream. Once you fix the problem, reprocess the entire small stream starting from the poison event, or the next event after that. The "entire" stream here might be just a handful of events.

Greg Young's Event Store (https://www.eventstore.com/) works this way (there's a $by_category projection that produces the aggregated streams).

Caveat: I haven't actually implemented this mechanism because I've been able to get away without it, because we have some legacy event streams that aren't split up in this way, and because nobody else has yet added support for it to the tools I'm using.