Hacker News new | ask | show | jobs
by impostervt 3551 days ago
I was looking into Event sourcing for a system I built recently, and the tooling just doesn't seem to be that widespread yet. How do you read out of the entire event stream to figure out the current state? While there are tols, they seem to be .net focused. Just didn't seem to be a "standard" answer yet.

We ended up going with microservices that pub/sub events into Kafka, but maintain their own databases. There's another microservice that lets you query past events for statistics.

4 comments

>Just didn't seem to be a "standard" answer yet.

This article was extremely helpful to me for understanding some solutions in this space.

http://www.confluent.io/blog/turning-the-database-inside-out...

We find that a simple in-memory synchronous message bus + event logging to files goes a long way. See e.g. https://github.com/robertreppel/hist for an in-memory bus + file system (and DynamoDB ...) helloworld which isn't .net.

Scaling that up by adding asynchronicity and more ambitious plumbing when needed seems reasonably straightforward. For something more out-of-the-box, see https://geteventstore.com/ . It has clients in a variety of languages. Comes with a nice HTTP API too.

I wouldn't normally read the entire event stream; usually, only the state of a particular object (aggregate, in Domain Driven Design speak) is of interest, E.g. the customer with id 12345. Events contain the aggregate ID, so the query to whatever event store you use would be "give me all events with aggregate ID 12345".

Are you using DynamoDB Streams at all? I've been toying the idea of using DynamoDB as an event store and having other services listen to a table's stream, allowing them to update caches/views (the read-side of CQRS), report analytics, perform asynchronous tasks, etc.
You could quite nicely use AWS Lambda for the materialized views, I think.
No, haven't tried them yet. Does look interesting for it, though.
You basically consider the event log as a big collection,and you "fold over" the events in order to incrementally build your state/projection, the same way you would do with finite collections in a Functional language (scala, haskell, ...).

GetEventStore documentation has some examples of how you can create projections (https://geteventstore.com/blog/20130212/projections-1-theory...), which you can use as inspiration to build your own projections.

Either you have fast queries and indexes, or you have a microservice that monitors for certain events, and keeps up to date state in a cache.