[delusional]

b) You're asking for occasional "eventual consistency" trouble when A's state lags or has moved on ahead of the event

If you allow A's state to lag behind it's own events, then how are you ever going to create a sane system? Surely A either has to be ahead or at the state that caused the event to emit, or events are pointless.

[SideburnsOfDoom]

> A's state to lag behind it's own events,

Real systems don't have just 2 services. There can be 100s and the "own events" assumption may not hold.

[delusional]

Sure, but in a thin events model someone would "own" the events since otherwise the subscriber wouldn't know where to query the actual data. What would you even do with an event saying a customer changes address if querying that address then produces the old one.

I'm genuinely curious how such an architecture would work. You don't have to respond directly here, but if you have any reference to further reading, I'd appreciate it.

[SideburnsOfDoom]

> I'm genuinely curious how such an architecture would work.

Complex systems are the way that they are because they got that way over time. It is not my goal to defend or even characterise a system that I did not create.

I am here telling you the issue that I saw: one event consumer, at an edge case, ran substantially behind another, and when they attempted to co-ordinate over http, this failed. And how it was successfully resolved: fatter events removed the need for co-ordination between these two altogether. This was IMHO a more elegant design - it avoided he issues of the the thin events.

[delusional]

Ah, so A and C where both subscribed to B, but during A's processing of the event it assumed C had already processed it and tried to look up some state. Is that correctly understood?

This sounds more like an architectural deficiency (as you say probably from architectural decay) than a systematic design edge case. I can't quite understand what information A would need to get from C that could be included in the fat event but not simply queried from B.

[SideburnsOfDoom]

> Ah, so A and C where both subscribed to B, but during A's processing of the event it assumed C had already processed it and tried to look up some state. Is that correctly understood?

yes, though you're down the rabbit-hole on this one issue. My point (aside from the fact we actually saw this specific issue and it took a long time to correctly diagnose) is that with thin events followed by a http query call-back, You're asking for occasional "eventual consistency" trouble. Data races will happen occasionally - this is inevitable in the design.

At the tail end of the latency distributions, too fast or too slow, or service A is now having a blip, or you now hit the new version just deployed or whatever, things will go wrong by mis-sequencing in surprising and hard to follow ways (example given that you're fixating on) in complex real systems, and it's a win to avoid that chaos entirely, with fat events.

[tremon]

If you allow A's state to lag behind it's own events

That's a mischaracterization. A's state is not lagging its emitted events; instead, A's state may have been changed at the time A's event is processed.

[SideburnsOfDoom]

The "own events" was the faulty assumption. it's not always the same service that both emits the events, and is the place to go to over http for data. It "seems logical" to also build that store from listening to events, but it can cause issues as mentioned.

[delusional]

The comment I quoted says:

> when A's state lags or has moved on ahead of the event

That sounds like it can EITHER be ahead or behind. Specifically, I do not understand it as A's state can either lag OR be ahead, not that "lags" is a synonym for "moved ahead"