[trevor-e]

Like anything it can be abused and sometimes folks go overboard with turning everything into an event. However, when done right, it is really amazing to work with.

As an event producer as long as you follow reasonable backwards-compatibility best practices then you should be pretty safe from breaking things downstream. As a consumer, follow defensive programming and allow for idempotency in case you need to reprocess an event. Pretty straightforward once you get the hang of things.

[majormajor]

> As an event producer as long as you follow reasonable backwards-compatibility best practices then you should be pretty safe from breaking things downstream.

That can protect you from "downstream can't even read the message anymore" but it doesn't help you with the much more common "downstream isn't doing the right thing with the message anymore" problem. Schema evolution is kinda like schema'd RPC calls vs plain JSON: it will protect you from "oops, we sent eventId instead of event_id" type of errors, but won't prevent you from making logical errors. In a larger org, this can turn into delayed-discovery nightmares.

A synchronous API call could give you back an error response and alert your immediately to something being wrong. The system notifies you directly.

A downstream event consumer may fail in ways entirely off of your team's radar. The downstream team starts getting alerts. Whether or not those alerts make it immediately obvious to them that it's your fault... that depends on a bunch of factors.

[pyrale]

Data consumers in unknown teams is a nightmare regardless of the architecture.

Events sent for readership you can’t control are ideally of the type «x changed», and the consumer must then fetch data on the relevant endpoint.

That or the company must have serious versioning policies.

[mattacular]

What's the benefit of using an asynchronous event driven system if you can't process any of those events without performing a synchronous query back on the same provider for all of the necessary data?

[pyrale]

You get relevant notifications without polling or needing to sub, and you don’t have to be strict about message versioning.

[gnat]

Do you have pointers to such best practices? Gratefully received etc.

[trevor-e]

Design for things to be easily failable. It should be trivial to have failed messages to go to a DLQ and then reprocess them later on, say after a bug fix.

Only make additive changes, don't change existing fields. For enums it's up to the consumer to ensure they don't fail when a new case is added.

Be very careful with including data (especially time/expiry stuff) in the message too. If you need to reprocess the event several hours later then it may no longer work or be stale. Rather than include the data in the message itself, we would include the database ID and then have the consumer query for that entry.

[mvc]

Rich Hickey's talk about "growth" (as opposed to change) of software systems is a good one for this.

Tldr: ok to add things. Not ok to remove things or change things