[polskibus]

Is there a good tutorial on modeling messages in idempotent way that uses non trivial examples and real life cases? Whenever I talk with others working on such systems it turns out that many messages aren't and that there are compensating actions that repair/sync system components when needed/on demand.

[hosh]

This isn't a tutorial, but this paper has been blowing me away: http://www.neilconway.org/docs/socc2012_bloom_lattices.pdf

I bet analyzing those messages using this framework would reveal exactly why compensating actions are needed.

The takeaway I am getting from this paper is that, you're not just looking for idempotency. Monotonicity is important, as well as commutativity and associativity. If it cannot be expressed in that way, then coordination is required.