[simskij]

Author of the post here. I definitely agree with that assessment. Having multiple stacks monitor each other is a really good, albeit somewhat resource intensive, way to cover all of the *likely* cases. The cost of covering that last percent very quickly approaches the point of diminishing returns.

[ethbr0]

You start towards it in your post, but don't explicitly call it out: the central task is decreasing complexity.

A "dumb" heartbeat message, and a check for its absence, is reliable precisely because of its simplicity.

So if the question is "How do I monitor my complicated observability stack?" then the answer, to me, starts with "What is the simplest method of accomplishing the minimal version of my goals?"