[not_kurt_godel]

Yup, the key point is they let you treat complex systems as a blackbox that either does or does not successfully support the end-user functionality/experience. Ideally the components of the blackbox are sufficiently self-regulating that the end-to-end functionality is never interrupted, but complex systems by their nature exhibit emergent behavior that may not register as anomalous at a per-component level.

Totally agree the nuances regarding levels of coverage between various end-to-end testing mechanisms is an interesting subject. Generally speaking I think it's preferable for canaries to essentially be continuously-run integ tests, but this is not universally applicable. Deep integ tests may be too resource-intensive to run continuously. Integ tests also aren't always ideal for monitoring behavior of long-lived resources as they typically start from a blank slate, create some new resources, test their functionality, and then clean them up. Canaries may be geared towards validating state that persists indefinitely by design.