[incidenrly]

Is it worth having psuedo production services that get chaosed monkeyed into another dimension at a random time and pager alerts on them.

Effectively: a drill!

[toast0]

I don't think chaos monkey works for incident drills. Anything the monkey can do is going to be easy to detect (probably).

You can do some amount of drills with periodic disaster recovery tests --- twice a year do a manual failover of a colo, etc.