[Jolter]

This is how my team used to work when I was on call in telecoms a decade ago. In the right engineering culture, and with management buy-in, it works really well.

We deployed a new system and had one week on call for each of five team members. The first couple of rotations were hell. Almost every night ended up with at least one wake up call. As we learned how to solve each type of outage, we then taught the first-line staff how to reboot the right components so we didn’t get as many wake-ups, while we spent our days fixing the bugs. And eventually the system stopped crashing.

The on-call pay was really good (nearly double for that week) and it was a pretty sweet reward to be able to rake that in as calls stopped coming. We broke out a bottle of champagne when the first week of no calls had passed.

Eventually on-call was cancelled.

Imagine how this story would have ended if management had incentivized us differently, for example if you only got the extra pay for the nights where you got pages.