Hacker News new | ask | show | jobs
by devuo 2748 days ago
At my company, specifically in my team we do on call, and:

1 - it's one week length

2 - it's paid extra

3 - it's optional, but you're a bit of a bad mate if you don't participate

4 - we try to have at least 6 people on rotation to ensure a full month between on call

Because we do several changes to production per day, our coverage is around > 99% for all our services and libraries (my team is responsible for about 30 of them). We have near zero live incidents, and whenever it does happen the phone rings, it ends up being just some unpredictable spike in load that self heals without intervention.

Because on call is not painful (as it shouldn't be!) and we support each other no one has any problem being on call.

4 comments

While what your company is doing is commendable (most don't pay extra or rotate in that fashion) #3 is a red flag for me because it sounds like the overly friendly but in the end passive aggressive and unprofessional atmosphere I've witnessed at startups and midsize companies who pretend they're startups. If on call is optional what's with the social penalty for people not wanting to do it.

IMO what companies should be doing is paying extra per hour until they get people that want to do it. As in, increase the price they pay "extra" until someone decides to give up their free time outside of normal development hours.

I agree on the preference that if it's not really optional, just don't make it optional.

On-call also varies a ton between companies. I was technically on-call all of the time in my last job, but it was a low throughput system. I had to be up at odd hours maybe once every 2-3 months. I slept pretty well. If you offered me free meals for the week, I wouldn't mind taking my turn on the watch regularly.

This job, I'm on call maybe one week every two months on a high throughput system, and even though it's only half of the day (we have an overseas team to take the night shift), it's generally acknowledged in the team that your sleep takes a hit and you get no real work done that week. If this were an optional part of my job, you'd have to pay me double for the week (basically a 10% raise).

evidently the pay must be adjusted to how painful the on call experience is.

as I said it is optional, no harm comes to you for not participating, and we have people with very good reasons for not helping the team support the code they themselves built and deployed themselves to production.

who said there is a social penalty? There are a number of reasons which I explained in a reply below.
> 3 - it's optional, but you're a bit of a bad mate if you don't participate

This seems to clearly state that you have a negative opinion of you for turning down extra work that is clearly undesirable. I assume that you're not the only one on your team that feels this way either, that's what I meant by social penalty.

Depending on who you ask it may not be a penalty but that would mean everyone on the team has to think this way. I personally don't think people should be considered a bad mate if they don't want to do an optional thing -- what if they value their time more than the pay+extra that's being offered?

I don't know if you've noticed, but time is the one thing you can never get back. Once you make a certain amount of money, you can live very comfortably -- priorities shift to things that you can't just buy, time is basically the most lucrative and rare yet abundant resource there is.

team effort is a thing, and we want everyone to pull their own weight. In my team we work at most 40h week and on call alerts are rare events, we work to keep it that way and we need solid team spirit to do that.

There are other companies to work at, and we make our expectations clear before the person is hired in relation to on call.

> If on call is optional what's with the social penalty for people not wanting to do it.

Because many optional activities have an impact on your peers and they are unlikely to judge you strictly based upon your job duties?

I think we don't have the same definition of optional -- like someone else has noted, maybe the better word was "flexible". The way you're using it is the super manipulative "yeah it's optional, but why would you want the rest of your team to suffer?". Does that not sound manipulative to you?

As far as your impact-on-peers argument -- you could "optionally" also stay 5 hours after when you normally go home to help reduce workload for your peers and help them, do you do that? No? What about 4 hours? what about 3? 2? Where is it fair to stop? The rest of the adult world calls this professionalism, and you stop at what's required of you as your job duty, put forth in your employment contract. In the course of fulfilling that duty you're expected to be reasonably courteous, not to subscribe to some weird hostage situation where the rest of your team suffers if you don't do something that was marked as optional.

> Does that not sound manipulative to you?

You seem to be assuming there is a lot of peer pressure placed on you if you don’t want to do it.

Why?

I’m simply saying there are always social costs. For example you probably won’t be listened to as much when there are conversations around improving system stability.

It’s like our after work Friday drinks are entirely optional - but lots of people build friendships and trust there and this can often lead to higher productivity.

If you can build these friendships another way or have a different path to an equivalently high productivity then not going doesn’t have an impact on you.

> For example you probably won’t be listened to as much when there are conversations around improving system stability.

That sounds like not listening to people about things they might be good at and know something about, because you want to punish them for something completely unrelated. Namely, punish them for not participating in "optional" activities. All the while you don't want to openly and transparently say what you expect from people.

Yes, it is manipulative and it is bad workplace.

> It’s like our after work Friday drinks are entirely optional - but lots of people build friendships and trust there and this can often lead to higher productivity.

It sounds sounds like nepotism where your ability to function and be promoted rests on your ability to make friends and be charming around beer.

No a meritocracy, but rather badly managed workplace.

-----------------

Seriously, you openly say that you would listen and judge system stability suggestions based on participation in supposedly optional activity unrelated to system stability. You also openly say that you trust people work based on Friday beer instead on how they act when working.

That sounds like horrible workplace for anyone who care about work and great workplace for charming bullshitters.

In all seriousness, you sound a little antisocial. I see where you’re coming from and I sympathize, but the environment described by the poster you’re replying to sounds very mildly manipulative at worst. I’m not sure you understand that the whole “bad mate” thing likely comes from his peers, not from management. Human beings are social animals, and you’ll be better off if you adapt to that reality rather than rail against it.
> would listen and judge system stability suggestions based on participation in supposedly optional activity unrelated to system stability.

In my experience they are closely related.

> You also openly say that you trust people work based on Friday beer

Sure - there is an incredible depth of research on trust building via outside of work/after work activities.

> That sounds like horrible workplace

Strange considering I work at companies regularly listed in “best companies to work for” surveys.

Why is it called optional if it really isn't?
it is optional, e.g if you have extra work activities that impede you, family reasons etc. and in these cases it's A-OK! Where it's not so OK it's when there's no reason other than not wanting to do just because you just don't want to be bothered.

This is a problem, because during the interviews the person was repeatedly told we did on call and he/she's ok with it.

For us, as a team, it's important because if you have the right to deploy to live whenever you want (after code review evidently), you have the obligation to keep it. When everyone shares the load, the load is lighter for everyone. And my experience tells me it just makes everyone much much more responsible and professional.

It sounds like you meant flexible, not optional.
If the interview makes sure that potential employees are okay with being on call, then there are no problems.

However, my employer has moral agency above me only insofar as I'm not committing a crime against them (ie fraud or embezzling company funds, etc). This does not include me not performing duties I'm paid for. If I don't do my work, then they don't pay me. This is a civil matter. This certainly doesn't include the reason why I'm deciding not to do optional work. My employer doesn't get to decide that I'm somehow a bad person because they don't agree with why I'm not doing extra non-required work.

Eventually, I'm going to be a corpse in the ground. I'm not missing out on my other life goals because you weren't satisfied with my priorities and it turns out that the money you were offering didn't help me accomplish what I want to accomplish.

> no one has any problem being on call.

How do you know this?

1:1s, meetings, general team feeling, retrospectives, amount of whining zero to none.
But do you keep track of how many people might not join because of the on-call? Or have exit interviews that check whether being on-call was a contributing factor?
If people don't join because of that, then I'd say that's a filter and I'm ok with that. In the Lisbon office of the company I work, on call was not a contributing factor for the people who have left. The vast majority was because they wanted to work in a different country, and not a problem with the company per se.
I think you should try to tune out the self healing spikes from your alerts as Alarm Fatigue is real and your mind gets programmed to treat it as yet another spike and either not taking it seriously or assuming that 'adding more resources' is going to be the solution instead of properly diagnosing the problem.
Beeping is a rare event in my team, might happen 1-3 times in a week, or there are weeks it doesn't beep at all, and we work to keep it that way.