Hacker News new | ask | show | jobs
by paxys 1885 days ago
I have also worked at Google (in an unrelated department) and completely disagree with this. Maintaining old code & services is a problem everywhere. Monorepo vs multirepo, monolith service vs microservices etc. all have nothing to do with it. There will always be a broken dependency, a service/API/library you rely on about to deprecate, new urgent security patches, an outage somewhere upstream or downstream which you have to investigate, an important customer hitting an edge case which was hidden for years. You will always need a dedicated team to support a live product regardless of how it is engineered.

The problem at Google was (and maybe still is) with lack of incentives at the product level to do any of this. You don't get a fat bonus and promotion for saying that you kept things working as they should, made an incremental update or fixed bugs. When your packet goes up to the committee (who don't know you and know nothing about your team or background), the only thing that works in your favor is a successful new product launch.

And as an engineer you still have multiple avenues to showcase your skills. That new product manager you just hired from Harvard Business School who is eager to climb the ladder does not. And due to the lack of a central cohesive product strategy, this PM also has complete control of your team's annual roadmap.

5 comments

The "you must make something new to get promoted" was a common meme (literally) at Google, but I never saw that myself. I got promoted, and I sat on promotion committees, and it didn't seem that important. I did sort of start a new project to get from 4 to 5 (rather a prototype was handed to me by my more senior team members), but it was clear to me that the path to 6 was not starting a new project -- it was increasing the reach and value of my existing project. I left before that happened (Google Fiber got canceled, found a new team but it wasn't really my thing), so I'll never know for sure, but I didn't feel any pressure to make something new for the sake of making something new. There was, of course, pressure to make the thing that you did work on good, and nobody was really going to stop you from making something new.

Basically, that whole eng ladder thing is really important. I looked at that a lot for my own promotions and for evaluating candidates for promotions. Just dealing with churn isn't really on there, so it's probably not something you should focus too much on. I'd say that's true at any job; customers aren't going to purchase your SaaS because you upgraded from Postgres 12 to 13. They give zero fucks about things like that. You do upgrades like that because they're just something you have to do to make actual progress on your project. Maybe unfortunate, but also unavoidable. Finding a balance is the key, as with anything in engineering.

The biggest problem I found with promotions is that people wanted one because they thought they were doing their current job well. That isn't promotion, that's calibration, and doing well in calibration certainly opens up good raise / bonus options. Promotion is something different -- it's interviewing for a brand new job, by proving you're already doing that job. Whether or not that's fair is debatable, but the model does make a lot of sense to me.

Things could have changed; I haven't worked at Google for 4 years. But this was a common complaint back then, and it just wasn't my experience in actually evaluating candidates for promotion.

"The biggest problem I found with promotions is that people wanted one because they thought they were doing their current job well. That isn't promotion, that's calibration, and doing well in calibration certainly opens up good raise / bonus options. Promotion is something different -- it's interviewing for a brand new job, by proving you're already doing that job. Whether or not that's fair is debatable, but the model does make a lot of sense to me."

Thanks for articulating this distinction so clearly; it's a simple enough idea, but it seems to elude so many.

> Promotion is something different -- it's interviewing for a brand new job, by proving you're already doing that job.

Every large corporation has a concept of levels. It makes sense to use levels as a progression (they are numeric after all) rather than a new job each time. That’s what job titles/roles are for.

I’m not convinced by this summary, and it seems anecdotal rather than realistic.

The reason things like this eludes so many people is because it never gets properly explained by anyone. What jrockway explained might be simple, but it is quite rare to see such an explanation.
With respect to your experience, the impact of promotion chasing was heavily felt by product teams and I wouldn't expect it to be that visible to people on the promo committees. I watched multiple fellow Googlers rush project work and cut corners in order to be able to "ship" and put the project in their promo package (and be frustrated when they missed promo). In some cases I got to watch them abandon the project and move on to something else even though it badly needed additional maintenance and cleanup due to all the corner-cutting. In one specific case, all the corner cutting led to multiple significant exploit chains (one of them delivered persistent root on Chromebooks)

Maybe it was just way worse in my org (Chrome).

> I watched multiple fellow Googlers rush project work and cut corners in order to be able to "ship" and put the project in their promo package (and be frustrated when they missed promo)

This kind of confirms my point -- the committee isn't looking for "created a disaster area a month before promo packets were due". They want a consistent track record of success at the next level.

I definitely encountered this problem at Google (there was a reason it was a meme), but it was far more prevalent at the EM/PM/director level, and so still directly affected the overall product strategy for the org and what you as an IC got to work on.
>Promotion is something different -- it's interviewing for a brand new job, by proving you're already doing that job

I've worked a couple places where getting a "meets expectations" on your annual review was expected

Their review processes were calibrated such that you should [almost] never get a 5 ("always exceeds")

A handful of 4s ("sometimes exceeds") was good - but not a requirement ('too many' 4s indicated you were in the wrong role, so titles/pay/etc would be adjusted)

More than one 2 ("sometimes doesn't meet") was reason for extra mentoring, one-on-ones, etc

There were no 1s ("fails to meet") - if you would otherwise have earned a 1 in any category, you'd've been let go already

I think monorepo makes it easier to update downstream dependencies atomically as part of an upstream change, and thus encourages a culture of unstable APIs.

That is, careful evolution of internal APIs is not given much weight, so modularity - in the sense of containing change - suffers.

I don't think monorepos must necessarily go this way, but expressing dependencies in terms of build targets rather than versioned artifacts has a strong gravitational effect. Change flows through the codebase quickly. That has upsides - killing off legacy dependencies more quickly - and downsides, wanting to delete code that isn't pulling its weight because of the labour it is inducing.

[I currently work at Google but I've only been here a few weeks. I certainly don't speak for the company.]

I think this is absolutely it. A lot of best practices go down the drain when the compiler compiles the bytecode. As such a lot of best practices are about people, not the computer. APIs can be just as stable behind a network or a library, but way more people are onboard with never break your APIs than they are never break your function.
Concerning the promotion thing, I hear this a lot from Googlers but isn't it the same everywhere else? Most tech companies (big tech at least) will promote on product achievements, not maintenance.
The main difference was/is that at Google your immediate manager, director, PM, peers and everyone else in your product unit (who you work with every day) have almost zero say in whether you get promoted or not. You have to essentially summarize everything you did in bullet points and send it over to an anonymous committee who don't know who you are. They will base their decision on this piece of paper without any additional background or context.

This does help in various ways – the process is more objective, there is less bias, less departmental/managerial politics etc. The drawback is that a lot gets lost in translation. There is too much burden on you as an engineer to pick and choose what you spend your time on so it looks good to the committee.

In other companies I have worked at getting promoted was a byproduct of doing a good job. At Google getting promoted is the job.

There have been some changes that make this entirely untrue for earlier promos and partially untrue for promotions to L6/Staff. There's considerably more locality at this point.
That’s not true anymore until you are going for 6/7 (depending on your org). Now, committees are based in your org and members are expected to be somewhat familiar with your work.
Sounds highly reminiscent of the Military's way of handing out promotions.
This sounds like any large organization, where each engineer is only one tooth on a very large gear.

I'm guessing, but do not have enough anecdotal experience myself, that just about any large tech company employee here is reading your description and thinking "sounds like my company."

I'm curious how sound my hypothesis/guess is. Can other large organization employees answer with a claim that this does NOT describe their situation?

This makes sense. I do not work at Google, we do not have monorepo, and yet many problems feel similar.

The guys who maintain the company infrastructure introduce some changes, send an e-mail notification, and call it a day. The maintenance you need to do at your existing project to keep up with these changes does not count as important work, because it is not adding new features. Therefore it is important to run away from the project as soon as it stops being actively developed.