Hacker News new | ask | show | jobs
by ALittleLight 1652 days ago
I worked at Amazon. While my boss was on vacation I took over for him in the "Launch readiness" meeting for our team's component of our project. Basically, you go to this meeting with the big decision makers and business people once a week and tell them what your status is on deliverables. You are supposed to sum up your status as "Green/Yellow/Red" and then write (or update last week's document) to explain your status.

My boss had not given me any special directions here so I assumed I was supposed to do this honestly. I set our status as "Red" and then listed out what were, I felt, quite compelling reasons to think we were Red. The gist of it was that our velocity was negative. More work items were getting created and assigned to us than we closed, and we still had high priority items open from previous dates. There was zero chance, in my estimation, that we would meet our deadlines, so I called us Red.

This did not go over well. Everyone at the Launch Readiness meeting got mad at me for declaring Red. Our VP scolded me in front of the entire meeting and lectured me about how I could not unilaterally declare our team red. Her logic was, if our team was Red, that meant the entire project was Red, and I was in no position to make that call. Other managers at the meeting got mad at me too because they felt my call made them look bad. For the rest of my manager's absence I had to first check in with a different manager and show him my Launch Readiness status and get him to approve my update before I was allowed to show it to the rest of the group.

For the rest of the time that I went to Launch Readiness I was forbidden from declaring Red regardless of what our metrics said. Our team was Yellow or Green, period.

Naturally, we wound up being over a year late on the deadlines, because, despite what they compelled us to say in those meetings, we weren't actually getting the needed work done. Constant "schedule slips" and adjustments. Endless wasted time in meetings trying to rework schedules that would instantly get blown up again. Hugely frustrating. Still slightly bitter about it.

Anyway, I guess all this is to say that it doesn't surprise me that Amazon is bad about declaring Red, Yellow, or Green in other places too. Probably there is a guy in charge of updating those dashboards who is forbidden from changing them unless they get approval from some high level person and that person will categorically refuse regardless of the evidence because they want the indicators to be Green.

12 comments

I had a good chuckle reading your comment. This is not unique to Amazon. Unfortunately, status indicators are super political almost everywhere, precisely because they are what is being monitored as a proxy for the actual progress. I think your comment should be mandatory reading for any leader who is holding the kinds of meetings you describe and thinks they are getting an accurate picture of things.
no need for it to be mandatory, everyone is fully aware of the game and how to play it.
I worked at AMZN and this perfectly captures my experience there with those weekly reviews. I once set a project I was managing as "Red" and had multiple SDMs excoriate me for apparently "throwing them under the bus" even though we had missed multiple timelines and were essentially not going to deliver anything of quality on time. I don't miss this aspect of AMZN!
How dare you communicate a problem using the color system. It hurts feelings, and feelings are important here.
We have something similar at my big corp company. I think the issue is you went from Green to Red in a flip of a switch. A more normal project goes Green...raise a red flags...if red flags aren't resolved in the next week or two, go to yellow...In these meetings everyone collaborates ways to keep your green or get you back to green if you went yellow.

In essence - what you were saying is your boss lied the whole time, because how does one go from a presumed positive velocity to negative velocity in a week?

Additionally assuming you're a dev lead, it's a little surprising that this is your first meeting of this sorts. As dev lead, I didn't always attend them but my input is always sought on the status.

Sounds like you had a bad manager, and Amazon is filled with them.

Exactly this. If you take your team from green to red without raising flags and asking for help, you will be frowned upon. It’s like pulling the fire alarm at the smell of burning toats. It will piss off people.
This is not unique. The reason is simple.

1) If you keep status green for 5 years, while not delivering anything, the reality is the folks at the very top (who can come and go) just look at these colors and don't really get into the project UNLESS you say you are red :)

2) Within 1-2 years there is always going to be some excuse for WHY you are late (people changes, scope tweaks, new things to worry about, covid etc)

3) Finally you are 3 years late, but you are launching. Well, the launch overshadows the lateness. Ie, you were green, then you launched, that's all the VP really sees sometime.

This explicitly supports what most of us assume is going on. I wont be surprised if someone with a (un)vested interest will be along shortly to say that their experience is the opposite and that on their team, making people look bad by telling the truth is expected and praised.
I once had the inverse happen. I showed up as an architect at a pretty huge e-commerce shop. They had a project that had just kicked off and onboarded me to help with planning. They had estimated two months by total finger in the air guessing. I ran them through a sizing and velocity estimation and the result came back as 10 months. I explained this to management and they said "ok". We delivered in about 10 months. It was actually pretty sad that they just didn't care. Especially since we quintupled the budget and no one was counting.
A punitive culture of "accountability" naturally leads to finger pointing and evasion.
I worked at an Amazon air-shipping warehouse for a couple years, and hearing this confirms my suspicions about the management there. Lower management (supervisors, people actually in the building) were very aware of problems, but the people who ran the building lived out of state, so they only actually went to the building on very rare occasions.

Equipment was constantly breaking down, in ways that ranged from inconvenient to potentially dangerous. Seemingly basic design decisions, like the shape of chutes, were screwed up in mind-boggling ways (they put a right-angle corner partway down each chute, which caused packages to get stuck in the chutes constantly). We were short on equipment almost every day; things like poles to help us un-jam packages were in short supply, even though we could move hundreds of thousands of packages a day. On top of all this, the facility opened with half its sorting equipment, and despite promises that we'd be able to add the rest of the equipment in the summer, during Amazon's slow season...it took them two years to even get started.

And all the while, they demanded ever-increasing package quotas. At first, 120,000 packages/day was enough to raise eyebrows--we broke records on a daily basis in our first holiday rush--but then, they started wanting 200,000, then 400,000. Eventually it came out that the building wouldn't even be breaking even until it hit something like 500,000.

As we scaled up, things got even worse. None of the improvements that workers suggested to management were used, to my knowledge, even simple things like adding an indicator light to freight elevators.

Meanwhile, it eventually became clear that there wasn't enough space to store cargo containers in the building. 737s and the like store packages mostly in these giant curved cargo containers, and we needed them to be locked in place while working around/in them...except that, surprise, the people planning the building hadn't planned any holding areas for containers that weren't in use! We ended up sticking them in the middle of the work area.

Which pissed off the upper management when they visited. Their decision? Stop doing it. Are we getting more storage space for the cans? No. Are we getting more workers on the airplane ramp so we can put these cans outside faster? No. But we're not allowed to store those cans in the middle of the work area anymore, even if there aren't any open stations with working locks. Oh, by the way, the locking mechanisms that hold the cans in place started to break down, and to my knowledge they never actually fixed any of the locks. (A guy from their safety team claims they've fixed like 80 or 90 of the stations since the building opened, but none of the broken locks I've seen were fixed in the 2 years I worked there.)

The problem here sounds like lack of clarity over the meaning of the colours.

In organisations with 100s of in-flight projects, it’s understandable that red is reserved for projects that are causing extremely serious issues right now. Otherwise, so many projects would be red, that you’d need a new colour.

I'd be willing to believe they had some elite high level reason to schedule things this way if I thought they were good at scheduling. In my ~10 years there I never saw a major project go even close to schedule.

I think it's more like the planning people get rewarded for creating plans that look good and it doesn't bother them if the plans are unrealistic. Then, levels of middle management don't want to make themselves look bad by saying they're behind. And, ultimately, everyone figures they can play a kind of schedule-chicken where everyone says they're green or yellow until the last possible second, hoping that another group will raise a flag first and give you all more time while you can pretend you didn't need it.

How about orange? Didn't know there was a color shortage these days.
but it's amz's color. it should carry a positive meaning #sacarsm
> While my boss was on vacation I took over for him in the "Launch readiness" meeting....once a week

Jeez, how many meetings did you go to, and how long was this person's vacation? I'm jelly of being allowed to take that much time off continuously.

You might be working at the wrong org? My colleagues routinely take weeks off at a time, sometimes more than a month to travel Europe, go scuba diving in French Polynesia, etc. Work to live, don’t live to work.
your story reminded me of the Challenger disaster and the "see no evil" bureaucratic shenanigans about the O-rings failing to seal in cold weather.

"How dare you threaten our launch readiness go/no-go?!"

Was Challenger the one where they buried the issue in a hundred-slide-long PowerPoint? Or was that the other shuttle?
Was this Amazon or AWS?
Based on the other comments in this comment thread I would say it's Amazon.