[mratzloff]

I'll share another perspective.

Execs view it as being able to make informed decisions about where to spend the company's time and money.

Let's say you have projects A and B you want to do, and you've estimated revenues from each at $11 MM and $4 MM. You really want to do A, since it would establish a business relationship with a new strategic partner, but estimates come back that A will take 16 weeks and B will take 4 weeks. You only have time to focus on one project at a time, so you choose B to try to capture a quick turnaround before moving onto A. Plus it would make an existing partner happy.

Execs know from experience that the engineering estimates usually suck. Sometimes they're really high and sometimes they're really low. Execs decide as long as B doesn't exceed 5 weeks, we're OK. They don't communicate this down the chain because if everyone knows 5 weeks is the "real" number they'll (grudgingly) accept, they are pretty sure someone somewhere in the chain will spend a week optimizing the test harness or refactoring the build process or something.

At week 3, engineering says they're gonna need 2 more weeks. Execs: Sure.

At week 4, engineering says they're gonna need 2 more weeks again. Execs are annoyed but now the work is mostly done, and everyone promises on their children's lives it will actually be done in 6 weeks, so they let it finish.

At week 5, we're still on track for week 6. Execs: Sure.

At week 6, we're gonna need another week. Execs: Whatever. At this point they've abandoned all faith in this engineering team. They re-task some other critical part of the business with beginning work on project A. Some part of the business suffers.

B actually takes 7 weeks when all is said and done. Execs wouldn't have bothered with it if they knew that up front. Everyone involved in B takes a reputation hit: the product manager, the project manager, the engineering manager, and the engineers themselves. The people who blame others get an even bigger reputation hit.

Now there's pressure from some sides of the exec table to make up time on project A and get the estimate below 16 weeks from a team that had nothing to do with B.

Happily for everyone but the first team, it turns out the other team is able to get project A out the door in just 12 weeks. Everyone on the first team takes another reputation hit.

You might say that the root problem was the estimate for B was too low. That is a problem, but the fact that the estimate for A was 25% is equally problematic. Had either estimate been more accurate, the company would have focuses on A first.

All of these execs are accountable to the CEO, who's accountable to the board. The CPO or CTO or whoever are not escaping unscathed from this, because at the next board meeting the CEO has to answer for why project A was delayed into Q2.

[valuearb]

We are just finishing a long refactor that strongly improved the product. But the planning process was poorly done. They took a couple of lead engineers, made them draw a bunch of diagrams we didn't need, and made them estimate all the work time, instead of involving the entire team. And all of the commitments in the schedule were not commmunicated to the entire team till the end.

From the exec's standpoint, it looks like we can't be trusted. But the truth is the process can't be trusted. Never give out time estimates. Use Agile and point planning, and relative cost estimates.

In your example, the team should have said that we estimate A is 4x as much work as B. Pick one, we'll get it done as fast as possible.

You can have a fixed time schedule, or a fixed project feature set. You can't have both, without the project being late, and features being compromised.

[mratzloff]

You described two problems.

You identified the first one correctly: they had the lead engineers estimate in isolation and not involve the team in any way until the end. Beyond being a bad way to estimate, it's a bad way to manage a team.

But then you said you should never provide a real estimate. That conclusion doesn't really follow. The leadership team has to come up with real estimates. "A is 4x as much work as B" is good information, but it's pretty useless when it comes to actual calendar planning.

The second issue, and the larger one, is that long, stop-the-world refactors (or rewrites) are the fastest way for the engineering team to lose all credibility in my experience. The business keeps moving even if new feature development doesn't. I've found incremental refactors to be the best way to go. And those are much easier to estimate.

As far as schedule vs. feature set, if you estimate properly and prioritize all of the least-needed features at the end, if you find yourself falling behind you just pop those tasks off. But you build that contingency in and communicate it up front.

[valuearb]

You contradict yourself st the end. If the organization can accept lower value features being deferred at the end, you never need a time estimate. Just a finish date.

In our case bogging down the entire team in time estimates would have wasted even more time and we would have accomplished even less. Had we kept to an agile process with sprints we could have met the fixed schedule expected, and delivered more benefits given that losing our leads in planning for weeks on end meant they contributed relatively little to the refactor.

[mratzloff]

Wow! If your planning is taking weeks, something has gone seriously wrong.

What I said was no contradiction. How does one arrive at the finish date? Finger in the air? There are essential features and nice-to-haves, and to get a delivery date you need to estimate them. But it's an estimate and so you plan for contingencies.

[valuearb]

Yes, something went seriously wrong. When it became clear we had no idea how much longer it would take (because the app wasn't in a fully testable state so our bug counts were meaningless), we got asked to spend half a day re-estimating task and bug times. Which was time not spent fixing bugs or making the product testable, or even making the schedule predictable, because fixing blocker bugs for testing was the only way to do it.

At the end they required twice a day reporting meetings, the standup to say what you were doing, and an end of day status meeting to say what you did so it could be communicated to the exacs. Two more meetings where zero development gets done.

I am not really disagreeing with you. It's reasonable to do basic estimation in most projects. Usually marketing/management need to know when something can ship in order to plan the launch plan. So you figure out how long you need to be really confident the essentials will be finished, and use the nice-to-haves as padding that can be deferred if necessary to make the date.

But if you don't have to hit a specific date, say the product is already shipping and just needs regular constant improvement, just do it in fixed schedule sprints. Focus on the highest priorities every sprint, get as many done as possible during it, ship the new release, rinse and repeat. A well run sprint can be great because eschewing time estimates means more time spent actually building the product.

[mratzloff]

Yeah, I think we're on the same page. That's the approach I follow.

[roel_v]

What is the core argument you're making here? I mean sure, I too can argue any point by making up examples about abstract projects A and B, but I'm not seeing what fundamental dynamic you're trying to illustrate here.

[mratzloff]

There's no argument; as I said, I'm sharing a perspective. Most engineers, and you can see it in the comments here, approach estimation as an us vs. them battle of wills. The perception battle that they deal with on a regular basis isn't going to improve that way.