[valuearb]

We are just finishing a long refactor that strongly improved the product. But the planning process was poorly done. They took a couple of lead engineers, made them draw a bunch of diagrams we didn't need, and made them estimate all the work time, instead of involving the entire team. And all of the commitments in the schedule were not commmunicated to the entire team till the end.

From the exec's standpoint, it looks like we can't be trusted. But the truth is the process can't be trusted. Never give out time estimates. Use Agile and point planning, and relative cost estimates.

In your example, the team should have said that we estimate A is 4x as much work as B. Pick one, we'll get it done as fast as possible.

You can have a fixed time schedule, or a fixed project feature set. You can't have both, without the project being late, and features being compromised.

[mratzloff]

You described two problems.

You identified the first one correctly: they had the lead engineers estimate in isolation and not involve the team in any way until the end. Beyond being a bad way to estimate, it's a bad way to manage a team.

But then you said you should never provide a real estimate. That conclusion doesn't really follow. The leadership team has to come up with real estimates. "A is 4x as much work as B" is good information, but it's pretty useless when it comes to actual calendar planning.

The second issue, and the larger one, is that long, stop-the-world refactors (or rewrites) are the fastest way for the engineering team to lose all credibility in my experience. The business keeps moving even if new feature development doesn't. I've found incremental refactors to be the best way to go. And those are much easier to estimate.

As far as schedule vs. feature set, if you estimate properly and prioritize all of the least-needed features at the end, if you find yourself falling behind you just pop those tasks off. But you build that contingency in and communicate it up front.

[valuearb]

You contradict yourself st the end. If the organization can accept lower value features being deferred at the end, you never need a time estimate. Just a finish date.

In our case bogging down the entire team in time estimates would have wasted even more time and we would have accomplished even less. Had we kept to an agile process with sprints we could have met the fixed schedule expected, and delivered more benefits given that losing our leads in planning for weeks on end meant they contributed relatively little to the refactor.

[mratzloff]

Wow! If your planning is taking weeks, something has gone seriously wrong.

What I said was no contradiction. How does one arrive at the finish date? Finger in the air? There are essential features and nice-to-haves, and to get a delivery date you need to estimate them. But it's an estimate and so you plan for contingencies.

[valuearb]

Yes, something went seriously wrong. When it became clear we had no idea how much longer it would take (because the app wasn't in a fully testable state so our bug counts were meaningless), we got asked to spend half a day re-estimating task and bug times. Which was time not spent fixing bugs or making the product testable, or even making the schedule predictable, because fixing blocker bugs for testing was the only way to do it.

At the end they required twice a day reporting meetings, the standup to say what you were doing, and an end of day status meeting to say what you did so it could be communicated to the exacs. Two more meetings where zero development gets done.

I am not really disagreeing with you. It's reasonable to do basic estimation in most projects. Usually marketing/management need to know when something can ship in order to plan the launch plan. So you figure out how long you need to be really confident the essentials will be finished, and use the nice-to-haves as padding that can be deferred if necessary to make the date.

But if you don't have to hit a specific date, say the product is already shipping and just needs regular constant improvement, just do it in fixed schedule sprints. Focus on the highest priorities every sprint, get as many done as possible during it, ship the new release, rinse and repeat. A well run sprint can be great because eschewing time estimates means more time spent actually building the product.

[mratzloff]

Yeah, I think we're on the same page. That's the approach I follow.