[judofyr]

The gist of this article is good. Fixing the code is often the smallest part of rolling out a bug fix. That's extremely important to be aware of. Using historical data can be a great way of seeing how fast you can actually roll out changes.

But I don't get how they went from that to "estimations are bad". It seems to me that they're just doing better estimations now by looking at historical data?

> I could tell confusion from the look of my team lead. How could I say it would take over five days for a change that we estimate to take 10 minutes!

Isn't this like comparing apples and oranges? The estimation of 10 minutes is clearly "the time it takes to flip the flag", but what they actually want to estimate is "the time it takes to make it into production in a safe manner".

Maybe the team leader is confused because they thought this was a man-hour estimation? And they're now afraid that one developer will work full-time five days on it?

> On Monday, we started with the stand-up and discussed picking up this task. Another engineer proposed fixing a bug on another component first before turning on the flag.

Doesn't this imply that their 10 minute estimation was done completely without asking the team?

> As humans, it is impossible to consider all the potential complexity. […] We did not consider a legacy system with bugs and scaling problems. […] We did not consider a combined release process with a Release Manager.

Really? You couldn't consider the complexity of a release process you designed yourself? You honestly thought this was a 10 minute task even though you knew you were dealing with a legacy system?

[brightball]

Fwiw, there are some pretty prominent authors that suggest you are better off just counting the number of tasks in the queue and tracking the average time per task, rather than estimating at all.

It’s just as reliable since everything averages out over time and the queue size is a leading indicator while cycle time is a trailing indicator.

[geoelectric]

It also auto-calibrates to the typical granularity of your tasks, assume you decompose with similar concerns across the board.

[snorberhuis]

Thank you for your comment!

> But I don't get how they went from that to "estimations are bad". It seems to me that they're just doing better estimations now by looking at historical data?

For me personally, I went from estimations are just really poor even for the most simple things and not worth the effort.

It is correct that is can still be seen as migrating to a better form of estimation. If you look at the conference talk by Allen Holub[1]. He tries to make the distinction between estimation and prediction is that the latter is only data driven.

> Maybe the team leader is confused because they thought this was a man-hour estimation? And they're now afraid that one developer will work full-time five days on it?

That could be the case. I never looked at it that way.

> Doesn't this imply that their 10 minute estimation was done completely without asking the team?

Yes, it was done with just the team lead and me. We migrated away from Scrum at that point and we did not do full team refinement with estimations sessions pre-sprint. We would collaborate on the technical solution for any story when we would pick it. This could be full team to two engineers. The engineer that brought up the bug would probably bring up the bug.

> Really? You couldn't consider the complexity of a release process you designed yourself? You honestly thought this was a 10 minute task even though you knew you were dealing with a legacy system?

Yes, I honestly did! It was just a simple code change in our terraform code to set the config from false to true and cycle the machines. I should clarify this more in the story.

[1] https://www.youtube.com/watch?v=QVBlnCTu9Ms&t=1s

[judofyr]

And thanks for replying!

> It is correct that is can still be seen as migrating to a better form of estimation.

Well, from my perspective there is still estimation going here: A project manager came to you asking for a timeline for a feature and you replied with a concrete number of days. I don't think it makes any difference to the project manager whether you call it an "estimate" or "prediction".

> For me personally, I went from estimations are just really poor even for the most simple things and not worth the effort.

The example you're presenting isn't showing this because it doesn't seem like you made any actual effort estimating it. It would have been better if you showed that you spent hours talking with various people for input and then in the end the Cycle Lead was more precise. In this example you did an estimate without even talking to the team! (Yes, you talked to the team leader, but apparently they didn't know that there was performance issues with enabling the flag so there's some communication lacking here.)

If you wanted to properly estimate it you would at least (1) ask in the stand-up if there was any concerns about flipping the flag and (2) communicate with the release manager to see that it would have been possible to do a release. And it looks like if you actually did this then you would end up with an estimate that is closer to the actual time. It might also have uncovered that there was another release process happening at the same time.

Considering that the standard deviation is two weeks (!) it seems more that you were lucky in this example that there wasn't more unplanned work. What if the performance issue required three days of work instead of one? What if the release manager told you that they were upgrading database servers and couldn't do a release for two days?

> Yes, it was done with just the team lead and me. We migrated away from Scrum at that point and we did not do full team refinement with estimations sessions pre-sprint.

There's a middle way between "full team refinement with estimation" and "only team leader and me estimates". You can ask around and double check before giving out estimates to project managers.

[eternalban]

The approach OP described was new for me. What I got from the article was that a statistical tool allows for fairly accurate estimates without the need to "[spend] hours talking with various people for input". Further, it is not a certainty that an elaborate exploration of the proposed changes with various stakeholders will address potential blindspots or simply unexpected events (e.g. some infra burning). If OP is corrent, the statistical estimate has all these factors baked into it and will only get better over time.

This is the value of the approach, imo. Takes the 'subjective' element out of the picture. [p.s. And also, it 'scales' with project size, likely O(1). The 'ask everyone involved' approach is at best O(N)'.]

Your point regarding the standard deviation is quite fair but doesn't indicate a failure of the approach: the approach would be broken if they ever exceed the sd, so it is still a nice, definite, limit to the task completion timeline. Also I assume that as more data is gathered, the sd will shrink.

[SideburnsOfDoom]

> But I don't get how they went from that to "estimations are bad".

If "estimations" means the usual, i.e. asking "Ok everyone, how long do you think this task will take to complete?" then yes, that is largely a waste of everyone's time. Those estimations are bad.

> they're just doing better estimations now by looking at historical data

Yes, better estimates are coming from data. The people aren't "doing estimation" any more, it's being automatically calculated. Doing estimation takes time and give wildly wrong results.

[qznc]

The distinction between effort (man hours) and lead time gets lost often. Clueless managers ask for the first and take it for later.

[ser0]

This is why definition of done is quite important, to get everyone one the same page about what it means to complete a task.