[akeefer]

There are two serious problems with this post, and it really saddens me that I see these sorts of posts so frequently here, with so many concurring voices.

First of all, cost absolutely 100% has to factor into prioritization decisions. That doesn't require absolute estimation, but it does demand relative estimation (which he mentions tangentially at the end of the post). If Feature A will deliver $100,000 in revenue but take 18 months and Feature B will deliver $10,000 in revenue but take 1 week, the choice is pretty obvious. What matters is never "return" but always "return on investment." If you don't know anything about the I side of the ROI equation, you're doomed to make bad decisions. With no estimate at all, and just a snarky "it'll take as long as it takes, shut up and let me work" response, you'll inevitably focus on the wrong things.

Secondly, many of us do in fact have deadlines, and they're not just total BS. If you have a customer that you're building something for, they have to be able to plan their own schedule, and just telling them "well, maybe we'll ship it to you in 10/2012 or maybe in 6/2013, you'll get it when you get it" doesn't fly. And it's totally reasonable that it doesn't fly: if they have to, say, install the software, or buy hardware to run it on, or train users, or migrate data from an old system, or roll out new products of their own that are dependent on the new system, they clearly can't plan or budget those activities if they have no clue whatsoever when they'll get the new system.

And if you do have a deadline, you kind of want to know, to the extent that you can, if you're going to make it or not so you can take corrective action (adding people, cutting scope, pushing the date) if need be. You can't do that if you have no estimates at all.

Relative estimation of tasks with empirical measurement of the team's velocity works pretty well; it doesn't work in all cases, but it's pretty much always better than absolutely nothing.

There's a huge, huge difference between doing relative point-based estimation and date-driven, pointy-haired-boss estimation, and it's a total disservice to the software community that so many engineers seem to not really understand that difference, and seem to think that the only two options are "unrealistic date-based estimates" and "no estimates."

TL;DR - Don't just rant for 3000 words about how estimation is horrible and then add in one sentence about relative estimation. You'll do the world much more good if you just help educate people how to do things the right way and spare the ranting.

[jacques_chester]

> There's a huge, huge difference between doing relative point-based estimation and date-driven, pointy-haired-boss estimation, and it's a total disservice to the software community that so many engineers seem to not really understand that difference, and seem to think that the only two options are "unrealistic date-based estimates" and "no estimates."

This was my beef with the article too. Basically on the one hand he proposes a strawman composed of known-worst practices (estimate-by-deadline, estimate-by-gut, ad hoc estimation and so on) and thereby tars all estimation with the brush ... except for the one alternative he approves.

This is the fallacy of dichotomy.

The root problem is the concept that estimates have to be accurate. Well, duh, they can't be. The bigger the project, the more people, the longer the timeframe, the less likely the project is to meet the estimate.

That's why you don't perform one estimate.

That's why you have confidence bands on estimates.

The whole blog article feels like a pastiche of criticism cribbed from agile books and not from a direct, thoughtful engagement with the primary or secondary literature on software project estimation.

I'm only 31. By any measure I'm still a young man. Why do I feel like such a curmudgeon all the time? Because apparently nobody reads books or papers any more. It's all blogs citing blogs who echoed the summary of the notes of the review of a single book.

One more thing. There's a difference between a plan and an estimate. Plan-and-control is not the same thing as performing an estimate; DeMarco's self-criticism is not directly applicable.

[jwhite]

I agree. What would you recommend as a good modern book on software estimation?

[jacques_chester]

As usual, Steve McConnell has done the hard yards of turning literature and research into something readable and instantly applicable.

http://www.amazon.com/Software-Estimation-Demystifying-Pract...

Every time I estimate for clients I always talk about the Cone of Uncertainty.

[reedlaw]

This is why I agree with the original article over these comment parents. I work in a small software agency for clients who would never grasp the Cone of Uncertainty. They are much closer to the pointy-haired boss type than the type of person who appreciates the finer points of software project estimation. While reading the literature is good, the average developer will seldom find the time to do so. And even if they do, an off-the-cuff estimate is often better than carefully planned specification documents that no business stakeholders will ever read.

Of course accurate estimates have tremendous business value. But in reality they often come at the expense of what the client really needs which is delivery of features. I have seen estimation and tight project control taking up substantially more time than delivering actual features. And it was exactly as the OP stated:

> Software projects that require tight cost control are the ones delivering little or no marginal value.

The lesser the project value the tighter the control leading to a vicious cycle of developers cutting corners and increased micro-management.

[jacques_chester]

> I have seen estimation and tight project control taking up substantially more time than delivering actual features.

It sounds to me that what you saw was a conflation of estimates and plans. Which is a common error.

[reedlaw]

Clients sometimes want an estimate of how long a single feature or fix will take, even when it will take only 15 minutes. The communication overhead and time spent estimating easily outweigh the time to implement.

[jwhite]

Thanks!

[akeefer]

I haven't read it myself, but http://www.amazon.com/Agile-Estimating-Planning-Mike-Cohn/dp... looks a good description of the story points/relative estimation techniques. They're really not something that should require a whole book to explain, but I can't say I've found any one blog post or article-length writeup that does a good job at it. The summary at http://epf.eclipse.org/wikis/openup/core.mgmt.common.extend_... is pretty good (though I'd ignore the bottom section on "Estimation of Effort"), and the wikipedia article on Planning Poker http://en.wikipedia.org/wiki/Planning_poker is a decent writeup as well.

It's unfortunate that so much of the literature on relative estimation/story points/velocity/planning poker ends up intertwined in agile-development-consultantese, so sometimes reading some of these things, you have to take it with a serious grain of salt and weed out all the BS and dogma to get to the useful and important bits. The important bits there are really pretty simple: * Estimate in terms of points, not days, and estimate tasks relative to one another * Use planning poker (or something like it) within the team to get more accurate estimates * Empirically measure how many points you do in a given period of time to come up with your "velocity". To do that, you have to be pretty honest about things being "done done" before you move on to the next task; otherwise it's too easy to fudge and your numbers are worthless. "Oh, yeah, we did 10 points of work, well, except for writing the tests or doing the styling . . ."

Remember that velocity is NOT a productivity measure, it'll change over time, and it'll change if the team members change or if other external factors intervene, like an increase in support requests or something. So this technique kind of only works if your organization isn't already dysfunctional: as soon as velocity is seen as a productivity measurement, you're pretty screwed.

That's pretty much it. The relative estimates let you prioritize work appropriately (i.e. "I'd rather have these five 1-point stories than this one 5-point story, so we'll do those first"), and the velocity lets you track how fast you're actually going and about when you'll be done with a given chunk of work, so you can adjust plans as needed.

Note that relative estimation doesn't work nearly so well for large-scale estimation, or for greenfield development where you don't know what you're doing yet. For large-scale planning, my company will generally just give a SWAG in terms of points (we think this feature is 100 points) to give us at least some initial idea of expected relative sizes of things, then we'll compare that initial SWAG to the actual points as we break things out into more bite-sized stories that we can estimate more accurately. If we feel like we're half way through the feature and we've already done 70 points of work, that's a signal that we need to up our estimate for the remainder of the work. Steve McConnell's book is good as well, though honestly we don't really do much in the way of confidence-based estimates at this point. My experience is that out of every 10 projects we do, 8 or 9 will be within 20% of our initial SWAG and 1 or 2 will blow out to 2 or 3x the initial estimate. Of course, we never know which ones will blow out, we just know something will. In other words, we don't bother with confidence intervals at the individual feature level, we just do it at the macro level. So if a team has a velocity of 10 and we have 26 weeks to deliver a release, giving us a hypothetical total capacity of 260 points, we'll "commit" to maybe 2/3 of that. So we say "Okay, this first 170 points we're pretty sure we can get done. Anything after that will be a bonus."

[jacques_chester]

I thought Cohn's book was good, but a bit padded out.

Basically all that agile really changed is how often folks take a sounding. Velocity is basically a productivity metric, it's a first derivative of project size (hence ... velocity).

But I mean you could always do that with traditional estimates.

That's pretty much what an Earned Value chart does.

Don't get me wrong: agile is an improvement. But it is an improvement that is evolved from what came before; not a categorical change. Looking at the historical literature still pays dividends to thoughtful engineers.

[arohner]

> What matters is never "return" but always "return on investment."

You're right. This is one of the primary advantages of startups with technical founders that talk to customers. Among many other flaws of the "OfficeSpace" work environment, one of the biggest is the fact that no single person in the organization knows 1) what is possible to build 2) how much it will cost to make and 3) what is the value of product to the customers.

At my last corporate job before I founded a startup, the CEO didn't know which software was possible to write, and didn't know which new features were easy vs. hard. The marketing people rarely told engineers what the customer wanted, and the engineers didn't talk to customers, and didn't know the cost (or value) of their time.

If there are any well-tested rules for producing good software, they're 1) small teams, 2) technical people with authority and responsibility, 3) talk to the customer.

[yason]

But it's already established that deadlines are moot. So giving a deadline for planning, training, and hardware merely reduces to a random guess.

They'd be better off telling the client the truth that "We don't know a honest date and we don't want to lie and come up with an arbitrary one. I can only say it's unlikely that the project would take more than six months and you should at least be prepared to order the hardware by the early summer. We'll keep you posted."

Emphasis on the last part.

Schedules are more accurate later on when some work has been done already. Instead of creating an arbitrary point in the future keep the customer posted with: "We've already finished X and Y in the first month, so we'll soon start working on Z which is a big task but on the other hand we noticed that we don't have to do Q at all since we can just build it in terms of X and Z combined which won't take more than a week instead of the month that was originally planned."

If you tell this every week then everybody has an up-to-date view of the runway ahead. You can also give a "deadline", like: "If you need a deadline for higher-ups or administrative planning, use October 31st but tell them to prepare to plunge in earlier. We all know we'll be done then by a far margin."

Most customers that I've ever had have intuitively understood this is how it works, even if they had to have a "deadline". YMMV.

[jbrechtel]

Are you agreeing with the premise of the article? (It's not clear if you are or if you're just disagreeing with the GP).

If you are then... "We'll keep you posted." "up-to-date view of the runway ahead."

Where do these things come from if you're not estimating effort?

[iridium]

Estimation becomes a lot easier a few weeks into any project. Once you have the framework and an overall story down, and you start getting your hands dirty with a few classes, the estimation numbers start to have more confidence and weight.

The only thing you can really confirm though, is what is already completed, and that is what you should keep the stakeholders posted on. And a description of work left, not necessarily the time it will take.

[jbrechtel]

I don't understand, why not just be fully transparent?

This is the work left, we think it's this size*, this is how much work we've been getting done. It looks like we might be done at this date but that's dependent on a) our estimation being correct and b) our pace continuing.

I think it's more about managing expectations than it is about managing the information you let stakeholders have.

[five18pm]

You took the words out of me.

Even if there are no external customers waiting for the release, a business' other departments have to plan their activities. When will marketing start their pre-release activities without engineering's estimates? When will sales start talking to customers about the new release? There is just too many things that need engineering's estimates.