[wai1234]

I have to laugh when I see these posts. So, software development is a special activity that should not be constrained by budgets or deadlines (or estimates)? Where do I sign up? There are so many strawmen here I can't possibly address them all. If you can't provide a credible estimate for a piece of work, you either a) aren't very good, b) don't properly understand the work to be done, or c) perceive some aspect of the job that is unknowable to you (through lack of experience or other novelty). Fixing each of these is up to you. 'a' is obvious (you either work hard to get better or find something else to do). 'b' means you need to ask more questions that allow you to sufficiently complete your understanding of the problem so an estimate is possible. 'c' means you need to isolate the novelty and explain that this part of the problem is new to you and you are not qualified to provide an estimate. Perhaps someone else can provide that insight or you can define an initial activity designed to help you better understand the novel problem until you can estimate the remaining work. NO area of engineering is so cut and dried that estimation is easy.

What IS common is a combination of poorly defined projects and a profound lack of self-awareness. By all means, push back against poorly defined projects ("I can't estimate the work until you tell me what it is."). Lack of self-awareness is squarely in YOUR lap ("I don't understand why it always takes longer than I thought it would..." Well, duh, maybe you should consider that reality for what it is and account for it next time).

Estimating work and committing to those estimates is what a professional does. Deal with it. If estimates are imposed on you that you do not agree with, that has nothing to do with the virtue of estimation. That's a social problem, not a technical one.

[_xnmw]

"So, software development is a special activity that should not be constrained by budgets or deadlines (or estimates)?"

Of course it is constrained, but how can you estimate it? Because "software development" is the continuous process of converting a real world idea into instructions that are so tiny and specific that a computer can understand them. Obviously you can't know in advance what those instructions are, or you've already written the program. When do you stop "pushing back against poorly defined projects"? You can literally continue asking for more specifics until the project is complete. This is different from other areas of engineering, because almost NOTHING demands preciseness like a computer does.

"Estimating work and committing to those estimates is what a professional does. Deal with it."

The reality is, we have been trying to deal with, for decades. And failing badly. The author is suggesting (probably rhetorically) that perhaps we need to abandon the idea entirely, because it is not working (for very large projects).

[sheepmullet]

We can estimate better but it's often not worth it.

It makes sense when there is a large cost to not meeting a deadline. Or when it's difficult to change scope.

[sheepmullet]

Most of the problems are organisational rather than with estimating per se.

If I'm doing prod support for three different products and build work for two products, then I'm naturally going to be juggling priorities. There are a lot of different and changing variables in the mix and it makes committing to deadlines very difficult.

If an organisation wants to isolate developers and have them work on only build work for a single product estimation becomes much more accurate. Of course most orgs would rather less accurate estimation instead.

[mattmcknight]

"Estimating work and committing to those estimates"

I agree with you that estimation has value, especially at the large scale when you are trying to budget for a multi-year project. (and at that level- the detail you think should be there simply isn't) You are also right that at some level it is a social problem, estimates and commitments are two very different things- and you don't just commit to an estimate. Estimates are predictions, statements of probability about the unknown future, hopefully based on solid data from past performance and some understanding of the work to be done. Commitments are promises to make things happen that are only partially within one's control. In many cases, an estimate is needed prior to getting the level of information that one would need to make a truly accurate prediction, so we respond with a range of dates to reflect that uncertainty, but this can be too early to make any kind of commitment.

If you estimate at the point in time where you are 50% likely to get the job done, and commit to that, half of the time you will be late, even though it is "average time" it would take to do that work. You can't take the easy way out and commit at 100% likely time, because you could be sick in the hospital tomorrow (and with a big enough project, someone eventually is). If you commit to an 90% confidence estimate, you get stuck with Parkinson's law type problems. Not to mention that those predictions with percentages are hard to base on real data...but if you make enough of them, you can actually get feedback on your accuracy.

The problem is that most audiences for estimates aren't able to deal with the uncertainty, and many givers of estimates are easily pushed into overcommitting. This is why some of the ideas popularized by the Agile movement seem to work- if you fix the time increment, it is much easier to flex on the scope, especially when it is not possible for it to be well-defined in advance. For example, there are many cases where an high confidence estimate can't be made- as is often the case in more research oriented projects, where the path to a working solution, or whether one will even be found, is unknown, and it is part of the job of the project to discover that. You are stuck with estimating how long it will take you to try approaches x, y, and z.

McConnell's classic "Software Estimation" probably says it better. See ch 1 of McConnell's book here- page one is on the commitment/target/estimation problem: http://www.stevemcconnell.com/estimation-01.pdf

[cageface]

The problem is that we still don't have the tools or language to specify software projects with the same degree of precision that other engineering disciplines do. Give me a spec as complete as precise and as a blueprint for a building or a bridge and I can give you a pretty good estimate.

What I get instead is a stack of vaguely worded "user stories" and, if I'm lucky, some wireframes that will invariable wind up looking nothing like the final design.

[careersuicide]

> Give me a spec as complete as precise and as a blueprint for a building or a bridge and I can give you a pretty good estimate.

I would argue that that would in fact be the source code for the program!

I know that sounds flippant, but - civil engineers and architects will correct me I'm sure - a blueprint to me is a set of pretty clear and hopefully unambiguous instructions, just not necessarily in a chronologically ordered list format. To torture the metaphor even further: the act of building the building from the blueprints is the same as running the program.

The key difference is that in the case of building a computer program the hard part is writing the instructions and in the case of building a building the hard part is execution of the instructions. Building designers have the advantage of centuries (millennia, even) of knowledge and some pretty hard limitations set by physics. Money and time aside, computer program designers are more or less limited by hardware and imagination.

I really do hope someday programmers will look back at conversations like this and think not that their counterparts of the past just needed to deal with it and be more professional but that we just didn't know any better way, yet.