[ThomPete]

Time estimations is an industrial way of thinking applied to a post-industrial world.

In the post industrial world time isn't the problem but rather project definition and scoping.

In the industrial world the problem was already solved (machine was built, market often established and output depended on a few factors that could be adjusted. Need more output add more of X)

In the post industrial world every project is about problem solving and scoping.

To put it into comparison.

If we apply post-industrial thinking to an industrial world. It means that each time a product needed to be done if not the factory, then the machines would have to be developed.

It will take many many years before time estimation will die, but it will happen.

[jongraehl]

What's needed may just be to solicit a wide confidence interval (e.g. "this task will take between 20 minutes and 1 month, and I'm right 90% of the time recently when making such estimates"). You can drop the lower-bound part, probably.

People absolutely should be able to provide a rough estimated amount of effort for a task. The trouble is in using a single point to describe the whole probability distribution. You may be right, in that nobody seems to have a great way of soliciting a probability distribution (or even a single probability) that makes sense. Something like a 70% confidence interval bracketing the amount of effort would be useful, but not sufficient.

I also agree that even if people could describe their beliefs about the required effort rigorously, you have to wonder how much planning/analysis they should spend trying to come up with a tight estimate. An 'outside view' - http://wiki.lesswrong.com/wiki/Outside_view - could give something reasonable in cases that aren't too novel.

I think estimates will never become useless. It's just that we may decide to replace them with "time by which we'll be late only 1/6 of the time", and provide feedback and incentives for people to correct this estimate so that it's, for someone who's experienced in the domain, eventually relatively unbiased.

[diegob]

What could replace it?

[ThomPete]

That is hard to say and it would be a book worthy to explain what could come next. All I know is that it is unsustainable.

The complexity is simply too high and it's not getting better. One of the reasons I think why you see the fail fast movement be so successful.

Once you accept that failure is part of the process, once you abandon the "zero mistake" policy that many large organizations instill internally and externally you will begin to approach projects differently.

The truth is that "zero mistake" organizations make as many mistakes as everyone else, they just have the financial strength to ignore them as long as economy of scale works in their favor.

I could write forever about projects that went wrong not because the developers where bad but because the premise that fuels product development is broken.

I blame primarily business schools and large parts of academia for this. But it could extend all the way into the way the stock market is structured.

If you buy my premise that post-industrial is different than industrial age. That project definition is primary and time is secondary today. Then it does put some doubt at least in me about whether the stock markets focus on growth and Q's is sustainable.

Nature seems to be doing a good job as pacing various processes. It takes nine months to give birth to a child. One cell at a time. But the process is ongoing.

Nature is the ultimate continues deployment strategy.

Edit: Fixed lame false modesty in the end!

[bendtheblock]

It sounds like you've thought deeply about this - have you written more on your ideas here elsewhere? Would be interested to hear how this would work practically as well as your ideas on how the focus on perpetual growth hinders successful project delivery.

[run4yourlives]

I'd love to see you write more on this as well. You've clearly thought about it prior to this conversation in a way that I don't think a lot of us have.

[jacques_chester]

I think you will find reading Hayek and Coase to be rewarding.

Hayek introduced the idea that no one individual planner can make flawless plans because the information to do so is widely dispersed; instead success or failure in the market gives rapid feedback about how to allocate resources. Indeed the market is a discovery process that unearths this information in a way that a planner never could[1].

Coase asked the question: if this is so, how do large firms emerge? He identified the cost of transactions to be the key. The higher the transaction costs, the higher the cost of loosely coupled economies. The size of the firm is thus based on the relative costs of transactions (searching, identifying, validating, negotiating etc) vs the inevitable waste caused by planning.

In essence, large companies are islands of command economics in the sea of the market.

[1] In a related argument, Mises said that even if a planner could know all the variables the resulting problem would simply be too large to solve rationally.

[bartonfink]

A serious attempt to provide a precise, correct answer to the ? "What do I really want done?"

[diegob]

So things like prototyping, UML, use cases, and Agile were not a "serious attempt" to answer that? I think it's also unwise to tell your manager "it'll be done when it's done."

[silvestrov]

UML is a description of the engine. It doesn't answer: "do we need our own transportation device instead of taking a taxi?"

[ThomPete]

You know there is a saying in the creative industry when project managers come knocking on your door.

"You want it know, or when it's done?"

My personal experience is that most deadlines are primarily perceptual.

[jdavid]

I think an under appreciated gift Apple has given the world is the patience to provide something when it's done, vs. possible.

[ThomPete]

Ohh Apple is on a completely different planet.

They follow the principle of "The best way to predict the future is to invent it"

so yes you are totally right about that.

[anthonyb]

Largely those are things done by developers, not by the business. Perhaps in cases where the business is on board with them (eg. Agile and rapid iterations), but mostly they'd rather handwave and hope for the best.

[netghost]

Instead of replacing estimation, I would say that you should embrace the problem in estimates - uncertainty.

Convey the uncertainty you have about a task to those you work with, and then you can start to factor in the risks of uncertain tasks.

I build project management software for a living at LiquidPlanner. Everything we do be it development, design, or marketing is based on ranged estimates. We don't always get the estimate right, but that's the beauty of a range, I it takes into account the fact that you will miss it some of the time.

The other thing that can replace an estimate is... lots of estimates. If you're planning something, update your estimates as you gain more knowledge about the problem.

[kgo]

Yep.

I used to work for big oil, and they would have 100-200 million dollar projects. Not software projects, actually building plants. Similar to software development, these were basically prototypes. Yes a lot of the technology was known, but they weren't exactly building cookie-cutter houses.

So the first round of estimates would be something crazy like +/- 80%, both money and timeframe.

If that seemed good, then they'ed fork over the money to get more specifications and more details, and come up with a new estimate. Maybe +/- 50%. Then they'd re-evaluate the viability of the project. And do that a few more times, spending more money and time each iteration, before they actually committed to the project or dumped it. At that point, the estimates were quite accurate.

So basically, the software version. How long will that feature take?

An experienced developer can spitball an answer. About a week. Then they start working on it. After a day-and-a-half, they're going to have a better idea if that's actually a three day project or a two-weeker or if it ain't gonna happen.

Or maybe they need a day-or-two or a week to do a spike to be able to provide that estimate. All well and good if you let them do the spike. Not so good if you demand an estimate when the developer has clearly said he has no idea.

[jacques_chester]

I absolutely agree that interval arithmetic is the best way to deal with this, accountants and their budget systems always require estimates to be reduced to fix figures.

[jongraehl]

Actual effort to complete can be quite surprising, but effort spent per day also varies tremendously. For example, when things are going well, there may be a burst of joyful effort.