Except some of us make a living selling development of projects and solutions to customers that require pretty accurate (cost) estimates prior to even getting the contract.
Absolutely this, clients need to know their costs upfront.
Meanwhile, I've been on the other side, working with a shop that insisted they were agile, and so refused to give an upfront cost.
That project ended up delivering a substandard project where key parts did not work well. Their answer was to tell us they would of course be happy to be paid for another 2 week sprint to fix those bugs...
I'm curious, did your contractors host regular demos, sprint retrospectives, etc.? If so, was it clear early on that the results were going to be substandard, or was it a surprise fairly late into the process?
I'm not going to debate about whether they were doing "true Agile", but it seems like even without settling upfront costs it would still be possible to control risks by tracking value added vs. cost on a sprint-to-sprint basis. Hopefully it should be evident early on if the project isn't worth it[1].
Of course, this requires that the contractors be able to deliver value incrementally, which might not be possible in all projects. In which case, upfront cost estimates (and more thorough planning) will probably be necessary.
The problem isn't with the requirement for fixed-time, fixed-bid contracts. The problem is that agile methods and tools, and all the advantages they carry, are incompatible with those requirements: You can't start fast with minimal specs. You can't apply what's been learned along the way unless, miraculously, it costs less and takes less time. You can't make changes to suit changing business goals without a major negotiation on change orders. You can't access most of the benefits of agile inside that box.
You CAN, however, find contract developers who have mastered the art of faking agile methods and being buzzword compliant while delivering no feedback that prevents you from having bad ideas implemented in bad ways.
You will also find contractors that will lead you down the garden path. You could call them "half agile." They won't warn you that you are asking for something dumb, or unworkable. They will treat your mistakes as revenue enhancement opportunities and lay on the agile talk pretty thick.
This is why the "no true agile Scotsman" argument is so easy to make: That's not agile. You have the wrong people. You fail at agile. That's an easy case to prove but it isn't very helpful.
Real agile is hard to do and doing it with compromised ingredients, like using low-bid contractors that want to minimize their effort and/or enhance their revenue, that you have stuffed into a fixed-bid box, is agile poison.
I don't think working fixed price quote would have given you a different outcome though? The team produced bad quality product, fixed cost or agile won't change that.
Selecting the team to build the product on price is often a bad idea, and that's all fixed cost gives you above agile (the ability to select on price).
A fixed cost makes it easier to turn and around not pay (in whole or part) if the product is really not up to the standard specified. It's harder to do that if the cost is not fixed but overrunning and the response is "well we just need more time".
In my experience it's easier and less risky doing it the agile way. You give them a 2-week sprint target. If they fail to hit that, then:
a. You don't pay them.
b. They're probably going to miss every other target.
If you paid the agile shop for everything they did no matter how crap, and withheld payment from the waterfall shop because quality control, then that's got nothing to do with agile vs waterfall, it's got to do with your QC.
Or it could be solid, perfect work that meets the best possible interpretation of the contract, but the client failed to communicate what they actually wanted and perceive it as subpar and unacceptable.
> ...customers that require pretty accurate (cost) estimates prior to even getting the contract.
If you require that accurate of an estimate, your business model cannot handle the risk of software development. Really, really wanting an accurate estimate does not make risk go away.
That's not exactly true, but it means holding the customer to exactly what they asked for, and having a formal "change request" process for any deviations. The track record of that model of development is not one of successful on-time delivery, but it does seem to make some types of customers more comfortable, even if it shouldn't.
Unfortunately, that doesn't change the fact that customers want estimates, and the more complex the project the more they need it for budgeting and scheduling purposes.
In the end, you have a few options for estimating:
1. Estimate the most optimistic scenario, knowing you'll end up using change requests and extensions to deliver (one of the worst options, but if you're responding to a proposal that's what the opposition is assuredly going to be doing);
2. Propose an initial phase to nail down requirements and deliver a schedule using traditional estimation tools (an easier sell if you're an internal team, tough if you're a vendor);
3. Use experience with similar projects to estimate, and then refine your assumptions with the client, using change orders to account for unexpected problems (roughly accurate to the degree of imported ambiguity, but now you have to risk going to war to get your change orders signed off on);
4. Use a top-down estimation model such as function points-based estimation to get in the ballpark, and then add buffer (and pray you aren't going to either lose money or blow the top off the client's budget levels -- or both);
5. Delphi the problem and get a bunch of developers and PMs to estimate the effort, then refine from there (useful for sanity checks, not useful as a real estimation process, yet one of the more common ways small shops build their numbers).
As you say, you can't make risk go away, you can only try to segregate it within your estimation. In my experience, clients really need stability in proposal numbers -- they're often willing to accept a significant embedded risk buffer in a proposal if it means they can limit unexpected costs. Conversely, you should also have a rough idea going into a project of what functionality can be cut and when so you can go/nogo those pieces when necessary.
> ...they're often willing to accept a significant embedded risk buffer in a proposal if it means they can limit unexpected costs.
So if they want 95% confidence in estimates, you need to pad the estimate a lot. It doesn't make risk go away. It just offloads risk onto the contractor.
That just means the contractor is assuming the risk of missing the 95% estimate. It also means the head of the contracting operation needs to be diligent about backing up the estimates of his employees. Or mitigating risk some other way (making sure each contract is a small fraction of the business, having enough cash to absorb risk, owning lots of other businesses, etc.).
Now, realistically, the risk is also absorbed by the developers in the form of unpaid overtime, stress, lost bonuses, and in other ways. That's just all the more reason for individual contributors to insist on honest estimates. Or find bosses that understand business models better and don't offload risk onto employees.
[] I understand that this is complicated and bosses don't typically mean to make employees miss bonuses and have terrible work/life balance. But to some degree it doesn't matter. Intentions don't make down payments on houses, make sure kids do their homework, or get us a full night of sleep.
Estimates can be fairly accurate if they're made for software that's a composition of previously iterated work.
If someone asks for a login system with X,Y,Z and you've done that 1000 times, you can be fairly accurate in stating what it will take to do it 1001 times.
The issue is when a piece of software is custom, and there's no domain expertise available for the custom bit then you cannot accurate estimate how long it will take because the developers will be learning while doing.
And some developers bring this on themselves. They could use the boring conservative technology that they know inside and out from the last project, or they could use something that's new and hot this month, which they have never done or have much less experience with (but is exciting).
Double whammy if it's an unfamiliar domain and a cutting-edge tech stack.
Customers don't need to know the cost, they need to know the price. The most important factor in determining that should be the value of the software to the customer, not the time it will take to produce it.
1. Cost-plus pricing (or contracts). Here, the vendor (that's you) gets costs plus some value. Which means that the customer probably wants to know what goes into determining those costs. This in turn requires a close familiarity of both parties. This was more common say, in the 1970s and earlier, and especially in government major-projects negotiation. But it's why negotiating contracts itself is a substantial activity.
If you're creating a one-off product, there isn't a market price. There's what you and a buyer agree to. Depending on circumstances, neither of you may be particularly free to walk away and go elsewhere.
2. Moen's Law of Bicycles: bad customers make for bad products. This is almost certainly a variant of Gresham's law, itself a generalisation of the point that information sufficient to distinguish product quality is itself difficult to come by. This manifests in multiple ways: crap quality cheap products flood markets, or flashy, easily-distinguished, but not materially significant features are added to products.
True quality, measurable in the use value of products, is often quite difficult to assess. This is tightly coupled to the Dunning-Kruger effect.
3. Map-terrain relation. A plan (or contract) is a map, it isn't the terrain itself. In many planning (and legal dispute) instances, there's a confusion of these. A contract which fails to encompass materially relevant information is at risk of being voided.
The upshot, again, is that contracts don't define software development schedules. Software development does.
It also means that you probably want to desgin the contract scope for developing complex products much as you do those products: iteratively, over time, with conscious trade-offs of features and costs.
Everything else is setting you up for a mismatch of model and frame to reality. Reality wins.
Let's say they figure out you only spent 100 hours to deliver them $50,000 of value. It's your job to persuade them that that guy on the internet who says they would sell them 100 hours of dev time for just $5,000 will only give them $5,000 of value (if that). If I have a choice between getting someone who delivers value at a rate of $500/hour versus someone who delivers value at $50/hour, and I need $50,000-worth of software, I should choose the more efficient developer, not the cheapest one.
For the kinds of projects my company tends to do, when we're getting into repeat business we're generally not competing with other contractors. Our clients know that the other contractors would have a ramp-up cost that we don't, and an unknown working relationship vs our good working relationship. Instead, they look for per-hour discounts and ways to 'cut costs' while delivering the same value. They also typically have their own development team who don't get paid nearly as much per-hour as my company charges, so they look to share more of the work with their internal team. Since we do product, services, and support, including training, it's hard to argue that the internal team can't or shouldn't do the work.
It gets complicated, but in the end we mostly come out alright. Our clients are well aware that taking on additional work reduces our likely development cost but increases overhead costs and raises project/schedule risks. It's a tradeoff they accept, and we work with them to ensure the project is successful.
My point is, in the real world you can't just charge by the value you provide rather than by the cost of providing that value. That's a simplistic point of view.
I do appreciate that the reality is that development contracts work that way. I've just spent a lot of time being frustrated at how often the business conversation around software assumes a basic hourly contract cost-plus basis.
I have no problem telling a customer that something along the same lines as a previous project will be more expensive today, because it was initially underestimated and I ate the additional cost. They realize they got a bargain the first time, and I don't have to eat the same costs to keep the customer. Of course, starting from the first project, I work very hard on building a trustful relationship to support such a claim.
I used to as well, until I started to refuse to do it. It truly is in the best interest of the customer to work this way, and I explain it to them. The ones that understand it and buy into this way of working are the ones I work with.
Meanwhile, I've been on the other side, working with a shop that insisted they were agile, and so refused to give an upfront cost.
That project ended up delivering a substandard project where key parts did not work well. Their answer was to tell us they would of course be happy to be paid for another 2 week sprint to fix those bugs...
As a customer it wasn't very satisfying.