[tectonic]

From today’s Orbital Index (https://orbitalindex.com/archive/2020-01-30-Issue-49/):

The Best Way to Make a Profit as an Aerospace Company is to Fail (https://qz.com/1784335/the-space-military-industrial-complex...), a compelling piece about how massive corporations like Northrop Grumman have little incentive to hit their contract budgets and are arguably incentivized not to. “Northrop Grumman […] won the James Webb Space Telescope contract in 1996 with a promise that the project would cost $500 million and be flight-ready in 2007. The telescope is now likely to launch in 2021 and is expected to cost nearly $10 billion. [...] [W]ith every delay and snafu, Northrop Grumman rakes in more money as missed deadlines extend the timeline and require more funding from the government. One delay in 2018 brought Northrop Grumman close to a billion dollars alone—twice the price the firm originally quoted to the government for the entire project.” According to this document (pdf) released on Jan. 28 by the Government Accountability Office, the JWST has only a 12% chance of launching in March 2021. The massive overruns by Boeing on SLS are a similar example.

[patorjk]

I worked at Northrop Grumman a decade ago on a contract that was missing it's deadlines. It was pretty choatic, and management was not very good (every new thing became a #1 priority). To try to get things done we were allowed to work as much overtime as possible. They even had white board showing who had worked the most hours (the top several people people worked over 2000 extra hours during a 12 month period). I almost burned out during this time period (though that's a different story). I didn't get the impression that there was an incentive to fail, quiet the opposite. People were genuinely worried about the reputation of the company. However, that was just my perspective as a ground level grunt. I don't know what was going on at the higher levels.

[perl4ever]

I briefly worked for a large federal contractor, far away from stuff like aerospace per se,* but I got the distinct impression that all the dysfunction and waste was inherent in the lack of communication between the government and the employees of the contractor. All around me were normal people trying to do a good job, but a lot of people seemed to be employed to prevent information from getting back and forth to the customer. You probably couldn't point to anyone specifically having malicious intent, but the feedback loop was so painful and convoluted that there must have been an enormous implicit cost.

*but oddly some major aerospace contractors had their fingers in the pie anyway.

[chii]

> a lot of people seemed to be employed to prevent information from getting back and forth to the customer.

that's because it's their job to _convey_ this information. But of course, their conveying is slow and/or wrong, whether by intention or by incompetence.

Also, if the customer and the sub/contractor were in direct communication, then these middle managers and project coordinators will no longer be needed! That threatens their job security!

[orpheansodality]

Sounds like a perfect example of an immoral maze: https://www.lesswrong.com/posts/zpA2Tnp2k38qSmr8J/how-to-ide...

[tristador]

> 2000 extra hours during a 12 month period

11.5 hour days, 7 days a week for a year?

> I almost burned out

No kidding. Seems like a recipe for much worse than burn out.

[patorjk]

I wasn't one of the 2000+ people, but yes, there were people doing that (working long days 7 days a week). One guy would actually sometimes sleep in his car. People weren't doing the same number of hours every day though. Also, there were some weird incentives. If you worked past 9PM, you got 10% extra pay on all of the hours you worked that day, so a lot of people (including myself - though not all of the time) would work past 9.

[p1necone]

> every new thing became a #1 priority

I know this well, this seems to be common in large software companies.

[alistairSH]

I never understood why there aren't performance requirements in government contracts. Same thing with infrastructure construction here in DC... new offices go up in a year, but changes to the intersection in front of said office take 5 years and cost multiple of the office itself. [hyperbole, but only just]

[spamizbad]

There used to be a lot more defense contractors during the cold war. Also, more accountability. So there was competitive pressure to be seen as an organization that could reliably deliver projects. Over the years, many of the major companies merged, then shifted strategies to geographically dispersing their jobs to bolster congressional support.

Take the Joint Strike Fighter / F-35 for example: 4 bids. Lockheed won the contract but subcontracts major parts to another JSF bidder (Northrop) and Boeing/MD ending up merging. The whole thing is so incestuous.

[perl4ever]

What I find humorous is a few years ago I observed apparently the federal government wanted to outsource some legal services and document management, and they went with one of these airplane makers (I forget which one). Who do you think of when you think of paperwork? The aerospace industry!

[chii]

"Most of our paper work is already for dealing with these contracting firms! Might as well outsource the paper work completely to them!"

[jariel]

"There used to be a lot more defense contractors during the cold war. Also, more accountability. "

Gear today is far more complicated. Schedule and cost increase geometrically with complexity.

Many of the 'innovations' on early era fighters were quick iterations made by tiny teams.

But it's no longer the 100's of fighters that will win, it's the one squadron with the best gear, radar, comms backed up by the rest of it (AWACS etc.) that will dominate. Of course, it has to actually work (!) but there's something to be said for that.

The basis of the 'over-budget makes money' is still reality however, there's no doubting that.

So it's going to be a matter of how we apply operational integrity in this new era of sophistication.

[spamizbad]

We have powerful computer software today that lets a single CAD engineer replace a room full of draftsman. We can spin up 20 different virtual designs then test them in virtual wind tunnels before a single physical mockup is made. Material science advances also can make your job easier in areas like friction and keeping weight down.

So while I agree things are orders of magnitude more complex, our tools are also much more powerful.

JSF studies started in 1993, when the fastest computer was a 235.8 GFLOPS behemoth -- roughly the speed of a midrange gaming GPU today.

[ska]

It's worth drawing a distinction between increasing costs and schedule complexity, and increasing overruns in both.

Regardless of how complex or expensive a project it, if the people doing it aren't strongly incentivized (positively or negatively) to keep these things under control, the will not. That doesn't matter what era you are talking about.

[ceejayoz]

The argument is that something the size and complexity of the JWST has never been attempted, which leaves a lot of Rumsfeld's famous "unknown unknowns" it's unfair to saddle the contractor with. There really needs to be a middle-ground on cost-plus contracts; something that allows for unanticipated issues to be dealt with, but punishes shoddy work or insanely silly lowball bids.

https://www.seattletimes.com/business/boeing-aerospace/shodd...

> In a vibration test of the telescope earlier this year in California by prime contractor Northrop Grumman, dozens of loose fasteners — some 70 pieces in all — came off. A few pieces are still missing and could well be inside the observatory. The locknuts were not tightened properly before the test, according to a report by the board.

This sort of thing (IIRC, this individual issue cost $150M) should come directly out of the contractor's profits.

[alistairSH]

I see your point for true research programs. But, we see similar patterns all the way down to building subways stations - massive overruns and delays and the taxpayer stuck with the bill. Building an above-grade rail should be a mostly solved problem. Just now, in DC metro, we have the Silver line expansion to Dulles - crap concrete is causing massive delays, but instead of forcing the contractor to replace it, the solution is just "eh, we'll just inspect it more often and hope it doesn't fall on anybody's head." And that doesn't solve the delay - it's still years overdue.

[coliveira]

The answer is the sunken cost fallacy. A government institution will view the sunken cost as too big, and solve that problem by throwing more money at it. The same thing that happens in Wall Street when they get into big problems.

[perl4ever]

Without actual knowledge, I'm certain your analysis is wrong, because it's too anthropomorphic. An institution isn't a person, and isn't subject to common fallacies, for better or for worse. Even if it appears to do things wrong that a person would do, the reasons aren't the same. You may say that there is a boss who is in control and to whom the explanation applies, but institutions don't work like that - they would never last if the leaders completely controlled them.

[scarejunba]

Organizations are even more susceptible to making poor decisions because they don't have information about the other participants. Take, for instance, a choice you and I have to make and where the best choice is either the one that gets you all the benefit or one that gets me all the benefit. e.g. a project that could be done by either of our teams. Instead of doing it that way, we will both choose the third (and worst) option: both our teams do the work and split the work and coordinate in meetings. This way we both get credit. This way the organization gets the worst possible outcome.

Now take the same thing but reward any attackable situation. What you get if you cancel the big losing project: You just threw out $100 million of work. The first mistake you make, I will capitalize on that to say why I should be director instead.

[fgonzag]

I'm starting to believe it'd be cheaper to just start out the same project with 2 or 3 companies, and then just keep the be st performing one and dump the other two as soon as its clear that one will be finishing the project in time and within budget. That would only double the cost (if you dump early enough) of a govt project, and since most projects seem to have a budget overrun much larger than that, it'd actually end up cheaper.

Companies with no competition and a guaranteed contract will never deliver the goods.

[pavon]

That is exactly what NASA did for Commercial Crew. The started out awarding 5 contracts, then at each stage down-selected to 4,3, and finally 2 suppliers, Boeing and SpaceX. Also commercial crew contracts are fixed-price, pay-for-performance, not the standard cost-plus contracts. So Boeing won't get a dollar to repeat this test; they have to pay for it out of their own pocket.

[ceejayoz]

> Also commercial crew contracts are fixed-price, pay-for-performance, not the standard cost-plus contracts. So Boeing won't get a dollar to repeat this test; they have to pay for it out of their own pocket.

LOL. Boeing knows how to handle minor obstacles like that.

https://spacenews.com/nasa-inspector-general-criticizes-addi...

"NASA paid Boeing nearly $300 million more than originally planned in its commercial crew contract in part because of agency concerns that the company might drop out of the program, a new report claims."

[njarboe]

SpaceX also has a contract with NASA to build a rocket and spaceship for astronauts to get to the space station. It has passed its last big test recently. If there are no further delays, NASA will be using it in a few months.

[wmf]

I've seen that done with DARPA contracts but I wonder if it just incentivizes contractors to back-load all the overages.

[wmf]

So if the contractor runs out of money and goes bankrupt when the project is 90% complete, then what? Spend far more money with a different contractor to try to pick up the pieces?

[ceejayoz]

For starters, don't give contractors projects so large they're an existential threat to the company. Northrop Grumman makes billions in profit every year, and the JWST has been in the works since 2003. They (and their insurance) could afford to pay for the fuckups.

When a bank collapses, the FDIC steps in, takes over operations, and ensures it keeps running while they find a different bank to take it over.

It's a rather fascinating process: https://www.npr.org/templates/story/story.php?storyId=102384...

[jessriedel]

How is your comment relevant to the OP topic? As mentioned by chrispauley and jdsully elsewhere in this thread, this "charge" is not Boeing charging NASA, but rather Boeing reserving some of its own money to cover a possible future expense.

https://news.ycombinator.com/item?id=22195084

[rory096]

The difference is that the projects mentioned by grandparent are under cost-plus contracts, while the Commercial Crew Program is under a fixed-price contract.

[markdown]

Always annoys me when talking about foreign aid to "3rd world" countries that you'll have people get on a high horse about 3rd world govts being so corrupt that money just ends up in the hands of cronies and the well-connected.

The reality is any such corruption is dwarfed by the corruption that goes on in the award of US govt contracts.

This is all an inevitable consequence of the US being one of the few countries on earth were bribery (by another name) is legal. So long as these corporations keep bribing the politicians, their contracts will continue to get extended.

[njarboe]

Yes. It is great that NASA is trying to get away from this model with the Commercial Crew Program. Boeing would eat this cost, just like SpaceX does when it has problems. Cost plus contracts, while maybe necessary in some situations, evolves huge cost bloat in defense, large public works, and health insurance sectors of the economy.

[garmaine]

Unfortunately Boeing has petitioned for, and received amendments to its “fixed-price” contract to cover these extra costs in the past. Boeing making this number public, which they would have every reason to keep secret for competitive reasons if it was their own cost, is the first step in normalizing the customer to the added cost. Watch as they threaten to pull out as they have done before...

[perl4ever]

Is that really a priori the wrong way to run things?

You have a project, you estimate the cost, you have a contract, then the customer just HAS to add more stuff, so you negotiate about what to add at what cost in time and money.

So, fixed price with adjustments, why wouldn't that be normal (in a vacuum)?

[garmaine]

The promise of fixed-price contracts is that the contractor is incentivized to get really creative and cut costs in order to maximize profits, like SpaceX has done. SpaceX is charging NASA about $220M per launch of the Falcon 9 -- the same rocket they price on their website for $62M -- and they get to keep the rocket and capsule for reuse too. Now at a marginal cost that's a minimum of 72% profit, and probably much higher. Now if SpaceX botched their demo flight and had to do another, would NASA be willing to pay the cost? No, and they shouldn't: the risk of cutting into their margins is what incentives them to keep on schedule, on budget, and minimize risk.

Yet Boeing has shown consistently that they are only able to operate in the old business model where "profits" disappear into "overhead" and contracts earn only a small margin of profit. Then as something unforeseen happens, as has happened multiple times, they threaten to pull out and force a renegotiation.

This defeats the entire spirit of the fixed-price contract, and costs the taxpayer. IMHO NASA contract officers should call the bluff, and give the contract to Sierra Nevada if Boeing does back out.

[ceejayoz]

> SpaceX is charging NASA about $220M per launch of the Falcon 9 -- the same rocket they price on their website for $62M...

To be fair, that doesn't include the cost of a Dragon capsule, nor the extra steps NASA flights involve. It's the "stick your ready-to-go satellite on top of our booster" price.

[scarejunba]

Sitting by and doing nothing is a sucker's game. I've always put money in NOC, LMT, and BA. They'll always go up because the government is easy to sucker and the people are easy to sucker. You just give them the latest spiel about who's a threat and bam, the people love it, and the government loves spending the people's money. Every one of them 5x-10x over the last decade.

Can't wait to hear how the Raider program takes forever haha. The US military is already way overpowered anyway, so the way I see it, I'm just getting the money I put in taxes back here plus a little from all the guys who don't play the game. And if you complain I can just seed something on Twitter about it being outsourced engineers who made the mistake. Then you'll rage for a few minutes and give me money.

Thanks, my dudes. Index funds are for losers.

[perl4ever]

Well, taking this as sincere, I would think your satisfaction would depend on them being benign tumors rather than turning out one day to be metastatic cancer. There's no way to have absolute security against that.

[scarejunba]

Yeah, if the US military starts sucking that would be bad but I'm not too worried about that. And to be honest, if they're going to suck at this, they're going to do that whether or not I put another $300k so I'd rather be wealthy and in trouble than poor and in trouble.

[perl4ever]

I sense that when you say "start sucking" you are envisioning a gradual process.

People and organizations can live in an unhealthy way for a long time before they have a crisis, but that doesn't mean there's a gradual ramp at the end.

[scarejunba]

I'm betting on seeing it coming.