[toast0]

Quality is always a trade-off. If you're deeply into economics, you wonder about the trade-offs of cost to find defects before shipping, difference in cost of addressing defects before and after shipping including costs of mitigation from consequences of defects, % of defects that will never be found after shipping (and are therefore a real cost savings), and in the long game costs of having a reputation for shipping product with defects that could have been reasonably detected.

In a lot of software organizations with rapidly changing and undocumented requirements, there's a good chance defects will go unnoticed until they're no longer relevant, so spending a lot to find them before they're shipped is a waste. Mitigation of many software defects is simple, but some aren't; hopefully you know which changes are expensive to fix if wrong, so you can more thoroughly vet those.

In Aerospace, addressing defects after shipping is very expensive, and mitigating the effects of defects is only approximate; you can't restore passengers from backup, economic damages don't really make families whole, but should be an incentive not to let reasonably detectable defects be shipped.

[wmidwestranger]

> Mitigation of many software defects is simple, but some aren't; hopefully you know which changes are expensive to fix if wrong, so you can more thoroughly vet those.

This assumes you're fortunate enough to have a defect at the outer edge of the system. Most times, these problems are created in the initial rush of pushing something out and then tax every effort that depends on them, forever, and ever.

[masfuerte]

Amen.

[lp4vn]

>In a lot of software organizations with rapidly changing and undocumented requirements, there's a good chance defects will go unnoticed until they're no longer relevant, so spending a lot to find them before they're shipped is a waste.

It's really a shame that a good percentage of these applications full of bugs and "rapidly changing and undocumented requirements" don't get scrapped and stay many decades afloat until they get replaced by another application also full of bugs and "rapidly changing and undocumented requirements".

I think that that's a very sad way of seeing things honestly.

In the past the USA put the man on the moon, today repeating the same feat looks almost impossible. I bet that a lot of managers at Boeing also think that building planes like a few decades ago looks almost impossible now.

[bbor]

I really find it disingenuous to imply that we can’t build moon rockets because we’re not good enough at engineering projects - I can think of a few engineering projects that took off last year, to say the least. And nasa doesn’t deserve the shade IMO. The kids of the people who built the Apollo program aren’t working at boeing or fighting for one of the few underpaid nasa positions - they’re building reusable rockets for the Twitter CEO, and, much more commonly, parasitic UX features for gig economy apps.

TL;DR: we’re fine at engineering, we’re terrible at resource allocation. Or at least that’s the more relevant cause. I post this knowing full well that this is HN and I might well be disagreeing with a senior nasa employee…

[TheOtherHobbes]

If we're fine at engineering, why are doors falling off planes?

I don't think you can say "We're fine at engineering but we're often terrible at management."

They're not separate things.

Engineering culture is about inventiveness, pride, craftsmanship, and getting the job done well. Bean counter culture is the opposite of all those. If that's the culture engineers work under not only do none of them happen, but they become less and less possible over time.

[bbor]

You’re making great points about the software/engineering industry and I don’t disagree about any of those specifics — engineering culture is paramount. I was just trying to point to (what I see as) the root cause: the engineering culture at Boeing didn’t fall apart because it’s run by lazy millennials or because the managers hadn’t read Mythical Man Month, it’s because they spent their money on stock buybacks and executive compensation.

Funnily enough, I went looking for Boeing stock buyback info and found this article… seems my hypothesis has some specific backing in this case!

Opinion | Did Stock Buybacks Knock the Bolts Out of Boeing? https://www.commondreams.org/opinion/boeing-safety-stock-buy...

[toast0]

Stock buybacks are tax efficient dividends. Boeing has been paying dividends fairly regularly since 1937, so paying stockholders can't really be the problem. Executive compensation is kind of a red herring too. If you don't like what the executives did, it probably reflects more on the choice of executives rather than the compensation of them, but you could maybe make an argument about how compensation incentives were setup.

MCAS and unbolted door plugs feel like two separate types of problems, IMHO. Both of them can be tied to Executives and culture, of course. MCAS comes from a desire to skirt regulations --- hiding automation from pilots in order to reduce certification requirements is a design error. OTOH; the unbolted door appears to come from production / rework corner cutting; the design is sound, the written process is sound (I think), but written process was not followed in order to meet schedule pressure.

You can have a company culture that encourages skirting regulations and cutting corners in production regardless of dividends/buybacks and executive compensation.

[bcrl]

The unbolted door was caused by a desire to avoid reinspecting the door were it removed using the official process. That's exactly the same problem as MCAS: they tried to skirt the regulations that required reinspection.

[bbor]

Again, great response, I think you understand the dynamics better than I. HN is gonna cut off this thread soon but I think your last line helps me sum up my point well:

  You can have a company culture that encourages skirting regulations and cutting corners in production regardless of dividends/buybacks and executive compensation.

IMO, a good way to avoid such a culture is to choose and cultivate quality engineers. A better way is to also entice them with high pay and/or prestigious projects. The best way, which is entirely foolproof, is just to have more money so that corner cutting doesn’t come up. And given the scale of Boeing’s behavior, I thinks it’s reasonable to say they could’ve expanded QA/safety/training/testing/etc budgets by a LARGE degree.

In other words: imagine I bought Airbus (lucky crypto run, ofc) and immediately transferred 10-20% of their liquid(izable) assets as cash into my bank account. And otherwise left the company running as is. Wouldn’t you think that’s the most relevant fact when discussing changes to their engineering culture in the preceding years?

Re: “it’s always been this way so if can’t be that”, I don’t think that’s a solid enough premise to support that conclusion. A) context changes, and b) I think it really is crazy out there these days. From Jacobin:

  In a 2017 article for The American Prospect that he coauthored with Sakinç, Lazonick identified one especially distressing instance in which Boeing neglected to follow through on its planned redesign of the 737 Max airplane, a project that was estimated to cost $7 billion at the time. That amount, he wrote, was what “on average, Boeing has been spending on stock buybacks annually since 2013.”

  In another analysis, Marie Christine Duggan, an economic historian at Keene State, concluded the amount that Boeing spent on buybacks in recent decades has generally outweighed spending on capital expenditures like upgrades and maintenance. Duggan found that in 2017, at the height of its buyback frenzy, “Boeing’s spending on dividends and stock buybacks was 66 percent of total spending, while only 9 percent of Boeing’s cash went into new equipment to manufacture planes.”

  Many of Boeing’s mass stock buybacks came after President Donald Trump’s 2017 tax cuts, despite the fact that Boeing and other major companies promised to invest their resulting tax savings on capital expenditure and innovation.

[maxerickson]

Quality is always a trade-off.

That's a pretty huge assumption.

For instance, compare firearms prior to replaceable parts to firearms after.

Better, cheaper, easier to make (because craft was replaced with process). Some up front cost, but absolutely not a trade off, it was a huge advancement.

Of course modern process control does more or less let you relax conformance rules to reduce cost, but it's farcical to call sacrificing reasonable conformance "quality".

Arguably, the idea that quality is obviously a trade off and you can make money by letting it slide is one of the sources of rot in our society.

[sokoloff]

> Some up front cost, but absolutely not a trade off, it was a huge advancement.

Any exchange of higher fixed costs for lower marginal costs (or other benefit) is a tradeoff.

This is a tradeoff that was/is massively beneficial, but it’s still a tradeoff.

[maxerickson]

It's not a quality trade off, which is at least implied to be what I am talking about in my comment.

Saying "Quality is always a trade-off" implies you can't actually make things better over time. I would accept something like "You can always trade off of the quality you are capable of producing to reduce costs", but the point is that capability can in fact change, and thus there are ways to improve quality that may not be a trade off (because they also improve the business/system along other dimensions). Even simple little things like aligning a process with the intended outcome can reduce costs while improving quality, you don't have to invent a revolutionary method of manufacturing.

[sokoloff]

Beyond a very early point in any engineering effort (where fruit might be not just low-hanging, but actually lying on the ground), nearly everything is a trade-off.

Those trade-offs (higher fixed costs in exchange for lower marginal costs and higher quality) can be wildly beneficial overall but finding large Pareto improvements against every dimension is rare in anything even remotely mature.

[Turing_Machine]

As the old saying goes: "Fast, cheap, or good. You can pick a maximum of two."