Hacker News new | ask | show | jobs
by jefe_ 2363 days ago
Boeing was still promising end of year FAA certification for 737 Max as recently as early December. Around that time, they had a spokesperson on CNBC who was asked directly how the outstanding steps (which indicated February at the earliest) aligned with their end of year statements and the position was essentially, 'we've been saying end of year and end of year could still happen.'

When your brand is suffering a crisis of confidence, you don't rebuild it by missing deadlines and dodging reality. The amount of hubris it takes to be so flippant in these circumstances is indicative of a deeply flawed corporate culture, and that falls at the feet of the CEO.

Also, how do you allow Starliner mission to occur without 100% confidence in ability of mission to be executed flawlessly? There are certainly complexities, but it almost seems like they passed the point when they could have quietly bumped the launch, and ended up in a literal PR gamble.

8 comments

The Starliner mission was a test. You wouldn't have 100% confidence that a test would be executed flawlessly. The purpose of the test is to find flaws in the system, and that's exactly what they did.
I"m not completely sure that's the case (and to be fair, I've not been following Starliner closely, so could well be wrong).

There are tests to prove out ideas during the course of development, but then there are tests which would perhaps be more accurately called "demonstrations"; where you're not trying to find flaws and refine your designs, but rather prove that you're [insert thing-name you're proving here] actually works the way you are representing it to work.

I understood this test to be more in the demonstration category, where Boeing would/should have had very high confidence, but they needed to prove to NASA that their spacecraft worked as advertised. If that's true, it was almost more a test on how much NASA should trust the confidence of the Boeing team than it was a test engineering and manufacture.

If that wasn't the character of the test, then I apologize for the distraction.

You are correct. The SpaceX version of this exact same test was literally called „Demo-1“. Ironically Boeing called it „Boeing Orbital Flight Test“.

https://en.m.wikipedia.org/wiki/Boeing_Orbital_Flight_Test

Boeing boefted that one
Doing the tests is fine, but Boeing is acting as though the tests were successful when they were not. Between this and the parachute issues from the pad abort test, I think actually approving Starliner for crewed use would constitute normalization of deviance. This is historically how astronauts die.
They were partially successful, which is a realistic goal. Few things work flawlessly the first or second go, even rocket science. I'm not sure why you're being so binary here. It's also why it wasn't manned, they have to test to work out the kinks. Overall Starliner was a bigger success than not.
I’m not saying to cancel the thing altogether, just maybe run some more unmanned tests first to make sure.
Reminds me of the news coverage of SpaceX’s first landing attempts, where they crashed into the barge and exploded and everyone was like “it’s a failure, it will never work”. Yeah but so much went right.

I think it’s just the (well deserved) media narrative Boeing is in now. When you kill hundreds of people and then pretend it wasn’t your fault, you get what’s coming to you.

The obvious difference is that SpaceX's landings were secondary objectives and something that nobody had ever done before. Getting in the proper orbit to the ISS was a primary objective for Starliner and something we've been doing for 20 years.
I don't know, space is hard and things go wrong. Just because we've done it hundreds of times doesn't mean we should expect perfection.
Yes it does?

These are known engineering problems with known engineering solutions. The explanation from Boeing was that a timer was set incorrectly. This sounds like a trivial error to me (though I'm not a "rocket scientist" just a "kerbal scientist", I guess, but we've been using timers for a long time afaik to properly manage burns to orbit).

Let's take a moment to consider the fact that apparently the MCAS uses input from only one of the two AoA sensors on a 737 MAX and swaps which one it takes the data from after each flight. I can't grasp how everyone involved could fail to realize that this statistically makes it less safe than only having one sensor.

I don't know how much the systemic issues that clearly compromised the design of the plane extend to the design of the capsule, but trivals errors seem to be very possible.

The procedures and how software systems handle changes to launch time are members of the set of hundreds of thousands of choices made during design and implementation that need to be validated. Yes, they feel like a "silly mistake" but ultimately most things that lead to failure will be in that category.
At some point we do, and that time is now. How qualified are you to give them a pass?
After SpaceX's failures, lots of people were saying things like "This is why they test— if you're not occassionally failing, you're not pushing the limits enough".

I would guess that at least in part the difference in attitudes is because SpaceX is considered more startup-y and Boeing is more associated with the "failure is not an option" ethos.

SpaceX still put satellites in orbit. From what I understood, SpaceX tests the entire software where they simulate launches. Something such as different timers should've been caught already.

Anyway, I do not care too much about pointing fingers elsewhere. The parachute thing was quite embarrassing and IMO this problem should have been prevented as well.

Boeing still put a capsule in orbit too. And I actually saw those excuses I'm referring to when the BFR prototype blew its lid, which didn't involved even a partially successful mission. Similar things were also said when the Crew Dragon blew up on the ground.

Regardless, testing all of the flight software is certainly not exclusive to SpaceX. I would bet any amount of money that Boeing also tested their software and performed simulated launches, moreover that testing is probably mandated by their contract with NASA. My uninformed guess is that one of two things happened:

1. Their tests were incomplete. e.g. they didn't find some edge case that would cause problems when the T-0 changed.

2. Someone goofed the procedures on the day of launch and didn't update their configuration properly.

That negative messaging was driven by ULA in a desperate bid to close the door on their new competitor. If they could kill off enough contracts, SpaceX might have run out of runway before they established their business model.
There were lots of scientists and ‘experts’ on our TV in the UK. Do you think they were part of that? That would be some impressive PRing.
It's easy to be indirectly involved in something like that. Your PR finds outside experts that agree with your world view, because journalists often ask you where they can find independent validation of your claims.

Then when something like this happens, you can have your PR call up journalists they know and suggest a name or two that is likely to say what you'd hope for.

Citation? That’s the first I’ve heard of it and that’s a pretty big accusation.
A significant share of the blame for the crashes accrues to the pilots who did not remember/follow emergency procedures for runaway trim. There was also the LA problem of putting an airplane back into service despite the critical malfunction on the earlier flight.

To reiterate, the procedure is:

1. restore trim to normal with the electric thumb switches

2. throw the stab trim cutoff switches

I know, sounds too simple to be true. But it is:

Boeing Emergency Airworthiness Directive

"Initially, higher control forces may be needed to overcome any stabilizer nose down trim already applied. Electric stabilizer trim can be used to neutralize control column pitch forces before moving the STAB TRIM CUTOUT switches to CUTOUT. Manual stabilizer trim can be used before and after the STAB TRIM CUTOUT switches are moved to CUTOUT."

https://theaircurrent.com/wp-content/uploads/2018/11/B737-MA...

This AD was sent to all MAX flight crews. It works because the first incident of MCAS failure (Lion Air) was safely dealt with by doing just this.

I know, sounds too simple to be true. But it is

No, it's not.

So here's some context. Boeing installed known not-to-spec structural components on the NG. Boeing installed known to fail prematurely slat tracks on the NG and 737 MAX. Boeing installed (probably known) not-to-spec pickle forks in the NG and 737 MAX. Boeing falsified repair documentation for an Air Canada 787. Oh, and of course, Boeing hid any mention of MCAS. Point being Boeing doesn't have a lot of credibility left.

With that in mind:

Manual stabilizer trim can be used before and after the STAB TRIM CUTOUT switches are moved to CUTOUT.

As the Ethiopian crew found out: it can't. The larger instruments of the NG required the hand cranks to shrink while the stabilizer itself grew. With the resulting lower mechanical advantage and increase in force required to move the stab itself the wheels became unusable. Sure, the Ethiopian crew went over the "maximum" speed but they were still under the max diving speed (Vd). That means the cranks were supposed to work.

It works because the first incident of MCAS failure (Lion Air) was safely dealt with by doing just this.

It worked because the first crew got lucky and had a third set of eyes that was free to dig through everything in search of a best guess.

> As the Ethiopian crew found out: it can't.

Whatever you read about that is simply wrong. (I've seen a LOT of misinformation in popular print about this.) You're correct that the hand cranks were unusable. But the electric thumb switches WERE usable and were pointed out in the AD.

Note that the crews of BOTH the LA and EA crashes had already used the thumb switches to restore normal trim, the LA crew did so 25 times.

> best guess

No guessing required. Follow the training, which is supposed to be a "memory item", meaning they weren't supposed to need to consult a checklist nor dig through anything nor guess.

I am not a pilot, but I would not consider myself fit to fly unless I knew by memory what every single switch in the cockpit does, ESPECIALLY the ones prominently located within easy reach. You can bet it's not the infotainment system.

For damn sure I would read every Emergency Airworthiness Directive for the airplane I'm the pilot of, most especially one issued in response to a crash.

The problem with this standpoint is that procedural fixes are the least preferable way of managing a hazardous condition. Engineering the hazard away is almost always the better option with procedural mitigation’s being a last resort. If engineering fixes were available and unused its indicative of poor safety engineering practices
> The problem with this standpoint is that procedural fixes are the least preferable way of managing a hazardous condition.

The way to make things safe is to address ALL points in the zipper that led to the accident. That includes the pilot error aspects.

Yes but the test was not supposed to be basic orbital insertion, it was supposed to be ISS rendezvous.
Imagine if it had taken out the ISS. Oopsie.
You're thinking of test, as in pilot. Everyone else is thinking of test, as in drive. I mean there are dramatically different expectations of what you, Boeing, and the public is supposed to believe what happened.
This is akin to saying the US Vanguard TV3 launch was a test. Yeah, it was, but it was also a response to their competitor having launched first and better, and a catastrophic failure blows up confidence in your programme.

https://en.wikipedia.org/wiki/Vanguard_TV-3

That is fair. When a test fails you would also expect them to repeat the test before putting humans on, we will see if that happens.
The article in the Washington Post hinted that NASA might not require them to pass this particular test (automated approach to the ISS and docking).

"It was unclear whether NASA would require Boeing to fly another test mission without crews onboard before allowing its astronauts to fly in the Starliner. Bridenstine said he wouldn’t rule out a mission with crews onboard, pointing out that the space shuttle had been piloted by astronauts, not computers."[0]

While Boeing makes noises about how there's no systemic problem with their ability to write software in general, earlier in the same article NASA comes to their defense by pointing out that if the craft was manned then the test would have been... saved? That seems strong to me but that is word used in the quote.

"NASA Administrator Jim Bridenstine said at a news conference Friday that the failure would not have been life-threatening had astronauts been onboard. He said that had the spacecraft been crewed, the mission might have been saved. “They are trained to deal with a situation where the automation is not working according to plan,” he said."

[0]: https://www.washingtonpost.com/technology/2019/12/20/boeing-...

During the post-event interview it was asked if testing docking was a hard requirement for NASA (with the implication the test would have to be redone before acceptance). The answer was no. If NASA wants this to be a new requirement they would likely have to pay extra for it
They’ll run remember to run PTP or NTP before launch next time, won’t they?
I am still a bit puzzled how cheerful some people seem about a product that was unable to fulfill its purpose: Starliner getting to its intended destination. Seems like they cheer that it stayed in one piece at least. Not too promising.
>I am still a bit puzzled how cheerful some people seem about a product that was unable to fulfill its purpose: Starliner getting to its intended destination.

Calling a space capsule a product and treating it like some kind of tech product launch and framing it's success that way is weird. It's a spacecraft and the mission was meant to test its capabilities and get data about it during an actual orbital flight, then go from there.

The reason everyone is trying to be so cheerful and optimistic is because it was a partial success - the capsule stayed together and landed, which were part of its mission parameters. But (clearly) the public perception of this mission is that it failed because it didn't get to the ISS, so I imagine people are trying to be cheerful to remind people that they're not upset cause they didn't fail, they just didn't fully succeed.

I'm no fan of Boeing or how they've handled 787 MAX or ULA or Starliner or their relationship with NASA, but calling this not too promising is a bit unfair.

It failed for essentially trivial reasons. Which suggests there are issues with the procedures that should eliminate those failures, and that similar failure modes are possible.

That's why it's not encouraging - and why it's not at all unfair to be scathing.

We launched the ISS over 20 years ago. If you look 20 years down the road from Kitty Hawk, we had already built tens of thousands of manned aircraft and fought a world war with them at that point in time.

Apparently, 20 years of progress and "promise" ain't what it used to be.

I think it's because it's a lot easier to go fast when there are huge potential rewards (winning a World War) and you're willing to accept fatalities (war). The rewards here are much more nebulous, and we're not willing to accept fatalities. So we go slow and careful.
I’m not at all puzzled that a bunch of people commenting in a hacker forum don’t understand very much about engineering like this. Astronautical and aeronautical engineering are difficult. The fact that we’ve been in space 50-odd years doesn’t mean we’ve perfected anything to the point where we should expect a flawless mission on the first go even for something like this.
The suggested failure/problem was in the software side.
Maybe not even in software, but procedures. Still, this is common. A big share of space failures are, on some level, failures of software and procedures.
Because it's a test that showed a large number of things went right, and otherwise didn't meet overall objectives for a small, trivial reason.

That is, the amount of project surface that it increased confidence in is as great or greater than the amount it called into question.

Well it did turn out better than the 737 MAX in that no one died...
At this point, management probably cannot get good information about the state of projects.

When a company creates a culture of sweeping problems under the rug, and then tries to go back and change that, it creates an impossible situation for people at all levels of the hierarchy. Where management once told people it was OK to hide things from them, now they've changed the rules and are telling them, hey, show us all the things we told you we didn't want to know about.

First of all, when you're told this, you have no way of knowing if they really mean it. Are they saying, hey, actually show us the dirt so we can deal with it? Or are they saying, hey, show us the dirt WINK WINK, but we want the answer to be that nothing too bad happened because that's the easy way out? (In other words, you have to judge whether they've had a sincere change of heart or you're playing a new phase of the same game as before.)

But even if you really believe them, now you have a difficult calculation to make. Maybe you will reveal some dirt and they will say, wait a second, we meant for you to hide dirt, but not dirt that was that bad, and that's your fault, and you were supposed to know that. And now you're in trouble for telling the truth. Unless management offers you complete immunity (if that's even possible), you still have a reason to keep hiding stuff to protect yourself.

TLDR, if your organization made sure everybody's closets are full of skeletons, then you're going to have a hard time getting people to open those closet doors.

The other side of this coin is it not good enough to just identify problems.

There is also the need for an organisation wide culture built on processes, procedures and channels that then enables the company to quickly and efficiently find solutions to these kind of problems.

When a company for a long time has being building a culture of ignoring problems, the effort required to rebuild that problem solving culture is massive.

It is not helped by the fact that the individuals holding the power at the different levels within the organisation tend to be the ones who oversaw the demise of that engineering culture in the first place.

This is why whistleblower protections are so important. Without a radical re-org, there will still be enough management within the company that prefers to cover things up. They might well have been hired for that skill.
This is a brilliant bit of insight.
> how do you allow Starliner mission to occur without 100% confidence in ability of mission to be executed flawlessly?

What space program with new hardware has ever executed 100% flawlessly?

> When your brand is suffering a crisis of confidence, you don't rebuild it by missing deadlines and dodging reality.

has worked for Elon, and now his stock is trading at an all time high. $420

But he is saving the world by filling it with toxic batteries and ugly trucks

Haven't you heard?

ISWYDT...
It really is a shame that it takes the deaths of hundreds to even budge the needle on getting back to being an engineering company rather than profit is #1 company and quality a distant 3rd. They should be ashamed to not have started from scratch on a new optimized plane design because they literally sell their planes for DECADES and almost have a captive market. Quarterly earnings focused governance killed those people as surely as faulty design.
> 'we've been saying end of year and end of year could still happen.'

Deliberately false and misleading statements that should be prosecuted by the SEC.

I filed an SEC complaint, own a single share of Boeing to have standing. Hopefully something comes of it.
Username checks out.
“If not me, then who?” ¯\_(ツ)_/¯
I think there were just waiting to fire him right before Christmas
nah they were planning for Hanukkah