Hacker News new | ask | show | jobs
by del82 2695 days ago
Based only on what I read in this article, I can kind of see Boeing's point? From what I understand, the procedure/checklist for an uncommanded nose down didn't change from the old to the new version, even with the addition of MCAS. So from the Pilot's perspective, there is nothing that they should do differently in the new vs. the older 737s when this happens-- follow the checklist, which will (eventually) cause you to flip the Stabilizer Trim Cutout Switches, and that will fix the problem.

So the interface didn't change, and the procedure's the same. Should Boeing and airlines update training every time they change something "under the hood", even when the procedure for pilots is the same? How about when they make software updates to already-flying models?

3 comments

One interface change was the effect of 'pulling hard back on the stick' in case of runaway stabilizers. That worked with the old system, but not with MCAS.

This seems to be exactly the interface change that lead to the crash.

To use a car analogy ...

You can always override cruise control by stomping hard on the brake (like to avoid an imminent crash). That's how it's always worked and you've gotten used to this, and done it on occasion when warranted.

Now imagine that the next generation of adaptive cruise-control/"auto-pilot"/whatever comes out, and stomping on the brakes no longer does anything. You have to first disable the cruise control by pressing a button on the steering wheel, and only then will inputs to the brake pedal do anything.

And then you don't tell drivers about this change.

You can totally see how, right in the lead-up to a fatal accident, a driver is going to be focused solely on stomping on that brake pedal in increasing panic, wondering right up to the moment of death why that's not doing anything. They won't consider the cruise control off button because it's not their most immediate need (braking is), and they've never needed to use it before.

But the difference is not only do the pilots of a commercial airliner have vastly more training and experience than your average driver, but they literally have a list of things to try (and extensive training on following the list) in the case of a malfunction like an uncommanded nose-down.

Imagine that the failure that caused the nose-down wasn't a failed AOA sensor giving bad readings to MCAS, but some other reason that _also_ wouldn't have been solved by pulling back on the yoke, but would have been by completing some later step on the checklist. Suppose the pilots didn't follow the checklist. Would that be enough information to say that Boeing is responsible?

To be sure, in this case the pilots may have followed the checklist! It may well be the case that Boeing is completely responsible! The checklist items might not have worked, or there may have been a good reason that the pilots didn't follow it, or the checklist might have been crazy, or they might not have had time to do what needed to be done, etc. There's still a whole lot that's not known (or hasn't been released) about what happened.

I'm just not sure that the current evidence, _viz_ that Boeing made an internal software change, that they didn't explicitly call it out to pilots, and that there's no difference in the actions prescribed in the event of an uncommanded nose-down pre- and post-change, is enough to say that the fault is entirely Boeing's for this accident.

More training and experience is a problem as previous plane model behaved differently. Pilot trained this situation and plane did X and now does Y - you don't really see the problem? - now pilot has extra burden to quickly determine if something else is wrong as plane behaves differently than pilot was trained to.

And you are writing this after many, many accidents that root cause was pilot not exactly knowing what or why something happens with a plane or plane autopilot (ie. AF 447)

Being apologetic of cost cutting on safety issues is dumb as it erodes culture of safety and encourages others to skimp on safety.

> Pilot trained this situation and plane did X and now does Y - you don't really see the problem? - now pilot has extra burden to quickly determine if something else is wrong as plane behaves differently than pilot was trained to.

The pilots did not train for a specific root cause of a fault. They trained for a symptom (uncommanded nose down), and the procedure for that situation was unchanged.

> extra burden to quickly determine if something else is wrong

This isn't an extra burden. Pilots aren't doing root cause analysis for failures while they are responding to them. They are trained to try actions in a specific order until something works. It's not like the checklist used to have one action on it and now it has two - the solution in this case was a standard action on the standard checklist.

Pilots do not look around, say "ah, electrical short in elevator actuator" or "ah, bad angle of attack sensor" and then take a single action.

They did train recovery scenario on simulator and know how plane should behave (based on old model knowledge) - now it behaves differently. All you wrote is they should ignore it as not important detail and stick to the checklist.

Unexpected situations are always extra cognitive burden.

I do not think this is a good analogy.

Firstly, there is no single equivalent to "slamming on the brakes" for uncommanded nose-down. This could be caused by a variety of faults, and pilots are trained to respond in a fashion that will be effective for even those in which the first thing to try doesn't work. There is a standard procedure in place to use in this type of situation - arguing that the pilots should only be expected to do one thing with increasing desperation is essentially arguing that they will not be able to respond to a whole host of emergencies causing uncommanded nose-down.

Second,

> Information from the flight data recorder shows that the plane’s nose was pitched down more than two dozen times during the brief flight, resisting efforts by the pilots to keep it flying level ... The standard checklist for dealing with that sort of emergency on the previous version of the 737 focuses on flipping the stabilizer trim cutout switches and using the manual wheels to adjust the stabilizers. [emph mine]

your argument essentially hinges on the assumption that pulling hard back on the stick is a sufficient solution for all the problems that may happen with a plane with the exception of a fault with MCAS (the new system). I don't think that is accurate. Even prior to the the 737 max! It sounds there were a lot of things that would require further action that pulling back on the stick.

Their response would have worked to return to level flight in non-MAX variants of the plane, though. Regardless of what the manual or checklist says, they had years of experience flying 737s that behaved in a specific way, they developed an unconscious/intuitive mental model of the plane based on those behaviors (that is faster and quicker to react than consulting checklists), and then a significant and deadly change was made to those behaviors and not communicated to them for cost-cutting reasons.

There's clearly a problem on Boeing's end here.

From the article:

"Older 737s had another way of addressing certain problems with the stabilizers: Pulling back on the yoke, or control column, one of which sits immediately in front of both the captain and the first officer, would cut off electronic control of the stabilizers, allowing the pilots to control them manually.

That feature was disabled on the Max when M.C.A.S. was activated — another change that pilots were unlikely to have been aware of. After the crash, Boeing told airlines that when M.C.A.S. is activated, as it appeared to have been on the Lion Air flight, pulling back on the control column will not stop so-called stabilizer runaway."

If the above is true, it's near criminal that Boeing didn't notify pilots about the change. It's also not just about following checklists but understanding the systems of the aircraft.

There's a Faustian bargain made by accepting so much abstraction from the true nature of an airplane by using software, and yet also that abstraction is not absolute. Pilots are made responsible for knowing all of the aircraft's behaviors, with full abstraction (normal mode), as well as partial abstractions (error and failure modes, of which there can be quite a few for a specific made/model, many of which must be inferred rather than being clearly displayed). If pilots get confused, however complicated and unintuitive the system behavior, they are blamed.

The more "piloting" computers are doing, it seems appropriate that they will be increasingly, properly, accused of the equivalent of "pilot error". And yet here is Boeing, taking more and more piloting responsibility, while still blaming the pilots when that doesn't work out. It's a deification of engineering: when things go properly and safely, praise engineering; when things fail, blame the pilots.

The point is that their response may have worked for this particular fault, but would not have worked in general. There is a reason that there are more procedures than "pull back on the stick" - pilots do not know what the fault is.

> they developed an unconscious/intuitive mental model of the plane

If the plane has a fault, it's frequently not going to behave like their unconscious model says it should. In the happy case, rely on your intuition. In the unhappy case, when things are working as they should, follow the emergency procedure.

Sorr, but this is a little like saying "follow standard procedure to put the landing gear down. If that procedure doesn't work, repeat your intuitive action over and over until hopefully the landing gear goes down." No - follow the emergency procedure in place for landing gear fails to descend.

> not communicated to them for cost-cutting reasons

Take it up with the FAA and the airlines?

> Take it up with the FAA and the airlines?

Sure. It's not just Boeings fault. I believe the FAA will course correct from this and probably all changes to flight controls will need to be reported to pilots. More lessons written in blood.

>Firstly, there is no single equivalent to "slamming on the brakes" for uncommanded nose-down.

Actually in the "old" version, there is.

From TFA:

Older 737s had another way of addressing certain problems with the stabilizers: Pulling back on the yoke, or control column, one of which sits immediately in front of both the captain and the first officer, would cut off electronic control of the stabilizers, allowing the pilots to control them manually.

Which makes the car analogy apt.

Indeed, any pilot will respond to nose down with ,pulling back on the yoke, or on the joystick, in other and older aircraft. This is a basic flying control, as basic as steering wheel and brakes, making this an excellent analogy. Any system that complicates this in the way described is a terrible system, for the reasons explained above.
Drivers aren't trained to follow checklists and usually don't have dozens of seconds to respond to mechanical emergencies. Cars also don't fall out of the sky if they break down. It's not a great metaphor.
> have dozens of seconds to respond

12 minutes, in this case.

And yet the pilots of the previous flight flipped the cutout switches.

EDIT: ... Presumably because it's literally the second thing on the list of things to do in the case of a runaway stabilizer.

I strongly agree. I've read most of the other comments here, and as an armature pilot myself, I think most of the responses to you are missing the point.

Its extremely unfortunate, but these pilots had ~10 minutes of flight time between their request to return to the airport noting aircraft control issues, and their crash, during which they completely failed to follow their checklists. Lion Air put those under-trained (if we're being generous) pilots into that cockpit and they are the only ones who carry responsibility for this crash.

Its anecdotal, but most of my older flight instructors have been lamenting the loss of critical pilot skills in the industry, and this appears to be just another example of that.

>your argument essentially hinges on the assumption that pulling hard back on the stick is a sufficient solution for all the problems that may happen with a plane with the exception of a fault with MCAS (the new system).

I don't read it that way. To me, it's specifically based on removal (without notice to pilots) of a function that would absolutely have terminated a trim runaway condition.

In other words, Boeing eliminated one well-known and trained-on trim runaway fix, and didn't tell the pilots. This is different from saying that stick pullback is a universal solution.

I think that's a pretty good analogy. If you do everything right, you'll get this malfunction under control. But it's easy to see how pilots whose plane is acting up unexpectedly can miss it, and it's reasonable to ask whether Boeing could've done better - and the regulators (note that according to the article, FAA and EASA were convinced by Boeing that this did not require re-training, while the Brazilian regulator wasn't).
> If you do everything right, you'll get this malfunction under control. But it's easy to see how pilots whose plane is acting up unexpectedly can miss it, and it's reasonable to ask whether Boeing could've done better

Absolutely reasonable to ask what Boeing could have done better. I'm just not sure the information about Boeing's actions contained in the article gets me all the way to "indefensible", paraphrasing another comment.

> That's how it's always worked and you've gotten used to this

but still, instead of going trough the full checklist they stopped and tried repeatedly whatever fixed the issue in the past/on the simulator.

a more complete analogy:

"follow these ten step do diagnose a bug on the software"

"but last time it was just a compilation flag, I'll check the compilation flags"

"bug persists"

"last time it was just a compilation flag, I'll check compilation flags"

"bug persists"

"last time it was just a compilation flag, I'll check the compilation flags"

"system halts"

From what I understand, part of the problem was that it helped temporarily... and then the system pointed the nose down again.
yeah I'm not siding with Boing here, what I'm saying is that pilots are given checklists for a reason, and going by the usual hunches instead of following the checklist as they were supposed to is as much a failure in training as is a failure in the plane user interface.
While the trim is not too far down, you can still counteract it with elevator (pulling the yoke). But then, MACS kicks in again and pushes the trim further and further down, and if you don't trim up and/or switch the trim cut-out, then you get into the situation where you can't recover anymore using the yoke.
"system halts"

That's one heck of an analogy! Brutal

Or that the system fights your stomping hard on the brake by accelerating. In the Lion Air crash, the system fought the pilots by aggressively nosing down. That it was doing this because of faulty sensor data, is different than how other 737's behave in the same situation.
For reference, here is the runaway stabilizer memory item (the "checklist") for 737:

1. Control column ............................. Hold firmly

2. Autopilot (if engaged) ..................... Disengage

3. IF the runaway stops:

------------------------ [done]

4. IF the runaway continues:

STAB TRIM CUTOUT switches (both) ...... CUTOUT

IF the runaway continues:

Stabilizer trim wheel ............ Grasp and hold

EDIT:

It's #4 that's of interest here. People saying that the interface changed are saying that it's fine if pilots stop after #1, even when dealing with runaway stabilizer for 12 minutes.

Accurate but irrelevant. This assumes the pilots knew they were dealing with a runaway stabilizer, and considering M.C.A.S. wasn't behaving similar to their previous training/experience/simulations with Runaway Stabilizers, it isn't clear they'd know they should follow this checklist. Now had they been actually trained on M.C.A.S. inc. faults, they may have known to do exactly this and we wouldn't even be discussing it.

This has been discussed in great detail on other flight forums.

It's not really possible to not know that you are dealing with a runaway stabilizer. MCAS (and every other automatic system to adjust trim) causes large physical wheels at the side of the pilot's knee to spin.
And these large wheels have small bell/clackers on them so you get a distinctly audible signal in addition to the large black wheels with white stripes on them.
Here's a video of the wheels in motion: https://vimeo.com/34501723

Disregard the text on the video description as it's blatantly wrong (describing that the trim tabs still move without the wheels moving; wrong on two accounts: the trim doesn't move without the wheels moving and the 737 uses a jackscrew for horizontal stabilizer trim rather than trim tabs [which is why the pilot can't simply override the aerodynamic force as they could with a trim tab])

Jackscrew operation: https://www.youtube.com/watch?v=rxPa9A-k2xY

The 7-3 has balance tabs to make the control forces lighter in the event of a hydraulics failure, but these are not trim tabs in any sense of the word.

As I read the article, it seemed like before, step 1 would suffice before. If this is not the case with MCAS, it could at the very least throw the pilot's diagnostics. "Is it runaway stabilizers? Well, no response from holding the control column firmly so I guess not. Lets check other things".

Sure, the correct response is to follow the checklist, but should we really rely on pilots always knowing the correct response? Especially when that goes against their previous experience? Not that I am defending these pilots not knowing the checklists, instead I am arguing we should take into account learning from experience rather than theory.

That's fair, but what does the checklist say? I mean, say there are a thousand different malfunctions that could cause an uncommanded nose-down. This one happens to be #359, but the pilots don't know that, they just know that they're pointed at the ground. Maybe it used to be the case that the first item on the checklist (pull back on the stick) fixed problem #359, and now it's the second. But there are several hundred other malfunctions that might have caused the problem that also aren't solved by pulling back on the stick, so the next move is to go to the next item on the checklist, right?
Pull back wasn't in the original manual but was a relied upon method. Boeing fucked up years ago by not documenting the function so it was allowed to be dropped transparently. How is that defensible?
Why was a method that wasn't in the official checklist "relied upon"? Pilots following some undocumented, non-standard procedure sounds like their fault rather than Boeing's.
Who knows? Have you ever been taught an undocumented procedure by the expert and been told to use it regardless of what the manual states? Happens all the time? Is a pilot in a position to affect Boeing beauracracy?
Sorry but how could Boeing design any plane if they can't assume pilots will follow the manual or any checklists?
The analogy to code would be if a library had a documented API to do something, but some people were using undocumented behavior in another part of the API to do the same thing, and the undocumented behavior changed. The difference is the consequences and how you prepare for them. With a library, there are many steps at which the change could be noticed by users: issue trackers, mailing list discussions, prerelease builds, integration tests, and test environments. Plus for most software nobody will be killed if the application goes down.

I see Boeing's point, too, but to me it just means both sides are at fault. The pilots are at fault for not following the emergency checklist. Boeing is at fault for abusing rules to slip in a change based on the assumption that pilots never rely on their own understanding of the aircraft, which I'm sure they know to be false. Air safety is all about human factors, and that's a pretty obvious one.

The interface isn’t opaque in airplanes like in software, everyone learns about how the internals of the aircraft work in order to troubleshoot problems. So pilots are reasoning based how how they think it works under the hood.