Boeing’s 737 Max Flight Tests Underestimated Risks, U.S. Investigators Say

Boeing Co.’s evaluation of the 737 Max system during development used an oversimplified test that didn’t anticipate the cacophony of alarms and alerts that actually occurred during a pair of deadly crashes, U.S. investigators concluded.

In the first official finding from a U.S. government review of the crashes that grounded Boeing’s best-selling airliner, the National Transportation Safety Board on Thursday issued seven recommendations calling on the Federal Aviation Administration to update how it assumes pilots will react in emergencies and to make aircraft more intuitive when things go wrong.

“We want them to step up how they certify these airplanes with regard to the human interface,” said Dana Schulze, the director of NTSB’s Office of Aviation Safety.

The NTSB, which is assisting inquiries in Indonesia and Ethiopia, where the two crashes occurred, stopped short of reaching conclusions on the causes. Under U.S. law, the agency investigates aviation accidents, but has no regulatory authority and relies on its recommendations to improve safety.

The agency’s 737 Max recommendations don’t call for any specific updates to the plane or other aircraft but could lead to sweeping and costly changes.

The way Boeing designed its flight tests were permitted under existing rules. But the NTSB is calling on the FAA to require more realistic assessments of complex emergencies during certification testing. Schulze said that it should be done before the Max flies again. Boeing is close to finalizing a redesign of the plane and has said it hopes that the grounding will be lifted before the end of the year.

The recommendations also ask the FAA to review all aircraft models to ensure that they don’t have similar safety issues lurking in the background, and to urge other nations to conduct similar reviews.

In addition, the NTSB wants the FAA to find more scientific methods to assess how pilots will perform in crises and to explore how to make aircraft warning systems more intuitive.

The FAA said in a statement that it would review the NTSB recommendations as it considers the proposed changes to the 737 Max: “The lessons learned from the investigations into the tragic accidents of Lion Air Flight 610 and Ethiopian Airlines Flight 302 will be a springboard to an even greater level of safety.”

Both the Lion Air crash last Oct. 29 and the Ethiopian Airlines crash on March 10 occurred after a malfunction prompted a safety feature known as the Maneuvering Characteristics Augmentation System to begin automatically and repeatedly pushing the planes into dives.

The crashes killed a total of 346 people.

Boeing’s 737s, including the Max models as well as earlier versions, have a relatively simple procedure for shutting off the motor that was driving down the nose. Yet pilots in the Indonesia crash didn’t perform it and the crew in Ethiopia started the procedure and then reversed course, according to preliminary investigation reports.

At least part of the reason for those miscues is that the underlying failure — faulty sensors that measure the angle of a plane’s nose relative to the oncoming air — triggered multiple loud alarms that were potentially confusing, the NTSB found.

In both cases, pilots faced a thumping warning of an aerodynamic stall, indications that their airspeed and altitude gauges weren’t accurate, as well as other alerts. And that was before MCAS began activating.

However, when Boeing assessed whether the system was safe during certification, test pilots flew far simpler simulator runs. They were tested on whether they could recover if MCAS began lowering the nose without any of the additional emergencies, according to the NTSB.

“This was the assumption that Boeing made in the certification process,” Schulze said. “It’s an assumption that’s permitted under the regulations.”

Years of research shows that pilots can be overwhelmed during complex emergencies with multiple alarms whooping and confusing, even contradictory, cockpit indications, Schulze added.

While Boeing hasn’t yet responded to the NTSB’s recommendations, the initial indications are that they are receptive, Schulze said. A special review by Boeing’s board of directors on Wednesday said the company should work with airlines to “re-examine assumptions around flight-deck design and operation,” particularly given shifts in demographics and “future pilot populations.”

Test Pilots

The FAA this week also urged the International Civil Aviation Organization, an arm of the United Nations, to consider stronger training requirements for similar emergencies.

One of the NTSB recommendations calls on the FAA to review how emergencies are displayed to pilots in an attempt to make it clearer how to respond.

More modern planes, such as the Airbus SE A350 or the Boeing 787, have computerized systems and automated checklists to help flight crews in emergencies. It might be more difficult to equip the 737 with such systems because it uses older technology.

Average Pilot

The recommendations urge the FAA to “develop more robust tools and methods” to assess how pilots will react in emergencies. Currently, test pilots try to determine reactions of average cockpit crews, but that may not be sufficient, according to the NTSB.

If the FAA follows the recommendation, one solution might be to begin using randomly selected pools of pilots from around the world to get a better sense of how they will behave, Schulze said.

While the Max’s design has been central to the investigations of the two crashes, the NTSB recommendations highlight how actions by the flight crews also played an important role in sending their aircraft into steep dives.

Human failures in such emergencies have been a recurring theme in accident investigations for decades, said Evan Byrne, the chief of NTSB’s Human Performance & Survival Factors Division.

“Through these investigations and with these recommendations, we’ve identified a gap with human-machine or human-airplane interface,” Byrne said. “What we’re trying to do is close that gap as it relates to multiple alerts going off simultaneously.”

–With assistance from Ryan Beene and Julie Johnsson.