NTSB Factual Report

HISTORY OF FLIGHT

On November 26, 2008, about 1930 coordinated universal time, a Boeing B777-232ER (B777), N862DA, operated by Delta Air Lines as flight 18, experienced an uncommanded thrust reduction (or “rollback”) of the right Rolls-Royce RB211 Trent 895 engine during cruise flight at flight level (FL) 390 (about 39,000 feet). Following the rollback, the flight crew descended to FL310 and executed applicable flight manual procedures, after which, the engine recovered and responded normally. The flight continued to Atlanta-Hartsfield International Airport (ATL), Atlanta, Georgia, where it landed without further incident. The airplane was not damaged, and no injuries were sustained by the flight crew or passengers. The flight was a regularly scheduled passenger flight from Shanghai Pudong International Airport (ZSPD), Shanghai, People’s Republic of China, to ATL, operating under the provisions of 14 Code of Federal Regulations (CFR) Part 121. Visual meteorological conditions prevailed for the flight, and an instrument rules flight plan had been filed.

There were no reported anomalies in flight from the departure from Shanghai through the initial portion of cruise. While crossing the northern Pacific Ocean, just south of the Aleutian Islands, the airplane passed a low pressure area with some light to moderate turbulence. Shortly after entering U.S. airspace in the vicinity of Seattle, Washington, air traffic control cleared the flight to climb to FL370. The flight crew executed the climb using VNAV (vertical navigation) mode and the CLB (maximum climb) power setting. About 15 minutes after reaching FL370, the en-route flight crew was relieved by the incident flight crew. At that time, the airplane was in clear weather over an undercast layer, and all flight deck indications were reported as normal. Shortly after the flight crews changed, the flight was cleared to climb to FL390. The incident flight crew executed the climb using the VNAV and CLB settings, and flight deck indications were reported as normal.

The first officer, who was the pilot flying, stated that, about 30 minutes after reaching FL390, he noted that the right engine pressure ratio (EPR) indication was “fluctuating” and that he saw corresponding fluctuations on the N1 and exhaust gas temperature (EGT) indications. He reported that his attention was drawn by the message, “ENG RESP,” displayed on the engine indicating and crew alerting system (EICAS); however, no message with this text exists on the B777. The flight crew reported that they were aware of a January 2008 accident involving a B777 at London Heathrow Airport that experienced engine rollbacks and, therefore, they suspected that “engine icing” was causing the indications. (Air Accidents Investigations Branch [AAIB] AAR 1/2010 “Report on the Accident to Boeing B777-236ER, G-YMMM, at London Heathrow Airport on 17 January 2008” is available on-line at http://www.aaib.gov.uk/sites/aaib/publications/formal_reports/1_2010_g_ymmm.cfm.)

The ENG RESPONSE (engine response) non normal checklist in the Delta B777 Operations Manual Quick Reference Handbook (QRH) had been revised on October 31, 2008, as a result of recommendations issued by the AAIB during the investigation of the G YMMM accident. The revised U (unannunciated) ENG RESPONSE checklist stated that it was to be used when “One or both engine(s) did not reach commanded thrust or rolls back after performing the Cold Fuel Operations supplementary procedure or after operating at high thrust settings. Note: The objective of this procedure is to clear ice from the fuel system by reducing engine fuel flow while descending, then checking for proper engine response.” The first item on the checklist was to descend with the thrust levers closed.

The flight crew reported that they had some questions about the applicability of the U ENG RESPONSE checklist to their situation because they were more than 3 hours from the top of the descent and had not yet performed the Cold Fuel Operations procedure, nor had they recently operated either engine at a high thrust setting. However, they retarded both thrust levers and executed a descent to FL310. The right engine recovered and responded normally for the remainder of the flight. The flight crew performed the rest of the U ENG RESPONSE checklist items after reaching FL310. The flight continued to ATL, where it landed without further incident.

Review of the DFDR (digital flight data recorder) and QAR (quick access recorder) information indicated that about 45 minutes after reaching FL390, the right engine EPR dropped from approximately 1.25 to 1.1. The data showed that the measured fuel flow for the right engine reduced to approximately 5000 pounds per hour, consistent with the reduced thrust of the engine. All the engine parameters, including rotor speeds, burner pressure, and stator vane position, were consistent with the measured fuel flow, indicating that the engine was operating as would be expected for the measured amount of fuel flow. Recorded data showed that the engine control system attempted to increase fuel flow in response to autothrottle commands by driving the fuel metering valve (FMV) fully open, but this did not result in any increase in the fuel flow.

The airplane’s Thrust Asymmetry Compensation (TAC) system automatically adjusted the rudder to compensate for the asymmetric thrust, caused by the reduced right engine power. The rollback condition persisted for about 23 minutes between the first indication of fuel flow reduction on the FDR data and the execution of the descent by the flight crew. The engine control system on the Rolls-Royce Trent 895 powered B777 is designed to display an ENG THRUST (sometimes termed “EPR shortfall”) caution message on the EICAS alerting the flight crew of a significant disagreement between the actual measured engine thrust and the commanded thrust. Examination of recorded engine controls data revealed that this message was not displayed on the EICAS (see Tests and Research section below.)

FDR and QAR data further indicated that about 55 minutes prior to the rollback, the oil temperature of the right engine began to rise, eventually increasing by about 31º C relative to that expected based on the previous portions of the flight. During the rollback period, the oil temperature decreased, but it still remained at a higher than expected value. Following the recovery, the oil temperature returned to a value consistent with the previous portions of the flight. Note that a consistent small difference in the oil temperature between the two engines was evident throughout the nominal portions of the flight, which was not considered abnormal, due to variances in manufacturing tolerances.

Recorded data indicated that, during the hour before the rollback, the outside (static) air temperature reached a minimum of -62º C and increased to about -57º C following the step climb to FL390. The total air temperature (due to the speed of the airplane) was about -25º C before the rollback, and it decreased slightly during the incident as the airspeed decreased. Main tank fuel temperatures went below 0° C about 3 hours into the flight and then progressively reduced to a minimum of -23° C. The airplane made four step climbs at fuel flows in excess of 11,000 pph before the restriction occurred. The third and fourth step climbs both occurred at fuel temperatures below 0° C. The third occurred shortly after the fuel temperature went below 0° C, and the fourth occurred just over 3 hours later when the fuel temperature was approaching -15° C. About 3 hours later, the airplane carried out a further step climb, with a maximum fuel flow of just over 11,000 pph. It was during this engine acceleration that the engine oil temperature was observed to rise, followed shortly thereafter by the rollback. Fuel temperature at the time of the rollback was -22° C. The AAIB report of the G-YMMM accident includes an appendix discussing data mining examining fuel temperatures and fuel flow on long-duration B777 flights.

PERSONNEL INFORMATION:

The captain, age 52, held an airline transport pilot (ATP) certificate, multi-engine land, and was type rated in the B777, B767, B757 and DC-9. He held a current Federal Aviation Administration (FAA) first-class medical certificate with no limitations. He had accumulated 13,409 total flight hours, 5,793 hours of which were as pilot-in-command and 87 hours of which were in the B777. All of his B777 time was accumulated in the 90 days before the accident. His last flight review was on September 25, 2008, in the B777.

The first officer, age 54, held an ATP certificate, multi-engine land, with type ratings in the B777, B767, B757 and TBM 700. He held a current FAA first-class medical certificate with no limitations. He had accumulated 10,412 total flight hours, 1,896 hours of which were in the B777. He had accumulated 117 hours in the B777 in the 90 days before the accident.

AIRCRAFT INFORMATION:

N862DA, a B777-232ER, manufacturer serial number 29734, company ship number 7003, was delivered new to Delta Air Lines in December 1999. At the time of the incident, the airplane had accumulated about 41,222 total flight hours and 5,670 flight cycles. The airplane was equipped with two Rolls-Royce RB211 Trent 895 engines. The left engine, serial number 51193, was manufactured in August 1999 and had 38,052 total flight hours. The right engine, serial number 51150, was manufactured in February 1999 and had 33,773 total flight hours. The fuel-oil heat exchangers (FOHE) were manufactured for Rolls-Royce by Sumitomo Precision Products of Japan. The FOHE on the left engine, serial number 366, was installed in March 2008, and the FOHE on the right engine, serial number 504, was installed in November 2007.

The airframe fuel supply system stores and delivers fuel to each of the engines. Fuel is stored in three fuel tanks: a center tank, a left main wing tank, and a right main wing tank. The center tank contains two (left and right) electrically driven override/jettison pumps, and each main fuel tank contains two electrically driven boost pumps, identified as forward and aft. Fuel pump design and normal operating procedure is for all of the pumps in the tanks, with fuel loaded, to be operating. The outlet port of each fuel boost pump contains a check valve and is connected to the fuel feed manifold, which runs across the airplane and connects to the engine fuel feed lines. The manifold is split between the left and right system by two crossfeed valves. When the crossfeed valves are closed and the center tank is the source of the fuel, the left override/jettison pump feeds the left engine and the right override/jettison pump feeds the right engine. Since the center tank pumps operate at a higher pressure than the main fuel tank boost pumps, the main tank boost pump outlet check valves will be closed so that the center tank delivers the fuel to the engines. When the center tank is nearly empty, the center tank fuel pumps are switched off and the main tank boost pumps continue the supply of fuel to the engines.

The Trent 800 series engine is fitted with a main engine-driven fuel pump, which has low-pressure (LP) and high-pressure (HP) stages. Fuel delivered from the LP stage passes through an FOHE before it is delivered to the HP stage. The FOHE is designed to cool hot engine oil by transferring heat to the cold fuel entering from the fuel tanks and to warm the fuel to prevent ice from clogging the fuel filter and sensitive components downstream. The design of the FOHE at the time of the incident passed fuel through a matrix comprising 1,180 tubes about 2 millimeters in diameter and the engine oil circulated around the outside of these tubes. The matrix of tubes was mounted into a casing with an upper and lower face plate to separate the fuel from the oil supply. The inlets to the fuel tubes protruded a few millimeters above the inlet face plate.

METEOROLOGICAL INFORMATION:

At the time of the rollback, the airplane was in visual meteorological conditions above an undercast layer. The flight crew reported no unusual weather phenomena during the cruise portion of the flight. Upper air data indicated a 165-knot jet stream off the northwest Pacific coast and a short wave trough along the route of flight. The airplane passed through a low pressure system south-southwest of the Gulf of Alaska resulting in light to moderate turbulence aloft. During the portion of the flight over the northwestern United States, the airplane passed north of an upper level low pressure system, with the tropopause sloping from about 37,000 to 34,000 feet. Recorded air temperatures during the period preceding the rollback indicated a static air temperature of about -57º C at FL390 with a slightly colder air mass below, corresponding with the tropopause level.

FLIGHT RECORDERS:

The airplane was equipped with a L3 Communications model 2100 digital flight data recorder (DFDR) and a Penny and Giles quick access recorder (QAR). Following the incident, Delta Air Lines removed the DFDR and QAR from the airplane. Delta Air Lines downloaded the DFDR and transmitted the data to Boeing. The airline also forwarded the QAR optical data disk to Boeing for download. The initial indications of this incident are not normally reportable to the National Transportation Safety Board (NTSB) per 49 CFR Part 830.5, and the data downloads were coordinated with NTSB staff, when it appeared that the incident had similarities to the G-YMMM accident. The NTSB’s Vehicle Recorder Division received the electronic data files for the DFDR and QAR from Boeing.

Details of the recorder data and plots of selected parameters are in the NTSB public docket. DFDR and QAR information was used as the basis of the performance studies and other sections of the report.

The airplane was equipped with a cockpit voice recorder; however, the data were overwritten during the continuation of the flight to Atlanta and, therefore, no incident information was available.

TESTS AND RESEARCH:

Postincident examination of the airplane’s engine fuel supply systems and re-examination of the systems in February 2009 following a flight from Shanghai to Atlanta found no evidence of preincident component failures or mechanical restriction in the fuel system. However, during the engine examination after the incident physical evidence similar to that seen on the HP pumps removed from both engines of G-YMMM was seen in the HP stage of the right engine fuel pump. Specifically, markings consistent with cavitation were observed at the outlet ports of the incident engine pump HP stage. Cavitation is the result of the vacuum created when the discharge capacity of the pump exceeds the replacement capacity to the pump inlet. A restriction in fuel flow upstream can result in the HP pump lowering its inlet pressure to the point at which a large quantity of air and vapor bubbles are formed, causing a mixture of fluid and gas bubbles to fill the gear teeth. As these gear teeth open to the HP outlet port of the pump, the gas bubbles collapse and the HP fuel in the outlet port rapidly fills the available space. Both of these actions result in shock waves being formed that cause damage to the outlet port and the delivery side of the bearing faces. (A detailed description of this effect is included in the AAIB accident report on G-YMMM.)

Fuel samples from the incident and the re-examination flights were taken and tested for water, contaminants, and other chemical characteristics. Additionally, records of fuel sample testing were received from Shanghai. No anomalies in the water content or chemical makeup of the fuel were found. A slight amount of bacterial growth was found in the wing tanks and fuel filters, but this was not considered significant.

In support of the AAIB investigation into the G-YMMM accident, Rolls-Royce and Boeing conducted extensive research and tests to understand the nature and location of the fuel flow restriction. The possibility that ice was in the fuel was considered because ice can form from water molecules normally present in jet fuel. Research demonstrated that water molecules entra

NTSB Probable Cause

An accumulation of ice in the fuel system, which formed from the water normally present in jet fuel during commonly encountered flight conditions, which accreted and released, restricting the fuel flow at the right engine fuel-oil heat exchanger inlet face.

Contributing to the incident were certification requirements (with which the aircraft and engine fuel systems were in compliance), which did not account for the possibility of ice accumulating and subsequently releasing in the aircraft and engine fuel feed system upstream of the fuel-oil heat exchanger.