Next Generation Air Transportation System
FAA and NASA Have Improved Human Factors Research Coordination, but Stronger Leadership Needed
Gao ID: GAO-10-824 August 6, 2010
To address challenges to the aviation industry's economic health and safety, the Federal Aviation Administration (FAA) is collaborating with the National Aeronautics and Space Administration (NASA) and other federal partners to plan and implement the Next Generation Air Transportation System (NextGen). NextGen will transform the current radar-based air traffic control system into a satellite-based system. Pilot and air traffic controller roles and responsibilities are expected to become more automated, thereby requiring an understanding of human factors, which studies how humans' abilities, characteristics, and limitations interact with the design of the equipment they use, environments in which they function, and jobs they perform. FAA and NASA are tasked with incorporating human factors issues into NextGen. As requested, this report discusses the extent to which FAA's and NASA's human factors research (1) is coordinated and (2) supports NextGen. To address these issues, GAO reviewed coordination mechanisms and planning documents and synthesized the views of nine aviation human factors experts.
While FAA and NASA officials are coordinating their NextGen human factors research efforts in a variety of ways, they lack a cross-agency human factors plan for coordination. FAA and NASA have participated in research advisory committees and interagency research transition teams, signed interagency agreements, and held cross-agency meetings and conferences focused on human factors issues. FAA also created a human factors portfolio to identify and address priority human factors issues but not a cross-agency human factors coordination research plan in cooperation with NASA, as previously recommended by FAA's Joint Planning and Development Office (JPDO)--an interagency organization responsible for planning NextGen. As a result, FAA has not established an agreed-upon set of initial focus areas for research that identifies and capitalizes on past and current research and establishes focus areas for human factors research and development, among other things. The experts GAO contacted generally agreed that FAA's and NASA's human factors research efforts adequately support NextGen, but made several suggestions, including enhancing human factors research leadership, for further incorporating human factors issues into NextGen systems. FAA and NASA have undertaken a variety of human factors efforts to support NextGen, including, among other things, creating planning documents detailing how human factors research will be incorporated into NextGen and dedicating financial resources specifically to NextGen human factors research. While the human factors experts GAO interviewed stated that these efforts support NextGen, a majority offered the following suggestions for further integrating human factors issues into NextGen: (1) Better ensure that human factors issues are fully integrated throughout the development of NextGen systems. FAA did not do this in the development of past systems, a fact that led to schedule slippages and cost increases. (2) Improve collaboration of human factors efforts within FAA departments. (3) Establish strong leadership. A 2008 National Academy of Public Administration's report identified leadership as the single most important element of success for large-scale systems integration efforts like NextGen. FAA has not prioritized consistently staffing the top two human factors positions. Specifically, the position of the Chief Systems Engineer for Human Factors (now referred to as the human factors integration lead) has been vacant since January 2010. Moreover, FAA did not have a permanent program director of its Human Factors Research and Engineering Group from January 2009 until June 2010. These two positions currently lack the authority to ensure that human factors issues are addressed early and throughout the NextGen system development process to prevent the need to redesign these systems after implementation, which can cause delays and add costs. As a result, FAA may lack consistent leadership with the sufficient authority to not only prioritize human factors issues but ensure that human factors issues are addressed throughout NextGen. FAA should (1) create a coordination plan and (2) give priority to filling vacant leadership positions and provide the positions with authority for prioritizing human factors. FAA agreed to consider the recommendations.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director:
Gerald L. Dillingham
Team:
Government Accountability Office: Physical Infrastructure
Phone:
(202) 512-4803
GAO-10-824, Next Generation Air Transportation System: FAA and NASA Have Improved Human Factors Research Coordination, but Stronger Leadership Needed
This is the accessible text file for GAO report number GAO-10-824
entitled 'Next Generation Air Transportation System: FAA and NASA Have
Improved Human Factors Research Coordination, but Stronger Leadership
Needed' which was released on August 6, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Committee on Science and Technology, House of
Representatives:
United States Government Accountability Office:
GAO:
August 2010:
Next Generation Air Transportation System:
FAA and NASA Have Improved Human Factors Research Coordination, but
Stronger Leadership Needed:
GAO-10-824:
GAO Highlights:
Highlights of GAO-10-824, a report to the Committee on Science and
Technology, House of Representatives.
Why GAO Did This Study:
To address challenges to the aviation industry‘s economic health and
safety, the Federal Aviation Administration (FAA) is collaborating
with the National Aeronautics and Space Administration (NASA) and
other federal partners to plan and implement the Next Generation Air
Transportation System (NextGen). NextGen will transform the current
radar-based air traffic control system into a satellite-based system.
Pilot and air traffic controller roles and responsibilities are
expected to become more automated, thereby requiring an understanding
of human factors, which studies how humans‘ abilities,
characteristics, and limitations interact with the design of the
equipment they use, environments in which they function, and jobs they
perform. FAA and NASA are tasked with incorporating human factors
issues into NextGen.
As requested, this report discusses the extent to which FAA‘s and
NASA‘s human factors research (1) is coordinated and (2) supports
NextGen. To address these issues, GAO reviewed coordination mechanisms
and planning documents and synthesized the views of nine aviation
human factors experts.
What GAO Found:
While FAA and NASA officials are coordinating their NextGen human
factors research efforts in a variety of ways, they lack a cross-
agency human factors plan for coordination. FAA and NASA have
participated in research advisory committees and interagency research
transition teams, signed interagency agreements, and held cross-agency
meetings and conferences focused on human factors issues. FAA also
created a human factors portfolio to identify and address priority
human factors issues but not a cross-agency human factors coordination
research plan in cooperation with NASA, as previously recommended by
FAA‘s Joint Planning and Development Office (JPDO)--an interagency
organization responsible for planning NextGen. As a result, FAA has
not established an agreed-upon set of initial focus areas for research
that identifies and capitalizes on past and current research and
establishes focus areas for human factors research and development,
among other things.
The experts GAO contacted generally agreed that FAA‘s and NASA‘s human
factors research efforts adequately support NextGen, but made several
suggestions, including enhancing human factors research leadership,
for further incorporating human factors issues into NextGen systems.
FAA and NASA have undertaken a variety of human factors efforts to
support NextGen, including, among other things, creating planning
documents detailing how human factors research will be incorporated
into NextGen and dedicating financial resources specifically to
NextGen human factors research. While the human factors experts GAO
interviewed stated that these efforts support NextGen, a majority
offered the following suggestions for further integrating human
factors issues into NextGen:
* Better ensure that human factors issues are fully integrated
throughout the development of NextGen systems. FAA did not do this in
the development of past systems, a fact that led to schedule slippages
and cost increases.
* Improve collaboration of human factors efforts within FAA
departments.
* Establish strong leadership. A 2008 National Academy of Public
Administration‘s report identified leadership as the single most
important element of success for large-scale systems integration
efforts like NextGen. FAA has not prioritized consistently staffing
the top two human factors positions. Specifically, the position of the
Chief Systems Engineer for Human Factors (now referred to as the human
factors integration lead) has been vacant since January 2010.
Moreover, FAA did not have a permanent program director of its Human
Factors Research and Engineering Group from January 2009 until June
2010. These two positions currently lack the authority to ensure that
human factors issues are addressed early and throughout the NextGen
system development process to prevent the need to redesign these
systems after implementation, which can cause delays and add costs. As
a result, FAA may lack consistent leadership with the sufficient
authority to not only prioritize human factors issues but ensure that
human factors issues are addressed throughout NextGen.
What GAO Recommends:
FAA should (1) create a coordination plan and (2) give priority to
filling vacant leadership positions and provide the positions with
authority for prioritizing human factors. FAA agreed to consider the
recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-10-824] or key
components. For more information, contact Gerald L. Dillingham at
(202) 512-2834 or dillinghamg@gao.gov.
[End of section]
Contents:
Letter:
Background:
Although FAA and NASA Are Generally Coordinating Their Human Factors
R&D, Key Areas Could Benefit from Increased Attention:
FAA and NASA Have Several Ongoing Human Factors R&D Efforts That
Support NextGen, but a Majority of Experts Suggest FAA and NASA Adopt
Additional Measures, Including Strengthening Human Factors Leadership:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Scope and Methodology:
Appendix II: GAO Contact and Staff Acknowledgments:
Table:
Table 1: Experts Providing Responses:
Figures:
Figure 1: FAA and NASA Funding for Human Factors R&D, Fiscal Years
2004-2009:
Figure 2: FAA and Human Factors Research, Engineering, and Development
Funding, Fiscal Years 2004-2009:
Figure 3: Key FAA and NASA Organizations Involved in Human Factors
Activities:
Abbreviations:
ATOP: Advanced Technologies and Oceanic Procedures:
DOD: Department of Defense:
DOT: Department of Transportation:
ERAM: En Route Automation Modernization:
FAA: Federal Aviation Administration:
HFREG: Human Factors Research and Engineering Group:
JDPO: Joint Planning and Development Office:
NASA: National Aeronautics and Space Administration:
NATCA: National Air Traffic Controllers Association:
NGII: NextGen Integration and Implementation:
R&D: research and development:
REDAC: Research, Engineering, and Development Advisory Committee:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
August 6, 2010:
The Honorable Bart Gordon:
Chairman:
The Honorable Ralph Hall:
Ranking Member:
Committee on Science and Technology:
House of Representatives:
The aviation industry is critical to the nation's economic health and
safety, accounting for over $1 trillion in economic activity annually
and handling about 50,000 flights per day while also maintaining a
high level of safety. The Department of Transportation's (DOT) Federal
Aviation Administration (FAA) and the aviation industry, however, face
several challenges to the industry's economic health and safety,
including increases in demand and congestion. To meet these
challenges, FAA is taking the lead in transforming the current air
traffic control system to a new system--the Next Generation Air
Transportation System (NextGen). Whereas the current system is radar-
based, NextGen is a more automated, aircraft-centric, satellite-based
system. FAA is charged with implementing NextGen by 2025 with the
cooperation and collaboration of the National Aeronautics and Space
Administration (NASA) and other federal partners.[Footnote 1]
Under NextGen, pilots and air traffic controllers will rely to a
greater extent on automation, changing their roles and
responsibilities in ways that will require a full understanding of
what are known as human factors issues. Human factors research and
development (R&D) studies how humans' abilities, characteristics, and
limitations interact with the design of the equipment they use, the
environments in which they function, and the jobs they perform. FAA
and NASA--the primary agencies responsible for integrating human
factors issues into NextGen--must ensure that NextGen explores human
factors issues so that controllers, pilots, and others will operate
NextGen components in a safe and efficient manner. Without adequate
consideration of human factors issues, the implementation of NextGen
could be delayed.
You requested that we study FAA's and NASA's efforts with regard to
human factors related to NextGen. This report provides information on
the extent to which (1) FAA's and NASA's human factors R&D efforts are
coordinated and (2) FAA's and NASA's human factors R&D efforts support
NextGen. To provide information on human factors R&D coordination, we
obtained and analyzed information provided by FAA and NASA officials
on mechanisms in place to coordinate human factors R&D. We assessed
these coordination efforts by comparing them with recommendations
issued by FAA's Joint Planning and Development Office (JPDO)--an
interagency organization responsible for long-term NextGen planning
efforts. We also obtained information from nine aviation human factors
experts who reviewed and assessed FAA's and NASA's process for
coordinating their human factors R&D. All the experts have been
involved in aviation human factors R&D specifically directed at
NextGen systems.[Footnote 2] To provide information on the extent to
which the human factors efforts support NextGen, we reviewed relevant
planning documents from FAA and NASA and asked FAA and NASA officials
in interviews to describe their human factors R&D efforts in detail.
We provided these planning documents and information from the
interviews to the nine aviation human factors experts, as well as
representatives from three aviation industry associations, and
obtained and synthesized their views regarding the extent to which
FAA's and NASA's human factors R&D efforts support NextGen. In
addition, we obtained views from officials representing the Aerospace
Industries Association, Air Transport Association, Air Line Pilots
Association, MITRE Corporation,[Footnote 3] National Air Traffic
Controller Association, Volpe National Transportation Systems Center,
[Footnote 4] JPDO, and the Boeing Corporation.
We performed our work from August 2009 to August 2010 in accordance
with generally accepted government auditing standards. Those standards
require that we plan and perform the audit to obtain sufficient and
appropriate evidence to provide a reasonable basis for our findings
and conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Background:
Human factors is a discipline concerned with, among other things,
designing products that are efficient for people to use. As such,
human factors combines features of many disciplines, including
psychology, engineering, anthropology, sociology, and linguistics.
Human factors R&D focuses on people as they interact with the design
of products. The goal of human factors is to minimize the potential
for design-induced error by ensuring that the equipment is suitable
for the users and their environment. The human factors discipline can
be described as having two components: human factors research, which
seeks to acquire information, and human factors engineering, which
seeks to apply the information gained from research to equipment,
systems, software, and training, among other things.
Recognizing the importance of human factors considerations, FAA issued
a Human Factors Policy Order in 1993 that requires human factors
issues to be integrated into the planning and execution of all FAA
activities associated with system acquisitions and operations.
[Footnote 5] FAA offers several guidance documents on implementing
human factors considerations, which, FAA officials told us, helped
aviation stakeholders, such as contractors and research institutions,
meet the requirement. For example, officials with the MITRE
Corporation told us that--in collaboration with FAA, airlines, and
others--they researched human factors issues in the development of the
Automatic Dependent Surveillance-Broadcast System, which is an
information-reporting technology that, when used in conjunction with
other navigation technologies, is expected to enable more precise
information about aircraft position. MITRE collected human factors
data on how pilots use the broadcast system, collaborated with human
factors engineers, and asked human factors personnel to observe
pilots' in-flight interaction with the system while it was being
tested.
FAA has several offices that are tasked with ensuring that FAA
programs integrate human factors issues. FAA's Human Factors Research
and Engineering Group (HFREG) is responsible for conducting the human
factors R&D for NextGen, with the program director serving as the
principal advisor to the FAA Administrator on human factors issues.
HFREG is divided into three R&D areas: (1) Flight Deck/Aviation
Maintenance/System Integration, which develops human performance
information that the agency uses in fulfilling its regulatory
responsibility and provides to the aviation industry for use in
designing and operating aircraft and training pilots and maintenance
personnel; (2) Air Traffic Control/Technical Operations, which
researches human factors issues with respect to the roles of air
traffic controllers, air traffic managers, and maintenance
technicians; and (3) general Human Factors Research and Engineering,
which attempts to ensure that the incorporation of human factors
engineering is explicit, timely, systematic, comprehensive, efficient,
and effective. In fiscal year 2009, HFREG conducted dozens of R&D
activities including the following:
* Mitigating fatigue in flight operations. Collecting data on fatigue
variables (such as sleep patterns, alertness, and mood) to develop
better fatigue-mitigating duty and rest schedules, and outline limits
of acceptable performance and flight safety.
* Improving pilots' visual approaches through perceptual training.
Investigating the skills pilots need in order to effectively conduct a
visual approach,[Footnote 6] and developing training and performance
metrics that will improve training and evaluation of pilots on visual
approach tasks.
* Assessing safety risks. Calculating the safety risks of an error
occurring in relation to the amount of time a controller spends on a
task.
In addition, FAA has assigned human factors experts to several offices
involved in the development of new systems and in the oversight of
aircraft operation and maintenance in order to ensure that human
factors issues are addressed. FAA has established chief systems
engineers to focus on agencywide, cross-cutting technical and
operational issues pertaining to NextGen. Because of the scope of
NextGen, FAA contracted with Volpe to provide a chief system engineer
for human factors to identify and help the agency better ensure that
human factors issues are integrated into the development of NextGen
aviation systems. As a result of the observations and recommendations
of that Volpe expert, FAA has designated a new position for human
factors integration lead and assigned that position to FAA's System
Engineering and Safety organization.
NASA has two units primarily responsible for ensuring human factors
consideration in aviation: the Airspace Systems Program and the
Aviation Safety Program, both within its Aeronautics Research Mission
Directorate. The Airspace Systems Program is the unit chiefly
responsible for NASA's input into NextGen. The primary research role
for the Airspace Systems Program is to contribute to the operations of
the airspace system by developing concepts, capabilities, and
technologies for high-capacity, efficient, and safe airspace systems.
The Aviation Safety Program is dedicated to improving the safety of
current and future aircraft operating in the national airspace system.
The research focus is on the way aircraft are designed, built,
operated, and maintained. Scientists and engineers in this program
develop concepts and tools to address aircraft aging and durability,
among other areas.
FAA and NASA have each invested about $121 million in human factors
R&D from fiscal year 2004 to fiscal year 2009 (see figure 1).[Footnote
7] Starting in fiscal year 2005, NASA adjusted the size of its human
factors research staff by reassigning some staff to other programs and
reducing the contractor and academic technical support for human
factors R&D. NASA reorganized its aeronautical research plan to focus
on what it calls "fundamental research," which takes a technology to a
point where it can be further matured by manufacturers and eventually
integrated into new aircraft or engine designs. FAA's investment in
human factors R&D is increasing, along with additional appropriations
for overall research development, though overall R&D appears to be
increasing at a higher rate (see figure 2). NASA takes the lead in
both identifying human factors concepts that need to be implemented to
support a particular technology or system and developing the human
factors engineering models and algorithms. NASA then works with FAA on
testing the new concept and hands off the responsibility to FAA to
make the concept operational. NASA officials told us that it generally
takes a concept 5 to 7 years to become operational after NASA
transfers responsibility to FAA. Furthermore, in June 2010, NASA
officials informed us of a new Integrated Systems Research Program
that is to focus on maturing and integrating NextGen technologies into
operational systems. The program began in fiscal year 2010 at a
funding level of $62.4 million.
Figure 1: FAA and NASA Funding for Human Factors R&D, Fiscal Years
2004-2009:
[Refer to PDF for image: multiple line graph]
Fiscal year: 2004;
FAA: $17 million;
NASA: $20 million.
Fiscal year: 2005;
FAA: $21 million;
NASA: $18 million.
Fiscal year: 2006;
FAA: $18 million;
NASA: $20 million.
Fiscal year: 2007;
FAA: $18 million;
NASA: $27 million.
Fiscal year: 2008;
FAA: $19 million;
NASA: $19 million.
Fiscal year: 2009;
FAA: $29 million;
NASA: $19 million.
Source: GAO analysis of FAA and NASA data.
Note: FAA subsequently informed us that for fiscal year 2009 it
invested $10 million for human factors related to another budget line
item.
[End of figure]
Figure 2: FAA and Human Factors Research, Engineering, and Development
Funding, Fiscal Years 2004-2009:
[Refer to PDF for image: multiple line graph]
Fiscal year: 2004;
FAA research, engineering, and development funding: $118 million;
FAA human factors research and development funding: $17 million.
Fiscal year: 2005;
FAA research, engineering, and development funding: $130 million;
FAA human factors research and development funding: $21 million.
Fiscal year: 2006;
FAA research, engineering, and development funding: $137 million;
FAA human factors research and development funding: $18 million.
Fiscal year: 2007;
FAA research, engineering, and development funding: $130 million;
FAA human factors research and development funding: $18 million.
Fiscal year: 2008;
FAA research, engineering, and development funding: $147 million;
FAA human factors research and development funding: $19 million.
Fiscal year: 2009;
FAA research, engineering, and development funding: $170 million;
FAA human factors research and development funding: $29 million.
Source: GAO analysis of FAA and NASA data.
Note: As previously cited, FAA subsequently informed us that for
fiscal year 2009 it invested $10 million for human factors related to
another budget line item.
[End of figure]
NextGen is a major transformation of the aviation system that will
have significant implications for human factors considerations.
NextGen will transform aviation procedures and the design of the
aviation system and introduce new technologies that pose dramatic
changes to the roles and responsibilities of both air traffic
controllers and pilots and change the way they interface with their
systems. According to FAA, under NextGen, a satellite-based system
would guide all phases of a flight, including climb, cruise, descent,
and taxi. Instead of monitoring aircraft movements using ground-based
radar and transmitting voice flight instructions to aircraft, air
traffic controllers would primarily monitor automated systems and
intervene when anomalies and emergencies occur. As a result, FAA and
NASA need to research the human factors considerations associated with
the new roles of both flight crew and air traffic management staff,
and incorporate the results into the implementation of the new system.
In addition, FAA and NASA will have to identify and develop the
training necessary for these changing roles, including the time frame
before NextGen is fully realized, when some aircraft will be equipped
with NextGen systems and others will not.
FAA and NASA structure their NextGen human factors R&D according to a
planned three-phase implementation of the NextGen system to align and
prevent duplication of NextGen R&D efforts. FAA--which is ultimately
in charge of implementing NextGen--is mainly responsible for the R&D
to help address near-term implementation (2009-2013), which addresses
the day-to-day promotion of the safe and efficient operation of the
current aviation system and the implementation of some NextGen
systems, and midterm implementation (2012-2018), which consists of
leveraging existing aircraft capabilities and introducing new aircraft
capabilities to establish a foundation for a longer-term evolution of
the aviation system. Within FAA, the Air Traffic Organization is
responsible for implementing near-and midterm improvements in
coordination with other FAA lines of business. Within the Air Traffic
Organization, several offices have different roles in the development
of NextGen. For example, within the NextGen and Operations Planning
Office, the NextGen Integration and Implementation (NGII) office is
tasked with monitoring the progress of NextGen development and
implementation and facilitating necessary coordination. These offices
are also responsible for ensuring that human factors R&D conducted by
HFREG is integrated into NextGen. NASA is responsible for conducting
research to help address far-term implementation (2018-2025). As
researchers better define system concepts, NASA officials inform FAA
officials about research results and FAA officials then use the
results to further develop the system. Figure 3 shows the key FAA and
NASA organizations involved in human factors activities.
Figure 3: Key FAA and NASA Organizations Involved in Human Factors
Activities:
[Refer to PDF for image: 2 organization charts]
FAA:
Top level:
* Air Traffic Organization;
* Joint Planning Development Office.
Second level, reporting to Air Traffic Organization:
* NextGen and Operations Planning.
Third level, reporting to NextGen and Operations Planning:
* NextGen Integration and Implementation;
* Research and Technology Development:
- Human Factors Research and Engineering Group;
* System Engineering and Safety:
- Chief Systems for Human Factors Engineer.
NASA:
Top level:
Aeronautics Research Mission Directorate.
Second level, reporting to Aeronautics Research Mission Directorate:
* Airspace System Program;
* Aviation Safety Program.
Sources: FAA and NASA.
Note: FAA also has human factors experts in other units, including
experts in the Air Traffic Organization and Aviation Safety.
[End of figure]
Although FAA and NASA Are Generally Coordinating Their Human Factors
R&D, Key Areas Could Benefit from Increased Attention:
Participation in Advisory and Coordinating Bodies and Other Efforts
Help FAA and NASA Align Their Human Factors R&D:
FAA and NASA officials take advantage of a number of existing
mechanisms to coordinate their human factors R&D efforts. First, they
use the Research, Engineering, and Development Advisory Committee
(REDAC), which advises on FAA's research, engineering, and development
activities with experts from industry, academia, and other government
agencies. REDAC was established in 1989 to advise the FAA
Administrator on research and development needs in human factors, air
traffic services, airport technology, aircraft safety, and
environmental issues. According to officials from both agencies, their
collaboration on REDAC helps to coordinate human factors R&D efforts.
One of the REDAC subcommittees is devoted to human factors, and
according to officials with HFREG and NGII, has provided important
perspectives on research management and coordination among agencies,
including human factors R&D. Several REDAC subcommittees have held
meetings at NASA to facilitate its participation and ensure that REDAC
is briefed on relevant NASA human factors projects as well as FAA's
human factors R&D efforts. NASA officials also use REDAC to brief FAA
officials on their human factors R&D efforts as well.
In 2007, FAA and NASA took steps to better coordinate their human
factors efforts as a direct result of REDAC's influence. The REDAC
human factors subcommittee recommended that FAA and NASA exchange
information about their human factors R&D efforts to better facilitate
research coordination, which FAA and NASA did. In addition, in 2009,
the subcommittee noted that while the agencies had improved
coordination of human factors R&D, they could further improve
coordination of FAA and NASA human factors R&D related to the NextGen
Controller Efficiency Program.[Footnote 8] In response, officials with
HFREG and NGII told us that they now review NASA human factors
research announcements to determine their applicability for FAA
NextGen R&D. NASA proposals encompass research that includes human
factors issues as part of the proposed work.
In addition, FAA and NASA take advantage of existing forums, meetings,
and interagency agreements to coordinate their human factors R&D
efforts. Officials with HFREG and NGII told us that FAA and NASA
exchange R&D results through reports, presentations, and joint panel
discussions at various seminars and professional conferences,
including the annual Human Factors and Ergonomics Society
conference.[Footnote 9] FAA officials added that they also attend
NASA's technical interchange meetings to share ideas, learn of NASA's
human factors research efforts, and coordinate research projects. FAA
also exchanges R&D planning documentation with NASA annually and as
needed to facilitate human factors R&D coordination activities.
The agencies also have undertaken specific efforts to coordinate human
factors R&D related to NextGen. FAA established research transition
teams to address research gaps and coordinate research between FAA and
NASA related to the primary NextGen systems. In September 2008, we
reported that FAA and NASA established four research transition teams
[Footnote 10] to outline how the two agencies will jointly develop
research requirements.[Footnote 11] These teams help FAA and NASA
identify R&D needed to implement NextGen and ensure that the research
is not only conducted but effectively transitioned to the implementing
agency. FAA is to provide requirements for users of the technologies,
while NASA is to conduct the research and provide an understanding of
the engineering rationale for design decisions. According to FAA,
these research transition teams facilitate coordination and transition
of new technologies and concepts related to NextGen, including human
factors components. For example, FAA and NASA are using the research
transition teams to coordinate human factors research on the roles and
responsibilities of air traffic controllers and pilots, as well as
their information needs and procedures, among other issues.
In addition, over the past several years, FAA and NASA officials have
established memorandums and interagency agreements that allow the
agencies to collaborate on research projects and coordinate human
factors R&D related to NextGen. The agreements include reimbursable
interagency agreements between HFREG and NASA to leverage resources.
According to interagency agreements and FAA officials, leveraging
activities include:
* researching, modeling, and testing the advanced technologies,
automation, and services and capabilities that are required for
successful implementation of NextGen with particular emphasis on the
issues associated with the NextGen flight deck,
* allowing collaborative research to develop NextGen data
communications, human factors collision avoidance requirements,
aircraft merging and spacing separation assurance systems, and
guidance for use of NextGen synthetic vision systems, enhanced flight
vision systems, and advanced cockpit vision technologies, and:
* developing models, simulations, and demonstrations that will
quantify efficiencies and benefits for the included programs, and
evaluate the operational feasibility of concepts.
HFREG has approved or initiated 35 human factors research activities
in partnership with NASA, universities, and private corporations.
Supporting flight deck human factors efforts for NextGen, HFREG has
approved or initiated 22 NextGen human factors research activities.
FAA funds the activities and plans to budget $45 million for them
between fiscal year 2009 and fiscal year 2011. In addition, HFREG has
approved or initiated 13 NextGen air traffic control human factors
research activities. NASA, the Volpe National Transportation Systems
Center, and academic and private research facilities and institutions
are conducting much of the research, with the goal of providing
scientific and technical information to support development of NextGen-
related standards, procedures, training, policy and other guidance as
well as human factors assessments of NextGen technologies and
procedures. The research includes projects related to NextGen
communication systems, automation and human roles and
responsibilities, risk and error management, decision making, aircraft
separation assurance and collision avoidance, ground operations,
aircraft trajectory management, instrument procedures, personnel
training and qualifications, and single pilot operations. NASA
officials have agreed to consult HFREG officials about their NextGen
human and automation roles and responsibility research and inform them
about the research. In addition, FAA signed two 5-year interagency
agreements with NASA in 2009 to provide NASA up to $19 million in
funding for human factors research projects covering both flight deck
and air traffic control issues.
FAA Has Not Developed a Cross-Agency Coordination Plan:
While FAA and NASA officials have taken many steps to coordinate their
human factors R&D, JPDO issued a report in April 2008 that raised
concerns regarding FAA and NASA coordination of human factors R&D for
supporting NextGen.[Footnote 12] Specifically, JPDO reported that
there was no cross-agency plan for identifying and addressing priority
NextGen human factors issues and recommended that FAA, in cooperation
with NASA, develop such a plan. JPDO recommended that FAA initiate an
effort across agencies, industry, and academia to develop a cross-
agency plan for NextGen human factors R&D that:
* establishes focus areas for human factors research and development;
* inventories existing capabilities and laboratories for conducting
human factors R&D;
* capitalizes on past and current human factors research and, where
appropriate, reorients it; and:
* ensures that the agencies perform the appropriate human factors R&D
during the initial phases of NextGen.
HFREG developed a human factors R&D portfolio in 2009 as part of its
effort to improve cross-agency coordination of NextGen human factors
R&D. Officials added that the portfolio is the beginning of their
attempt to meet JPDO's recommendation to develop a cross-agency human
factors research plan. The portfolio lists and describes all past,
ongoing, and planned NextGen human factors R&D projects. HFREG
officials stated that the portfolio demonstrates the extent to which
FAA and NASA human factors R&D efforts are aligned, and described the
portfolio as a repository of NextGen human factors R&D. They added
that the portfolio is intended to assist NextGen researchers in
developing concepts, establishing requirements, identifying research
gaps, and determining additional research and engineering
considerations.
FAA's human factors portfolio is a good step toward better
coordinating human factors R&D, but does not currently satisfy JPDO's
cross-agency plan recommendation. Our review of the FAA portfolio
indicates that it is a listing and description of R&D projects and
results, but not a cross-agency plan with features characteristic of
plans, such as role definitions, goals, and time frames. Likewise, the
DOT Inspector General reported in April 2010 that FAA has not
developed a cross-agency research plan to identify and address how
NextGen will affect the roles of controllers and pilots and help
ensure that new concepts and technologies can be safely implemented.
[Footnote 13] The Inspector General observed that such a plan would
establish an agreed-upon set of initial focus areas for research,
provide inventories of existing facilities for research, and
capitalize on past and current research because both NASA and FAA
conduct human factors work specifically for air traffic management.
A cross-agency plan could help better ensure that FAA and NASA follow
key collaboration practices. We have previously reported that federal
agencies must effectively collaborate in order to deliver results more
efficiently and in a way that is consistent with their multiple
demands and limited resources.[Footnote 14] We identified several
practices that could enhance and sustain collaboration efforts,
including agreeing on roles and responsibilities, establishing
mutually reinforcing or joint strategies, and establishing compatible
policies, procedures, and other means to operate across agency
boundaries, among other things. A cross-agency coordinating plan that
establishes an agreed-upon set of initial focus areas for research,
inventories existing facilities for research, and capitalizes on past
and current research would help FAA and NASA more closely follow key
practices for enhancing and sustaining collaboration.
Experts' Opinions on Sufficiency of Human Factors Coordination Efforts
Vary:
Our panel of nine human factors experts had mixed views about FAA's
and NASA's efforts to improve coordination of their human factors R&D
efforts. While some experts told us that the steps the agencies have
taken in response to JPDO and REDAC recommendations are sufficient,
others suggested that FAA and NASA could do more to improve their
human factors coordination. Similarly, officials representing two
aviation associations had mixed views regarding coordination; one
association stated that NASA and FAA are well coordinated, while
another stated that FAA and NASA need to provide more clarity and
consensus on their coordination plans.[Footnote 15]
Four of the nine experts stated that FAA and NASA were coordinating
well on human factors research related to NextGen and did not suggest
further actions the agencies could take to better coordinate research.
However, five experts stated that FAA and NASA could better coordinate
human factors research. They suggested hosting additional human
factors conferences to improve coordination, and prioritizing
coordination of NextGen human factors research. More specifically, two
experts told us that while the agencies have held conferences and
research workshops (as previously discussed), they have not held
conferences specifically devoted to human factors research for
supporting NextGen. According to FAA officials, hosting such
conferences is very expensive, so HFREG tries to leverage hosting
sessions at external conferences and annual meetings. For example, FAA
officials sponsored a session on human factors issues related to
NextGen at the Human Factors and Ergonomics Society's Aerospace
Systems Technical Group meeting in May 2008 and plan to hold another
similar session at this year's annual meeting in September.
FAA and NASA Have Several Ongoing Human Factors R&D Efforts That
Support NextGen, but a Majority of Experts Suggest FAA and NASA Adopt
Additional Measures, Including Strengthening Human Factors Leadership:
FAA and NASA Have Undertaken a Variety of Human Factors R&D Efforts
and Designated Funding to Support NextGen:
FAA and NASA have created and shared planning documents for how the
agency will incorporate human factors R&D into NextGen. As previously
noted, FAA has taken steps to standardize the way it integrates human
factors considerations into all aviation projects. To this end, FAA
developed a NextGen Human System Integration Roadmap to identify and
address human factors R&D needs for supporting NextGen in particular.
In addition, as previously discussed, FAA created the Human Factors
Portfolio, which lists and describes all past, ongoing, and planned
NextGen human factors R&D projects. According to FAA, the portfolio
was intended to identify potential gaps and unfunded R&D needs across
midterm and potential far-term operational improvements for NextGen.
Although we find it currently lacking as a coordination tool, it does
enumerate the NextGen projects that are under way, which could be
useful in terms of monitoring the efforts of other stakeholders.
In addition, HFREG officials told us that FAA has a range of human
factors R&D initiatives that support NextGen. FAA not only conducts
focus groups and interviews with a panel of human factors experts, but
also conducts live simulations and field trials to evaluate system and
human performance in different scenarios. For example, FAA conducted
human simulations with pilots and air traffic controllers in fiscal
year 2008 and planned further simulations for its High Density Airport
Capacity and Efficiency Improvement Project in fiscal year 2009.
[Footnote 16] The agency also conducts field surveys and interviews of
operational personnel that are extensively used to address major
NextGen and other aviation human factors issues that have an impact on
the workforce. For example, FAA plans to conduct a survey to assess
the degree of fatigue in the controller workforce.
NASA also has human factors research efforts that support NextGen.
Officials told us that NASA experiments with early concept
technologies that will involve human interaction, thereby fully
leveraging the strengths and mitigating the weaknesses of both the
human and automated components. NASA staff then conduct simulations to
test human compatibility and subsequently help FAA develop the
technologies that prove themselves capable of supporting NextGen.
Over the last 2 years, FAA has also dedicated financial resources
specifically to incorporating human factors R&D into NextGen. Prior to
fiscal year 2008, FAA used funding from its overall human factors R&D
budget for NextGen projects, one of various types of human factors
R&D; however, since fiscal year 2008, FAA has had a specific human
factors research and development budget for NextGen. To incorporate
human factors issues into NextGen, for example, conducting additional
human simulations and field trials, FAA invested $25.5 million in
human factors R&D specifically dedicated to NextGen from fiscal year
2008 through fiscal year 2010,[Footnote 17] and has requested
additional funding for fiscal year 2011 to fiscal year 2013. NASA
officials told us that NASA conducts applied human factors research
across its Aviation Safety and Airspace Systems programs and does not
have a specific line item budget for NextGen. According to these
officials, this research addresses human factors considerations for
new concepts and technologies applicable to NextGen. In addition,
NASA's Aeronautics Research Mission Directorate programs were
realigned in 2006, causing difficulty in assessing funding trends
across several years of similar research activities.
Experts Cited Suggestions Regarding Human Factors Efforts Supporting
NextGen:
For the most part, aviation human factors experts we interviewed
stated that FAA's and NASA's human factors R&D efforts adequately
support NextGen. For example, experts commended FAA and NASA for
appropriately conducting human factors R&D according to the three-
phase implementation structure for NextGen systems. As previously
mentioned, FAA is mainly responsible for R&D to support near-term
implementation and midterm implementation, while NASA conducts much of
the research to address far-term implementation. One expert also told
us that FAA, in response to REDAC input, has developed a good method
for understanding likely human performance. NASA also has modeled
NextGen systems to predict how beneficial NextGen systems will be to
users. However, a majority of experts offered suggestions for further
incorporating human factors issues into NextGen. Experts specifically
identified the following suggestions:
Better ensure that human factors issues are fully integrated
throughout design and development of NextGen systems. Human factors
must be considered and integrated throughout the design and
development of aviation systems. Failure to fully consider human
factors issues at all stages can increase costs and delay projects.
Six of nine experts and a senior official at the Volpe National
Transportation Systems Center were concerned that NextGen developers
may not be adequately considering human factors R&D throughout the
entire NextGen planning and implementation process.[Footnote 18] FAA
has not fully integrated human factors considerations into the
development of some aviation systems. For example, FAA did not fully
address human factors considerations in developing the En Route
Automation Modernization (ERAM) system, which FAA plans to complete by
2010.[Footnote 19] According to the National Air Traffic Controllers
Association (NATCA), air traffic controllers involved in initial
operations capabilities tests at an air traffic control center in Salt
Lake City have come across significant problems with using the system.
According to NATCA, controllers have found the new formats cumbersome,
confusing, and difficult to navigate, thus indicating that FAA did not
adequately involve those who operate the system (controllers) in the
early phases of system development. As a result, to better ensure
optimal performance of ERAM, FAA will have to address these human
factors issues before it deploys the new system. This could increase
the costs or delay the implementation of other components of NextGen,
such as the previously mentioned Automatic Dependent Surveillance-
Broadcast System, since the operation of numerous NextGen components
will depend on this new system. FAA officials within the En Route
Automation Modernization office agreed with NATCA's views on the new
system and added that the simulation capabilities of its Technical
Center in Atlantic City, New Jersey, where the agency conducts human
factors testing, were not robust enough to capture all of the problems
subsequently identified by controllers.[Footnote 20] In May 2010,
however, FAA announced the building of an Aviation Research and
Technology Park near FAA's Technical Center to provide a central
location for partners in academia, industry, and other state and
federal government agencies to work on NextGen. According to FAA, the
park is being built with no direct cost to FAA and has amassed $3.5
million in grant funding. In June 2010, FAA issued a task order to
MITRE Corporation to conduct a programmatic review of the ERAM problem
and make an assessment of what circumstances led to the current delay,
among other things. The MITRE Corporation is expected to issue a final
report on October 1, 2010.
Similarly, in reviewing the development of the Operational and
Supportability Implementation System,[Footnote 21] the Department of
Transportation's Inspector General reported that FAA identified a
number of significant human factors concerns with the system, such as
inadequately addressing weather information.[Footnote 22] The
Inspector General concluded that system developers did not adequately
consider human factors research throughout design and development,
thereby contributing to the delay of the system's implementation.
Similarly, as noted in a report we issued in 2005,[Footnote 23] FAA's
failure to provide adequate attention to human factors issues when
implementing the Standard Terminal Automation Replacement System
resulted in schedule slips and a significant cost increase of $500
million.[Footnote 24] As noted, however, since fiscal year 2008, FAA
has designated funding solely for human factors R&D supporting
NextGen. It remains to be seen if FAA's added emphasis on human
factors research and engineering will better ensure that human factors
issues are fully integrated into the development of future NextGen
components.
Ensuring the mitigation of human factors issues also involves
oversight of contractors. HFREG officials told us that they do not
track vendors to make sure they are considering human factors R&D
issues in their development, as this is a responsibility of the
program managers who lead procurement efforts for FAA systems.
However, once contracts are awarded, contractors are supposed to
follow the contract specifications, which can include human factors
system performance requirements. HFREG officials told us that in the
past they collaborated with program office human factors coordinators
to assess outside vendors' compliance with human factors issues; they
found that the contractors were not in compliance in all aspects,
particularly human factors. In April 2010, the Department of
Transportation's Inspector General also expressed concern about FAA's
ineffective oversight of a contractor in developing NextGen systems,
adding that NextGen implementation will require significant contract
oversight.[Footnote 25] Furthermore, FAA's post-implementation review
of the Advanced Technologies and Oceanic Procedures system concluded
that FAA and the contractors who developed the system did not, from a
human factors perspective, develop the system to meet FAA's needs. The
post-implementation review recommended that for future systems, FAA
should ensure that it articulates to contractors in unambiguous terms
the human factors-related characteristics that the proposed system
must meet. According to the Chief Scientist for NextGen and Operations
Planning, a contractor developing an aviation system may have
implemented human factors designs that were originally flawed or may
have had a flawed methodology for incorporating human factors issues
into system development. FAA program offices and contractors often
support the incorporation of human factors consideration in a system
by convening a panel of controllers and obtaining their feedback. Such
a method may result in the controllers providing information regarding
their preferences instead of information regarding the usability of
the system to the controller panel. An alternative method may be to
conduct a modeling effort that analyzes data on human performance for
certain components of the system. HFREG officials also noted that
under the best of circumstances, all major and most human factors
issues should be identified and mitigated during system development,
making it unusual for additional problems to arise when a system is
being implemented. To address this issue, experts stated FAA should
ensure system developers consider human factors in all phases of the
development of aviation systems (as required by the Human Factors
Policy Order). Having oversight of system developers (including
contractors) that develop NextGen systems to make sure they adhere to
FAA's Human Factors Policy Order would significantly reduce the
possibility of expensive and untimely delays. FAA has taken action to
improve its oversight of contractors. For example, in its June 2010
letter to MITRE, FAA requested an assessment of the ERAM contractor's
program management procedures and practices as part of an overall
review of the program.
Improve collaboration of human factors efforts across FAA departments.
Collaboration within FAA departments is important to ensure that
aviation systems are designed and developed with agency input from
human factors researchers. Several experts we interviewed stated that
system development projects with a human factors research component
take place in different departments and offices at FAA, and that those
developing the systems do not always collaborate. While HFREG provides
R&D and engineering support, HFREG officials told us that there is no
requirement for program offices or developers to consult with HFREG.
HFREG conducted a post-implementation review of the Advanced
Technologies and Oceanic Procedures that implied that system managers
did not properly consider human factor issues.[Footnote 26] This
suggests that the system managers either did not consult human factors
stakeholders (including HFREG) or did not fully address their human
factors issues through a collaborative working relationship. As a
result, the post-implementation review concluded that from a human
factors perspective, the system that was implemented in the field was
not the system FAA had asked for. FAA's experience in developing the
Advanced Technologies and Oceanic Procedures is an indication of what
can happen when system developers fail to collaborate with human
factors specialists and develop a comprehensive human factors program.
To improve collaboration, HFREG officials also told us that the Chief
Scientist of the NextGen and Operations Planning unit sponsored a
technical interchange meeting in January 2010 to better ensure that
all FAA units involved in NextGen development are aware of the need to
fully consider human factors in their work. The Chief Scientist plans
to host another technical interchange meeting on July 29, 2010.
Strong Leadership Would Improve Consideration of Human Factors Issues:
A majority of the experts we interviewed agree that strong leadership
is needed to provide adequate consideration of human factors issues
within NextGen. Furthermore, a September 2008 National Academy of
Public Administration's report identified leadership as the single
most important element of success for large-scale systems integration
efforts like NextGen. That report highlighted leadership as a NextGen
implementation challenge. The critical impact of human factors issues
on NextGen indicates that human factors issues require strong
leadership to ensure they are a priority for NextGen. FAA has not
prioritized consistently staffing the top two leadership positions
within FAA that are formally responsible for human factors R&D.
Specifically, the Chief Systems Engineer for Human Factors position
has been vacant since the previous chief retired in January 2010.
[Footnote 27] Moreover, FAA did not assign a permanent program
director of HFREG for 16 months, from January 2009 until FAA filled
the position in June 2010.[Footnote 28]
The leadership void was the issue most frequently identified by the
nine experts. Seven of nine experts we interviewed told us that the
lack of leadership within FAA is a significant challenge in ensuring
that human factors R&D supports NextGen.[Footnote 29] Although a
majority of the experts were concerned that the leadership void could
have prevented human factors issues from being fully considered for
NextGen, subsequently delaying the implementation of a system, none
could identify any specific examples. Nevertheless, FAA officials
emphasized the importance of both positions. FAA officials told us
that the Chief System Engineer position could be pivotal in
integrating and maximizing the effectiveness of human factors in
support of NextGen and is thus critical to prioritizing NextGen
research and resources within FAA. JPDO officials we interviewed
stressed that the program director of HFREG is the single most
important position needed to ensure that the necessary human factors
R&D is conducted and that the results are integrated into the
development of NextGen systems.
According to FAA officials, FAA has not had a chance to fill the
position of Chief System Engineer--which FAA now refers to as the
human factors integration lead--because of a hiring freeze and
uncertainty as to which unit to put the position. FAA has resolved
those issues and plans to begin the process for filling the position.
Officials cautioned, however, that it may take a long time to find a
qualified candidate with the right human factors expertise and other
relevant skill sets. Nonetheless, FAA would like to fill the position
by the close of fiscal year 2010. FAA officials also told us that it
took a long time to fill the position of program director for HFREG,
in part because of the long process of completing required personnel
administrative procedures. The new program director of HFREG was
formerly the acting program director and had been in that position
since the previous program director left. The assignment to program
director involved a change in the position classification that
involved several time-consuming administrative procedures to address,
according to HFREG officials and an FAA senior executive.
Experts also told us in filling these positions that the new leaders
should have adequate authority to make sure that human factors issues
are considered (particularly early in system development) and
prioritized during all phases of NextGen development. These positions
currently lack the authority to ensure that human factors issues are
addressed early and throughout the NextGen system development process.
Such authority could mitigate the need to redesign these systems after
implementation has begun, which can cause delays and add costs. For
example, as previously discussed, it has been found that FAA's human
factors plans have not adequately addressed how humans will use newly
developed NextGen weather information. One of the experts we consulted
who has worked extensively with FAA on human factors R&D told us that
a program director of the HFREG or Chief System Engineer who has
adequate authority could have reviewed the weather information to
ensure that human factors were fully integrated into that and other
NextGen systems. However, in filling the position of program director
of HFREG, FAA did not authorize the new program director with
additional authority to review NextGen programs and ensure that human
factors issues are addressed. HFREG officials told us that FAA is
conducting a review of distribution among HFREG, service units, and
other offices for responsibility and authority to conduct human
factors activities to better serve the human factors needs of NextGen.
Conclusions:
Human factors research must be incorporated into NextGen to ensure
that controllers, pilots, and other aviation system users can operate
NextGen in a safe and efficient manner. To this end, FAA and NASA have
pursued a wide range of efforts to incorporate human factors R&D into
NextGen. However, these and future efforts will require a sustained
focus not only across agencies but from the beginning to the end of
the long process of developing a complex system like NextGen. Some
suggest that FAA can meet this challenge by incorporating two elements
into its human factors R&D efforts:
* a cross-agency plan developed in cooperation with NASA to identify,
prioritize, and coordinate NextGen human factors issues, and:
* strong and consistent leadership with the authority to not only
prioritize human factors issues but ensure that they are taken into
account throughout NextGen.
Recommendations for Executive Action:
We recommend that the Secretary of Transportation direct the FAA
Administrator to take the following two actions:
* create a cross-agency human factors coordination plan in cooperation
with NASA, as JPDO has previously recommended, that establishes an
agreed-upon set of initial focus areas for research, inventories
existing facilities for research, and capitalizes on past and current
research of all NextGen issues, and:
* assign a high priority to filling the vacancy of human factors
integration lead and structure that position and the program director
of HFREG position in a manner that provides the authority to ensure
that human factors research and development is coordinated,
considered, and prioritized in all phases of NextGen development.
Agency Comments:
We provided a draft of this report to the Department of Transportation
and NASA for review and comment. NASA had no comments. DOT agreed to
consider the recommendations and provided technical clarifications,
which we incorporated into the report as appropriate.
We are sending copies of this report to the Secretary of
Transportation, FAA, NASA, and interested congressional committees.
The report is also available at no charge on GAO's Web site at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-2834 or dillinghamg@gao.gov. Contact points
for our Offices of Congressional Relations and Public Affairs may be
found on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix II.
Signed by:
Gerald L. Dillingham, Ph.D.,
Director, Physical Infrastructure Issues:
[End of section]
Appendix I: Scope and Methodology:
In response to your request, this report provides information on the
status of the Department of Transportation's Federal Aviation
Administration's (FAA) and National Aeronautics and Space
Administration's (NASA) efforts to incorporate human factors issues
into the Next Generation Air Transportation System (NextGen). In
particular, we sought to identify the extent to which (1) FAA's and
NASA's human factors research and development (R&D) is coordinated,
and (2) FAA's and NASA's human factors R&D supports NextGen.
In determining the extent to which FAA's and NASA's human factors R&D
is coordinated, we obtained and analyzed information provided by FAA
and NASA officials on mechanisms in place to align human factors R&D
efforts. We asked FAA and NASA officials to describe the mechanisms
that are in place to coordinate the agency's human factors R&D. We
assessed the information FAA and NASA officials provided us regarding
their coordination mechanisms by comparing those efforts with
recommendations issued by the Joint Planning and Development Office
(JPDO)--an interagency organization responsible for planning NextGen.
In 2008, JPDO issued a cross-agency gap analysis that found FAA and
NASA lacked a cross-agency plan for identifying and addressing
priority NextGen human factors issues. We also assessed FAA's and
NASA's coordination efforts by summarizing the views of nine external
aviation human factors experts who reviewed and assessed FAA's and
NASA's coordination mechanisms. See our discussion below for more
detail regarding the nine aviation human factors experts. We also
obtained the views of several aviation industry officials, including
officials from the Aerospace Industries Association, Air Transport
Association, Air Line Pilots Association, MITRE Corporation,[Footnote
30] National Air Traffic Controller Association, JPDO, Volpe National
Transportation Systems Center, and the Boeing Corporation.[Footnote
31] We also reviewed relevant reports issued by GAO, the Inspector
General of the Department of Transportation, and the National Academy
of Public Administration.
In determining the extent to which FAA's and NASA's human factors R&D
supports NextGen, we obtained relevant planning documents from FAA and
NASA and had FAA and NASA officials provide us with detailed
descriptions of their human factors R&D efforts. We provided this
information and other related planning documents to nine aviation
human factors experts and representatives from three aviation industry
associations and asked them about their views on the extent to which
FAA's and NASA's human factors research supports NextGen. The experts
provided suggestions that FAA and NASA could adopt to better
incorporate human factors issues in developing NextGen, and we
reported the suggestions that a majority of experts recommended FAA
and NASA adopt. In addition, we obtained the views of several aviation
industry officials identified above.
In assessing FAA and NASA human factors R&D coordination and human
factors R&D supporting NextGen, we summarized the views of nine
aviation human factors experts. We took several steps to identify
potential aviation human factors experts. First, we identified experts
in human factors R&D that GAO had consulted in the past. We then asked
cognizant FAA and NASA officials responsible for and knowledgeable
about aviation-related human factors R&D to recommend experts in
aviation-related human factors R&D. In addition, we conducted
comprehensive Internet searches for human factors aviation experts.
Finally, we asked experts identified in the first four steps to
recommend other human factors aviation experts. Taking these steps
enabled us to identify 25 potential experts.
To make our final expert selection, we narrowed our selection of the
25 potential experts based on the following criteria:
* knowledge of aviation-related human factors research as determined
by published research, such as human factors research related to
aviation development, and:
* knowledge of NextGen planning and implementation needs as determined
by research, published work, and participation in NextGen seminars,
conferences, and workshops.[Footnote 32]
Applying the criteria listed above to the 25 potential experts
resulted in a final selection of 11 experts who have significant
knowledge in both aviation-related human factors R&D and more
specifically human factors R&D pertaining to NextGen. We obtained and
synthesized responses from 9 of the 11 aviation human factors experts.
The experts we obtained responses from are listed in table 1. We
interviewed an additional selected expert prior to finalizing our
methodology and incorporated the expert's views where appropriate in
this report.
Table 1: Experts Providing Responses:
Expert: Deborah A. Boehm-Davis;
Title and position: George Mason University Professor and Chair of the
Department of Psychology, Department of Psychology, Human Factors, and
Applied Cognition.
Expert: Kim Cardosi;
Title and position: Volpe National Transportation Systems Center
National Expert, Aviation Human Factors.
Expert: Frank Durso;
Title and position: Georgia Institute of Technology Professor of
Psychology.
Expert: R. John Hansman;
Title and position: Massachusetts Institute of Technology Professor of
Aeronautics and Astronautics, Department of Aeronautics and
Astronautics.
Expert: John Lauber;
Title and position: Airbus Industries Senior Vice President (retired)
Chief Product Safety Officer.
Expert: Raja Parasuraman;
Title and position: George Mason University Director PhD Program in
Human Factors and Applied Cognition.
Expert: Amy Pritchett;
Title and position: Georgia Institute of Technology Associate
Professor, School of Aerospace Engineering Joint Associate Professor,
School of Industrial and Systems Engineering .
Expert: Nadine Sarter;
Title and position: University of Michigan Associate Professor
Department of Industrial and Operations Engineering.
Expert: Christopher Wickens;
Title and position: University of Illinois Professor Visual Cognition
and Human Performance.
Source: GAO.
[End of table]
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
GAO Contact:
Gerald L. Dillingham, Ph.D., (202) 512-2834 or dillinghamg@gao.gov:
Staff Acknowledgments:
In addition to the contact above, other key contributors to this
report were Ed Laughlin, Assistant Director; Samer Abbas; Bert
Japikse; Richard Hung; Michael Mgebroff; Tina Paek; and Amy Rosewarne.
[End of section]
Footnotes:
[1] These partners include the Departments of Commerce (particularly
its National Oceanic and Atmospheric Administration), Defense,
Homeland Security, and Transportation, and the White House Office of
Science and Technology Policy.
[2] Appendix I lists the experts who provided information to GAO.
[3] MITRE is a not-for-profit organization chartered to work in the
public interest. MITRE manages four federally funded research and
development centers, including one for FAA. MITRE has its own
independent research and development program that explores new
technologies and new uses of technologies to solve problems in the
near term and in the future.
[4] The Volpe Center is part of the Department of Transportation's
Research and Innovative Technology Administration and is a federal,
fee-for-service organization whose mission is to improve the nation's
transportation system. Volpe's work is performed for the Department of
Transportation, as well as other federal, state, local, and
international agencies and entities.
[5] FAA Order 9550.8.
[6] A visual approach occurs when air traffic control authorizes an
aircraft to proceed visually to the airport of intended landing.
[7] FAA also uses other sources of funding to support human factors
functions. In addition, this amount does not include funding for human
factors R&D conducted at the MITRE federally funded research and
development center.
[8] The NextGen Controller Efficiency Program examines the roles of
the various actors in the national airspace system, including
controllers, pilots, dispatchers, and maintainers, to ensure safe
operations at increased capacity levels and how those roles are best
supported by allocation of functions between human operators and
automation.
[9] The society was founded in 1957 to promote the discovery and
exchange of knowledge concerning the characteristics of human beings
that are applicable to the design of systems and devices of all kinds.
[10] GAO, Next Generation Air Transportation System: Status of Systems
Acquisition and the Transition to the Next Generation Air
Transportation System, [hyperlink,
http://www.gao.gov/products/GAO-08-1078] (Washington, D.C. Sept. 11,
2008).
[11] The research transition teams are Efficient Flow into Congested
Airspace, Multi-Sector Planner, Dynamic Airspace Configuration, and
Integrated Arrival/Departure/Surface.
[12] JPDO consists of representatives from the Departments of
Transportation, Defense, Commerce, and Homeland Security, and NASA.
[13] Department of Transportation Inspector General, Testimony Before
the Committee on Transportation and Infrastructure, Subcommittee on
Aviation, U.S. House of Representatives, Challenges in Meeting FAA's
Long-Term Goals for the Next Generation Air Transportation System,
(Washington, D.C.: Apr. 21, 2010).
[14] GAO, Results-Oriented Government: Practices That Can Help Enhance
and Sustain Collaboration among Federal Agencies, [hyperlink,
http://www.gao.gov/products/GAO-06-15], (Washington, D.C.: Oct. 21,
2005).
[15] Officials from a third aviation association did not have an input
regarding FAA and NASA coordination.
[16] The High Density Airport Capacity and Efficiency Improvement
Project attempts to take advantage of existing ground technologies and
functionality as a first step toward trajectory-based operations. It
leverages airborne navigational capabilities that already exist on
most commercial production and many in-service airplanes.
[17] FAA also uses other sources of funding to support human factors
functions for NextGen. In addition, this amount does not include
funding for human factors R&D conducted at the MITRE federally funded
research and development center.
[18] The Volpe National Transportation Systems Center assesses the
activities and needs of the transportation community, evaluates
research and development activities in the technology community, and
assists in the application and deployment of new transportation
technologies and policies. It also addresses major national and
international transportation issues related to safety, security,
environment, mobility, and economic growth and trade.
[19] The En Route Automation Modernization program will replace the
primary computer system used to control air traffic. The new system
will replace software and hardware in the host computers at FAA's 20
en route air traffic control centers, which provide separation,
routing, and advisory information.
[20] Officials from FAA's Air Traffic Organization also stated that
the inclusion of human factors best practices for ERAM was extensive
and followed FAA's human factors policy order. They noted that air
traffic controllers and technical operations specialists were
extensively involved through structured human factors activities and
design and development of the system from very early in the design and
throughout the process. According to these officials, the current
problems with ERAM stem from the quality of information presented to
the operator.
[21] The Operational and Supportability Implementation System replaces
workstation consoles, among other things, at automated flight service
stations. It furnishes up-to-the-minute weather graphics by
integrating real-time weather and flight planning data with an overlay
of flight routes. It also provides operational support, retrieves
reports, and supplies lighting data and icing images.
[22] Department of Transportation Inspector General, Report on
Automated Flight Service Stations: Significant Benefits Could Be
Realized by Consolidating Sites in Conjunction With Deployment of
OASIS, (Washington, D.C.: Dec. 7, 2001).
[23] GAO, National Airspace System: FAA Has Made Progress but
Continues to Face Challenges in Acquiring Major Air Traffic Control
Systems, [hyperlink, http://www.gao.gov/products/GAO-05-331]
(Washington, D.C.: June 10, 2005).
[24] The Standard Terminal and Automation Replacement System is a
joint program of FAA, the Department of Defense (DOD), and DOT. It
replaces aging FAA and DOD terminal systems with state-of-the-art
terminal air traffic control systems. The system is designed to
prevent duplication of development and logistic costs.
[25] FAA, Advanced Technologies & Oceanic Procedures (ATOP): Human
Factors Post Implementation Review, (Washington, D.C.: July 2008).
[26] The Advanced Technologies and Oceanic Procedures system replaced
FAA's systems and procedures responsible for separating aircraft over
the oceans, enabling controllers to reduce spacing between aircraft in
flight. Now fully deployed, ATOP is an integrated air traffic control
system for the U.S.-controlled oceanic airspace. In fiscal year 2006,
ATOP was in place at all three oceanic sites: the Oakland, California;
New York, New York; and Anchorage, Alaska, Air Route Traffic Control
Centers.
[27] The previous chief was also operating on a part-time basis.
[28] FAA appointed a HFREG program director in January 2009, but the
official was detailed to the Department of Transportation and never
served in the position. That official is now the JPDO Director. FAA
officials told us that personnel rules precluded assignment of another
person to permanently fill the program director position while the
appointed official was on detail.
[29] An eighth expert stated that filling the positions would benefit
NextGen only if those positions had more authority.
[30] MITRE is a not-for-profit organization chartered to work in the
public interest. MITRE manages four federally funded research and
development centers, including one for FAA. MITRE has its own
independent research and development program that explores new
technologies and new uses of technologies to solve problems in the
near term and in the future.
[31] Despite several attempts, we were unable to obtain interviews
with representatives from other aviation associations, including the
Air Traffic Control Association and RTCA Inc.--a private, not-for-
profit corporation that develops consensus-based recommendations on
communications, navigation, surveillance, and air traffic management
issues.
[32] For both selection criteria, we used Internet searches to
determine the extent to which identified experts had knowledge of
aviation-related human factors research and NextGen planning and
implementation.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: