Federal Aviation Administration
Stronger Architecture Program Needed to Guide Systems Modernization Efforts Gao ID: GAO-05-266 April 29, 2005The Federal Aviation Administration's (FAA) mission is to promote the safe, orderly, and expeditious flow of air traffic in the U.S. airspace system. To this end, FAA is modernizing its air traffic control systems, a multibillion dollar effort that GAO has designated as a high-risk program. GAO's research into the practices of successful public- and private-sector organizations has shown that developing and using an enterprise architecture, or blueprint, to guide and constrain systems investments is crucial to the success of such a modernization effort. GAO was asked to determine whether FAA has established effective processes for managing the development and implementation of an enterprise architecture.
FAA has two architecture projects--one for its National Airspace System (NAS) operations and one for its administrative and mission support activities--that together constitute its enterprise architecture program. However, it has established only a few of the management capabilities for effectively developing, maintaining, and implementing an architecture. For example, the agency reports that it has allocated adequate resources to the projects, and it has established project offices to be responsible for developing the architecture, designated a chief architect for each project, and released Version 5.0 of its NAS architecture. But the agency has yet to establish other key architecture management capabilities--such as designating a committee or group that represents the enterprise to direct, oversee, or approve the architecture, and establishing an architecture policy. FAA agreed that the agency needs an effective enterprise architecture program and stated that it plans to improve its management of both projects. For example, the agency intends to establish a steering committee; develop a policy that will govern the development, maintenance, and implementation of the architecture program; and have an approved architecture project management plan for the non-NAS architecture. GAO's experience in reviewing other agencies has shown that not having an effective enterprise architecture program can be attributed to, among other things, an absence of senior management understanding and support and cultural resistance to having and using one. It has also shown that attempting major systems modernization programs like FAA's without having and using an enterprise architecture often results in system implementations that are duplicative, are not well integrated, require costly rework to interface, and do not effectively optimize mission performance.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-266, Federal Aviation Administration: Stronger Architecture Program Needed to Guide Systems Modernization Efforts
This is the accessible text file for GAO report number GAO-05-266
entitled 'Federal Aviation Administration: Stronger Architecture
Program Needed to Guide Systems Modernization Efforts' which was
released on May 31, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
April 2005:
Federal Aviation Administration:
Stronger Architecture Program Needed to Guide Systems Modernization
Efforts:
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-05-266]:
GAO Highlights:
Highlights of GAO-05-266, a report to congressional requesters
Why GAO Did This Study:
The Federal Aviation Administration‘s (FAA) mission is to promote the
safe, orderly, and expeditious flow of air traffic in the U.S. airspace
system. To this end, FAA is modernizing its air traffic control
systems, a multibillion dollar effort that GAO has designated as a high-
risk program. GAO‘s research into the practices of successful public-
and private- sector organizations has shown that developing and using
an enterprise architecture, or blueprint, to guide and constrain
systems investments is crucial to the success of such a modernization
effort.
GAO was asked to determine whether FAA has established effective
processes for managing the development and implementation of an
enterprise architecture.
What GAO Found:
FAA has two architecture projects”one for its National Airspace System
(NAS) operations and one for its administrative and mission support
activities”that together constitute its enterprise architecture
program. However, it has established only a few of the management
capabilities for effectively developing, maintaining, and implementing
an architecture. For example, the agency reports that it has allocated
adequate resources to the projects, and it has established project
offices to be responsible for developing the architecture, designated a
chief architect for each project, and released Version 5.0 of its NAS
architecture. But the agency has yet to establish other key
architecture management capabilities”such as designating a committee or
group that represents the enterprise to direct, oversee, or approve the
architecture, and establishing an architecture policy. FAA agreed that
the agency needs an effective enterprise architecture program and
stated that it plans to improve its management of both projects. For
example, the agency intends to establish a steering committee; develop
a policy that will govern the development, maintenance, and
implementation of the architecture program; and have an approved
architecture project management plan for the non-NAS architecture.
GAO‘s experience in reviewing other agencies has shown that not having
an effective enterprise architecture program can be attributed to,
among other things, an absence of senior management understanding and
support and cultural resistance to having and using one. It has also
shown that attempting major systems modernization programs like FAA‘s
without having and using an enterprise architecture often results in
system implementations that are duplicative, are not well integrated,
require costly rework to interface, and do not effectively optimize
mission performance.
FAA‘s Mission:
[See PDF for image]
[End of figure]
What GAO Recommends:
To support the agency in its efforts to develop and implement an
enterprise architecture, GAO is making recommendations to the Secretary
of Transportation for establishing an effective enterprise architecture
management program, beginning with demonstrating senior management
commitment and support for the program.
The Department of Transportation provided technical comments on a draft
of this report. GAO has incorporated these, as appropriate, in the
report.
www.gao.gov/cgi-bin/getrpt?GAO-05-266.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Randolph C. Hite, 202-512-
3439, hiter@gao.gov or David Powner, 202-512-9286, pownerd@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
FAA Has Yet to Establish Key Architecture Development, Maintenance, and
Implementation Processes:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendixes:
Appendix I: Objective, Scope, and Methodology:
Appendix II: Assessment of Architecture Management Efforts for the
National Airspace System:
Appendix III: Assessment of Architecture Management Efforts for the Non-
National Airspace System:
Appendix IV: GAO Staff Acknowledgments 40:
Tables:
Table 1: Responsibilities of FAA's Staff Offices:
Table 2: Responsibilities of FAA's Lines of Business:
Table 3: GAO's Framework for Enterprise Architecture Management
Maturity (Version 1.1):
Table 4: GAO's Framework for Enterprise Architecture (EA) Management
Maturity (Version 1.1):
Figures:
Figure 1: Summary of the Air Traffic Control System over the
Continental United States and Oceans:
Figure 2: Simplified Diagram of FAA's Organizational Structure:
Abbreviations:
CIO: chief information officer:
COO: chief operating officer:
EA: enterprise architecture:
FAA: Federal Aviation Administration:
FEA: Federal Enterprise Architecture:
FEAF: Federal Enterprise Architecture Framework:
IG: Office of Inspector General:
IT: information technology:
NAS: National Airspace System:
OMB: Office of Management and Budget:
SRA: Systems Research and Applications International, Inc.
Letter April 29, 2005:
The Honorable Tom Davis:
Chairman, Committee on Government Reform:
House of Representatives:
The Honorable Adam H. Putnam:
House of Representatives:
The Federal Aviation Administration's (FAA) mission is to promote the
safe, orderly, and expeditious flow of air traffic in the U.S. airspace
system--commonly referred to as the National Airspace System (NAS). To
accomplish this mission, it relies on air traffic control systems to
provide such services as controlling takeoffs and landings and managing
the flow of traffic between airports. It also relies on its
administrative and mission support systems (non-NAS) to perform other
activities needed to achieve its mission, such as accident and incident
investigations and security inspections. To support its mission
performance, FAA is modernizing its air traffic control
systems.[Footnote 1]
Our research into the practices of successful public-and private-sector
organizations has shown that developing and using a well-defined
modernization blueprint--an enterprise architecture--is essential to an
organization's ability to transform its operations and supporting
systems in a way that eliminates duplication, promotes
interoperability, reduces costs, and optimizes mission performance.
You asked that we evaluate whether FAA is following best practices in
key information technology (IT) management areas, such as enterprise
architecture, investment management, and software/system development.
This report is one in a series of reports responding to your
request.[Footnote 2] As agreed, the objective of our review was to
determine whether FAA has established effective processes for managing
the development and implementation of an enterprise architecture. To
accomplish this objective, we analyzed documents, interviewed agency
officials, and compared FAA's architecture development, maintenance,
and implementation practices against our enterprise architecture
management maturity framework.[Footnote 3] We performed our work in
accordance with generally accepted government auditing standards.
Details on our objective, scope, and methodology are in appendix I.
Results in Brief:
FAA has taken steps to develop an enterprise architecture through two
architecture projects that are intended to cover its two core business
areas--NAS operations and non-NAS administrative and mission-support
operations. However, it has yet to establish most of the key management
structures, processes, and controls that are necessary to effectively
manage either of these two architecture projects. For example, for the
NAS architecture project, the agency has devoted resources, established
a project office, designated a chief architect, and issued Version 5.0
of its architecture, but it has not implemented other key management
capabilities, such as designating a committee or group representing the
enterprise to direct, oversee, or approve the architecture. Similarly,
for the non-NAS architecture project, the agency has devoted resources,
established a project office, and designated a chief architect.
However, it has not yet implemented other key management capabilities,
such as establishing a written and approved architecture policy. FAA
officials agreed that management improvements are needed for both
projects, and they told us that they are in the early stages of
implementing these improvements, including establishing a steering
committee, developing an architecture policy, and having an approved
architecture project management plan for the non-NAS architecture.
Our experience in reviewing other agencies has shown that not having an
effective enterprise architecture program can be attributed to, among
other things, an absence of senior management understanding and support
of an architecture, and cultural resistance to having and using one.
Our experience also shows that attempting major system modernization
programs, like FAA's, without having and using a well-defined
enterprise architecture often results in system implementations that
are duplicative, are not well integrated, require costly rework to
interface, and do not effectively optimize mission performance.
To support FAA in managing its efforts to develop and implement an
enterprise architecture, we are making recommendations to the Secretary
of the Department of Transportation related to establishing an
effective enterprise architecture management program. In comments on a
draft of this report provided by the Department's Director of Audit
Relations, the department neither agreed nor disagreed with our
conclusions and recommendations. The Director provided technical
comments, which we have incorporated as appropriate in the report.
Background:
FAA's Mission and Organizational Structure:
FAA's primary mission is to provide a safe, secure, and efficient
global airspace system that promotes airspace safety in the United
States and contributes to national security. The agency's roles include
regulating civil aviation, developing and operating a system of air
traffic control and navigation for civil and military aircraft, and
researching and developing the NAS, which consists of more than 19,000
airports, 750 air traffic control facilities, and about 45,000 pieces
of equipment.
FAA's mission performance depends on the adequacy and reliability of
the nation's air traffic control system. The air traffic control
system, the primary component of the NAS, is a vast network of computer
hardware, software, and communications equipment. This system consists
of automated information processing and display, communication,
navigation, surveillance, and weather resources that permit air traffic
controllers to view key information--such as aircraft location,
aircraft flight plans, and prevailing weather conditions--and to
communicate with pilots. These resources reside at, or are associated
with, several air traffic control facilities--towers, terminal radar
approach control facilities, air route traffic control centers (en
route centers), flight service stations, and the System Command Center.
Figure 1 shows a visual summary of the air traffic control system over
the continental United States and oceans.
Figure 1: Summary of the Air Traffic Control System over the
Continental United States and Oceans:
[See PDF for image]
[End of figure]
FAA's mission performance also depends on the skills and expertise of
its work force, composed of over 50,000 staff who provide aviation
services--including air traffic control; maintenance of air traffic
control equipment; and certification of aircraft, airline operations,
and pilots. In fiscal year 2005, FAA's budget authority to support its
mission was approximately $14 billion.[Footnote 4] According to FAA
officials, approximately 95 percent of the agency's total spending is
in support of the NAS. Further, FAA estimates that it will spend $7.6
billion over the next two years to complete key modernization projects.
As figure 2 illustrates, FAA has twelve staff offices to accomplish its
mission--including the Office of International Aviation and the Office
of Information Services/Chief Information Officer--and four lines of
business--Air Traffic Organization, Commercial Space Transportation,
Airports, and Regulation and Certification. Tables 1 and 2 provide
additional information about the responsibilities of these offices and
lines of business.
Figure 2: Simplified Diagram of FAA's Organizational Structure:
[See PDF for image]
[End of figure]
Table 1: Responsibilities of FAA's Staff OfficesA:
Office: Chief Counsel;
Responsibilities: Provides legal services and representation to FAA's
Administrator and all agency organizations at the headquarters,
regional, and center levels.
Office: Civil Rights;
Responsibilities: Advises, represents, and assists FAA's Administrator
on civil rights and equal opportunity matters such as unlawful
discrimination; program beneficiaries; and valuing, using, and managing
the differences that individuals bring to the workplace.
Office: Government and Industry Affairs;
Responsibilities: Advises and represents FAA's Administrator on matters
concerning the Congress, aviation industry groups, and other
governmental organizations. Works with other FAA offices to develop and
review plans and strategies involving these groups.
Office: System Safety;
Responsibilities: Develops and implements improved tools and processes,
such as hazard identification and risk management tools and processes,
in order to coordinate safety issues and facilitate more effective use
of safety data, both inside and outside the agency.
Office: Public Affairs;
Responsibilities: Provides the public with information about the
agency's mission, policies, activities, and operations.
Office: Human Resource Management;
Responsibilities: Advises and assists FAA's Administrator in directing,
coordinating, communicating, and ensuring the adequacy of agency plans,
programs, and initiatives associated with, among other things, human
capital planning, measurement, and evaluation.
Office: International Aviation;
Responsibilities: Works with key aviation partners and the
International Civil Aviation Organization to support the adoption of
international safety standards and to implement harmonized air traffic
procedures and technologies. Also works through various international
organizations and programs to share civil aviation safety information.
Office: Financial Services;
Responsibilities: Advises the FAA offices about plans and programs for
budget, financial management, and performance management.
Office: Security and Hazardous Materials;
Responsibilities: Ensures the integrity of those individuals who work
in or support the NAS and protects FAA employees and facilities from
criminal and terrorist acts.
Office: Regions and Center Operations;
Responsibilities: Provides corporate shared services, including
financial systems and operations; emergency readiness; enterprisewide
information services and business application development; and
logistics services such as acquisition, real estate, and supply
support.
Office: Aviation Policy, Planning and Environment;
Responsibilities: Leads the agency's strategic policy and planning
efforts, coordinates FAA's reauthorization before the Congress, and is
responsible for national aviation policies and strategies in the
environment and energy arenas.
Office: Information Services/Chief Information Officer;
Responsibilities: Provides policy and direction for the agency in the
areas of IT strategic planning, IT investment analysis, process
engineering, information management, information security, and
enterprise architecture.
Source: FAA.
[A] These twelve staff offices will provide input that will be used to
develop the agency's non-NAS architecture.
[End of table]
Table 2: Responsibilities of FAA's Lines of Business:
Line of business: Air Traffic Organization/Chief Operating Officer[A];
Responsibilities: Moves air traffic safely and efficiently and manages
the results of these efforts through objectives, goals, customer
service standards, and targets for improved cost and performance.
Line of business: Commercial Space Transportation;
Responsibilities: Ensures the protection of the public, property, and
the national security and foreign policy interests of the United States
during a commercial launch or re-entry activity and encourages,
facilitates, and promotes U.S. commercial space transportation.
Line of business: Airports;
Responsibilities: Provides leadership in planning and developing a safe
and efficient national airport system to satisfy the needs of the
aviation interests of the United States.
Line of business: Regulation and Certification;
Responsibilities: Establishes aviation safety standards, monitors
safety performance, conducts aviation safety education and research,
and issues and maintains aviation certificates and licenses.
Source: FAA.
[A] The Air Traffic Organization was formed on February 8, 2004, by
combining the Air Traffic Services and the Research and Acquisitions
units, which had been primarily responsible for managing air traffic
services within FAA. It is headed by a chief operating officer, whose
responsibilities include establishing and maintaining organizational
and individual goals, a 5-year strategic plan that includes the air
traffic control system mission and objectives, and a framework
agreement with FAA's Administrator to establish the organization's
relationships with other agency organizations.
[End of table]
An Enterprise Architecture Is Critical to Successful Systems
Modernization:
Effective use of enterprise architectures, or modernization blueprints,
is a trademark of successful public and private organizations. For more
than a decade, we have promoted the use of architectures to guide and
constrain systems modernization, recognizing them as a crucial means to
a challenging goal: agency operational structures that are optimally
defined in both business and technological environments. The Congress,
the Office of Management and Budget (OMB), and the federal Chief
Information Officer (CIO) Council have also recognized the importance
of an architecture-centric approach to modernization. The Clinger-Cohen
Act of 1996[Footnote 5] mandates that an agency's CIO develops,
maintains, and facilitates the implementation of an IT architecture.
Further, the E-Government Act of 2002[Footnote 6] requires OMB to
oversee the development of enterprise architectures within and across
agencies.
Enterprise Architecture: A Brief Description:
Generally speaking, an enterprise architecture connects an
organization's strategic plan with program and system solution
implementations by providing the fundamental business and technology
details needed to guide and constrain investments in a consistent,
coordinated, and integrated fashion. As such, it should provide a clear
and comprehensive picture of an entity, whether it is an organization
(e.g., federal agency) or a functional or mission area that cuts across
more than one organization (e.g., air traffic control). This picture
consists of snapshots of both the enterprise's current or "As Is"
environment and its target or "To Be" environment, as well as a capital
investment road map for transitioning from the current to the target
environment. These snapshots further consist of "views," which are
basically one or more architecture products that provide conceptual or
logical representations of the enterprise.
The suite of products and their content that form a given entity's
enterprise architecture are largely governed by the framework used to
develop the architecture. Since the 1980s, various frameworks have
emerged and been applied. For example, John Zachman developed a
structure or "framework" for defining and capturing an
architecture.[Footnote 7] This framework provides for six windows from
which to view the enterprise, which Zachman terms "perspectives" on how
a given entity operates: those of (1) the strategic planner, (2) the
system user, (3) the system designer, (4) the system developer, (5) the
subcontractor, and (6) the system itself. Zachman also proposed six
abstractions or models associated with each of these perspectives:
these models cover (1) how the entity operates, (2) what the entity
uses to operate, (3) where the entity operates, (4) who operates the
entity, (5) when entity operations occur, and (6) why the entity
operates.
In September 1999, the federal CIO Council published the Federal
Enterprise Architecture Framework (FEAF), which is intended to provide
federal agencies with a common construct for their respective
architectures, to facilitate the coordination of common business
processes, technology insertion, information flows, and system
investments among federal agencies. FEAF describes an approach,
including models and definitions, for developing and documenting
architecture descriptions for multiorganizational functional segments
of the federal government. Similar to most frameworks, FEAF's proposed
models describe an entity's business, the data necessary to conduct the
business, applications to manage the data, and technology to support
the applications.
More recently, OMB established the Federal Enterprise Architecture
(FEA) Program Management Office to develop a federated enterprise
architecture according to a collection of five "reference models, and a
security and privacy profile overlaying the five models."
* The Performance Reference Model is intended to describe a set of
performance measures for the major IT initiatives and their
contribution to program performance. Version 1.0 of the model was
released in September 2003.
* The Business Reference Model is intended to describe the federal
government's businesses, independent of the agencies that perform them.
It serves as the foundation for the FEA. Version 2.0 of the model was
released in June 2003.
* The Service Component Reference Model is intended to identify and
classify IT service (i.e., application) components that support federal
agencies and promote the reuse of components across agencies. Version
1.0 of the model was released in June 2003.
* The Data Reference Model is intended to describe, at an aggregate
level, the types of data and information that support program and
business line operations and the relationships among these types.
Version 1.0 of the model was released in September 2004.
* The Technical Reference Model is intended to describe the standards,
specifications, and technologies that collectively support the secure
delivery, exchange, and construction of service components. Version 1.1
of the model was released in August 2003.
* The Security and Privacy Profile is intended to provide guidance on
designing and deploying measures that ensure the protection of
information resources. OMB has released Version 1.0 of the profile.
Although these various enterprise architecture frameworks differ in
their nomenclatures and modeling approaches, they consistently provide
for defining an enterprise's operations in both (1) logical terms, such
as interrelated business processes and business rules, information
needs and flows, and work locations and users and (2) technical terms,
such as hardware, software, data, communications, and security
attributes and performance standards. The frameworks also provide for
defining these perspectives for both the enterprise's current or "As
Is" environment and its target or "To Be" environment, as well as a
transition plan for moving from the "As Is" to the "To Be" environment.
The importance of developing, implementing, and maintaining an
enterprise architecture is a basic tenet of both organizational
transformation and IT management. Managed properly, an enterprise
architecture can clarify and help to optimize the interdependencies and
relationships among an organization's business operations and the
underlying IT infrastructure and applications that support these
operations. Employed in concert with other important management
controls, such as portfolio-based capital planning and investment
control practices, architectures can greatly increase the chances that
an organization's operational and IT environments will be configured to
optimize its mission performance. Our experience with federal agencies
has shown that making IT investments without defining these investments
in the context of an architecture often results in systems that are
duplicative, not well integrated, and unnecessarily costly to maintain
and interface.[Footnote 8]
Our Prior Work Has Emphasized the Need for FAA to Establish
Architecture Management Capabilities:
In November 2003,[Footnote 9] we reported the results of our
governmentwide survey of agencies' progress--including FAA's--in
establishing key enterprise architecture management capabilities as
described in Version 1.1 of our architecture management maturity
framework.[Footnote 10] This framework associates specific architecture
management capabilities with five hierarchical stages of management
maturity, starting with creating enterprise architecture awareness and
followed by building the enterprise architecture management foundation,
developing the enterprise architecture, completing the enterprise
architecture, and leveraging the enterprise architecture to manage
change. Table 3 provides a more detailed description of the stages of
Version 1.1 of the framework.
Table 3: GAO's Framework for Enterprise Architecture Management
Maturity (Version 1.1):
Maturity stage: Stage 1: Creating enterprise architecture awareness;
Description: Organization does not have plans to develop and use an
architecture, or it has plans that do not demonstrate an awareness of
the value of having and using an architecture. While stage 1 agencies
may have initiated some architecture activity, these agencies' efforts
are ad hoc and unstructured, lack institutional leadership and
direction, and do not provide the management foundation that is
necessary for successful architecture development.
Maturity stage: Stage 2: Building the enterprise architecture
management foundation;
Description: Organization recognizes that the architecture is a
corporate asset by vesting accountability for it in an executive body
that represents the entire enterprise. At this stage, an organization
assigns architecture management roles and responsibilities and
establishes plans for developing enterprise architecture products and
for measuring program progress and product quality; it also commits the
resources necessary for developing an architecture--people, processes,
and tools.
Maturity stage: Stage 3: Developing the enterprise architecture;
Description: Organization focuses on developing architecture products
according to the selected framework, methodology, tool, and established
management plans. Roles and responsibilities assigned in the previous
stage are in place, and resources are being applied to develop actual
enterprise architecture products. The scope of the architecture has
been defined to encompass the entire enterprise, whether organization-
based or function-based.
Maturity stage: Stage 4: Completing the enterprise architecture;
Description: Organization has completed its enterprise architecture
products, meaning that the products have been approved by the
architecture steering committee or an investment review board and by
the CIO. Further, an independent agent has assessed the quality (i.e.,
completeness and accuracy) of the architecture products. Additionally,
evolution of the approved products is governed by a written
architecture maintenance policy approved by the head of the
organization.
Maturity stage: Stage 5: Leveraging the enterprise architecture to
manage change;
Description: Organization has secured senior leadership approval of the
enterprise architecture products and a written institutional policy
stating that IT investments must comply with the architecture unless
they are granted an explicit compliance waiver. Further, decision
makers are using the architecture to identify and address ongoing and
proposed IT investments that are conflicting, overlapping, not
strategically linked, or redundant. Also, the organization tracks and
measures architecture benefits or return on investment, and adjustments
are continuously made to both the architecture management process and
the enterprise architecture products.
Source: GAO.
[End of table]
Based on information provided by FAA, we reported that the agency had
not established an architecture management foundation; as a result, we
rated the agency to be at stage 1 of our framework. Specifically, we
reported that it had not (1) allocated adequate resources and (2)
established a framework, methodology, and automated tools to build the
enterprise architecture. According to our framework, effective
architecture management is generally not achieved until an enterprise
has a completed and approved architecture that is being effectively
maintained and is being used to leverage organizational change and
support investment decision making. An enterprise with these
characteristics would need to have satisfied all of the stage 2 and 3
core elements and most of the stage 4 and 5 elements.
Our Prior Work Has Also Emphasized the Need for FAA to Institutionalize
Other Key IT Management Controls:
In August 2004,[Footnote 11] we reported that FAA had established most-
-about 80 percent--of the basic practices needed to manage its mission-
critical investments, including many of the foundational practices for
selecting and controlling IT investments. However, we reported that
weaknesses still existed in the process. For example, FAA had not
involved its senior IT investment board in regular reviews of
investments that had completed development and become operational, and
had not implemented standard practices for managing its mission-support
and administrative investments. Because of these weaknesses, we
concluded that agency executives could not be assured that they were
selecting and managing the mix of investments that best met the
agency's needs and priorities. Accordingly, we made several
recommendations, including that the agency develop and implement a plan
aimed at addressing the weaknesses identified in our report. FAA
generally concurred with our conclusion and recommendations.
In addition, in August 2004,[Footnote 12] we reported that FAA had made
progress in improving its capabilities for acquiring software-intensive
systems, but that there were still areas that needed improvement.
Specifically, we reported that it had recurring weaknesses in the areas
of measurement and analysis,[Footnote 13] quality assurance, and
verification. We concluded that these weaknesses prevented FAA from
consistently and effectively managing its mission-critical systems and
increased the risk of cost overruns, schedule delays, and performance
shortfalls. We made several recommendations, including that FAA address
these specific weaknesses and institutionalize its process improvement
initiatives by establishing a policy and plans for implementing and
overseeing process improvement initiatives. FAA generally concurred
with our conclusion and recommendations.
Our Prior Work Has Identified Problems with the Air Traffic Control
Modernization Program:
FAA has a long and well-documented history of problems with its air
traffic control modernization program, including cost overruns,
schedule delays, and performance shortfalls. We first identified this
program as an area at high risk in 1995 because of the modernization's
size, complexity, cost, and problem-plagued past. Over the past decade,
we have continued to report on these problems. The program remains on
our high-risk list today.[Footnote 14]
In March 1999,[Footnote 15] we testified that FAA had had some success
in deploying new modernization systems over the past two decades, but
that the agency had not delivered most of its major air traffic control
systems in accordance with its cost, schedule, and performance goals,
due largely to its failure to implement established guidelines for
acquiring new systems. Specifically, we testified that the agency had
not fully implemented an effective process for monitoring the cost,
schedule, benefits, performance, and risk of its key projects
throughout their life cycles. We also noted that FAA lacked an
evaluation process for assessing outcomes after projects had been
developed, in order to help improve the selection and monitoring of
future projects. Moreover, we testified that the agency's problems in
modernizing its systems resulted from several root causes, including
the agency's attempt to undertake this modernization without the
benefit of a complete NAS architecture to guide its efforts. We
concluded that the agency would continue to experience problems in
deploying new systems until it had fully implemented solutions that
addressed these root causes of its modernization problems and
strengthened controls over its modernization investments.
In February[Footnote 16] and October 2003,[Footnote 17] we testified
that FAA had taken steps to improve the management of its air traffic
control modernization, but that systemic management issues, including
inadequate management controls and human capital issues, were
contributing to the continued cost overruns, schedule delays, and
performance shortfalls that major air traffic projects have
consistently experienced. We stated that to overcome these problems,
FAA would need to, among other things, improve its software
capabilities by requiring that all systems achieve a minimum level of
progress before they would be funded, and improve its cost estimating
and cost accounting practices by incorporating actual costs from
related system development efforts in its processes for estimating the
costs of new projects. We testified that until these issues had been
resolved, resources would not be spent cost-effectively, and
improvements in capacity and efficiency would be delayed.
FAA's Enterprise Architecture Program: A Brief Description:
According to FAA, its enterprise architecture initiative is intended to
influence the agency's ongoing initiatives in E-Government, data
management, information systems security, capital planning, investment
analysis, and air traffic control and navigation and is to benefit the
agency by aligning business processes with IT processes; improving
flight safety; reducing the development and maintenance costs of
systems; decreasing airline delays; guiding IT investments; and
improving the security, interoperability, and data usage of these
systems. FAA officials told us that the agency plans by April 2006 to
have a comprehensive version of its enterprise architecture to guide
and constrain the agency's investment decisions.
The Assistant Administrator for Information Services, who is the
agency's CIO, has been assigned responsibility for developing and
maintaining the agency's enterprise architecture. The CIO has
designated a program director to oversee this effort. Two project
offices are responsible for developing the NAS and non-NAS segments of
the enterprise architecture, respectively, in coordination with the
program director. Brief descriptions of the NAS and non-NAS
architecture projects are provided below.
NAS Architecture Project:
According to FAA, the NAS architecture is intended to be the agency's
comprehensive plan for improving NAS operations through the year 2015
and is to address how FAA will replace aging equipment and introduce
new systems, capabilities, and procedures. The NAS architecture, which
FAA reports is being developed in collaboration with the aviation
community, is intended to achieve several objectives. For example, it
is to (1) ensure that the NAS can handle future growth in aviation
without disrupting critical aviation services, (2) improve flight
safety and the use of airspace, (3) decrease airline delays, and (4)
improve systems integration and investment planning.
The agency is developing the NAS architecture in a series of
incremental versions. It released the first version of the NAS
architecture in September 1995. In 1999, FAA released Version 4.0 of
this architecture, which, according to the agency, was the first
version to include a 15 to 20-year view (a "To Be" view) and support
budget forecasts. According to FAA, the current version of the NAS
architecture (Version 5.0) shows how the agency intends to achieve the
target system described by 2015.
The chief operating officer (COO) for the Air Traffic Organization is
responsible for developing and implementing the NAS segment of the
architecture. The COO has tasked the Operations Planning/Systems
Engineering group within FAA's Air Traffic Organization with the day-
to-day activities involved in this effort. This group is headed by the
Vice President for Operations Planning, who reports directly to the
COO. The COO has also designated a chief architect, who reports to the
Director of Systems Engineering, to develop and maintain the NAS
architecture and to provide technical leadership and guidance, as
necessary, to support investment decision making. The Operations
Planning/Systems Engineering group receives input from several FAA
organizations, but primarily from business units within the Air Traffic
Organization.
Non-NAS Architecture Project:
According to FAA, the non-NAS architecture will cover the agency's
administrative services and mission support activities--the process
areas, data, systems, and technology that support such functions as
budget and finance, as well as all of the other governmental air
transportation missions and functions that are unique to the agency
(e.g., certification of aircraft). FAA initiated a project to develop
the non-NAS architecture in March 2002 and, according to FAA, the
agency plans to have, by January 2005, an initial baseline architecture
that will describe the "As Is" and "To Be" environments. According to
FAA, it plans to incrementally build on this baseline and have a
version of the non-NAS architecture by April 2006 that will also
include a sequencing plan.
According to FAA, the Information Management Division within the Office
of Information Services/CIO is responsible for developing and
maintaining the non-NAS architecture. FAA has designated a chief
architect, who reports to the program director, to oversee the day-to-
day program activities for developing and maintaining the non-NAS
architecture. To develop the non-NAS architecture, this division will
receive input from the agency's twelve staff offices and four lines of
business.
FAA Has Yet to Establish Key Architecture Development, Maintenance, and
Implementation Processes:
FAA recognizes the need for and has begun to develop an enterprise
architecture; however, it has yet to establish key architecture
management capabilities that it will need to effectively develop,
maintain, and implement the architecture. As previously stated, the
agency has set up two separate project offices and tasked each with
developing one of the two architecture segments (NAS and non-NAS) that
together are to compose FAA's enterprise architecture. The agency also
reports that it has allocated adequate resources to these project
offices and that chief architects have been assigned to head the
architecture projects. However, FAA has not established other key
architecture management capabilities, such as designating a committee
or group representing the enterprise to direct, oversee, or approve the
architecture effort; having an approved policy for developing,
maintaining, and implementing the architecture; and fully developing
architecture products that meet contemporary guidance and describe both
the "As Is" and "To Be" environments and a sequencing plan for
transitioning between the two.
According to FAA officials, attention to and oversight of the
enterprise architecture program have been limited in the past, and the
agency has not documented its architecture management policies,
procedures, and processes; but this is changing. For example, by the
end of this fiscal year, FAA plans to issue a policy governing its
enterprise architecture efforts and to establish a steering committee
to guide and direct the program. By April 2005, the agency also plans
to approve an architecture project management plan for the non-NAS
architecture. In addition, it plans to have a framework for developing
the NAS architecture by September 2005.
Based on our experience in reviewing other agencies, not having an
effective enterprise architecture program is attributable to, among
other things, limited senior management understanding and commitment
and cultural resistance to having and using an architecture. The result
is an inability to implement modernized systems in a way that minimizes
overlap and duplication and maximizes integration and mission support.
FAA Has Yet to Implement Key Best Practices for Managing Its NAS
Architecture Project:
As we first reported in 1997, it is critical that FAA have and use a
comprehensive NAS architecture to guide and constrain its air traffic
control system investment decisions. To effectively develop, maintain,
and implement this architecture, FAA will need to employ rigorous and
disciplined architecture management practices. Such practices form the
basis of our architecture management maturity framework; the five
maturity stages of our Version 1.1 framework are described in table 3.
Some of these key practices or core elements associated with each of
the stages are summarized below. For additional information on these
key practices or core elements, see the framework.
For stage 2, our framework specifies nine key practices or core
elements that are necessary to provide the management foundation for
successfully launching and sustaining an architecture effort. Examples
of stage 2 core elements are described below.
* Establish a committee or group, representing the enterprise, that is
responsible for directing, overseeing, or approving the enterprise
architecture. This committee should include executive-level
representatives from each line of business, and these representatives
should have the authority to commit resources and enforce decisions
within their respective organizational units. By establishing this
enterprisewide responsibility and accountability, the agency
demonstrates its commitment to building the management foundation and
obtaining buy-in from across the organization.
* Appoint a chief architect. The chief architect should be responsible
and accountable for the enterprise architecture, supported by the
architecture program office, and overseen by the architecture steering
committee. The chief architect, in collaboration with the CIO, the
architecture steering committee, and the organizational head is
instrumental in obtaining organizational buy-in for the enterprise
architecture, including support from the business units, as well as in
securing resources to support architecture management functions such as
risk management, configuration management, quality assurance, and
security management.
* Use a framework, methodology, and automated tool to develop the
enterprise architecture. These elements are important because they
provide the means for developing the architecture in a consistent and
efficient manner. The framework provides a formal structure for
representing the enterprise architecture, while the methodology is the
common set of procedures that the enterprise is to follow in developing
the architecture products. The automated tool serves as a repository
where architectural products are captured, stored, and maintained.
* Develop an architecture program management plan. This plan specifies
how and when the architecture is to be developed. It includes a
detailed work breakdown structure, resource estimates (e.g., funding,
staffing, and training), performance measures, and management controls
for developing and maintaining the architecture. The plan demonstrates
the organization's commitment to managing architecture development and
maintenance as a formal program.
Our framework similarly identifies key architecture management
practices associated with later stages of architecture management
maturity. For example, at stage 3--the stage at which organizations
focus on architecture development activities--organizations need to
satisfy six core elements. Examples of these core elements are
discussed below.
* Issue a written and approved organization policy for development of
the enterprise architecture. The policy defines the scope of the
architecture, including the requirement for a description of the
baseline and target architectures, as well as an investment road map or
sequencing plan specifying the move between the two. This policy is an
important means for ensuring enterprisewide commitment to developing an
enterprise architecture and for clearly assigning responsibility for
doing so.
* Ensure that enterprise architecture products are under configuration
management. This involves ensuring that changes to products are
identified, tracked, monitored, documented, reported, and audited.
Configuration management maintains the integrity and consistency of
products, which is key to enabling effective integration among related
products and for ensuring alignment between architecture artifacts.
At stage 4, during which organizations focus on architecture completion
activities, organizations need to satisfy eight core elements. Examples
of these core elements are described below.
* Ensure that enterprise architecture products and management processes
undergo independent verification and validation. This core element
involves having an independent third party--such as an internal audit
function or a contractor that is not involved with any of the
architecture development activities--verify and validate that the
products were developed in accordance with architecture processes and
product standards. Doing so provides organizations with needed
assurance of the quality of the architecture.
* Ensure that business, performance, information/data,
application/service, and technology descriptions address security. An
organization should explicitly and consistently address security in its
business, performance, information/data, application/service, and
technology architecture products. Because security permeates every
aspect of an organization's operations, the nature and substance of
institutionalized security requirements, controls, and standards should
be captured in the enterprise architecture products.
At stage 5, during which the focus is on architecture maintenance and
implementation activities, organizations need to satisfy eight core
elements. Examples of these core elements are described below.
* Make the enterprise architecture an integral component of the IT
investment management process. Because the road map defines the IT
systems that an organization plans to invest in as it transitions from
the "As Is" to the "To Be" environment, the enterprise architecture is
a critical frame of reference for making IT investment decisions. Using
the architecture when making such decisions is important because
organizations should approve only those investments that move the
organization toward the "To Be" environment, as specified in the road
map.
* Measure and report return on enterprise architecture investment. Like
any investment, the enterprise architecture should produce a return on
investment (i.e., a set of benefits), and this return should be
measured and reported in relation to costs. Measuring return on
investment is important in order to ensure that expected benefits from
the architecture are realized and to share this information with
executive decision makers, who can then take corrective action to
address deviations from expectations.
Table 4 summarizes our framework's five stages and all of the
associated core elements for each.
Table 4: GAO's Framework for Enterprise Architecture (EA) Management
Maturity (Version 1.1):
Stage: Stage 1: Creating EA awareness;
Core elements:
* Agency is aware of EA.
Stage: Stage 2: Building the EA management foundation;
Core elements:
* Adequate resources exist;
* Committee or group representing the enterprise is responsible for
directing, overseeing, or approving EA;
* Program office responsible for EA development and maintenance exists;
* Chief architect exists;
* EA is being developed using a framework, methodology, and automated
tool;
* EA plans call for describing the "As Is" and "To Be" environments and
a sequencing plan;
* EA plans call for describing the enterprise in terms of business,
information/data, application/service, and technology;
* EA plans call for business, performance, information/data,
application/service, and technology descriptions to address security;
* EA plans call for developing metrics for measuring EA progress,
quality, compliance, and return on investment.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core elements:
* Written and approved organization policy exists for EA development;
* EA products are under configuration management;
* EA products describe or will describe the enterprise's business,
performance, information/data, application/service, and the technology
that supports them;
* EA products describe or will describe the "As Is" and the "To Be"
environments and a sequencing plan;
* Business, performance, information/data, application/service, and
technology descriptions address or will address security;
* Progress against EA plans is measured and reported.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core elements:
* Written and approved organization policy exists for EA maintenance;
* EA products and management processes undergo independent verification
and validation;
* EA products describe the "As Is" and the "To Be" environments and a
sequencing plan;
* EA products describe the enterprise's business, performance,
information/data, application/service, and the technology that supports
them;
* Business, performance, information/data, application/service, and
technology descriptions address security;
* Organization's chief information officer has approved current version
of EA;
* Committee or group representing the enterprise or the investment
review board has approved current version of EA;
* Quality of EA products is measured and reported.
Stage: Stage 5: Leveraging the EA to manage change; (includes all
elements from stage 4);
Core elements:
* Written and approved policy exists for IT investment compliance with
EA;
* Process exists to formally manage EA change;
* EA is integral component of IT investment management process;
* EA products are periodically updated;
* IT investments comply with EA;
* Organization head has approved current version of EA;
* Return on EA investment is measured and reported;
* Compliance with EA is measured and reported.
Source: GAO.
[End of table]
For its NAS architecture project, FAA is currently at stage 1 of our
maturity framework. The NAS project office has satisfied three of the
core elements associated with "building the enterprise architecture
management foundation"--stage 2 of our framework---and three of the
elements associated with "developing enterprise architecture products"--
stage 3 of our framework. It has not satisfied other stage 2 and 3 core
elements or any core elements associated with stages 4 and 5. According
to the framework, effective architecture management is generally not
achieved until an enterprise has a completed and approved architecture
that is being effectively maintained and is being used to leverage
organizational change and support investment decision making; having
these characteristics is equivalent to having satisfied all of the
stage 3 core elements and many of the stage 4 and 5 elements.
For the stage 2 core elements, FAA reports that it has allocated
adequate resources for developing a NAS architecture. Further, it has
established a project office that is responsible for architecture
development and maintenance and has assigned a chief architect to the
project. However, the agency has not satisfied other core elements for
stage 2, such as assigning responsibility for directing, overseeing, or
approving the architecture to a committee or group representing the
enterprise. Without such an entity to lead and be accountable for the
architectural effort, there is increased risk that the architecture
will not represent a corporate decision-making tool and will not be
viewed and endorsed as an agencywide asset.
With respect to stage 3, according to the CIO, FAA plans to build on
the current version of the NAS architecture (Version 5.0) to ensure
that architecture products are developed that meet contemporary
guidance and standards. According to FAA officials, including the CIO
and the chief scientist for the NAS project office, the current NAS
architecture does not conform to contemporary architecture guidance or
standards--including OMB's FEA reference models and GAO's enterprise
architecture management maturity framework--because it predates them
and has not been updated to comply with them. However, the CIO stated
that future versions of the architecture will conform to this guidance.
Among other things, this guidance calls for products that describe the
"As Is" and "To Be" business, performance, information/data,
applications/services, technology, and security environments as well as
a sequencing plan for transitioning from the "As Is" to the "To Be"
states. Other stage 3 core elements nevertheless have not been met,
such as having a written and approved architecture development policy.
Further, none of the stage 4 and 5 core elements have been met,
although the CIO stated that FAA has recently begun to take steps
associated with meeting some of these core elements. The detailed
results of our assessment of the NAS project office's progress in
implementing the core elements associated with the five maturity stages
are provided in appendix II.
In addition, FAA's senior enterprise architecture officials, including
the program director, stated that attention to and oversight of the
enterprise architecture program have been limited in the past and that
the agency has not documented its architecture management policies,
procedures, and processes. These officials stated that the agency
recognizes the need to establish an effective NAS architecture project
and that it intends to do so. To this end, FAA currently plans to have,
by September 2005, a framework for developing the architecture and an
approved enterprise architecture policy requiring the development,
maintenance, and implementation of an enterprise architecture. The CIO
also stated that the agency plans to update its NAS architecture to
reflect current architecture standards and guidance.
Our research of successful organizations and our experience in
reviewing other agencies' enterprise architecture efforts show that not
having these controls is, among other things, a function of limited
senior management understanding of and commitment to an enterprise
architecture and cultural resistance to having and using one. Until
such barriers are addressed and effective architecture management
structures and processes are established, it is unlikely that an agency
will be able to produce and maintain a complete and enforceable
architecture and thus implement modernized systems in a way that
minimizes overlap and duplication and maximizes integration and mission
support. Given the size and complexity of FAA's air traffic control
systems and their importance to FAA's ability to achieve its mission,
it is critical that FAA develop a well-defined architecture that can be
used to guide and constrain system investment decisions.
FAA Has Yet to Implement Key Best Practices for Managing Its Non-NAS
Architecture Project:
Similar to its NAS architecture effort, FAA's attempt to develop,
maintain, and implement its non-NAS architecture needs to be grounded
in the kind of rigorous and disciplined management practices embodied
in Version 1.1 of our architecture management maturity framework.
(Tables 3 and 4 provide a description of the framework's five maturity
stages and the key practices or core elements associated with each
stage.)
For its non-NAS architecture project, FAA is currently at stage 1 of
our maturity framework. The non-NAS project office has satisfied three
of the core elements associated with "building the enterprise
architecture management foundation"--stage 2 of our framework--and four
of the core elements associated with stages 3 and 5. According to the
framework, effective architecture management is generally not achieved
until an enterprise has a completed and approved architecture that is
being effectively maintained and is being used to leverage
organizational change and support investment decision making; having
these characteristics is equivalent to having satisfied all of the
stage 2 and 3 core elements and many of the stage 4 and 5 elements.
For stage 2 core elements, FAA reports that it has allocated adequate
resources, and it has established a project office and assigned a chief
architect. However, the agency has not satisfied several of the stage 2
core elements that are critical to effective architecture management.
For example, the agency has not established a committee or group
representing the enterprise to guide, direct, or approve the
architecture. Having such a corporate entity is critical to overcoming
cultural resistance to using an enterprise architecture. As previously
stated, the absence of such an entity increases the risk that the
architecture will not represent a corporate decision-making tool and
will not be viewed and endorsed as an agencywide asset.
Concerning stage 3, FAA has not satisfied three of the six core
elements. For example, although the agency is developing architecture
products, it does not have a written and approved policy for
architecture development. Without such a policy, which, for example,
identifies the major players in the development process and provides
for architecture guidance, direction, and approval, FAA will be
challenged in overcoming cultural resistance to using an enterprise
architecture and achieving agencywide commitment and support for an
architecture.
The agency has not implemented any of the stage 4 core elements, but it
has implemented one core element--architecture products are
periodically updated--associated with stage 5 of our framework. For
example, FAA has not (1) documented and approved a policy for
architecture implementation, (2) implemented an independent
verification and validation function that covers architecture products
and architecture management processes, and (3) made the architecture an
integral component of its IT investment management process. The
detailed results of our assessment of the non-NAS project office's
progress in implementing the core elements associated with the five
maturity stages are provided in appendix III.
According to FAA's senior enterprise architecture officials, including
the chief architect, the attention to and oversight of the enterprise
architecture program have been limited in the past, and the agency has
not documented its architecture management policies, procedures, and
processes. FAA officials, including the CIO and the chief architect for
the non-NAS project, agreed with our assessment of the project office's
current architecture management capabilities. These officials stated
that the agency recognizes the need to establish an effective non-NAS
architecture project, and it intends to do so. For example, the
agency's strategic plan includes the goal of having an approved
enterprise architecture policy requiring the development, maintenance,
and implementation of an enterprise architecture by September 2005, and
the agency intends to establish a steering committee. In addition, the
chief architect stated that FAA plans to have an approved architecture
project management plan by April 2005, and a comprehensive version of
the non-NAS architecture by April 2006.
As previously stated, our research and our experience show that not
having these controls is, among, other things, attributable to limited
senior management understanding of and commitment to an enterprise
architecture and cultural resistance to having and using one. Until
such barriers are addressed and effective architecture management
structures and processes are established, it is unlikely that any
agency will be able to develop and maintain a complete and enforceable
architecture and thus implement modernized systems in a way that
minimizes overlap and duplication and maximizes integration and mission
support.
Conclusions:
Having a well-defined and enforced enterprise architecture is critical
to FAA's ability to effectively and efficiently modernize its NAS and
non-NAS systems. To accomplish this, it is important for FAA to
establish effective management practices for developing, maintaining,
and implementing an architecture. Currently, FAA does not have these
practices in place. Establishing them begins with agency top management
commitment and support for having and using an architecture to guide
and constrain investment decision making.
Recommendations for Executive Action:
To ensure that FAA has the necessary agencywide context within which to
make informed decisions about its air traffic control system and other
systems modernization efforts, we recommend that the Secretary of the
Department of Transportation direct the FAA Administrator to ensure
that the following four actions take place.
* Demonstrate institutional commitment to and support for developing
and using an enterprise architecture by issuing a written and approved
enterprise architecture policy.
* Ensure that the CIO, in collaboration with the COO, implements, for
the NAS architecture project, the best practices involved in stages 2
through 5 of our enterprise architecture management maturity framework.
* Ensure that the CIO focuses first on developing and implementing a
NAS architecture.
* Ensure that the CIO implements, for the non-NAS architecture project,
the best practices involved in stages 2 through 5 of our enterprise
architecture management maturity framework.
Agency Comments:
In commenting on a draft of this report, the Department of
Transportation's Director of Audit Relations stated via e-mail that FAA
is continuing its NAS architecture efforts. The Director also provided
technical comments, which we have incorporated as appropriate in the
report. The Director's comments did not state whether the department
agreed or disagreed with the report's conclusions and recommendations.
As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the report date. At that time, we will send copies to interested
congressional committees, the Director of OMB, the Secretary of the
Department of Transportation, the FAA Administrator, FAA's CIO, and
FAA's COO. We will also make copies available to others upon request.
In addition, the report will be available at no charge on the GAO Web
site at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions on matters discussed in this
report, please contact Randolph C. Hite at (202) 512-3439 or
[Hyperlink, hiter@gao.gov], or David A. Powner at (202) 512-9286 or
[Hyperlink, pownerd@gao.gov]. Major contributors to this report are
acknowledged in appendix IV.
Signed by:
Randolph C. Hite:
Director:
Information Technology Architecture and Systems Issues:
Signed by:
David A. Powner:
Director:
Information Technology Management Issues:
[End of section]
Appendixes:
Appendix I: Objective, Scope, and Methodology:
Our objective was to determine whether the Federal Aviation
Administration (FAA) has established effective processes for managing
the development and implementation of an enterprise architecture.
To address our objective, we used our enterprise architecture
management maturity framework, Version 1.1,[Footnote 18] which
organizes architecture management best practices into five stages of
management maturity. Specifically, we compared our framework to the
ongoing efforts of FAA's two project offices to develop the National
Airspace System (NAS) and non-NAS segments of the architecture. For
example, for the NAS architecture, we reviewed program documentation,
such as the acquisition management system policy, the Joint Resources
Council's investment management guidance for NAS investments, and FAA's
NAS architecture development process. We reviewed, for the non-NAS
architecture, program documentation, such as the methodology FAA is
using to develop this architecture, a Systems Research and Applications
report[Footnote 19] on the agency's efforts to implement management
processes and controls over its architecture development activities,
and the Department of Transportation's Enterprise Architecture
Subcommittee and Architecture Review Board charters. We then compared
these documents with the elements in our framework.
To augment our documentation reviews of FAA's architecture management
efforts, we interviewed various officials, including the chief
information officer, the program director, the chief architects for the
NAS and non-NAS architectures, and the chief scientist for the NAS
architecture, to determine, among other things, the agency's plans to
develop an enterprise architecture. Specifically, we inquired about (1)
the agency's plans for developing an enterprise architecture, including
the key milestones and deliverables for completing the two segments of
the architecture, (2) the content of the NAS and non-NAS architecture
segments (i.e., architecture products that have been developed to
date), and (3) the strategy to be used to align the NAS and non-NAS
architectures.
We conducted our work at FAA headquarters in Washington, D.C. We
performed our work from June 2004 to March 2005, in accordance with
generally accepted government auditing standards.
[End of section]
Appendix II: Assessment of Architecture Management Efforts for the
National Airspace System:
Stage: Stage 1: Creating enterprise architecture (EA) awareness;
Core element: Agency is aware of EA;
Satisfied? Yes;
Comments: The Federal Aviation Administration (FAA) strategic plan
includes the goal of having an approved EA policy requiring the
development, maintenance, and implementation of an EA by September
2005.
Stage: Stage 2: Building the EA management foundation;
Core element: Adequate resources exist (funding, people, tools, and
technology);
Satisfied? Yes;
Comments: According to the chief scientist and the chief architect for
the NAS architecture, the agency has adequate project funding. FAA
reports that fiscal year 2005 funding for the National Airspace System
(NAS) architecture is about $2.6 million. In addition, the agency
reports that it has skilled staff, including contractor support, for
its NAS architecture project. Furthermore, FAA is using automated tools
and technology, such as Rational Rose by Rational Software
Corporation/IBM Software Group, CORE by Vitech Corporation, and Dynamic
Object Oriented Requirements System by Telelogic.
Stage: Stage 2: Building the EA management foundation;
Core element: Committee or group representing the enterprise is
responsible for directing, overseeing, or approving the EA;
Satisfied? No;
Comments: FAA has not assigned responsibility for directing,
overseeing, or approving a NAS architecture to a group or committee
representing the enterprise.
Stage: Stage 2: Building the EA management foundation;
Core element: Program office responsible for EA development and
maintenance exists;
Satisfied? Yes;
Comments: In 1997, FAA established a project office that is responsible
for developing and maintaining a NAS architecture.
Stage: Stage 2: Building the EA management foundation;
Core element: Chief architect exists;
Satisfied? Yes;
Comments: In February 2004, FAA designated the chief architect for the
NAS architecture project.
Stage: Stage 2: Building the EA management foundation;
Core element: EA is being developed using a framework, methodology, and
automated tool;
Satisfied? No;
Comments: According to the chief scientist, the NAS architecture is
being developed using a framework that focuses on strategically
supporting FAA's investment management process. The chief scientist
stated that, unlike other architecture frameworks, this framework is
not fully developed or documented. Further, FAA has yet to provide us
with any documentation on this framework and on how it is being
implemented to support the agency's investment management process.
According to the chief information officer (CIO), the agency plans to
select an architecture framework by September 2005. FAA does not have a
methodology that defines the standards, steps, tools, techniques, and
measures that it is following to develop, maintain, and validate a NAS
architecture. As stated above, FAA is using automated tools to build a
NAS architecture.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for describing both the "As Is" and the "To
Be" environments of the enterprise, as well as a sequencing plan for
transitioning from the "As Is" to the "To Be.";
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for describing both the "As Is" and the "To
Be" environments in terms of business, performance, information/data,
application/service, and technology;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for business, performance,
information/data, application/service, and technology descriptions to
address security;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for developing metrics for measuring EA
progress, quality, compliance, and return on investment;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans
and metrics for measuring NAS architecture progress, quality,
compliance, and return on investment.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: Written/approved organization policy exists for EA
development;
Satisfied? No;
Comments: FAA has yet to develop a written/approved policy for
developing a NAS architecture. However, FAA's strategic plan includes
the goal of developing an approved enterprise architecture policy by
September 2005.
Stage 3: Developing EA products (includes all elements from stage 2);
Core element: EA products are under configuration management;
Satisfied? No;
Comments: FAA has yet to establish a configuration management process.
Stage 3: Developing EA products (includes all elements from stage 2);
Core element: EA products describe or will describe both the "As Is"
and the "To Be" environments of the enterprise, as well as a sequencing
plan for transitioning from the "As Is" to the "To Be.";
Satisfied? Yes;
Comments: According to the chief information officer (CIO), future
versions of the NAS architecture will conform to contemporary guidance.
Such guidance describes, among other things, products that describe the
"As Is" and "To Be" environments and a sequencing plan.
Stage 3: Developing EA products (includes all elements from stage 2);
Core element: Both the "As Is" and the "To Be" environments are
described or will be described in terms of business, performance,
information/data, application/service, and technology;
Satisfied? Yes;
Comments: According to the CIO, future versions of the NAS architecture
will conform to contemporary guidance. Such guidance describes, among
other things, products that describe the "As Is" and "To Be"
environments in terms of business, performance, information/data,
application/service, and technology.
Stage 3: Developing EA products (includes all elements from stage 2);
Core element: Business, performance, information/data,
application/service, and technology descriptions address or will
address security;
Satisfied? Yes;
Comments: According to the CIO, future versions of the NAS architecture
will conform to contemporary guidance. Such guidance includes, among
other things, business, performance, information/data,
application/service, and technology descriptions that address security
in both the "As Is" and "To Be" environments.
Stage 3: Developing EA products (includes all elements from stage 2);
Core element: Progress against EA plans is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans
and metrics; therefore, progress against plans is not measured and
reported.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: Written/approved organization policy exists for EA
maintenance;
Satisfied? No;
Comments: FAA has yet to develop a written/approved policy for
maintaining a NAS architecture. However, FAA's strategic plan includes
the goal of developing an approved enterprise architecture policy by
September 2005.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: EA products and management processes undergo independent
verification and validation;
Satisfied? No;
Comments: FAA has yet to establish an independent verification and
validation process.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: EA products describe both the "As Is" and the "To Be"
environments of the enterprise, as well as a sequencing plan for
transitioning from the" As Is" to the "To Be.";
Satisfied? No;
Comments: FAA has yet to develop NAS architecture products that
describe both the "As Is" and the "To Be" environments and a sequencing
plan.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: Both the "As Is" and the "To Be" environments are
described in terms of business, performance, information/data,
application/service, and technology;
Satisfied? No;
Comments: FAA has yet to develop NAS architecture products that
describe both the "As Is" and the "To Be" environments in terms of
business, performance, information/data, application/service, and
technology.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: Business, performance, information/data,
application/service, and technology descriptions address security;
Satisfied? No;
Comments: FAA has yet to develop business, performance,
information/data, application/service, and technology descriptions that
address security in both the "As Is" and "To Be" environments.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: Organization CIO has approved current version of EA;
Satisfied? No;
Comments: FAA has yet to develop a version of the NAS architecture for
the CIO to approve that conforms to contemporary guidance and
standards.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: Committee or group representing the enterprise or the
investment review board has approved current version of EA;
Satisfied? No;
Comments: FAA has yet to develop a version of the NAS architecture that
conforms to contemporary guidance and standards for a committee or
investment review board to approve.
Stage: Stage 4: Completing EA products (includes all elements from
stage 3);
Core element: Quality of EA products is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop NAS architecture product metrics;
therefore, product quality is not measured and reported.
Stage: Stage 5: Leveraging the EA for managing change (includes all
elements from stage 4);
Core element: Written/approved organization policy exists for IT
investment compliance with EA;
Satisfied? No;
Comments: FAA has yet to develop a written/approved policy requiring IT
investments to comply with a NAS architecture. However, FAA's strategic
plan includes the goal of developing an approved enterprise
architecture policy by September 2005.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: Process exists to formally manage EA change;
Satisfied? No;
Comments: FAA has yet to establish a formal process for managing
changes to a NAS architecture.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: EA is integral component of IT investment management
process;
Satisfied? No;
Comments: According to the CIO, FAA has recently begun to consider
architecture compliance as part of its Joint Resources Council process
and the CIO's approval of Exhibit 300 budget exhibits for NAS
investments, and he anticipates that over the next couple of years the
NAS architecture will become integral to the investment process.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: EA products are periodically updated;
Satisfied? No;
Comments: FAA has yet to complete development of NAS architecture
products.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: IT investments comply with EA;
Satisfied? No;
Comments: According to the CIO, FAA has recently begun to consider
investment compliance with the architecture, and the CIO expects this
compliance determination to expand and evolve over the next couple of
years.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: Organization head has approved current version of EA;
Satisfied? No;
Comments: FAA has yet to complete development of a NAS architecture for
the Administrator to approve.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: Return on EA investment is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop metrics and processes for measuring
NAS architecture benefits; therefore, return on investment is not
measured and reported.
Stage 5: Leveraging the EA for managing change (includes all elements
from stage 4);
Core element: Compliance with EA is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop metrics for measuring compliance with
the NAS architecture; therefore, compliance with an architecture is not
measured and reported.
Source: GAO analysis of FAA data.
[End of table]
[End of section]
Appendix III: Assessment of Architecture Management Efforts for the
Non- National Airspace System:
Stage: Stage 1: Creating enterprise architecture (EA) awareness;
Core element: Agency is aware of EA;
Satisfied? Yes;
Comments: The Federal Aviation Administration (FAA) strategic plan
includes the goal of having an approved EA policy requiring the
development, maintenance, and implementation of an EA by September
2005.
Stage: Stage 2: Building the EA management foundation;
Core element: Adequate resources exist (funding, people, tools, and
technology);
Satisfied? Yes;
Comments: According to the chief architect, the agency has adequate
project funding. FAA reports that fiscal year 2005 funding for the non-
National Airspace System (NAS) architecture is $1.5 million. In
addition, the agency reports that it has skilled staff (two government
employees, six full-time contractors, and additional contractor staff
as needed) working to develop its non-NAS architecture. FAA is also
using automated tools, such as Rational Rose by Rational Software
Corporation/IBM Software Group, Microsoft Visio and an Oracle portal
server.
Stage: Stage 2: Building the EA management foundation;
Core element: Committee or group representing the enterprise is
responsible for directing, overseeing, or approving the EA;
Satisfied? No;
Comments: FAA has not assigned responsibility for directing,
overseeing, or approving the non-NAS architecture to any group or
committee. According to the chief architect, FAA plans to assign
responsibility for directing the non-NAS architecture to its
Information Technology Executive Board by April 2005.
Stage: Stage 2: Building the EA management foundation;
Core element: Program office responsible for EA development and
maintenance exists;
Satisfied? Yes;
Comments: In January 2003, FAA established a project office that is
responsible for developing and maintaining the non-NAS architecture.
Stage: Stage 2: Building the EA management foundation;
Core element: Chief architect exists;
Satisfied? Yes;
Comments: In January 2003, FAA designated a chief architect for the non-
NAS architecture.
Stage: Stage 2: Building the EA management foundation;
Core element: EA is being developed using a framework, methodology, and
automated tool;
Satisfied? No;
Comments: According to the chief architect, FAA is using the Federal
Enterprise Architecture Framework and the Office of Management and
Budget's Federal Enterprise Architecture reference models to develop
the non-NAS architecture. FAA also has a methodology for developing the
architecture, but the methodology does not define the standards, steps,
tools, techniques, and measures that it is following to develop,
maintain, and validate the non-NAS architecture. However, according to
the chief architect, FAA will update its methodology to describe
management activities by March 2005. As stated above, FAA is using
automated tools to build the non-NAS architecture.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for describing both the "As Is" and the "To
Be" environments of the enterprise, as well as a sequencing plan for
transitioning from the "As Is" to the "To Be.";
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans.
However, the chief architect stated that the agency intends to have an
approved plan by April 2005 and the plan will call for describing both
the "As Is" and the "To Be" environments of the enterprise, as well as
a sequencing plan.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for describing both the "As Is" and the "To
Be" environments in terms of business, performance, information/data,
application/service, and technology;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans.
However, the chief architect stated that the agency intends to have an
approved plan by April 2005 and the plan will call for describing both
the "As Is" and the "To Be" environments in terms of business,
performance, information/data, application/service, and technology.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for business, performance,
information/data, application/service, and technology descriptions to
address security;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans.
However, the chief architect stated that the agency intends to have an
approved plan by April 2005 and the plan will call for the business,
performance, information/data, application/service, and technology
descriptions to address security for both the "As Is" and "To Be"
environments.
Stage: Stage 2: Building the EA management foundation;
Core element: EA plans call for developing metrics for measuring EA
progress, quality, compliance, and return on investment;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans
and metrics. According to the chief architect, the agency intends to
have an approved plan by April 2005 and the plan will include metrics
for measuring non-NAS architecture progress, quality, compliance, and
return on investment.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: Written/approved organization policy exists for EA
development;
Satisfied? No;
Comments: FAA has yet to develop a written/approved policy for
developing the non-NAS architecture. However, FAA's strategic plan
includes the goal of developing an approved enterprise architecture
policy by September 2005.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: EA products are under configuration management;
Satisfied? No;
Comments: FAA has yet to develop non-NAS architecture products and a
configuration management process has not been established. However,
according to the chief architect, FAA plans to update its methodology
to address how changes to all architecture products will be documented
by March 2005.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: EA products describe or will describe both the "As Is"
and the "To Be" environments of the enterprise, as well as a sequencing
plan for transitioning from the "As Is" to the "To Be.";
Satisfied? Yes;
Comments: The chief architect stated that the agency intends to have an
approved plan by April 2005 and that the plan will call for describing
both the "As Is" and the "To Be" environments of the enterprise, as
well as a sequencing plan. The chief architect also stated that the
agency will have a comprehensive non-NAS architecture by April 2006.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: Both the "As Is" and the "To Be" environments are
described or will be described in terms of business, performance,
information/data, application/service, and technology;
Satisfied? Yes;
Comments: The chief architect stated that the agency intends to have an
approved plan by April 2005 and that the plan will call for describing
both the "As Is" and the "To Be" environments in terms of business,
performance, information/data, application/service, and technology. The
chief architect also stated that the agency will have a comprehensive
non-NAS architecture by April 2006.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: Business, performance, information/data,
application/service, and technology descriptions address or will
address security;
Satisfied? Yes;
Comments: The chief architect stated that the agency intends to have an
approved plan by April 2005 and that the plan will call for the
business, performance, information/data, application/service, and
technology descriptions to address security for both the "As Is" and
"To Be" environments. The chief architect also stated that the agency
will have a comprehensive non-NAS architecture by April 2006.
Stage: Stage 3: Developing EA products (includes all elements from
stage 2);
Core element: Progress against EA plans is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop architecture project management plans
and metrics; therefore, progress against plans is not measured and
reported. However, according to the chief architect, the agency intends
to have an approved plan by April 2005 and progress against the plan
will be measured and reported.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: Written/approved organization policy exists for EA
maintenance;
Satisfied? No;
Comments: FAA has yet to develop a written/approved policy for
maintaining the non-NAS architecture. However, FAA's strategic plan
includes the goal of developing an approved enterprise architecture
policy by September 2005.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: EA products and management processes undergo independent
verification and validation;
Satisfied? No;
Comments: FAA has yet to establish an independent verification and
validation process.[A] However, according to the chief architect, the
non-NAS architecture products and architecture management processes
will undergo independent verification and validation by December 2005.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: EA products describe both the "As Is" and the "To Be"
environments of the enterprise, as well as a sequencing plan for
transitioning from the" As Is" to the "To Be.";
Satisfied? No;
Comments: The current non-NAS architecture products do not yet fully
describe both the "As Is" and the "To Be" environments of the
enterprise, or a sequencing plan. However, according to the chief
architect, FAA will have a comprehensive non-NAS architecture that
describes both the "As Is" and the "To Be" environments of the
enterprise, as well as the sequencing plan by April 2006.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: Both the "As Is" and the "To Be" environments are
described in terms of business, performance, information/data,
application/service, and technology;
Satisfied? No;
Comments: The current non-NAS architecture products do not yet fully
describe both the "As Is" and the "To Be" environments in terms of
business, performance, information/data, application/service, and
technology. However, according to the chief architect, FAA will have a
comprehensive non-NAS architecture that describes these terms by April
2006.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: Business, performance, information/data,
application/service, and technology descriptions address security;
Satisfied? No;
Comments: According to the chief architect, the non-NAS architecture
does not yet contain complete business, performance, information/data,
application/service, and technology descriptions that address security
for both the "As Is" and "To Be" environments. However, FAA will have a
comprehensive non-NAS architecture comprised of these terms by April
2006.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: Organization chief information officer (CIO) has approved
current version of EA;
Satisfied? No;
Comments: FAA has yet to develop a non-NAS architecture for the CIO to
approve. According to the chief architect, the first comprehensive
version of the non-NAS architecture is scheduled for release in April
2006.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: Committee or group representing the enterprise or the
investment review board has approved current version of EA;
Satisfied? No;
Comments: FAA has yet to develop a non- NAS architecture for the
committee or investment review board to approve. According to the chief
architect, the first comprehensive version of the non-NAS architecture
is scheduled for release in April 2006; In addition, FAA has yet to
establish a committee or investment review board that will be
responsible for approving the non-NAS architecture.
Stage: Stage 4: Completing EA products; (includes all elements from
stage 3);
Core element: Quality of EA products is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop metrics and assess the quality of the
non-NAS architecture products that it is currently developing;
therefore, product quality is not measured and reported. However,
according to the chief architect, the agency intends to have an
approved plan by April 2005 and the plan will include metrics for
measuring the quality of the non-NAS architecture products.
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: Written/approved organization policy exists for IT
investment compliance with EA;
Satisfied? No;
Comments: FAA has yet to develop a written/approved policy requiring
that IT investments comply with the architecture. However, FAA's
strategic plan includes the goal of developing an approved enterprise
architecture policy by September 2005.
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: Process exists to formally manage EA change;
Satisfied? No;
Comments: FAA has yet to establish a formal process for managing
changes to the non-NAS architecture. However, according to the chief
architect, the agency intends to have an approved architecture project
management plan by April 2005 and the plan will include a formal
process for managing architecture changes.
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: EA is integral component of IT investment management
process;
Satisfied? No;
Comments: FAA has yet to complete development of a non-NAS
architecture, and it is not an integral component of the IT investment
management process.[B].
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: EA products are periodically updated;
Satisfied? Yes;
Comments: FAA updates the non-NAS architecture products annually to
reflect the agency's investment decisions.[B].
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: IT investments comply with EA;
Satisfied? No;
Comments: FAA has yet to complete development of a non-NAS
architecture; therefore, IT investments are not evaluated for
compliance with the architecture. However, the first version of the non-
NAS architecture is scheduled for release in April 2006.[B].
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: Organization head has approved current version of EA;
Satisfied? No;
Comments: FAA has yet to complete development of a; non-NAS
architecture for the Administrator to approve. However, the first
version of the non-NAS architecture is scheduled for release in April
2006.[B].
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: Return on EA investment is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop metrics and processes for measuring
non-NAS architecture benefits; therefore, return on investment is not
measured and reported. However, the first version of the non-NAS
architecture is scheduled for release in April 2006.[B].
Stage: Stage 5: Leveraging the EA for managing change; (includes all
elements from stage 4);
Core element: Compliance with EA is measured and reported;
Satisfied? No;
Comments: FAA has yet to develop metrics for measuring compliance with
the non- NAS architecture. However, the first version of the non-NAS
architecture is scheduled for release in April 2006.[B].
Source: GAO analysis of FAA data.
[A] In January 2004, Systems Research and Applications (SRA)
International, Inc. reviewed FAA's architecture management processes.
The results of this review were provided to the chief architect, but a
corrective action plan was not developed. According to the chief
architect, the weaknesses in the SRA report will be addressed as the
agency establishes additional architecture management capabilities and
refines its investment management process to include non-NAS
investments.
[B] According to the chief architect, these core elements will be
addressed in the enterprise architecture policy that FAA plans to issue
by September 2005.
[End of table]
[End of section]
Appendix IV: GAO Staff Acknowledgments:
Staff Acknowledgments:
Staff who made key contributions to this report were Kristina Badali,
Joanne Fiorino, Michael Holland, Anh Le, William Wadsworth, and Angela
Watson.
(310284):
FOOTNOTES
[1] In 1995 we designated the air traffic control modernization program
as high risk because of the program's size, importance, and complexity
and because of the cost and the numerous problems it had encountered in
systems acquisition. It has remained on our high-risk list since that
time.
[2] See also, GAO, Information Technology: FAA Has Many Investment
Management Capabilities in Place, but More Oversight of Operational
Systems Is Needed, GAO-04-822 (Washington, D.C.: Aug. 20, 2004); and
GAO, Air Traffic Control: System Management Capabilities Improved, but
More Can Be Done to Institutionalize Improvements, GAO-04-901
(Washington, D.C.: Aug. 20, 2004).
[3] GAO, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management (Version 1.1), GAO-03-
584G (Washington, D.C.: April 2003).
[4] Federal Aviation Administration: Budget in Brief Fiscal Year 2006
(Washington, D.C.: February 2005).
[5] The Clinger-Cohen Act of 1996, 40 U.S.C. 11315(b)(2).
[6] E-Government Act of 2002, Pub. L. No. 107-347 (Dec. 17, 2002).
[7] J. A. Zachman, "A Framework for Information Systems Architecture,"
IBM Systems Journal 26, no. 3 (1987).
[8] See, for example, GAO, Homeland Security: Efforts Under Way to
Develop Enterprise Architecture, but Much Work Remains, GAO-04-777
(Washington, D.C.: Aug. 6, 2004); DOD Business Systems Modernization:
Limited Progress in Development of Business Enterprise Architecture and
Oversight of Information Technology Investments, GAO-04-731R
(Washington, D.C.: May 17, 2004); Information Technology: Architecture
Needed to Guide NASA's Financial Management Modernization, GAO-04-43
(Washington, D.C.: Nov. 21, 2003); DOD Business Systems Modernization:
Important Progress Made to Develop Business Enterprise Architecture,
but Much Work Remains, GAO-03-1018 (Washington, D.C.: Sept. 19, 2003);
and Information Technology: DLA Should Strengthen Business Systems
Modernization Architecture and Investment Activities, GAO-01-631
(Washington, D.C.: June 29, 2001).
[9] GAO, Information Technology: Leadership Remains Key to Agencies
Making Progress on Enterprise Architecture Efforts, GAO-04-40
(Washington, D.C.: Nov. 17, 2003).
[10] GAO-03-584G.
[11] GAO-04-822.
[12] GAO-04-901.
[13] The purpose of measurement and analysis is to develop and sustain
a measurement capability that is used to support management information
needs.
[14] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.:
January 2005).
[15] GAO, Air Traffic Control: Observations on FAA's Air Traffic
Control Modernization Program, GAO/T-RCED/AIMD-99-137 (Washington,
D.C.: March 25, 1999).
[16] GAO, National Airspace System: Reauthorizing FAA Provides
Opportunities and Options to Address Changes, GAO-03-473T (Washington,
D.C.: Feb. 12, 2003).
[17] GAO, Air Traffic Control: FAA's Modernization Efforts--Past,
Present, and Future, GAO-04-227T (Washington, D.C.: Oct. 30, 2003).
[18] GAO-03-584G.
[19] Systems Research and Applications International, Inc., Independent
Validation and Verification Final Report, Version 1.0, Jan. 30, 2004.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
