Defense Acquisitions
Assessments of Major Weapon Programs
Gao ID: GAO-04-248 March 31, 2004
Although the weapons that the Department of Defense (DOD) develops have no rival in superiority, there still remain ways in which they can be improved. GAO's reviews over the past 20 years have found consistent problems with weapon acquisitions--cost increases, schedule delays, and performance shortfalls--along with underlying causes, such as pressure on managers to promise more than they can deliver. DOD can resolve these problems by using a knowledge-based approach derived from the best practices of successful product developments. GAO's goal for this report is to provide congressional and DOD decision makers with an independent, knowledge-based assessment of selected defense programs that identifies potential risks and offers an opportunity for action when a program's projected attainment of knowledge diverges from the best practice. It can also highlight those programs that employ practices worthy of emulation by other programs. GAO plans to update and issue this report annually.
GAO assessed 51 defense programs ranging from the Missile Defense Agency's Airborne Laser to the Army's Warfighter Information Network. GAO's assessments are anchored in a knowledge-based approach to product development that reflects best practices of successful programs. This approach centers on attaining high levels of knowledge in three elements of a new product or weapon--technology, design, and production. If a program is not attaining this level of knowledge, it incurs increased risk of technical problems, with potential cost and schedule growth. If a program is falling short in one element, like technology maturity, it is harder to attain knowledge in succeeding elements. Most of the programs GAO assessed proceeded with less knowledge at critical junctures than suggested by best practices, although several came close to meeting best practice standards. GAO also found that programs generally did not track statistical process control data, a key indicator for production maturity. Program stakeholders can use these assessments to recognize the gaps in knowledge early and to take advantage of opportunities for constructive intervention--such as adjustments to schedule, trade-offs in requirements, and additional funding. GAO has summarized the results of its assessments in a 2-page format. Each 2-page assessment contains a profile of the product that includes a description; a timeline of development; a baseline comparison of cost, schedule, and quantity changes to the program; and a graphical and narrative depiction of how the product development knowledge of an individual program compared to best practices. Each program office submitted comments and they are included with each individual assessment as appropriate.
GAO-04-248, Defense Acquisitions: Assessments of Major Weapon Programs
This is the accessible text file for GAO report number GAO-04-248
entitled 'Defense Acquisitions: Assessments of Major Weapon Programs'
which was released on March 31, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
March 2004:
DEFENSE ACQUISITIONS:
Assessments of Major Weapon Programs:
GAO-04-248:
GAO Highlights:
Highlights of GAO-04-248, a report to congressional committees
Why GAO Did This Study:
Although the weapons that the Department of Defense (DOD) develops have
no rival in superiority, there still remain ways in which they can be
improved. GAO‘s reviews over the past 20 years have found consistent
problems with weapon acquisitions”cost increases, schedule delays, and
performance shortfalls”along with underlying causes, such as pressure
on managers to promise more than they can deliver. DOD can resolve
these problems by using a knowledge-based approach derived from the
best practices of successful product developments.
GAO‘s goal for this report is to provide congressional and DOD decision
makers with an independent, knowledge-based assessment of selected
defense programs that identifies potential risks and offers an
opportunity for action when a program‘s projected attainment of
knowledge diverges from the best practice. It can also highlight those
programs that employ practices worthy of emulation by other programs.
GAO plans to update and issue this report annually.
What GAO Found:
GAO assessed 51 defense programs ranging from the Missile Defense
Agency‘s Airborne Laser to the Army‘s Warfighter Information Network.
GAO‘s assessments are anchored in a knowledge-based approach to product
development that reflects best practices of successful programs. This
approach centers on attaining high levels of knowledge in three
elements of a new product or weapon”technology, design, and production.
If a program is not attaining this level of knowledge, it incurs
increased risk of technical problems, with potential cost and schedule
growth (see figure). If a program is falling short in one element, like
technology maturity, it is harder to attain knowledge in succeeding
elements.
Attainment of Product Knowledge:
[See PDF for image]
[End of figure]
Most of the programs GAO assessed proceeded with less knowledge at
critical junctures than suggested by best practices, although several
came close to meeting best practice standards. GAO also found that
programs generally did not track statistical process control data, a
key indicator for production maturity. Program stakeholders can use
these assessments to recognize the gaps in knowledge early and to take
advantage of opportunities for constructive intervention”such as
adjustments to schedule, trade-offs in requirements, and additional
funding.
GAO has summarized the results of its assessments in a 2-page format.
Each 2-page assessment contains a profile of the product that includes
a description; a timeline of development; a baseline comparison of
cost, schedule, and quantity changes to the program; and a graphical
and narrative depiction of how the product development knowledge of an
individual program compared to best practices. Each program office
submitted comments and they are included with each individual
assessment as appropriate.
www.gao.gov/cgi-bin/getrpt?GAO-04-248.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Paul Francis at (202)
512-4841 or francisp@gao.gov.
[End of section]
Contents:
Foreword:
Letter:
A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes:
Knowledge-Based Assessments:
General Observations:
Assessments of Individual Programs:
Airborne Laser (ABL):
Aegis Ballistic Missile Defense (Aegis BMD):
Advanced Extremely High Frequency Satellite (AEHF):
Active Electronically Scanned Array Radar (AESA):
Advanced Precision Kill Weapon System (APKWS):
Advanced SEAL Delivery System (ASDS):
Advanced Threat Infrared Countermeasure/Common Missile
Warning System (ATIRCM/CMWS):
Advanced Wideband Satellite/Transformational Satellite (AWS/TSat):
B-2 Radar Modernization Program (B-2 RMP):
C-130 Avionics Modernization Program (C-130 AMP):
C-5 Avionics Modernization Program (C-5 AMP):
C-5 Reliability Enhancement and Reengining Program (C-5 RERP):
Cooperative Engagement Capability (CEC):
CH-47F Improved Cargo Helicopter (CH-47F):
Comanche Reconnaissance Attack Helicopter (RAH-66):
Future Aircraft Carrier CVN-21:
DD(X) Destroyer:
E-10A Multi-Sensor Command and Control Aircraft (E-10A):
E-2 Advanced Hawkeye (E-2 AHE):
EA-18G Growler (EA-18G):
Evolved Expendable Launch Vehicle --Atlas V, Delta IV (EELV):
Expeditionary Fighting Vehicle (EFV):
Extended Range Guided Munition (ERGM):
Excalibur Precision Guided Extended Range Artillery Projectile:
F/A-22 Raptor:
Future Combat Systems (FCS):
Global Hawk Unmanned Aerial Vehicle:
Ground-Based Midcourse Defense (GMD):
Joint Air-to-Surface Standoff Missile (JASSM):
Joint Helmet Mounted Cueing System (JHMCS):
Joint Common Missile:
Joint Strike Fighter (JSF):
Joint Standoff Weapon (JSOW):
Joint Tactical Radio System (JTRS):
Littoral Combat Ship (LCS):
Long-term Mine Reconnaissance System (LMRS):
Minuteman III Guidance Replacement Program (MM III GRP):
Minuteman III Propulsion Replacement Program (MM III PRP):
Mobile User Objective System (MUOS):
National Polar-Orbiting Operational Environmental Satellite System
(NPOESS):
Guided Missile System Air Defense (Patriot) PAC-3 Program:
MQ-9 Predator B:
Space Based Infrared System (SBIRS) High:
Small Diameter Bomb (SDB):
RQ-7A Shadow 200 Unmanned Aerial Vehicle System (Shadow 200):
Space Tracking and Surveillance System (STSS):
Theater High Altitude Area Defense (THAAD):
Tactical Tomahawk Missile:
V-22 Joint Services Advanced Vertical Lift Aircraft (V-22):
Wideband Gapfiller Satellites (WGS):
Warfighter Information Network-Tactical (WIN-T):
Agency Comments:
Scope of Our Review:
Appendixes:
Appendix I: Comments from the Department of Defense:
Appendix II: Scope and Methodology:
Appendix III: Technology Readiness Levels:
Appendix IV: GAO Contact and Acknowledgments:
Related GAO Products:
Figures:
Figure 1: Building Knowledge at Key Points in Product Development
Reduces the Risk of Unknowns:
Figure 2: Depiction of a Notional Weapon System Program's Knowledge as
Compared with Best Practices:
Abbreviations:
ACTS: AEHF Comsec/Transec System:
AEA: Airborne Electronic Attack:
DOD: Department of Defense:
EKV: exoatmospheric kill vehicle:
GAO: General Accounting Office:
GEO: geosynchronous:
GPS: global positioning system:
HEO: highly elliptical orbit:
HLV: heavy lift vehicle:
IMIS: Integrated Maintenance Information System:
ISO: International Organization for Standardization:
KSDI: Key System Development Integration:
MDA: Missile Defense Agency:
MEADS: Medium Extended Air Defense System:
NASA: National Aeronautics and Space Administration:
NOAA: National Oceanic and Atmospheric Association:
SAR: synthetic aperture radar:
SDACS: Solid Divert Attitude Control System:
SDD: system development and demonstration:
TBD: to be determined:
TF/TA: Terrain Following and Terrain Avoidance:
USAF: United States Air Force:
USMC: United States Marine Corps:
USN: United States Navy:
Foreword March 31, 2004:
Congressional Committees:
The Department of Defense (DOD) is in the midst of a modernization and
transformation effort that will drive its spending priorities well into
the next decade. DOD is investing heavily in programs that it believes
will provide a new portfolio of military capabilities to decisively
combat the full spectrum of threats to U.S. security. Investment in the
research, development, and procurement of major weapon systems is
expected to grow considerably as these efforts progress, rising from
$135 billion in fiscal year 2004 to a projected $166 billion in 2009.
DOD's total investment will, in fact, approach almost $1 trillion
during the same period. These efforts to transform and modernize major
weapon systems will not achieve their full potential if they are
stymied by the cost growth and schedule delays that have limited the
buying power of the defense investment dollar in the past.
For this reason alone, DOD needs to seek better outcomes from its new
investments. It is also possible that these outcomes could have a
significant affect on yet more urgent challenges to be faced by the
federal budget in the forthcoming years. Health-care costs are growing
at double-digit rates, and spending on homeland security will likely
grow as the United States seeks to defeat terrorism worldwide. This
country also faces an oncoming demographic tidal wave--by 2035, the
number of people who are 65 or over will have doubled. These and other
factors will substantially increase the demand on funding for
associated entitlement programs, as well as create further pressures on
discretionary funding--such as investments in weapon systems.
Therefore, it is critical that DOD get the most out of these:
investments for the amounts budgeted. We believe that this report can
provide useful insights on key risks in weapons development, allow
decision makers to take corrective actions, and place needed and
justifiable programs in a better position to succeed.
Signed by:
David M. Walker:
Comptroller General of the United States:
Letter March 31, 2004:
Congressional Committees:
The Department of Defense (DOD) develops weaponry that is unmatched in
levels of technological sophistication and lethality. In an effort to
transform the military, DOD is on the threshold of several major
investments in improved weapon systems that are likely to dominate the
budget and doctrinal debates well into the next decade. These programs
include such systems as the Missile Defense Agency's suite of land,
sea, air, and space defense systems; the Army's Future Combat Systems;
the Air Force's, Marine Corps', and Navy's Joint Strike Fighter; and
overarching systems, such as the Advanced Wideband Satellite/
Transformational Satellite.
Despite their superiority, these weapon systems will routinely take
much longer to field, cost more to buy, and require more support than
provided for in investment plans. An alternative approach must be found
to develop these systems. Our work on best practices has found that
programs managed within a knowledge-based approach--where high levels
of product knowledge are demonstrated at critical points during
development--are better positioned to deliver superior performance
within cost and schedule estimates. We believe that by employing this
approach, DOD can get similar outcomes from its weapon system programs.
This annual report is one step in our effort to help DOD adopt a more
knowledge-based approach. In this current report, we assess 51 major
weapon systems whose combined program costs exceed $672 billion. Each
assessment is presented in a 2-page summary that analyzes each
program's attainment of knowledge as compared with best practices,
along with its cost and schedule status. Our objective is to provide
decision makers with an independent, knowledge-based assessment of
individual systems that identifies potential risks and allows decision
makers to take early actions, if warranted, to put programs in a better
position to succeed.
A Knowledge-Based Approach Can Lead to Better Acquisition Outcomes:
Over the last several years, we have undertaken a body of work that
examines weapon acquisition issues from a different, more cross-cutting
perspective--one that draws lessons learned from best system
development practices to see if they apply to weapon system
development. We found that successful product developers employed
specific practices to ensure that a high level of knowledge regarding
critical facets of the product was achieved at key junctures in
development. We characterized these junctures as three knowledge
points. We also identified key indicators that can be used to assess
the attainment of knowledge. When tied to major events on a program's
schedule, they can disclose whether gaps or shortfalls exist in
demonstrated knowledge, which can presage future cost, schedule, and
performance problems. These knowledge points and associated indicators
are defined as follows.
Knowledge point 1: Resources and needs are matched. This level of
knowledge is attained when a match is made between a customer's needs
and the developer's technical, financial, and other resources.
Achieving a high level of technology maturity at the start of system
development is a particularly important best practice. This means that
the technologies needed to meet essential product requirements have
been demonstrated to work in their intended environment.
Knowledge point 2: The product design is stable. This level of
knowledge is attained when the product's design is shown to meet the
customer's requirements. A best practice is to achieve design stability
at the system-level critical design review, usually held midway through
development. Completion of engineering drawings at the system design
review provides tangible evidence that the design is stable.
Knowledge point 3: Production processes are mature. This level of
knowledge is attained when it is demonstrated that the product can be
manufactured within cost, schedule, and quality targets. A best
practice is to achieve production maturity at the start of production.
This means that all key manufacturing processes produce output within
statistically acceptable limits for quality.
As illustrated in figure 1, the attainment of each successive knowledge
point builds on the preceding one. While the knowledge itself builds
continuously without clear lines of demarcation, the attainment of
knowledge points is sequential. In other words, production maturity
cannot be attained if the design is not mature, and design maturity
cannot be attained if the key technologies are not mature.
Figure 1: Building Knowledge at Key Points in Product Development
Reduces the Risk of Unknowns:
[See PDF for image]
[End of figure]
For the most part, all three knowledge points are eventually attained
on a completed product. The difference between highly successful
product developments--those that deliver superior products within cost
and schedule projections--and problematic product developments is how
this knowledge is built and how early in the development cycle each
knowledge point is attained. If a program is attaining the desired
levels of knowledge, it has less risk--but not zero risk--of future
problems. Likewise, if a program shows a gap between demonstrated
knowledge and best practices, it indicates an increased risk--not a
guarantee--of future problems.
Knowledge-Based Assessments:
Our assessment of each program is summarized in two components--(1) a
system profile and (2) a product knowledge assessment.
The system profile presents a general description of the product in
development; a picture of the product or of one of its key elements; a
schedule timeline identifying key dates in the program; a table
identifying the prime contractor, the program office location, and the
funding remaining from fiscal 2004 through completion, if available;
and a table summarizing the cost, schedule, and quantity changes to the
program.
The rest of the assessment analyzes the extent to which product
knowledge at the three key knowledge points has been attained. We
depict the extent of knowledge in a stacked bar graph and provide a
narrative summary at the bottom of the first page. The second page is
devoted to a narrative assessment of technology, design and production
maturity, as well as other program issues identified and comments from
the program office.
As shown is figure 2, the knowledge graph is based on the three
knowledge points and the key indicators for the attainment of
knowledge. A "best practice" line is drawn based on the ideal
attainment of the three types of knowledge at the three knowledge
points. As can be seen, knowledge about the technology, design, and
production of a new product builds over time. The closer a program's
attained knowledge is to the best practice line, the more likely the
weapon will be delivered within its estimated cost and schedule. A
knowledge deficit at the start of development--indicated by a gap
between the technology knowledge attained by the weapon system and the
best practice line--means the program proceeded with immature
technologies and may face a greater likelihood of cost and schedule
increases as technology risks are discovered and resolved.
The first knowledge point on the best practice line represents two
facts: a commitment to a new system development has been made and the
key technologies needed for the new product are mature. The orange bar
indicates the actual technology maturity attained for a program's key
technologies as measured at the start of development--normally
milestone II or milestone B in DOD's acquisition process.[Footnote 1]
The second major point on the best practice line captures technology
maturity plus:
design maturity. A green bar indicates the design knowledge attained by
a weapon system program. A design is considered mature when 90 percent
of the engineering drawings have been released or deemed releasable to
manufacturing. The third major point on the best practice line captures
the sum of technology maturity, design maturity, and production
maturity. A blue bar indicates the production knowledge attained by a
weapon system program. Production is considered mature when all key
production processes are in statistical control.[Footnote 2] The blue
bar is stacked on top of the orange and green bars to indicate whether
any cumulative technology, design, and production gaps exist at the
time production begins. In some cases, we obtained projections from the
program office of future knowledge attainment. These projections are
depicted as dashed bars.
Figure 2: Depiction of a Notional Weapon System Program's Knowledge as
Compared with Best Practices:
[See PDF for image]
[End of figure]
An interpretation of this notional example would be that the system
development began with key technologies immature, thereby missing
knowledge point 1. Knowledge point 2 was not attained at the design
review as some technologies were still not mature and only a
small percentage of engineering drawings had been released. Projections
for the production decision show that the program is expected to
achieve greater levels of maturity, but will still fall short. It is
likely that this program would have had significant cost and schedule
increases.
We found two situations in which programs were unable to provide key
knowledge indicators. We used two types of labels in the knowledge
graphs to depict those situations. Programs with these labels are
distinguished from those that have elected not to collect data that can
be used to assess progress against best practices. First, some programs
were unable to reconstruct the relevant knowledge indicator because the
event happened too many years ago. In these situations, we annotate the
graph with the phrase "Data not available." Second, a few programs have
not followed the traditional acquisition model. For example, some
programs combined the development start decision with the production
decision. Other programs used commercial off-the-shelf components,
which negated the need to monitor production processes. In these
situations, we annotate the graph with the phrase "Not applicable.":
We conducted our review from June 2003 through March 2004 in accordance
with generally accepted government auditing standards. Appendix II
contains detailed information on our methodology.
General Observations:
Most of the programs we assessed proceeded with lower levels of
knowledge at critical junctures and attained key elements of product
knowledge later in development. In addition, while most programs were
able to assess technology maturity using technology readiness levels
and were able to track the status of engineering drawings, few programs
collected or analyzed information on production process controls. We
did find some programs that attained relatively high levels of key
product knowledge. Examples of programs that demonstrated relatively
high levels of technology, design, and production maturity are provided
below, along with examples of programs where levels of product
knowledge were low. While DOD has announced the cancellation of the
Comanche program to reallocate resources, the program still
demonstrated relatively high levels of design and production knowledge.
The examples below include Comanche because it remains a good example
of attaining key product knowledge.
Technology Maturity:
The following programs attained a greater level of technology maturity
before entering system development than most weapon systems we
assessed:
* The B-2 Radar Modernization program demonstrated full technology
maturity in advance of the start of system development. A formal
technology readiness assessment is planned for completion prior to the
start of development in May 2004. The program has already built and
tested some transmit/receive modules, and several key elements of the
modules were already tested in an operational environment.
* The MQ-9 Predator B aircraft program has matured three of the
program's four technologies, and the fourth--an avionics subsystem
designed to integrate and store data necessary to launch munitions--is
comprised of several off-the-shelf components and is being evaluated in
a laboratory environment.
In some programs, the consequences of proceeding with immature
technologies have already been felt. For example:
* The Extended Range Guided Munition program began system development
in 1996 with only 1 of its 20 critical technologies mature. While
progress has been made, full technology maturity was still not
demonstrated at the time of the design review in 2003. The lack of
mature technologies contributed to cost increases, schedule delays, and
test failures. These test failures later led the program to miss a Navy
deadline that required successful completion of two land-based flight
tests by November 2003. The Navy is conducting an independent
assessment of the program's readiness to proceed with further
flight-testing. The Navy has also issued a solicitation for alternative
precision-guided munition concepts that could offer cost savings.
* The Advanced SEAL Delivery System began system development over
9 years ago, and currently has technologies that are not fully mature.
During that time, total program costs increased 571 percent. While
progress has been made within the past year, the technologies are not
expected to reach maturity until the second boat is built in 2008.
* The Advanced Wideband Satellite/Transformational Satellite program
has only matured one of its five critical technologies, with the
remaining four scheduled to reach maturity in early 2006. This is more
than 2 years after the planned start of development. While the
program's acquisition strategy allows for concurrent technology and
system development, concern over this aggressive acquisition strategy
led the Air Force to schedule an interim review for November 2004. This
review will determine whether the program's technology development has
progressed sufficiently or whether alternative action should be taken.
To date, program costs have increased 148 percent.
Design Maturity:
In a number of these programs, having mature technology at the start of
system development resulted in having more design stability at the time
of the design review. Some examples include:
* The Theater High Altitude Area Defense System program attained full
technology and design maturity in advance of the design review in
December 2003. This program made significant strides following a
problematic preliminary development phase where the delayed
demonstration of technologies and components, and reliance on
full-system testing to discover problems, nearly caused the
cancellation of the program. The program has since structured a system
development phase with a much greater emphasis on risk reduction,
including the use of technology readiness levels. The program achieved
design stability by releasing 100 percent of engineering drawings
before the design review.
* The National Polar-Orbiting Operational Environmental Satellite
System achieved 86 percent technology maturity before committing to
system development, and the program has completed half of the currently
identified drawings well in advance of the design review in April 2006.
The program is also taking steps to reduce program risk by
demonstrating three critical sensors on a demonstrator satellite prior
to their inclusion on the new satellite.
* The Comanche Reconnaissance Attack Helicopter program released
84 percent of design drawings by the time of its design review.
Additionally, the tools used to gather and validate knowledge on the
Comanche's design were required by contract, with targeted award
fees that provided additional incentives for building knowledge.
Other programs proceeded with their design review without having the
requisite level of technology knowledge. This lack of knowledge
affected the level of design stability attained. For example:
* The F/A-22 Fighter program began system development in 1991 without
having mature technologies--deferring knowledge point 1--and
subsequently attained only a quarter of the desired amount of
engineering drawings at the time of the design review in 1995. While
the program now has mature technology and design stability, the program
experienced substantial cost increases and schedule delays in the
latter stages of development.
* The Guided Missile System Air Defense (Patriot) PAC-3 program
attained less technology maturity and design maturity than best
practices suggest. At the time of the design review in 1996, the
program only had 23 percent design maturity, and the technologies were
still not mature. The seeker technology did not demonstrate maturity
until close to the production decision. The cost of the seeker
increased by 76 percent and contributed to a 2-year delay in the
program's schedule.
* The Advanced Threat Infrared Countermeasure/Common Missile Warning
System held its design review in 1997 with only 22 percent design
maturity. While the basic design of the system is now complete, it was
not until 2 years after the design review that 90 percent of the
drawings were released and the design was considered stable. This
resulted in inefficient manufacturing, rework, additional testing, and
a 3-year schedule delay.
Production Maturity:
Unlike technology readiness levels, which can be applied at any time,
and engineering drawing release data, which is captured on all
programs, few programs collected statistical process control data.
While the absence of this data does not necessarily mean that
production processes were immature, it does prevent an assessment
against an objective standard. Other indicators of production maturity,
such as scrap and rework rates, can indicate positive trends, but are
not prospective--that is, they are not useful in guiding preparations
for production. To some extent, statistical process control data is not
collected because DOD is delegating more responsibility to prime
contractors and reducing the amount of data requested. The lack of such
data may put program offices in a disadvantaged position to gain
insights about a contractor's production progress. Some programs,
however, have started changing this trend, making the collection of
statistical process control data part of the contract requirements. For
example:
* The Comanche Reconnaissance Attach Helicopter program called for
collecting more knowledge about production processes and maturity than
we have seen on many programs. Specifically, the Army planned
to collect information on control over the production processes and
reliability and included these requirements in the Comanche contract.
In addition, the contractor had established reliability growth plans
and goals and had started conducting reliability growth testing.
* The Tactical Tomahawk missile program has begun collecting
statistical control data from the assembly of components for the first
low-rate production cycle. Initial data in support of verifying
critical process compliance is expected in March 2004. Program
officials plan to establish preliminary boundaries for upper and lower
control limits by the full-rate production decision in June 2004, and
metrics are expected to be fully stable by the completion of the low-
rate deliveries in November 2004.
Assessments of Individual Programs:
Our assessments of the 51 weapon systems follow.
[End of section]
Airborne Laser (ABL):
MDA's ABL element is being developed in incremental, capability-based
blocks to destroy enemy missiles during the boost phase of their
flight. Carried aboard a highly modified Boeing 747 aircraft, ABL
employs a beam control/fire control subsystem to focus the beam on a
target; a high-energy chemical laser to rupture the skin of enemy
missiles; and a battle management subsystem to plan and execute
engagements. We assessed the Block 2006 configuration. Program
officials expect this block to provide an initial capability, but not
before 2006.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Arlington, Va.
Funding to complete through 2009:
R&D: $3,274.3 million:
Procurement: $0.0 million:
Total funding: $3,274.3 million:
Procurement quantity: 0:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
Latest cost includes all costs from the program's inception through
fiscal year 2009. Procurement funding and quantities have yet to be
determined. NA = not applicable:
[End of table]
Only one of ABL's seven critical technologies is fully mature, yet MDA
has released about 93 percent of the current block's engineering
drawings. Program officials plan to use the first ABL block to
demonstrate these technologies, but until this occurs, the potential
for design change remains. Additional drawings may also be needed if
the design is enhanced during the next block. The program experienced a
$242-million cost increase during fiscal year 2003, mainly because of
difficulties manufacturing components that could meet requirements.
Program officials recently postponed the procurement of the second
aircraft because of testing delays. This postponement allowed them to
shift funds to cover fiscal year 2003 cost overruns associated with
efforts to build the first aircraft.
[See PDF for image]
[End of figure]
ABL Program:
Technology Maturity:
Only one of ABL's seven critical technologies--managing the high power
beam--is fully mature. The remaining six other technologies--the six-
module laser, missile tracking, atmospheric compensation, transmissive
optics, optical coatings, and jitter control--are not fully mature. The
last three technologies are the least mature. All the above
technologies are necessary for generating and directing laser energy
onto a boosting missile.
While the program office has assessed the six-module laser as being
close to reaching full maturity, the power generated by grouping six
laser modules together must be demonstrated before this assessment can
be validated. The transmissive optics, optical coatings, and jitter
control are the least mature and consist of prototype technologies that
have only been tested in the laboratory, or demonstrated through
analysis and simulation. They have not been tested during the operation
of the six-module laser. The program plans to prove that all
technologies will work in an operational environment during a flight
test when ABL will attempt to shoot down a short-range ballistic
missile. Because the program cannot replicate an operational
environment on the ground, this flight test will provide the first
opportunity for many technologies to demonstrate their maturity.
Design Maturity:
The ABL program has completed 93 percent, or over 9,900, of the
expected 10,631 engineering drawings for the first block. Although
releasing this percentage of drawings suggests that ABL's design is
stable, it is a measurement of the current block's design stability
rather than the stability of future ABL blocks. Technology maturation
and future enhancements may lead to more design changes.
Production Maturity:
We did not assess the production maturity of ABL's current block
because of the limited quantity of hardware being produced.
Accordingly, statistical process control data is not available. Program
officials explained that it has been difficult to maintain a stable
manufacturing base for some subcomponents and that this problem has not
been resolved.
Other Program Issues:
Program officials recently identified performance of the ABL system
being developed during the current block as one of their greatest risks
toward achieving an initial capability. Between October 2002 and
September 2003, development costs increased by about $242 million.
Program officials attributed the cost overruns to difficulties with
component manufacturing and integration. They noted, for example, that
the leading cause of cost growth in the current effort is the
difficulty in manufacturing advanced optics and laser components.
Planned testing of the six integrated laser modules continues to slip,
and as of early February, the program had not rescheduled the test.
Program officials attribute the delays to the complexity and volume of
integration activities. This delay could affect subsequent program
events and has already caused the program to postpone procurement of a
second aircraft. The delay allowed program officials to shift those
funds, along with funds intended for other program activities, to cover
fiscal year 2003 cost overruns associated with efforts to build the
first aircraft.
Program Office Comments:
In commenting on a draft of this assessment, MDA maintained that the
current design is stable despite the assessed technology maturity.
Officials told us that because the ABL operational environment is
impractical to duplicate on the ground, the technology maturity
assessment process will understate actual maturity until after 100
percent of the drawings are released. While the officials expect
changes to future blocks as part of spiral development, they believe
the basic design will directly migrate to subsequent blocks.
With respect to the timing of the purchase of the second aircraft,
officials said the decision is still under deliberation as MDA
constantly assesses progress toward all objectives, including technical
maturity.
[End of section]
Aegis Ballistic Missile Defense (Aegis BMD):
MDA's Aegis BMD element of missile defense is being developed in
incremental, capability-based blocks to protect deployed U.S. forces
and other assets from ballistic missiles. Its two missions are long-
range surveillance and tracking in support of the Ballistic Missile
Defense System and engagement of short-and medium-range ballistic
missiles using the Standard Missile-3 (SM-3). We assessed the maturity
of the Block 2004 SM-3 missile.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin/Raytheon:
Program office: Crystal City, Va.
Funding to complete through 2009:
R&D: $3,918.1 million:
Procurement: $0.0 million:
Total funding: $3,918.1 million:
Procurement quantity: 0:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Latest cost includes all costs from the program's inception through
fiscal year 2009. Procurement funding and quantities have yet to be
determined. NA = not applicable:
According to the program office, the SM-3 technologies are mature and
the design is stable. However, the technology that enables the
interceptor's kill vehicle to maneuver itself to hit and destroy its
target has not been fully demonstrated. This "divert" technology
succeeded in ground testing but failed during a flight test in June
2003. MDA expects Aegis BMD to perform long-range surveillance and
tracking in support of the Ground-based Midcourse Defense element
beginning in September 2004. Agency plans call for Aegis BMD to be
capable of engaging short-and medium-range ballistic missiles by
December 2005.
[See PDF for image]
[End of figure]
AEGIS BMD Program:
Aegis BMD Element-Block 2004:
SM-3 development began with the "ALI" Program, a series of intercept
flight tests to demonstrate critical technologies of an interceptor
launched from a Navy cruiser. The SM-3 interceptor builds upon the SM-
2 missile, a two-stage missile in operational use by the U.S. Navy, but
incorporates a third stage rocket motor and a kinetic warhead--the kill
vehicle.
The third stage rocket motor and the infrared seeker of the kill
vehicle have been demonstrated in previous flight tests. However, while
the new solid divert attitude control system (SDACS) passed a series of
ground tests, it failed during its first flight test in June 2003.
According to program documents, the most likely cause of the failure
was a defective component within SDACS. The Aegis BMD Program Office
expects to resolve the issue by early 2004.
Design Maturity:
The SM-3 missile design is stable. At the time of the critical design
review in May 2003, 98 percent of the total expected drawings were
releasable to the manufacturer.
Production Maturity:
To meet a presidential directive requiring the fielding of an initial
missile defense capability beginning in 2004, five SM-3 missiles are
being developed in fiscal year 2004. These missiles are accelerated
test assets that could also be used, if needed, in a national
emergency. However, the missiles will not have a fully functional
SDACS. We did not assess statistical control processes for the five
missiles because these missiles are not production representative.
Other Program Issues:
Another component of the Aegis BMD program involves an upgrade of the
Aegis Weapon System--an operational asset comprised of the AN/SPY-1
Radar and Weapon Control System software--to accommodate the BMD
mission. Program officials told us that development and delivery of the
Aegis Weapon System are proceeding on schedule.
Program Office Comments:
In commenting on a draft of this assessment, the program office
generally concurred with the information presented. It added that the
latest flight test, held in December 2003, provided a successful
demonstration of SDACS.
GAO Comments:
While the program did conduct an SDACS test in December, this test did
not fully address SDACS issues because the divert system operated in
sustain mode. In sustain mode, the system does not use its two pulse
motors to steer the warhead during the final minutes before reaching
the target. To be considered fully functional, SDACS will require
successful testing using the two pulse motors.
[End of section]
Advanced Extremely High Frequency Satellite (AEHF):
The Air Force's AEHF satellite system is intended to replenish the
existing Milstar system with higher capacity, survivable, jam-
resistant, worldwide, secure communication capabilities for strategic
and tactical warfighters. The system also includes a mission control
segment. Terminals used to transmit and receive communications are
acquired separately by each service. AEHF is an international
partnership program that includes Canada, United Kingdom, and the
Netherlands. First launch is scheduled for December 2006.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin:
Program office: El Segundo, Calif.
Funding needed to complete:
R&D: $2,452.0 million:
Procurement: $473.6 million:
Total funding: $2,925.6 million:
Procurement quantity: 1:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The AEHF satellite program demonstrated most of its critical
technologies at development start and expects to have all technologies
demonstrated by the design review. The program has released two-thirds
of its drawings and expects to complete 90 percent by the design
review. In September 2003, the program office decided to delay the
launch of the first two satellites by 4 months. The delay was necessary
to accommodate changing security requirements and resolve fabrication
issues relating to the critical cryptological equipment.
[See PDF for image]
[End of figure]
AEHF Program:
Technology Maturity:
Eleven of the critical technologies identified by the program office
are mature. The remaining three technologies have engineering models
and are undergoing testing in a relevant environment to simulate both
launch and space atmosphere. Two of these three technologies have
mature backup technologies. Only one technology, a component of the
phased array antenna, does not have a backup technology that meets
operational requirements. Program officials expect all technologies to
be mature by the design review scheduled for April 2004.
Design Maturity:
The program office has released over two-thirds of its expected
drawings. The program office expects to release 90 percent of the
expected drawings by the scheduled design review. In addition,
preliminary design reviews are complete and the program office has
initiated the subsystem design reviews. The program is also developing
early software builds for the ground and space segments.
Production Maturity:
The production maturity could not be assessed because the program
office does not have statistical process control data. The Air Force
currently plans to buy only three satellites. However, there have been
some problems in producing a critical system component. The AEHF
Comsec/Transec System (ACTS) is a suite of cryptological equipment
installed in both the satellites and the terminals to limit access to
authorized users. ACTS has already experienced significant cost growth
and schedule delays due to changes in satellite architecture design,
interface, and other requirements changes. ACTS consists of computer
chips whose fabrication is more technically challenging than producing
other computer chips. The challenge results from a security requirement
to have separate foundries produce components of the chips that must be
integrated together. During a major functional test in September 2003,
a problem was discovered, and the program is evaluating ways to resolve
the problem.
Concurrent development of ACTS and the AEHF satellite payload has
resulted in a 4-month delay in the launch of the first two AEHF
satellites, now scheduled for April 2007 and April 2008, respectively.
ACTS is managed by the National Security Agency and is on the AEHF
satellite payload critical path. The program office stated the launch
delay was necessary to accommodate changes in ACTS security
requirements and resolve ACTS production issues.
Other Program Issues:
The current development contract includes the first two satellites and
the mission control segment. A decision to buy a third satellite is
planned after the design review. In December 2002, two satellites were
deleted from the program because the newly developed Transformational
Communications Architecture calls for the Transformational Satellite,
assessed elsewhere in this report, to replace these AEHF satellites.
Because the Transformational Satellites are early in development and
may not progress in time to meet the military need, the Air Force has
scheduled a progress review and decision point in early fiscal year
2005 to determine if additional AEHF satellites will be needed to meet
operational requirements.
Program Office Comments:
In commenting on a draft of this assessment, the program office noted
that the AEHF program continues to progress through the system
development and demonstration phase, meeting all scheduled milestones
and is projected to meet all key performance parameters.
[End of section]
Active Electronically Scanned Array Radar (AESA):
The Navy's AESA radar is one of the top upgrades for the F/A-18E/F
aircraft. It is to be the aircraft's primary search/track and weapon
control radar and is designed to correct deficiencies in the current
radar. According to the Navy, the AESA radar is key to maintaining the
Navy's air-to-air fighting advantage and will improve the effectiveness
of the air-to-ground weapons. When completed, the radar will be
inserted in new production aircraft and retrofitted into the existing
aircraft.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: McDonnell Douglas, Corp.
Program office: Patuxent River, Md. Funding needed to complete:
R&D: $219.9 million:
Procurement: TBD:
Total funding: TBD:
Procurement quantity: TBD:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The AESA radar's technology appears mature and the design is stable.
The program entered system development with technologies leveraged from
other DOD programs. However, the program identified four critical
technologies new to radar application. These technologies were not
mature at the start of system development or at the design review, but
they are now mature. Design changes have been identified as a result of
completed development tests, and more changes are anticipated as
development and operational tests continue during production. The
program anticipates retrofitting 135 aircraft with the radar at a cost
of about $424 million. These numbers could increase if operational
evaluation is delayed. Program officials estimate that the AESA radar's
first low-rate production units will exceed the cost target by 27
percent. Cost reduction initiatives are underway.
[See PDF for image]
[End of figure]
AESA Program:
Technology Maturity:
The AESA radar program utilizes technologies from other DOD programs.
Four critical technologies are new to radar application. These
technologies were evaluated using technology readiness levels and
determined to be mature based on initial testing. Software maturity
must be increased before this radar can be fully tested.
Design Maturity:
The AESA radar's design is stable. The program had 67 percent of
drawings released at design review. Additional drawings, however, may
be needed to address engineering design changes evolving out of ongoing
development tests and to address immature hardware and software that
existed during recently completed development tests. Software
development, to support development tests and technical evaluation, is
planned through most of fiscal year 2004. Operational evaluation, to
determine radar effectiveness and suitability, will be completed in the
summer of 2006. A recent initial operational test identified a number
of risks that will need to be addressed through development tests
before operational evaluation of the radar.
The program is tracking a number of technical, cost, and schedule risks
and challenges. First, the AESA radar places excessive loads on the
environmental control system. Second, parallel F/A-18E/F development
efforts may affect AESA integration and tests and delay production and
delivery schedules. Third, AESA radar operations could degrade
performance of other subsystems, resulting in unacceptable weapon
system performance.
Production Maturity:
We could not assess production maturity because statistical control
data was not available.
Other Program Issues:
The AESA radar's first low-rate production units are expected to exceed
the cost target by 27 percent. Most of the cost increase is
attributable to subcontractor development cost. The increase will not
affect the three favorably negotiated low-rate production lot options.
Cost increases during full-rate production, however, will occur if cost
reduction initiatives are not pursued. Cost reduction initiatives are
underway to reduce the cost overruns once the fix priced options
expire, but the initiatives are not fully funded. A recent cost
estimate, however, projects the program to be fully funded throughout
the 5-year defense plan.
Delivery of the first production AESA radars, for insertion into F/A-
18E/F aircraft on the production line, is scheduled for fiscal year
2005. This will result in only 8 of the planned 45 F/A-18E/F aircraft
in that fiscal year being equipped with the AESA radar on the
production line. A match between AESA radar production and F/A-18E/F
production will occur, with deliveries in fiscal year 2008. As a result
of the mismatch, 135 of the radars will need to be retrofitted into
already produced aircraft at a projected cost of $424 million. This
cost does not include the cost of legacy radars that must be installed
on aircraft that are not receiving the AESA radar. The need to retrofit
could be reduced if more radars were made available sooner. However,
while excess radar production capacity exists, program management does
not want to ramp up this production beyond current plans because it
would add risk to the program and take the radar into production prior
to completion of operational evaluation in mid fiscal year 2006. Delay
of operational evaluation would result in greater retrofit numbers.
Program Office Comments:
The AESA program office concurred with this assessment and provided
clarifying comments. The AESA radar received approval for the second
low-rate initial production effort from the Assistant Secretary of the
Navy, Research Development and Acquisition in January 2004. The first
F/A-18F with the AESA radar installed recently demonstrated high
resolution synthetic aperture radar (SAR) modes at 3 times the
resolution and 2-1/2 times the range of the current operationally
deployed F/A-18 radar. This high resolution SAR mode capability
represents the first step in multiple areas that the AESA radar will
greatly improve the F/A-18E/F Super Hornet's air-to-air and air-to-
ground radar capabilities in addition to adding modes not currently
available to the fleet.
[End of section]
Advanced Precision Kill Weapon System (APKWS):
The Army's APKWS is a precision-guided, air-to-surface missile designed
to engage soft and lightly armored targets. The system will add a new
laser-based seeker to the existing Hydra 70 Rocket System and is
expected to provide a lower cost, accurate alternative to the Hellfire
missile. Future block upgrades are planned to improve system
effectiveness. We assessed the laser guidance technology used in the
new seeker.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: General Dynamics:
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: $93.5 million:
Procurement: $1,530.0 million:
Total funding: $1,777.4 million:
Procurement quantity: 89,420:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The APKWS entered system development before demonstrating that its
critical guidance technology was fully mature. Program officials
currently project that the technology will not demonstrate maturity
until after the system design review. The program has released about
half of expected drawings, and program officials expect all will be
released by the time of the design review in March 2004. If immature
technology persists at the design review, risks of redesign and
modification of drawings late in development will be incurred.
[See PDF for image]
[End of figure]
APKWS Program:
Technology Maturity:
The APKWS' critical laser guidance technology has not demonstrated full
maturity. Although a prototype guidance system was successfully
demonstrated under the Low Cost Precision Kill Advanced Technology
Demonstration, the current design for the guidance system includes
numerous hardware changes to improve system cost, performance, and
producibility. The new design will not be fully integrated and tested
until June 2004, 3 months after the design review. Program officials
noted that although the prototype system design exists, reverting to
that design would increase cost and degrade the system's performance
and producibility.
Design Maturity:
Program officials expect to release 100 percent of the drawings by the
system-level design review in March 2004. At the time of our review,
the program had released only 55, or about 48 percent, of the 115 total
planned drawings to manufacturing.
Production Maturity:
Program officials expect that there will be nine key processes
associated with manufacturing the APKWS. The program plans to collect
statistical data on these processes when production begins.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that it demonstrated the technology maturity required by DOD
acquisition system policy during the Low Cost Precision Kill Advanced
Technology Demonstration. The APKWS technologies were successfully
demonstrated in both a high fidelity hardware-in-the-loop test facility
and a live-fire flight test environment. Program officials also stated
that although the system's final design requires some modification to
meet affordability, producibility, and operational requirements, these
design changes are consistent with the intent of the system development
and demonstration phase.
[End of section]
Advanced SEAL Delivery System (ASDS):
The Special Operations Forces' ASDS is a battery-powered, dry interior
minisubmarine developed for clandestine insertion and extraction of
Navy SEALs and their equipment. It is carried to its deployment area by
a specially configured SSN-688 class submarine. ASDS is intended to
provide increased range, payload, on-station loiter time, and endurance
over current submersibles. The 65-foot long, 8-foot diameter ASDS is
operated by a two-person crew, and equipped with a lock out/lock in
chamber to allow divers to exit and reenter the vehicle.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman:
Program office: Washington, D.C.
Funding needed to complete:
R&D: $35.8 million:
Procurement: $1,268.0 million:
Total funding: $1,341.4 million:
Procurement quantity: 5:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Two of ASDS' three critical technologies, the battery and the
propulsion, are not fully mature, even though system development began
over 9 years ago. Key technical problems with the battery and the
propeller were discovered late--during testing on the first boat--
rather than in component-or subsystem-level testing. Although
significant progress has been made in the past year, all critical
technologies have not achieved maturity and will not reach maturity
until the second ASDS boat is produced, currently estimated to be in
2008. However, program officials believe technology maturity may be
reached as early as 2005. In April 2003, DOD designated ASDS as a major
defense acquisition program, entailing greater oversight by high-level
decision makers. Most of the engineering drawings are complete;
however, these will be updated after the contract is awarded for the
second ASDS boat.
[See PDF for image]
[End of figure]
ASDS Program:
Technology Maturity:
Two of ASDS' three critical technologies--the battery and the
propulsion--have not reached maturity, and they are not expected to be
mature before the production decision for additional boats.
The silver-zinc propulsion battery has experienced premature failures
and short demonstrated life. Although the Navy continues to mature the
silver-zinc battery for the first boat, it intends to replace it with a
lithium-ion battery. The Navy has three contractors exploring this
technology. Two contracts were awarded to identify and test viable
lithium-ion battery technology for a battery that can be housed inside
the existing ASDS titanium battery bottles. Program officials expect to
receive battery samples in early 2004. A third contractor is developing
an alternative design for a battery that is contained in fiberglass
housings and will fit in the same area as the existing silver-zinc
battery. Lithium-ion battery technology, like silver-zinc, is not new;
however, the challenge lies in adapting the technology to ASDS' size
and environment.
The most significant noise offender, the propeller, was replaced with a
composite propeller before operational test and evaluation. However,
acoustic measurements have not been made, and other acoustic signature
issues still need to be addressed. The acoustic requirement has been
deferred until delivery of the second ASDS boat.
Design Maturity:
About 99 percent of the 4,999 engineering drawings have been released
to manufacturing for the second ASDS boat. After contract award for the
second ASDS boat, the contractor will prepare revised and new drawings
to account for part item substitutions, and to reflect updates in
commercial off-the-shelf equipment availability, especially for the
integrated control and display system.
The first ASDS boat has not demonstrated the ability to meet all of the
program's key performance parameters. Specifically, the first boat is
not quiet enough to meet acoustic stealth requirements, and compliance
with survivability requirements has not yet been verified and approved.
In addition, the Navy's operational evaluation of ASDS included
numerous recommendations to correct deficiencies and vulnerabilities
and recommended additional operational testing and evaluation to verify
corrections prior to full operational capability.
According to the program office, the follow-on ASDS boats--numbers two
through six--will be substantially similar to ASDS-1. It believes the
above changes and the change to a lithium-ion battery will have only
minor affects on the design. However, until survivability issues are
addressed, technical problems are solved, and testing is completed, we
believe the ASDS' final design will remain uncertain and may have cost
and schedule implications.
Other Program Issues:
Future testing issues could affect the program, but these results will
not be known before the production decision for additional boats
scheduled in early 2004. For example, the Commander, Operational Test
and Evaluation Force, recommended an additional phase of operational
test and evaluation to verify that deficiencies and vulnerabilities
identified during the May 2003 operational evaluation are corrected
prior to full operational capability. In addition, since the program's
first cost estimate was originally approved in 1994, research and
development costs have more than tripled, and the Navy has not yet
issued an updated cost estimate for follow-on boats and has not
provided a life-cycle cost estimate for the ASDS program.
Program Office Comments:
The ASDS program office provided technical comments, which were
incorporated as appropriate.
[End of section]
Advanced Threat Infrared Countermeasure/Common Missile Warning System:
The Army's and the Special Operations' ATIRCM/CMWS is a component of
the integrated infrared countermeasures suite planned to defend U.S.
aircraft from advanced infrared-guided missiles. The system will be
employed on Army and Special Operations aircraft. The system includes
an active infrared jammer, a missile warning system, and a
countermeasure dispenser capable of loading and employing expendables,
such as flares, chaff, and smoke.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: BAE Systems North America:
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: $61.4 million:
Procurement: $2,608.6 million:
Total funding: $2,670.0 million:
Procurement quantity: 2,673:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The ATIRCM/CMWS program entered production in November 2003 with
technologies mature, designs stable, and production processes in
control. The CMWS portion of the program entered limited production in
February 2002 to meet urgent deployment requirements. However, full-
rate production for both components was delayed because of reliability
problems. Over the past several years, the program has had to overcome
cost and schedule problems brought on by shortfalls in knowledge: key
technologies were demonstrated late in development and only a small
number of design drawings were completed by design review.
[See PDF for image]
[End of figure]
ATIRCM/CMWS Program:
Technology Maturity:
The ATIRCM/CMWS' five critical technologies are mature. However, they
did not mature until after the design review in February 1997. Most of
the early technology development effort was focused on the application
to rotary wing aircraft. When system development began in 1995, the
requirements were expanded to include Navy and Air Force fixed wing
aircraft. This change caused problems that largely contributed to cost
increases of more than 150 percent to the development contract. The
Navy and the Air Force subsequently dropped out of the program,
rendering the extra effort needless.
Design Maturity:
The basic design of the system is complete with 100 percent of the
drawings released to manufacturing. The design was not mature at the
time of the design review, with only 22 percent of the drawings
complete. This was primarily due to the expanded requirements. It was
not until 2 years after the design review that 90 percent of the
drawings were released and the design was considered stable. This
resulted in inefficient manufacturing, rework, additional testing, and
a 3-year schedule delay.
Production Maturity:
The ATIRCM/CMWS program has all 15 key manufacturing processes in
control. The Army entered limited CMWS production in February 2002 to
meet an urgent need of the Special Operations Command. The ATIRCM
subsystem's production was delayed due to reliability testing failures.
The program is implementing reliability fixes to six production
representative subsystems that will be used for initial operational
test and evaluation. The subsystems will be delivered in March 2004.
The full-rate production decision for the complete system is now
scheduled for 2005.
Other Program Issues:
The Army procured an initial 32 systems for use on Special Operations'
CH-47 helicopters in fiscal year 2002 that only included CMWS. The Army
plans to procure a total of 99 systems to outfit Special Operations'
aircraft between fiscal years 2003 and 2009.
Program Office Comments:
The ATIRCM/CMWS program office concurred with this assessment and
provided technical comments, which were incorporated where appropriate.
Additionally, it commented that the Army acquisition executive approved
the Army Systems Acquisition Review Council's recommendation that
ATIRCM/CMWS transition from system development and demonstration to
production and deployment. Initial operational tests and evaluation
will be completed in fiscal years 2004 and 2005. A full-rate production
decision review is planned in August 2005.
[End of section]
Advanced Wideband Satellite/Transformational Satellite (AWS/TSat):
The AWS/TSat system is designed to provide improved, survivable, jam-
resistant, worldwide, secure, and general purpose communications to
support DOD in conjuction with systems that support NASA and the
intelligence community. It will replace the current Milstar satellite
system and supplement the AEHF satellite system, reviewed elsewhere in
this report. It will include multiple satellite systems and be a
cornerstone of the new DOD communications architecture.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: In Competition:
Program office: El Segundo, Calif.
Funding needed to complete:
R&D: TBD:
Procurement: TBD:
Total funding: TBD:
Procurement quantity: 8:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The AWS/TSat program entered system development in December 2003 with
only one of its five critical technologies mature. The remaining four
technologies are not expected to reach maturity until 2006. The product
development period requires concurrent technology maturation and
product development activities to maintain schedule.
[See PDF for image]
[End of figure]
AWS/TSat Program:
Technology Maturity:
Of the five AWS/TSat critical space technologies, one is mature while
the other four are scheduled to reach maturity in early 2006, more than
2 years after the planned start of development. Three of the four
immature technologies have a backup technology available in case of
development difficulties. However, use of any of the backup
technologies would degrade overall system performance. The Single
Access Laser Communications technology has no backup, and according to
program officials, any delay in maturing this technology would cause
the expected first satellite launch date to slip beyond 2011.
Other Program Issues:
The AWS/TSat acquisition strategy allows the system's technology
development and product development to be conducted concurrently prior
to the production decision. Because the military users expect new
communications capability by 2011 and they were concerned with the
aggressive acquisition strategy of the AWS/TSat program, the Air Force
scheduled an interim review point in November 2004. The review is
intended to determine if technology development has progressed
sufficiently to ensure the military users' needs can be met no later
than 2011. If not, the Air Force must decide on alternatives, one of
which is to buy an additional AEHF satellite. Air Force officials have
not defined the evaluation criteria they intend to use to assess AWS/
TSat's progress or determine alternatives.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the current AWS/TSat development plan matures all critical path
technologies sufficiently before the preliminary design review. The Air
Force believes this is consistent with both government and commercial
best practices. Furthermore, it noted that nearly all technologies that
are not now mature have backup technologies that provide significantly
increased capability to the warfighter. The only exception is the laser
communications subsystem that it believes is a low risk for production.
[End of section]
B-2 Radar Modernization Program (B-2 RMP):
The Air Force's B-2 RMP is designed to modify the current radar system
to resolve potential conflicts in frequency band usage between the B-2
and a commercial communication satellite system under development. To
comply with federal requirements, the frequency must be changed to a
band where the B-2 will be designated as a primary user. The modified
radar system is being designed to support the B-2 stealth bomber and
its combination of stealth, range, payload, and near precision weapons
delivery capabilities.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $696.1 million:
Procurement: $498.9 million:
Total funding: $1,195.0 million:
Procurement quantity: 21:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The B-2 RMP's two critical technologies are fully mature well in
advance of development start, scheduled for May 2004. A date has not
been set for the final design readiness review. The program plans to
build six radar units during development for pilot training with the B-
2 operational wing. These prototypes will later become operational
units on the B-2 aircraft.
[See PDF for image]
[End of figure]
B-2 RMP Program:
Technology Maturity:
The B-2 RMP's two critical technologies, the transmit/receive modules
of the AESA antenna and the beam steering controller software, appear
mature. A formal technology readiness assessment is planned for
completion prior to the start of development in May 2004. In an effort
to further reduce risk, the program has already built and tested some
transmit/receive modules. In addition, several key elements of the
modules have already been tested in an operational environment. Over
half of the beam steering controller software has been demonstrated on
prior AESA upgrade programs.
Design Maturity:
The contractor built and tested some transmit/receiver modules as part
of a proof-of-design phase prior to the start of development. However,
the contractor has not released any manufacturing drawings because the
program is not scheduled to start development until May 2004. A date
has not yet been set for the final design readiness review.
Production Maturity:
Both the prime contractor and the major subcontractor plan to collect
manufacturing process control data. Production is scheduled to begin in
January 2007. The program plans to conduct a production readiness
review prior to the planned start of production. The program is also
involved in a proof-of-manufacturing effort to demonstrate that the
transmit/receive modules can be built to specifications.
Other Program Issues:
The program plans to build six radar units during development and later
modify these units for placement on operational B-2 aircraft. The Air
Force needs these six radar units when the current B-2 radar frequency
becomes unavailable, in order to continue vital air crew training and
proficiency operations. Building these six units early in development
adds risk because most of the radar flight-test activity will not occur
until after these units are built.
Program Office Comments:
The B-2 Program Office concurred with this assessment.
[End of section]
C-130 Avionics Modernization Program (C-130 AMP):
The C-130 AMP standardizes the cockpit configurations and avionics for
all 14 different mission designs of the C-130 fleet. It consolidates
and installs the mandated DOD navigation/safety modifications, the
Global Air Traffic Management systems, and the C-130 broad area review
requirements. It also incorporates other reliability, maintainability,
and sustainability upgrades and provides increased situational
awareness capabilities.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $900.2 million:
Procurement: $2,876.1 million:
Total funding: $3,776.3 million:
Procurement quantity: 479:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The C-130 AMP is utilizing commercial and modified off-the-shelf
technologies, and it entered system development with all but one of its
six critical technologies mature. The remaining technology is nearing
full maturity; however, there is concern that it may not meet current
performance requirements. Program officials are working with the user
to lower requirements to match resources. Program officials plan to
release 90 percent of engineering drawings by the design review. The
program office recently delayed program milestones in response to
funding reductions. While this delay provides extra time to achieve
design stability, it reduces the time available to achieve production
knowledge. Plans to accelerate the installation on Special Operations
aircraft and software integration challenges are placing additional
pressure on the compressed schedule.
[See PDF for image]
[End of figure]
C-130 AMP Program:
Technology Maturity:
Five of the C-130 AMP's critical technologies are fully mature. The
program utilizes primarily proven commercial and modified off-the-shelf
technology for all AMP capabilities. The remaining critical technology,
the Terrain Following and Terrain Avoidance (TF/TA) capability, has
been demonstrated through the Air Force Research Lab's Quiet Knight
advanced technology demonstration program, and it is nearing full
maturity. There is a risk that the TF/TA technology may not meet a key
requirement to operate at 250 feet. Program officials are working with
the user to lower the requirements to operate between 250 and 1,000
feet, which will more closely match the capability of the TF/TA
technology. Failure to make this change may necessitate a redesign.
Design Maturity:
Currently, 14 percent of the design drawings are complete and could be
released to manufacturing. Program officials stated they are committed
to meeting the required 90 percent drawing release by design review,
now scheduled for May 2006.
The modernization effort is divided into a number of capability spirals
due to the various aircraft designs. The first spiral will outfit C-130
aircraft with core capabilities and an integrated defensive system.
Future spirals are planned for Special Operations Command's C-130
aircraft because they require additional, unique defensive systems
integration and enhanced situational awareness.
Other Program Issues:
Funding reductions in fiscal years 2003 and 2004 delayed the C-130
AMP's development program and resulted in a rescheduling of program
milestones. The design review, low-rate initial production, and
production readiness decisions have all been delayed. Program officials
stated that the delay in schedule would provide more time to resolve
issues with the TF/TA technology and software. Despite this additional
time, the time available for system integration has been compressed by
9 months, giving less time to reduce manufacturing risks. There is also
a new plan to accelerate Special Operations Command aircraft deliveries
by 12-14 months, further compressing an already optimistic timeline.
The program is also at risk if less software is reused than originally
estimated, which may cause an increase in development costs and delay
the program's schedule. Software integration remains a risk due to its
complexity, number of suppliers, potential for developmental growth,
certification of a secure operating system, and software safety
standards. The program office is working to mitigate these risks
through modeling and simulation, utilizing the systems integration
laboratory built by the contractor, and through flight demonstrations.
Program Office Comments:
In commenting on a draft of this assessment, program officials stated
that the technology demonstrated during the Quiet Knight demonstration
is mature and that the remaining technological challenge lies in
integrating all TF/TA system components and coupling them with the
other avionics functions. An early study identified a risk that the TF/
TA system may not meet a key requirement to operate at 250 feet caused
by errors attributable to the integrated subsystems. The study
identified fixes to minimize errors, and program officials stated they
worked closely with the user and the contractor for implementation to
ensure a match between requirements and the TF/TA System capabilities.
Only this key engineering requirement was loosened to match the
capability of the currently fielded systems. Program officials further
stated that the risk of the compressed schedule should be reduced by a
robust predevelopment test and evaluation TF/TA flight demonstration
and having two aircraft in development testing.
[End of section]
C-5 Avionics Modernization Program (C-5 AMP):
The Air Force's C-5 AMP is the first of two major upgrades for the C-5
to improve the mission capability rate and transport capabilities and
reduce ownership costs. The AMP implements Global Air Traffic
Management, navigation and safety equipment, modern digital equipment,
and an all-weather flight control system. The second major upgrade, the
C-5 Reliability Enhancement and Reengining Program (RERP), replaces the
engines and modifies the electrical, fuel, and hydraulic systems. We
assessed the C-5 AMP.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $76.3 million:
Procurement: $280.1 million:
Total funding: $356.3 million:
Procurement quantity: 45:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The program office considers the C-5 AMP's critical technologies to be
mature because they are relying on commercial off-the-shelf
technologies that are installed in other commercial and military
aircraft. The main challenge involves the development and integration
of software. The Air Force plans to modify 55 of the 112 C-5 aircraft,
and the program office has let the production contract for the first 8
C-5 AMP modifications. The Air Force is also seeking funding to modify
the remaining 57 C-5s; however, that decision will not be made until
the Air Force determines whether it will use C-17s instead of the C-5s
to meet its airlift requirements. If the Air Force decides to use the
C-17s, it may not upgrade some, or all, of the remaining 57 C-5s.
[See PDF for image]
[End of figure]
C-5 AMP Program:
Technology Maturity:
We did not assess the C-5 AMP's critical technologies as the program
used commercial technologies that are considered mature. Program
officials indicated that the technologies are in use on other aircraft.
For example, the new computer processors are being used in the Boeing
777, 717, other commercial aircraft, the KC-10, and a Navy
reconnaissance aircraft.
Design Maturity:
The design appears stable as the contractor has released 100 percent of
the drawings for the AMP. In addition, the seven major subsystem-level
design reviews were completed before the December 2003 system-level
design review. Demonstration of these integration activities is
scheduled during development test and evaluation, which was started in
December 2002 and should be completed in October 2004.
Production Maturity:
We could not assess the production maturity because most components are
readily available as commercial off-the-shelf items. This equipment is
being used on other military and commercial aircraft. In addition, the
C-5 AMP is incorporating many other off-the-shelf systems and
equipment, such as the embedded global positioning system (GPS), the
inertial navigation system, and the multifunction control and display
units. To ensure production maturity, the program office is collecting
data regarding modification kit availability and installation
schedules.
Other Program Issues:
Program officials indicated the greatest risk to the AMP is software
development and integration. Several new software programs must be
developed and integrated with several other commercial off-the-shelf
software packages. Program officials stated that the software
development risks stem from a variety of issues, including an
aggressive cost and schedule baseline and a geographically diverse
software development team. To overcome these problems, the prime
contractor added additional staff. Program officials are confident that
the problems will be satisfactorily resolved within the current
schedule.
The C-5 aircraft must undergo the AMP modifications prior to the RERP
modifications. However, only 55 aircraft have been approved for the AMP
upgrades, while 112 are awaiting RERP upgrades. The Air Force needs to
determine how many of the remaining 57 C-5s will receive the AMP
upgrades. That decision will not be made until the Air Force determines
the correct mix of C-5 and C-17 aircraft that are needed to meet DOD's
airlift needs. Until it is decided whether to use C-17s to replace
some, or all, of the earlier 57 C-5s, the number of aircraft to undergo
the AMP and RERP modernization will remain uncertain.
Program Office Comments:
In commenting on the draft of this assessment, the program office said
that the cost comparison of the November 1998 AMP position to the
latest AMP position, for the purpose of calculating a percentage change
to the unit cost, does not accurately portray the program's cost
growth. The November 1998 position represents the original 126-aircraft
program. The program has since been restructured to a 55-aircraft
program. Such a change would increase costs by a large amount because
it would be less expensive, on a unit cost basis, to procure a greater
number of aircraft than it would be to procure fewer aircraft.
GAO Comments:
While the program has established a new cost and performance baseline
since the November 1998 decision to begin development, the comparison
presented provides an accurate picture of change since that major
decision. Although DOD may update its baseline for management purposes,
our goal is to provide an aggregate or overall picture of the program's
history.
[End of section]
C-5 Reliability Enhancement and Reengining Program (C-5 RERP):
The Air Force's C-5 RERP is one of two major upgrades for the C-5
aircraft. The RERP is designed to enhance the reliability of the
aircraft by replacing engines and modifying subsystems such as the
electrical, fuel, hydraulic, and flight controls systems, while the C-
5 AMP is designed to enhance the avionics. These upgrades are part of a
two-phased modernization effort to improve the mission capability rate
and transport capabilities and reduce ownership costs. We assessed the
C-5 RERP.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $1,093.2 million:
Procurement: $7,514.1 million:
Total funding: $8,610.7 million:
Procurement quantity: 109:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The program ensured that the technology was mature and that the design
was stable at critical points in development, closely tracking best
practice standards. The RERP is utilizing demonstrated commercial off-
the-shelf components that require little or no modification. The major
challenge to the RERP is software development and integration, which
has experienced problems. Also, the RERP is dependent on the number of
aircraft approved to undergo the C-5 AMP upgrades. Until additional
aircraft are approved for the C-5 AMP, it is uncertain how many
aircraft will undergo the RERP.
[See PDF for image]
[End of figure]
C-5 RERP Program:
Technology Maturity:
The C-5 RERP technologies are mature based on an independent technology
readiness assessment conducted in October 2001. New engines account for
64 percent of the expected improvement in mission capability rate for
the aircraft. The new engines are commercial jet engines currently
being used on numerous airlines. According to the Air Force technology
assessment, these engines have over 70 million flying hours of use.
Design Maturity:
The C-5 RERP design is mature. As of November 2003, 98 percent of the
design drawings were complete. In addition, the seven major subsystem-
level design reviews were completed before the December 2003 system-
level design review.
According to program officials, the greatest risk to the C-5 RERP is
software development and integration activities. Several new software
programs must be developed and integrated together as well as with
other commercial off-the-shelf software packages. The program has
experienced problems during software development and integration, and
it believes these problems are linked to pressures caused by an
aggressive cost and schedule baseline and different geographical
locations of the software development team. A program official stated
that the prime contractor has started to take actions to improve
program software development activities.
Production Maturity:
We did not assess the C-5 RERP production maturity because the Air
Force is buying commercially available items. However, we expect that
production maturity would be at a high level.
Other Program Issues:
The C-5 RERP is dependent on the C-5 AMP, as the aircraft has to
undergo avionics modernization prior to the RERP. The C-5 RERP has been
authorized for 112 of the C-5 aircraft, but the AMP has only been
authorized for 55 aircraft. Therefore, until the Air Force decides on
how many C-5 aircraft will undergo avionics modernization, it is
uncertain how many aircraft will undergo the RERP upgrades.
Program Office Comments:
In commenting on the draft of this assessment, the program office
stated that the cost comparison of the November 2001 RERP position to
the latest RERP position, for the purpose of calculating a percentage
change to the unit cost, does not accurately portray the program's cost
growth. The November 2001 position represents the original 126-aircraft
program. The program has since been restructured to a 112-aircraft
program. It further stated that such a change would increase costs by a
large amount because it would be less expensive, on a unit cost basis,
to procure a greater number of aircraft than it would be to procure
fewer aircraft.
GAO Comments:
While the program has established a new cost and performance baseline
since the November 2001 decision to begin development, the comparison
presented provides an accurate picture of change since that major
decision. Although DOD may update its baseline for management purposes,
our goal is to provide an aggregate or overall picture of the program's
history.
[End of section]
Cooperative Engagement Capability (CEC):
The Navy's CEC is designed to connect radar systems to enhance
detection and engagement of air targets. Ships and planes equipped with
their version of CEC hardware and software will share real-time data to
create composite radar tracks--allowing the battle group to see the
same radar picture. A CEC-equipped ship can then detect and launch
missiles against targets its radar cannot see. We assessed the current
shipboard and airborne versions of CEC.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon Systems Corporation:
Program office: Washington, D.C.
Funding needed to complete:
R&D: $487.0 million:
Procurement: $1,353.1 million:
Total funding: $1,840.1 million:
Procurement quantity: 221:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
CEC's production maturity could not be assessed because the government
does not collect the necessary data on the commercially available
portions of the shipboard and airborne versions of CEC. However,
program and contractor officials consider the production processes to
be capable of producing a quality product on time and within cost. The
technologies and design of both the shipboard and airborne versions are
fully mature. In April 2002, the shipboard version was approved for
full-rate production, and the airborne version was approved for
continued low-rate initial production.
[See PDF for image]
[End of figure]
CEC Program:
Technology Maturity:
All six of CEC's critical technologies are considered mature, based on
operational assessments by the Office of Naval Research issued in
January 2002. While the shipboard and airborne versions have different
hardware, they share the same six critical technologies.
Design Maturity:
CEC's basic design appears stable. All drawings needed to build the
shipboard and airborne versions have been released to manufacturing.
CEC program officials noted that new drawings for both versions will
continue to be released. They explained that as commercially available
technologies, which comprise approximately 60 percent of the CEC
hardware, become more advanced, portions of the system will need to be
redesigned to incorporate those advances.
Production Maturity:
We could not assess production maturity as data was not available.
According to program officials, the noncommercially available portions
of CEC do not involve any critical manufacturing processes. Officials
indicated that they do not have insight into the manufacturing
processes for the commercially available portion, including whether
these processes are critical and whether the contractor has them under
statistical control.
Program officials and the contractor are confident that a quality
product can be delivered on time and within cost based on the
contractor's adherence to industry standards and past performance on
low-rate initial production contracts for the shipboard version. Also,
according to program officials, a production readiness review of the
airborne version is planned for the second quarter of fiscal year 2004.
Other Program Issues:
In November 2003, the Navy announced plans to resolve outstanding
issues associated with CEC's interoperability by pursuing open
architecture and functionality changes in coordination with the Joint
Single Integrated Air Picture Systems Engineering Organization (JSSEO).
The CEC Program Office then discontinued planning for a Block 2
development effort and began working with JSSEO to jointly engineer
sensor measurement and radar tracking management solutions. According
to the CEC Program Office, the JSSEO's goal is to have a common set of
solutions available to all services to implement, thereby ensuring
optimum interoperability across the battlespace.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that it generally concurred with our assessment. The program office
also noted it will be incorporating a new antenna assembly, which is a
critical technology, into the shipboard version starting in fiscal year
2005. This antenna assembly will eliminate the current need for two
antenna arrays on some ships. The new antenna array, which is expected
to be less expensive, will be produced using commercial processes. The
program office plans to hold a production readiness review on the new
antenna assembly in the second quarter of fiscal year 2004.
[End of section]
CH-47F Improved Cargo Helicopter (CH-47F):
The Army's CH-47F heavy lift helicopter is intended to provide
transportation for tactical vehicles, artillery, engineer equipment,
personnel, and logistical support equipment. It is also expected to
operate in both day and night. The purpose of the CH-47F program is to
enhance performance and extend the useful life of the CH-47 helicopter.
This effort includes installing a digitized cockpit, rebuilding the
airframe, and reducing aircraft vibration.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing Helicopters:
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: $10.4 million:
Procurement: $5,594.9 million:
Total funding: $5,605.3 million:
Procurement quantity: 330:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The CH-47F helicopter began low-rate production in December 2002,
although key production processes were not in control. Program
officials believe that production is low risk because no new technology
is being inserted into the aircraft, two prototypes have been produced,
and the production process has been demonstrated during the development
phase. The CH-47F technologies and design appear mature, although a low
percentage of engineering drawings were released at the design review.
In 2002, production unit costs more than doubled due to contractor rate
increases, new system requirements, and initial underestimation of
program cost.
[See PDF for image]
[End of figure]
CH-47F Program:
Technology Maturity:
Although we did not assess technology maturity in detail, the CH-47F
helicopter is a modification of the existing CH-47D helicopter. Program
officials believe that all critical technologies are mature and have
been demonstrated prior to integration into the CH-47F development
program.
Design Maturity:
The CH-47F design is complete, with 100 percent of the drawings
released to manufacturing. However, at the design review only 37
percent of the system's engineering drawings were complete. Since that
time, the number of drawings completed increased substantially. The
majority of the new drawings were instituted to correct wire routing
and installation on the aircraft, changes the program office believed
could not be determined until after the first prototype was developed.
Production Maturity:
The CH-47F's production maturity could not be determined because
statistical process control data was not available. According to
Boeing, this data is available at the contractor; however, the CH-47F
program is not reviewing it. Although program office officials believe
the CH-47F's production is low risk, because two prototypes were
produced during development, there is no evidence to show that its
critical manufacturing processes are under control. In the absence of
this data, the program office started its second low-rate initial
production in December 2003.
Other Program Issues:
In addition to the cost increases experienced in the CH-47F program
last year, further cost increases and schedule delays are expected due
to DOD's direction to remanufacture more helicopters for the Special
Operations Command, which have not yet been reflected in the costs of
the program. According to the CH-47F deputy program manager, DOD
directed the Army to remanufacture an additional 16 MH-47G helicopters
for the Special Operations Command before the start of the Army's low-
rate initial production for its CH-47F helicopters. The program office
maintains that DOD's decision affected the program's cost and schedule
estimates and resulted in a schedule rebaselining of the CH-47F
program. The restructuring will result in a schedule slippage of 15
months and a cost increase of about $630 million, the majority of which
will go toward replacing helicopters provided to the Special Operations
Command.
Program Office Comments:
In commenting on a draft of this assessment, the CH-47F product manager
generally concurred with this assessment, but provided clarifying
comments. Regarding manufacturing control processes, the CH-47F product
manager stated that quality control measures that are more realistic
than the statistical process control metrics are in place and are being
monitored to ensure production maturity. For example, both contractor
and government personnel are inspecting all flight safety parts, and
the program office reviews this data monthly.
The product manager believes that the initial program cost increase
associated with the additional helicopters for the Special Operations
Command has been absorbed into the current program and will have a
minimal effect on average unit cost. Further, a revised program
deviation report addressing the schedule slip for the first unit-
equipped date is pending approval by the Army acquisition executive.
[End of section]
Comanche Reconnaissance Attack Helicopter (RAH-66):
The Army has terminated the Comanche program to reallocate resources.
It was the Army's next generation armed reconnaissance aircraft system
and its technology would have provided the Army with a system capable
of operating in adverse weather conditions across a wide spectrum of
threat environments. It would have replaced AH-1, OH-6, and OH-58A/C/D
helicopters. We have retained the Comanche assessment because it
remains a good example of attaining key product knowledge.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing Sikorsky:
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: $4,968.8 million:
Procurement: $21,955.4 million:
Total funding: $26,962.7 million:
Procurement quantity: 646:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Most critical technologies have demonstrated acceptable levels of
maturity, and the program has confirmed the stability of the first
design iteration through its recent critical design review. This level
of maturity follows years of difficult development. In 2002, the
program was restructured to incorporate an evolutionary acquisition
approach and reduce concurrency and lower overall risk. The
restructured program includes aspects of a knowledge-based acquisition
approach that provides better balance in the program by spreading out
requirements and adding resources and time for development.The prime
contract calls for the collection of drawing release data and
statistical process control data; its award fee provides incentives for
demonstrating increasing levels of design stability.
[See PDF for image]
[End of figure]
Comanche Program:
Technology Maturity:
Seven of the Comanche's eight critical technologies are considered
mature. However, the radar cross-section technology, needed for low
observability, requires additional development. The Army does not
expect this technology will reach maturity until fiscal year 2005--1
year before the production decision.
Design Maturity:
The Comanche program essentially attained design stability for the
initial configuration of the aircraft. At the completion of the design
review, 84 percent of the helicopter's engineering design drawings were
complete and released to the manufacturer.
Prior to the 2002 program restructuring, integration of critical
technologies was considered high risk, even though most of the
technologies had demonstrated individual maturity. The restructuring
adopted an evolutionary acquisition approach, realigned program
requirements, added about $4.0 billion for additional testing, added
time and testing capabilities, and adopted methods for improving
contractor performance. The additional resources, coupled with fewer
initial requirements, allowed the program to build more design
knowledge before committing to production--thereby reducing risks. The
tools used to gather and validate design knowledge are required by the
contract, and targeted award fees provide additional incentives for
building knowledge.
One remaining design risk is that development testing of a fully
integrated Comanche will not take place until after the production
decision. Discovering and correcting design problems during production
will be much costlier than problems discovered during development.
Production Maturity:
The restructured Comanche program calls for collecting more knowledge
about production processes and production maturity than we have seen on
many programs. Specifically, the Army plans to collect information on
control over the Comanche's production processes and reliability, and
it has included these requirements in the Comanche contract. In
addition, the contractor has established reliability growth plans and
goals and has started conducting reliability growth testing. At this
point, two risks for demonstrating production maturity remain: (1) it
is not clear that all key aircraft characteristics will be identified
by the critical design review and (2) the Army has not set a standard
for what constitutes an acceptable level of production process control.
Program Office Comments:
In commenting on a draft of this assessment, the program office
generally concurred with the information presented in this report. It
noted however that comparing the current program with the acquisition
program baseline dated October 2002 would show essentially no variance.
GAO Comments:
While the program has established a new cost and performance baseline
since the July 2000 decision to begin development, the comparison
presented provides an accurate picture of change since that major
decision. Although DOD may update its baseline for management purposes,
our goal is to provide an aggregate or overall picture of the program's
history.
[End of section]
Future Aircraft Carrier CVN-21:
The Navy's CVN-21 class is the successor to the Nimitz-class aircraft
carrier and includes a number of advanced technologies in propulsion,
aircraft launch and recovery, weapons handling, and survivability.
These technologies will allow for increased sortie rates and decreased
manning rates as compared to existing systems. Many of the technologies
were intended for the second ship in the class, but they were
accelerated into the first ship in a December 2002 restructuring of the
program.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Newport News:
Program office: Washington, D.C.
Funding needed to complete:
R&D: $1,862.9 million:
Procurement: TBD:
Total funding: TBD:
Procurement quantity: 1:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The CVN-21 is expected to enter system development in April 2004 with
very few of its critical technologies fully mature. This is due in part
to DOD's decision to accelerate the installation of a number of
technologies from the second ship into the first ship. The accelerated
technologies are at much lower levels of maturity. Program officials
state that the extended construction and design period that ends in
2014 allows further time for technology development. Program officials
established a risk reduction strategy that includes decision points for
each technology's inclusion based on demonstrated maturity level. These
decision points coincide with key design milestones and include mature
backup technologies for all but two technologies.
[See PDF for image]
[End of figure]
CVN-21 Program:
Technology Maturity:
Program officials currently estimate that 3 of the 13 critical
technologies will be mature by system development, and that 3 more will
be approaching maturity. An additional seven will be at much lower
levels of readiness. The technologies vary widely in maturity due to a
mix of factors, including decisions by acquisition officials, standard
practices in Navy shipbuilding, and feasibility of sea-based testing.
Of the six critical technologies identified at or just below
recommended maturity levels by system development, all were a part of
the original acquisition approach for the first ship. These
technologies were well into development by December 2002 when the
program was restructured and technologies were accelerated from the
second ship into the first ship. For example, the original technologies
included the reverse osmosis desalinization plant, critical to the
functioning of the nuclear propulsion system as well as daily
functions. Through a series of land-and sea-based tests, this
technology has been brought to recommended maturity levels.
In contrast, technologies accelerated during the restructuring are at
lower levels of maturity. For example, the advanced weapons elevator
will exceed the current elevator load capacity by 70 percent or more
while increasing sortie rates and decreasing operating costs.
Development of this technology did not begin until February 2003, and
it will not be fully mature by the start of system development.
Program officials stated that the risk associated with development of
some CVN-21 technologies is manageable due to the nature of ship
construction. Critical technologies, such as the radar systems and the
advanced arresting gear, reside in the upper decks of the ship and are
not slated for ship installation until late in the process.
Program officials stated that it is not possible to mature some systems
to the best practices standard early in development. One such system is
the electromagnetic aircraft launch system, a replacement for the
current steam catapult system used to launch aircraft off carriers.
This system has been in development since the late 1990s, but due to
the size and complexity of the system it cannot be prototype tested
aboard a surrogate ship. While land-based demonstrations of form, fit,
and function are possible, a full operational environment is not
reasonably achievable.
The program's risk reduction strategy defines a timeline for making
decisions about a technology's maturity. For the majority of the
technologies, a readiness review will occur in early fiscal year 2005.
Other technologies, primarily those included in later stages of design,
will be assessed later. If technologies are not ready for inclusion,
fallback technologies will be used. The program has mature fallback
technologies for all systems except nuclear propulsion and
desalinization systems.
Other Program Issues:
System development was delayed by the decision to restructure the
program; however, the dates for construction start and commissioning
the ship have not been moved. The date for delivering the ship to the
fleet is driven by the decommissioning of the U.S.S. Enterprise, which
will reach the end of its service life in 2014. This schedule
compression raises the risk of costly redesign late in development.
Program Office Comments:
In commenting on a draft of this assessment, the program office
emphasized that the CVN-21 program has established a technology
development strategy to manage the risk associated with bringing new
technologies into the design. Each new technology has a development
timeline with identified decision points for evaluating technology
maturity. These decision points are linked to key events in the
platform design schedule and the technology development schedule.
Program officials stated that if sufficient maturity has not been
demonstrated at the decision points, an "off-ramp" can be selected to a
fallback technology. Fallback plans identify existing, mature
technologies that can be incorporated into the design within ship
delivery schedule constraints. Program officials indicated that in some
cases selection of an off ramp would result in a loss of projected
operational capability, but at least equal current capability.
Technologies that do not mature in time will continue development for
follow-on ships.
[End of section]
DD(X) Destroyer:
The Navy's DD(X) is a multimission surface combatant designed to
provide advanced land attack capability in support of forces ashore and
contribute to U.S. military dominance in littoral operations. In
November 2001, the Navy restructured the DD(X) program to focus on
developing and maturing a number of transformational technologies.
These technologies will provide a baseline to support development of a
range of future surface ships such as the future cruiser and the
Littoral Combat Ship.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Ship Systems:
Program office: Washington, D.C. Funding needed to complete:
R&D: $7,452.1 million:
Procurement: TBD:
Total funding: TBD:
Procurement quantity: TBD:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Costs increased due to changes in cost estimates, technology
development programs, and program restructuring. Current estimate
includes detailed design and construction of the lead ship.
DD(X) is scheduled to enter system development with none of its 12
critical technologies fully mature. The program is pursuing risk
mitigation by constructing and testing engineering development models
for its critical technologies. The acquisition strategy calls for
engineering development model construction and testing concurrent with
system design. Because of schedule slippage, only two models will be
mature by the award of the lead ship construction contract, currently
planned for September 2005. Backups are available for only 2 of the 12
technologies. Program progress has been hampered by changes in desired
ship size and capabilities.
[See PDF for image]
[End of figure]
DD(X) Program:
Technology Maturity:
None of the 12 critical technologies for DD(X) are fully mature. The
Navy does not anticipate any of these technologies reaching maturity
prior to entering system development. At the time of the first ship
production decision, the Navy expects to have only two critical
technologies sufficiently tested to demonstrate maturity. Only two
backup technologies exist, one for the integrated power system and one
for the hull form. While the backup technology for the integrated power
system is mature, the alternate hull form remains in development. If
other critical technologies do not mature as planned, system redesign
would occur.
The DD(X) Program Office is managing risk in part by constructing and
testing engineering development models for each of the 12 critical
technologies. The program's acquisition strategy scheduled these models
to be fully built and tested concurrent with system design and
completed before authorizing construction of the first ship. Current
testing schedules call for the integrated power system, dual band radar
suite, total ship computing environment, and peripheral vertical
launching system to continue development beyond lead ship production
decision.
A second element in the risk reduction strategy is "design budgeting."
According to the program manager, this approach consists of designing
the requirements for technologies with a margin for growth. The DD(X)
program allows for a 10 percent margin to account for necessary
increases in size, weight, or manpower discovered through testing of
the engineering development models. If the 10 percent margin is
exceeded, system redesign would occur.
Modifications to ship size and capabilities affected the progress of
the technology maturation process. In June 2003, the weight of the ship
was reduced, prompting redesign of the advanced gun system and hull
form engineering development models. Multiple reevaluations of radar
characteristics contributed to a delay in the development of the dual
band radar engineering development model.
Other Program Issues:
The DD(X) acquisition strategy focuses on developing and maturing
technologies that could be leveraged across multiple ship classes. If
DD(X) critical technologies do not reach maturity or are delayed, risks
will increase for other programs in development. For example, the delay
associated with the DD(X) dual band radar suite has already affected
the CVN 77 Nimitz class aircraft carrier program. As a result, the
aircraft carrier was forced to use a legacy radar system, leading to
costly redesign and rework.
Program Office Comments:
In commenting on a draft of this analysis, the program office stated
that the ability of DD(X) to deliver revolutionary capabilities with
reduced crew necessitates some element of development and production
risk. Program officials expect that the spiral development approach
adopted in 2001, combined with robust testing of the engineering
development models, will mitigate that risk. Officials indicated that,
since the 2002 contract award, the only significant schedule change was
due to dual band radar changes.
The program office also stated that the time required to design and
build a ship makes the process unique from other weapon systems. DOD
policy states that ship technologies must be mature in time for
installation, and the program office stated that all DD(X) engineering
development models will meet this requirement. At design review, the
program expects that most engineering development models will be
nearing maturity, and that design budgeting will enable incorporation
of changes.
GAO Comments:
The program will be integrating technologies into a ship-level system
design at the same time that it is maturing individual technologies.
Should any of these innovative technologies encounter challenges that
cannot be accommodated by design budgeting, redesign of other
technologies and of the integrated system may be needed. Redesign would
likely result in additional costs and schedule delays as well as affect
the planned installation schedule.
[End of section]
E-10A Multi-Sensor Command and Control Aircraft (E-10A):
The Air Force's E-10A aircraft (formerly known as the MC2A) is planned
to provide the next generation of airborne surface surveillance
capability and focused air surveillance for cruise missile defense. It
will consist of a modified, commercial Boeing 767 airframe, an active
electronic scanned array radar, and a battle management, command and
control computer mission subsystem. Development of the radar and
funding of the first airframe have begun. We assessed only the radar.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman (prime)/Boeing/Raytheon:
Program office: Hanscom Air Force Base, Mass.
Funding to complete through 2009:
R&D: $1,907.9 million:
Procurement: $1,311.2 million:
Total funding: $3,219.1 million:
Procurement quantity: 2:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The latest costs reflect all costs from the program's inception through
fiscal year 2009, and are for the entire E-10A system.
Only the radar subsystem of the E-10A aircraft has entered system
development. Six of the radar's nine critical technologies are fully
mature. The remaining three are nearing full maturity, but they are not
expected to reach full maturity until the first E-10A flight in 2009.
The entire E-10A weapon system is scheduled to enter system development
in July 2005. At that time, the program plans to integrate the radar
with the airframe and the battle management, command and control
computer mission subsystem. The Air Force has identified ongoing
changes to requirements and software development as high risks. The
program is projected to provide initial operational capability by 2013,
1 year later than required, due to a fiscal year 2003 congressional
funding reduction.
[See PDF for image]
[End of figure]
E-10A Program:
Technology Maturity:
At the start of the radar's product development in December 2003, six
of the nine critical technologies were mature and had been demonstrated
in an operational environment. The remaining three technologies are
nearing full maturity, but they are not expected to reach full maturity
until the first E-10A flight in 2009.
Design Maturity:
We did not assess the design maturity of the E-10A radar as the number
of releasable drawings is not yet available.
Other Program Issues:
The development of the entire E-10A platform includes the radar, the
Boeing 767 airframe, and the battle management, command and control
computer mission subsystem and is scheduled to begin in July 2005. At
that time, the computer mission subsystem must achieve software and
hardware maturity to demonstrate the machine-to-machine communications
capability needed to operate with legacy command and control systems.
The radar and antenna need to be incorporated into the Boeing 767, as
do other capabilities, such as adding air-refueling, hardening the
airframe hull against electro-magnetic interference, strengthening the
cabin floor, and increasing the onboard electric power generation.
Hosting the radar on the Boeing 767 involves incorporating an open
systems architecture and interfaces that have yet to be designed.
The program office identified a number of high risks in the program.
For example, design changes to the platform may be needed to address
weight and drag issues, which can affect range and time on station.
Ongoing reviews of the operational requirements, and changes to the
requirements, may also affect system function and design. In addition,
software development is considered a high risk because of the large
number of lines of code, range of applications needed, and changing
requirements.
The Office of the Secretary of Defense recently directed that the Air
Force delay the start of development for the E-10A from July 2004 to
July 2005. This was done to better align program reviews with the
delivery of the test bed aircraft in December 2005 and to provide
sufficient time to complete a study on ground-moving target indicator
capability.
The program office implemented a spiral development approach to
incrementally deliver E-10A capability. Program officials stated that
the E-10A will reach its initial capability in 2013, 1 year later than
the operational need date of 2012, due to a $343 million congressional
cut in fiscal year 2003 program funding.
Program Office Comments:
In commenting on a draft of this assessment, program officials stated
that the E-10A program is on track to provide the initial capability of
the next-generation airborne surface surveillance, and is focused on
surveillance for cruise missile defense, to the warfighter by 2013 in
accordance with the current program schedule and funding.
[End of section]
E-2 Advanced Hawkeye (E-2 AHE):
The Navy's E-2 AHE is an all-weather, twin engine, carrier-based
aircraft designed to extend early warning surveillance capabilities. It
is the next in a series of upgrades the Navy has acquired for the E-2C
Hawkeye platform since its first flight in 1971. The E-2 AHE is
designed to improve battle space target detection and situational
awareness, especially in littoral areas; support Theater Air and
Missile Defense operations; and improve operational availability.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Corp.
Program office: Patuxent River, Md.
Funding needed to complete:
R&D: $3,123.3 million:
Procurement: $9,483.2 million:
Total funding: $12,606.5 million:
Procurement quantity: 69:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The E-2 AHE program entered system development in June 2003 without
demonstrating that its four critical technologies had reached full
maturity. Program officials do not expect to achieve maturity on those
critical technologies until at least a third of the way through product
development. The program office plans to have the majority of drawings
completed by the time of design review in November 2005. However, until
the technologies are mature, the potential for design changes remains.
[See PDF for image]
[End of figure]
E-2 AHE Program:
Technology Maturity:
None of the E-2 AHE's four critical technologies are fully mature. The
four critical technologies are the rotodome antenna, the Silicon
Carbide-based transistor for the Power Amplifier Module to support E-2
UHF radio operations, the Multi-channel Rotary Coupler for the antenna,
and the Space Time Adaptive Processing algorithms and associated
processor. The program expects to have these technologies matured after
critical design review but before production, which is scheduled to
start in March 2009.
More mature backup technologies exist for three of those technologies
(the rotodome antenna, the Silicon Carbide-based transistor, and the
Multi-channel Rotary Coupler) and are currently being flown on a larger
test platform. However, use of the backup technologies would result in
degraded system performance or reduced ability to accommodate future
system growth due to size and weight constraints. While there is no
backup for the fourth technology (Space Time Adaptive Processing
algorithms and associated processor), the program office is confident
that the technology will operate well on the test aircraft in 2005.
Design Maturity:
While none of the engineering drawings are complete, program officials
project that they will have 81 percent completed by the time of
critical design review in November 2005 and that 100 percent will be
completed by the time of the production decision in March 2009.
However, the technology maturation process may lead to more design
changes.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the AHE program entered system development and demonstration after
6 years of research development and 18 months of presystem development
work. Program officials also stated that in preparation for entry into
system development a technology readiness assessment was performed
using industry, academic, and government experts and that the results
of that assessment were approved in accordance with DOD's acquisition
guidance. The program office noted that the maturity of the
technologies examined in that assessment was primarily based on
demonstrations conducted in 1997 and 1999 and did not include recent
accomplishments, including AHE test-bed flights conducted through the
summer of 2003.
The next AHE technology readiness assessment is to be performed prior
to the production decision for the system in fiscal year 2008, and the
program office anticipates that the critical technologies will be
mature at that time. In addition, program officials noted that a mature
risk process, with mitigation plans, exists for the entire AHE program,
including critical technologies, which focuses on risks associated to
operational requirements.
[End of section]
EA-18G Growler (EA-18G):
The Navy's EA-18G is an electronic attack aircraft designed to jam
enemy radar and communications and conduct electronic warfare as part
of a battle group. The program was approved as a replacement for the
EA-6B aircraft, and will integrate its electronic warfare technology
into the F/A-18F platform. Because of the heavy use of the aging EA-6B
aircraft, a large number are being retired due to wear. To prevent a
gap in electronic war-fighting capabilities, DOD intends to begin
fielding the EA-18G in 2009.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Patuxent River, Md.
Funding needed to complete:
R&D: $1,632.2 million:
Procurement: $6,030.5 million:
Total funding: $7,662.7 million:
Procurement quantity: 90:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The EA-18G entered system development without demonstrating that its
five critical technologies had reached full maturity. Three
technologies were very close to maturity and two technologies require
substantial adjustments. In addition to the mechanical challenges of
integration, the program also faces risks with software integration.
The EA-18G will rely on technological upgrades developed for the EA-6B,
which could increase program risk.
[See PDF for image]
[End of figure]
EA-18G Program:
Technology Maturity:
None of EA-18G's five critical technologies are fully mature. While
they are similar to the mature technologies on the EA-6B and the F/A-
18F, integrating the technologies into the EA-18G requires significant
modification. Three critical technologies, the ALQ-99 pods, the F/A-18F
platform, and the tactical terminal system, are approaching full
maturity. The remaining two technologies, the receiver system and the
communications countermeasures set, are not mature.
Both of the less mature technologies, the receiver system and the
communications countermeasures set, require substantial modification
to operate on the EA-18G. The receiver system will be similar to the
system on the EA-6B, with adjustments to allow it to fit onboard the F/
A-18F platform. Several of the receiver's components, such as the
antenna preselectors, will also need to be upgraded, mainly because
some have become obsolete. The communications countermeasures set on
the EA-6B is no longer in production, and a contractor will be selected
to develop a new set for the EA-18G. While the new set will be based on
existing technology, there is additional risk to the program until the
new set is produced and demonstrated to work in the EA-18G.
The electronic warfare equipment on the EA-18G will be subject to a
more severe operating environment than on the EA-6B. Advanced
technologies will be needed to counter the higher levels of vibration.
Other Program Issues:
The EA-18G program plans to build one-third of its aircraft during low-
rate initial production due to the need to begin replacing retiring EA-
6Bs by 2009. Any problems that arise during production could result in
costly modifications to the already produced aircraft.
Additionally, the increased weight and vibration caused by the
electronic warfare equipment added to the F/A-18F platform may limit
the life span of the aircraft. Although the program office asserts that
the design will meet life span requirements, it plans to conduct
additional testing and design work to further extend the life span of
the aircraft.
Program Office Comments:
The EA-18G weapon system integrates proven EA-6B Airborne Electronic
Attack (AEA) systems onto the combat proven F/A-18F platform. Due to
the maturity of the systems, the EA-18G program risk is significantly
less than a new weapon system development. To date, the program has not
identified any major technical inabilities to achieving the current
design approach within cost and schedule constraints. Program officials
believe that all five critical items are fully mature, including the
ALQ-99 pods that have been in existence for 30 years, and the F/A-18F
platform and tactical terminal, which are both in production for the
Navy.
GAO Comments:
While the ALQ-99 pods have existed for 30 years, they are being
physically modified to be compatible with the F/A-18F pylons and will
have a pod interface unit added to them for communications with the F/
A-18F platform. The F/A-18F platform is being modified to support the
installation of the AEA suite and to increase auxiliary memory. The
tactical terminal will be modified to fit inside the F/A-18F platform
and will have a new antenna. Because of these changes to form and fit,
these systems, while approaching full maturity, are not yet fully
mature.
[End of section]
Evolved Expendable Launch Vehicle --Atlas V, Delta IV (EELV):
The Air Force's EELV program is an industry partnership to acquire
commercial satellite launch services from two competitive families of
launch vehicles--Atlas V and Delta IV. The program's goal is to meet
the government's launch requirements while reducing the life-cycle cost
of space launches by at least 25 percent over existing systems.
Different types of lift vehicles may be used, depending on the
particular mission. We assessed both the Atlas V and the Delta IV.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin & Boeing Launch Services:
Program office: El Segundo, Calif. Funding needed to complete:
R&D: $45.3 million:
Procurement: $15,854.7 million:
Total funding: $15,899.9 million:
Procurement quantity: 173:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Implicit in the government's decision to purchase launch services, is
the assumption that the Delta IV and Atlas V launch vehicles are
capable of carrying satellite payloads. The program office does not
believe it needs formal information on EELV's technology, design, and
production maturity because it is buying the service. It does have
access to this data, however. The core launch vehicles are mature, and
since August 2002, there have been six successful launches--two
government and four commercial. However, the heavy lift vehicle (HLV)
has yet to complete production and fly a demonstration mission.
[See PDF for image]
[End of figure]
EELV Program:
Technology Maturity:
We could not assess the technology maturity of the EELV because the Air
Force has not formally contracted for information on the technology
maturity of the EELV launchers from its contractors. Program officials
state that they ensure that all government missions are on track for
their currently scheduled launch dates through daily insight and
interaction in contractors' development, engineering, manufacturing,
and operations processes.
Design Maturity:
We could not assess the design maturity of the EELV because the Air
Force was not able to provide information needed to conduct this
assessment.
Production Maturity:
We could not assess the production maturity of the EELV because the Air
Force was not able to provide information needed to conduct this
assessment.
Other Program Issues:
Initial plans for the EELV program projected a much more robust
commercial launch market. However, the decline in the commercial launch
market since the late 1990s significantly reduced the anticipated
number of Atlas V and Delta IV launches, making the government the
primary customer for both launch vehicles. This reduction, in turn,
caused anticipated prices for government launch services to increase
significantly. According to the Air Force, EELV production rates vary
and depend on the overall condition of the launch market. Contractors
do not begin producing a launch vehicle until they receive an order for
a launch service--usually about 2 years before launch.
The EELV program has recently experienced schedule and program cost
changes. The program milestone schedule has slipped more than 6 months
for the HLV demonstration mission and first operation flights.
According to the Air Force, the delay occurred because of other launch
priorities, slips in launch dates of the first three Delta IV missions,
and modifications to the HLV launch pad.
A requirement to maintain two viable launch contractors over the next 5
years and efforts to improve government oversight have contributed to a
$539-million increase in program costs. Other factors that contributed
to the cost increase included an increase in launch price due to
reallocating missions among the EELV contractors, an anticipated award
of four additional missions, increases in satellite weight growth, and
increases in support costs for a West Coast launch pad.
Although the EELV concept of launch vehicle families emphasizes
commonality of hardware and infrastructure, EELV program officials are
currently addressing technical risks. Both Delta IV and Atlas V use
versions of the RL-10 upper stage engine, meaning an engine flaw could
ground both vehicles. Until production of the Russian made RD-180
propulsion technology starts in the United States, the Atlas will
continue to rely on this engine.
Program Office Comments:
In commenting on a draft of this assessment, the program office
generally concurred with our assessment. It acknowledged that it did
not contract for technology, design, or production maturity
deliverables, but daily government insight, interaction, and access to
contractor data ensure readiness. Also, cost and schedule changes
primarily resulted from a downturn in the commercial market and the
addition of funding to maintain two viable launch competitors. The
anticipated number of launches decreased significantly, increasing
prices for government launches. The Delta IV HLV demonstration slipped;
however, officials said they are ready to provide required launch
services.
[End of section]
Expeditionary Fighting Vehicle (EFV):
The Marine Corps' EFV (formerly called the Advanced Amphibious Assault
Vehicle) is designed to transport troops from ships off shore to their
inland destinations at higher speeds and from farther distances than
the existing AAV-7. It is designed to be more mobile, lethal, reliable,
and effective in all weather conditions. EFV will have two variants--a
troop carrier for 17 Marines and a command vehicle to manage combat
operations in the field. We assessed both variants.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: General Dynamics:
Program office: Woodbridge, Va.
Funding needed to complete:
R&D: $882.9 million:
Procurement: $7,378.4 million:
Total funding: $8,306.2 million:
Procurement quantity: 1,012:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
EFV demonstrated most technology and design knowledge at critical
junctures in the program. At the start of the program, all but one of
the critical technologies were mature. The design was close to meeting
best practice standards at the design review, signifying the design was
stable. Early development of fully functional prototypes and other
design practices facilitated design stability. However, the remaining
technology has not matured as expected, which may lead to some
redesign. Also, the demonstration of production maturity remains a
concern because the program does not plan to use statistical process
controls to achieve quality. The EFV production decision is not
scheduled until September 2005. Remaining efforts include
developmental, operational, and reliability testing.
[See PDF for image]
[End of figure]
EFV Program:
Technology Maturity:
Four of EFV's five critical technologies are mature. The remaining
technology, the moving map navigation technology, is not expected to
reach maturity until the summer of 2004. This is a 1-year delay from
what was reported last year on the EFV program. The moving map
navigation is to provide situational awareness. As of November 2003 the
technology had been demonstrated in a high fidelity laboratory
environment on representative EFV system hardware. By next year, it
should be demonstrated in an operational environment.
Design Maturity:
The EFV has released nearly all of its drawings for the development
prototype currently being manufactured. At the time of the critical
design review in 2001, 77 percent of the drawings had been released,
signifying the design was stable. After building the first seven
development prototypes, the program identified changes that would
affect about 10 percent of the drawings. Program officials said the
changes to the drawings are mostly to attain better manufacturing
efficiencies in producing the EFV and will be incorporated into the
last two of the nine development prototypes. Program officials stated
they will have additional design reviews prior to starting low-rate and
full-rate production and that additional changes may result from
ongoing development testing. Finally, until the moving map technology
has been demonstrated and incorporated into the EFV design, the
potential exists for additional design changes.
Within the last year, the program delayed the start of developmental
testing by 3 months to fix defects in test vehicles. Based on lessons
learned earlier in the program, the contractor put the initial EFV test
prototypes through a short shakedown period before sending them to the
developmental test location. The shakedown was intended to identify
problems that could affect EFV availability during testing to avoid
unnecessary increases in the testing costs. Also, reliability testing
remains to be done.
Production Maturity:
The program expects a low-rate production decision in September 2005,
but does not require the contractor to use statistical process controls
to ensure its critical processes are producing high quality and
reliable products. Instead, the program has directed the contractor to
develop a production readiness plan to ensure its critical processes
are in control. The plan consists primarily of collecting
postproduction quality data on items produced.
Because the final EFV production facility is not ready, the contractor
is using the planned manufacturing processes to build prototypes at the
development facility. This will provide verification of these
manufacturing processes. However, when production moves to the new
facility, processes will need to be validated again to ensure they work
as expected. Furthermore, skills gained by staff working in the
development facility may be lost if different people are hired at the
production facility.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the acquisition plan is based on four design, build, and test
iterations of EFV to mature the design and to prove its readiness for
production and operation. The second iteration is currently underway as
part of the system development and demonstration (SDD) phase of the
program. The improvements from the first-generation prototypes will be
demonstrated during extensive testing of the second generation of SDD
vehicles. This testing began during the third quarter of fiscal year
2003 and will continue through a comprehensive operational assessment
in fiscal year 2005.
General Dynamics is working toward certification to the International
Organization for Standardization (ISO) 9001:2000 quality management
standard. Various quality assurance methods are being implemented to
meet the ISO standard. Statistical process control is one of the
approaches to be used where applicable during low-rate and full-rate
production.
[End of section]
Extended Range Guided Munition (ERGM):
The Navy's ERGM is a rocket-assisted projectile that is fired from a
gun aboard ships. It can be guided to targets on land at ranges of
between about 15 and 50 nautical miles to provide fire support for
ground troops. ERGM is expected to offer increased range and accuracy
compared to the Navy's current gun range of 13 nautical miles. ERGM
requires modifications to existing 5-inch guns, a new munitions-
handling system (magazine), and a new fire control system. We assessed
the projectile only.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon:
Program office: Washington, D.C.
Funding needed to complete:
R&D: $50.8 million:
Procurement: $156.8 million:
Total funding: $207.6 million:
Procurement quantity: 3,055:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The ERGM program began development with very few of its critical
technologies mature, and while progress has been made, program
officials do not expect to achieve maturity on all critical
technologies until at least February 2004. No production representative
engineering drawings were released to manufacturing by the design
review; however, over half of these drawings have since been released.
The program office expects to have a complete and updated drawing
package by October 2004. Finally, due to several test failures, the
program did not meet a Navy deadline that required successful
completion of two land-based flight tests by November 2003. The Navy is
conducting an independent assessment of the program's readiness to
proceed with further flight-testing. The Navy has also issued a
solicitation for alternative precision-guided munition concepts that
could offer cost savings.
[See PDF for image]
[End of figure]
ERGM Program:
Technology Maturity:
Fifteen of ERGM's 20 critical technologies have demonstrated
technological maturity. The remaining 5 technologies are approaching
maturity, and program officials expect that all 20 critical
technologies will be demonstrated in an operational environment by
February 2004, almost 8 years after the start of system development.
Four of these five technologies are related to the unitary warhead
design change, which was made in January 2002. In our May 2003
assessment, the program office projected that these technologies would
be mature by the end of 2003. However, a series of flight test failures
prevented the program from demonstrating these technologies as
projected.
Design Maturity:
The program released approximately 54 percent of drawings, and the
program office plans to have all production representative drawings
complete by October 2004, over 1 year after the design review. This
updated and mature drawing package will reflect knowledge gained from
18 flight tests and qualification tests and will be used to build
production representative operational test rounds.
At the May 2003 design review, none of ERGM's 128 production
representative engineering drawings had been released. Instead, the
program conducted this review with less mature drawings and used them
to validate the design of the development test rounds.
According to program officials, seemingly minor design and quality
assurance problems have been responsible for the recent test failures.
For example, one of the causes of a June 2003 test failure was a design
flaw in the rocket motor's igniter, a .012-inch gap between two parts,
which caused it to fall out after gun launch. This problem was
addressed, and the igniter functioned properly during three later
flight tests. Another critical test was delayed when excessive paint on
the round made it slightly too large to fit in the gun barrel. As a
result of these test issues and others, the program office failed to
meet a Navy deadline that required the successful completion of two
land-based flight tests by November 2003. In February 2004, a
component-level flight test of the rocket motor was also unsuccessful.
As a result, ERGM guided flight tests, scheduled for February 2004,
have been postponed. An independent failure investigation, which will
determine the program's readiness to proceed with further guided flight
tests, has been initiated.
Production Maturity:
Since the ERGM program will not begin to build production
representative rounds until October 2004, Raytheon has not started to
collect information on production process maturity. The manufacturing
plan states the contractor will identify key product characteristics
and then determine how to implement statistical process control.
However, it is not clear when this will occur.
Other Program Issues:
Future program costs are not accurately reflected in the latest program
cost estimate because the estimate is based on a much lower production
quantity than is contained in current program documents and the Navy
has yet to establish a firm ERGM inventory requirement. A new program
baseline with revised cost and quantity information will not be
available until at least March 2004.
In October 2003, the Navy issued a solicitation for alternative
precision-guided munition concepts that could be a complement or
competitor to ERGM. In particular, the Navy is concerned about the unit
cost of the ERGM round and is looking to develop alternatives that
could offer cost savings. The Navy plans to spend $35 million in fiscal
years 2004 and 2005 to pursue a technology demonstration of other
extended range munition concepts by September 2005.
Program Office Comments:
In commenting on a draft of this assessment, the program office noted
that it is investigating a number of options for restructuring the ERGM
program to address technical, budget, and schedule issues. The program
office also provided separate technical comments, which were
incorporated as appropriate.
[End of section]
Excalibur Precision Guided Extended Range Artillery Projectile:
The Army's Excalibur is a family of global positioning system-based,
fire-and-forget, 155-mm cannon artillery precision munitions. It is
intended to improve the accuracy and range of cannon artillery. Also,
the Excalibur's near vertical angle of fall is intended to reduce the
collateral damage area around the intended target, making it more
effective in urban environments than the current artillery projectiles.
It would allow the Future Combat Systems' non-line-of-sight cannon to
fire from farther away and defeat threats more quickly.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon:
Program office: Picatinny Arsenal, N.J. Funding needed to complete:
R&D: $425.4 million:
Procurement: $3,407.4 million:
Total funding: $3,832.8 million:
Procurement quantity: 76,408:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The Excalibur program's critical technologies are not fully mature,
even though product development began over 6 years ago. Currently,
about one-half of the drawings are at a level that could be released to
manufacturing. Program officials expect to have technological maturity
and design stability by the design review in 2005. The program has
encountered a number of challenges since development began, including a
decrease in planned quantities, a relocation of the contractor's plant,
early limited funding, technical problems, and changes in program
requirements. This past year, it completed a major restructuring by
merging with the Trajectory Correctable Munition program.
[See PDF for image]
[End of figure]
Excalibur Program:
Technology Maturity:
None of the Excalibur's three critical technologies--the guidance
control system, the airframe, or the warhead--are fully mature.
According to program officials, all three have been demonstrated in a
relevant environment and are expected to reach full maturity before the
design review in June 2005. The warhead was not considered a critical
technology in 1997 because the Excalibur design called for a warhead
that was under production for other munitions. At the Army's direction,
the program has undertaken development of a different warhead that is
undergoing testing.
Design Maturity:
Currently, 55 percent of the Excalibur's engineering drawings are
releasable to manufacturing. The program office plans to have all
drawings complete by the design review in June 2005. The program
recently successfully conducted a preliminary design review to verify
that the Excalibur's initial design has the potential to satisfy system
requirements.
Other Program Issues:
The program has gone through many changes since the beginning of
product development in May 1997. It was almost immediately restructured
due to limited funding, and it was restructured again in 2001. In the
past year, the program was again restructured and merged with a joint
Swedish and U.S. program known as the Trajectory Correctable Munition.
This merger has helped the Excalibur deal with design challenges,
including issues related to its original folding fin design. In May
2002, due to the cancellation of the Crusader, the Army directed the
restructure of the program to include the Future Combat Systems' non-
line-of-sight cannon. In December 2002, the Secretary of Defense
approved the early fielding plan, which includes fielding to the Joint
Lightweight 155mm cannon in fiscal year 2006, the non-line-of-sight
cannon in fiscal year 2008, and the enhanced unitary round in fiscal
years 2010-2011.
The net effect of these changes has been to increase the program's
schedule and to substantially decrease planned procurement quantities.
As a result, the program's overall costs and unit costs have
dramatically increased.
Program Office Comments:
The Excalibur program office provided technical comments, which we
incorporated as appropriate.
[End of section]
F/A-22 Raptor:
The Air Force's F/A-22, originally planned to be an air superiority
fighter, will also have air-to-ground attack capability. It is being
designed with advanced features, such as stealth characteristics, to
make it less detectable to adversaries and capable of high speeds for
long ranges. It has integrated aviation electronics (avionics) designed
to greatly improve pilots' awareness of the situation surrounding them.
It is designed to replace the Air Force's F-15 aircraft.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $3,642.2 million:
Procurement: $29,004.1 million:
Total funding: $33,081.2 million:
Procurement quantity: 225:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The F/A-22 Raptor entered production without assurance that production
processes were in control. The Air Force expects to have about 27
percent of the aircraft on contract prior to the full-rate decision in
December 2004, yet quality issues remain. For example, the F/A-22 has
not achieved important reliability goals, and some components, like the
canopy, are not lasting as long as expected. Technology and design
matured late in the program, which contributed to numerous problems.
Avionics have experienced major development problems, which caused
large cost increases and testing delays. The potential for further cost
increases and schedule delays exists as a significant amount of testing
remains. Additionally, production costs could increase if the assumed
$25 billion in offsets from cost reduction plans is not realized.
[See PDF for image]
[End of figure]
F/A-22 Raptor Program:
Technology Maturity:
The three critical F/A-22 technologies (supercruise, stealth, and
integrated avionics) appear to be mature. However, two of these
technologies, integrated avionics and stealth, did not mature until
several years after the start of the development program. Integrated
avionics has been a source of major problems, delaying developmental
testing and the start of initial operational testing. Since 1997, the
development costs of avionics have increased by over $980 million. The
avionics is still considered unstable, and initial operational testing
has not started. Until testing demonstrates the avionics work as
intended, the program is subject to additional delays and cost
increases.
Design Maturity:
The F/A-22 design is essentially complete, but it matured slowly,
taking over 3 years beyond the critical design review to meet best
practice standards. The late drawing release contributed to parts
shortages, work performed out of sequence, delayed flight testing, and
increased costs. Design changes have resulted from flight and
structural tests. For example, problems with excessive movement of the
vertical tails and overheating problems in the fuselage and engine bay
required design modifications. The Air Force is still conducting
development testing and has not started operational testing. Until
testing is completed, now scheduled for September 2004, the possibility
of additional design changes remains.
Production Maturity:
The program office stopped collecting process control information in
November 2000. The contractor estimated that nearly half of the key
processes had reached a marginal level of control, but not up to best
practice standards. The Air Force has 52 production aircraft on
contract with 22 more scheduled for contract before full-rate
production approval, expected in December 2004. The contractor
continues to revise its manufacturing process to gain greater
efficiency and quality. However, the Air Force has not demonstrated the
F/A-22 can achieve its reliability goal of 3 hours mean time between
maintenance. It does not expect to achieve this goal until 2008 when
most of the aircraft will have been bought. Best practices strive to
achieve reliability requirements before entering production. As of mid-
January 2004, the Air Force had only demonstrated about 18 percent of
the reliability required at maturity.
Other Program Issues:
The Air Force is counting on over $25 billion in future cost reduction
plans to offset estimated cost growth and enable the program to meet
the latest production cost estimate. If these cost reduction plans are
not achieved, production costs could increase.
The Integrated Maintenance Information System (IMIS), a paperless
computerized maintenance system, is used by the Air Force to maintain
the F/A-22. The system collects and analyses problem data and develops
a maintenance solution. The system has not functioned properly, causing
unnecessary maintenance actions. This has affected the Air Force's
ability to fly the test aircraft on schedule. The Air Force expects new
software, planned to be released in February 2004, to address many of
the errors generated by IMIS.
Program Office Comments:
In commenting on a draft of this assessment, the program office
recommended technical changes. We incorporated these comments where
appropriate. The program office also pointed out that while only 18
percent of the reliability requirement had been demonstrated to date,
corrections had been identified that should increase the value to 28
percent, once they are implemented. The program office also pointed out
it has an interim reliability goal of 1.95 hours mean time between
maintenance for the end of development.
[End of section]
Future Combat Systems (FCS):
The Army's FCS is a family of systems composed of advanced, networked
combat and sustainment systems, unmanned ground and air vehicles, and
unattended sensors and munitions. Within a system-of-systems
architecture, FCS will eventually feature 18 major systems and other
enabling systems. Increment one currently includes 14 systems, and it
will rely on an overarching network for information superiority and
survivability. Additional systems and new technologies will be
introduced as they mature and funding is available.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Warren, Mich.
Funding needed to complete:
R&D: $18,214.6 million:
Procurement: $59,987.8 million:
Total funding: $78,811.4 million:
Procurement quantity: 15:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Quantities refer to complete brigade-sized Units of Action. Each unit
contains many FCS systems or platforms.
The FCS program began system development with only 4 of its 52 critical
technologies mature and only 2 additional technologies are expected to
reach full maturity by the time of the design review in July 2006, more
than half way through product development. The program expects product
maturation to continue throughout system development and full
integration to be demonstrated at the time of operational testing.
Maturing technologies concurrently with product development increases
the risk of cost growth and schedule delays. Since FCS will dominate
Army investment accounts over the next decade, cost growth and schedule
delays could affect all Army acquisitions. While system development
began in May 2003, the program will be reviewed in November 2004 to
determine if the Army should continue the development phase and to
authorize prototypes.
[See PDF for image]
[End of figure]
FCS Program:
Technology Maturity:
Only 4 of the FCS program's 52 critical technologies are mature and
only 2 additional technologies are expected to be mature at the time of
the design review in July 2006. By maturing technology while developing
the FCS products, the Army has increased the risk of cost growth and
schedule delays.
Design Maturity:
The FCS program projects that about 80 percent of the estimated 42,750
drawings will be released to manufacturing by the time of the design
review for increment one in July 2006. However, DOD may authorize
developmental prototype production as early as November 2004, about 20
months prior to the design review and before these production drawings
are available. These developmental prototypes, which are not intended
to be production representative, will be used, along with simulations,
in tests conducted before the 2008 initial production decision, to
generate additional acquisition knowledge needed to help mitigate cost
and schedule risks.
The FCS program represents a major integration effort, both at the
weapon systems platform level and at the networked systems level. The
total program involves over 33 million lines of software code and 14
weapon systems or platforms networked together. Given the size of the
program, it will be a challenge to demonstrate the maturity of the
entire system of systems.
Other Program Issues:
The concept of an FCS equipped brigade-sized combat unit, known as a
Unit of Action, represents a major departure in the way the Army has
conducted combat operations and is a major part of the Army's
transformation efforts. To successfully develop FCS, the Army faces a
number of technological and programmatic challenges. One challenge is
to equip Units of Action with a common family of networked vehicles and
other systems. These vehicles and systems are expected to be a fraction
of the weight of existing heavy fighting vehicles to improve
transportability. For example, vehicles must be light and small enough
to be airlifted by a C-130 transport, which could require lighter armor
on each vehicle than existing vehicles.
Another challenge involves developing multiple systems and a network in
less time than DOD typically needs to develop a single advanced system.
The schedule for developing FCS is challenging and currently focuses on
obtaining an initial operating capability in 2010. Even though the
weapon systems have yet to be clearly defined, DOD may authorize
prototype builds for testing as early as November 2004 to generate
additional information needed for the 2008 production decision.
Combined with the projected state of design maturity, this could result
in the prototypes being significantly different than production units.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the Army guideline for technology maturity is less stringent than
that recommended by GAO and that DOD is monitoring the Army's efforts
to mature critical technologies to that level. The DOD approved FCS
acquisition strategy indicates that critical technologies should be at
the maturity level required by the Army at the time of the program's
preliminary design review in April 2005 and at the maturity level
recommended by us prior to the FCS production decision in 2008.
[End of section]
Global Hawk Unmanned Aerial Vehicle:
The Air Force's Global Hawk is a high altitude, long endurance unmanned
aerial vehicle with integrated sensors and ground stations providing
intelligence, surveillance, and reconnaissance capabilities. Following
a successful technology demonstration, Global Hawk entered system
development and limited production in March 2001. Identified as a
transformational system, the program was restructured in 2002 to
implement an evolutionary acquisition strategy intending to more
quickly develop and field a larger and more capable air vehicle.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Integrated Systems:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $1,634.6 million:
Procurement: $2,717.5 million:
Total funding: $4,469.6 million:
Procurement quantity: 45:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Due to program restructuring to more quickly field the larger and more
capable system, key product knowledge on Global Hawk is now less than
it was in March 2001. Officials originally planned to first produce
systems very similar to technology demonstrators and then slowly
develop and acquire more advanced systems. Technology and design
maturity approached best practice standards for this plan. However,
program restructuring in 2002 accelerated deliveries, overlapped
development and production schedules, and added the new, larger air
vehicle with enhanced sensors. These actions increased development and
program unit costs. Technology and design knowledge for the
restructured plan are below best practices, but they should be
increased by the full-rate production decision date. Production
maturity is not known; statistical process controls are being planned
but are not yet in place.
[See PDF for image]
[End of figure]
Global Hawk Program:
Technology Maturity:
Four of 14 critical technologies associated with the Global Hawk system
are mature, another 4 technologies are approaching maturity, and 6 are
less mature. Overall, technology maturity is less than it was in March
2001 when the Global Hawk program was approved for product development
and low-rate production. At that time, the plan was to acquire air
vehicles similar to technology demonstrators in operation and whose
maturity levels for its three critical technologies approached best
practice standards.
The restructured program acquires 7 air vehicles similar to the
demonstrators (RQ-4A) and 44 larger and more capable models (RQ-4B).
The RQ-4B air vehicle has not been built or tested, and only 1 of its
11 critical technologies is considered mature. It is to have a 50
percent larger payload capacity and incorporate advanced capabilities
that depend on new sensors and other enhancements in various stages of
development. In particular, three critical technologies to meet user
requirements--two signals intelligence sensors and an improved radar
capability--are not expected to be demonstrated until after a
significant number of RQ-4Bs are already produced. Officials intend to
develop and integrate new technologies in a series of spiral
developments, adding them to the production line as they mature.
Production approval for the air vehicles with the most advanced sensors
is planned for fiscal year 2007.
Design Maturity:
Design maturity for the Global Hawk has not yet been achieved and
varies between the two models. Engineering drawings are complete for
the RQ-4A, the first seven production units. About 60 percent of the
drawings for the RQ-4B have been released to manufacturing. Officials
project that almost 80 percent of the drawings will be complete by the
design review date in March 2004. This approaches the best practices
standard of 90 percent.
The restructured program and the evolutionary acquisition approach
accelerated deliveries and increased concurrency of development and
production activities, resulting in greater risks to cost, schedule,
and performance. Testing of the basic design of the new, larger RQ-4B
will not be completed until 13 are on order and advanced procurement
awarded for seven more. Problems found late in development, while
production activities are taking place, may require more time, money
and effort to fix. Delays or failures in developing, producing, or
testing enhanced sensor capabilities, especially new signals
intelligence and radar components, could severely affect cost and
schedule. Production decisions for the advanced payloads will be made
later as the technologies mature.
Production Maturity:
Statistical process control is not yet in place at the assembly
facility. As a result, Global Hawk entered low-rate production with no
assurance that production processes were in control. Program officials
said that the contractor is in the process of planning and collecting
data to implement control techniques for key manufacturing tasks.
Manufacturing performance is currently monitored by such quality
control measures as manufacturing defects per opportunity and rework
data. The quality data for the second production vehicle shows
improvement over the first vehicle. Contract performance data indicates
that work is slightly behind schedule and over cost.
Program Office Comments:
In commenting on a draft of this assessment, the program office
generally concurred and provided the following statements on
acquisition strategy and risk management. A successful technology
demonstration supported a coordinated development and initial
production start. The evolutionary acquisition strategy was implemented
to deliver an early combat capability followed by time-phased
incremental improvements. Program risk is managed through incremental
production decisions, tailored testing, interim management reviews, and
contract awards of each new capability. The program benefited by
operational experiences gained in the technology demonstration and in
extensive combat missions in the war on terror. These experiences
helped refine operational needs and allowed user-requested improvements
to be incorporated into first deliveries with minimal program impact.
The Global Hawk system transforms military operations providing
persistent, near real-time intelligence to combat commanders.
[End of section]
Ground-Based Midcourse Defense (GMD):
MDA's GMD element is being developed in incremental, capability-based
blocks to defend the United States against limited long-range ballistic
missile attacks. The first block consists of a collection of radars and
an interceptor--a three-stage booster and an exoatmospheric kill
vehicle (EKV)--integrated by a central control system that formulates
battle plans and directs the operation of GMD components. We assessed
the initial capability to be fielded in September 2004, and Block 2004
to be completed by December 2005.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing Company:
Program office: Huntsville, Ala.
Funding to complete through 2009:
R&D: $9,532.8 million:
Procurement: $0.0 million:
Total funding: $9,532.8 million:
Procurement quantity: TBD:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Latest cost includes all costs from the program's inception through
fiscal year 2009. Procurement funding and quantities have yet to be
determined. NA = not applicable:
Three of GMD's 10 critical technologies are mature, and the design
appears stable. Three technologies are expected to be fully mature by
the third quarter of fiscal year 2004. Of the remaining four
technologies, three are expected to reach maturity by December 2005,
but it is not known when the final technology will reach maturity. The
program has released about 89 percent of system engineering drawings,
but until all technologies are demonstrated, the potential for design
change remains. By beginning integration before these technologies have
been demonstrated, MDA has accepted higher risks associated with
potential cost growth, schedule slippage, or decreased performance.
Finally, questions over whether the contractors can produce the
interceptor at planned rates and problems with one of the boosters
raise concerns about the program's ability to field the expected
capability by December 2005.
[See PDF for image]
[End of figure]
GMD Program:
Technology Maturity:
Only 3 of GMD's 10 critical technologies are mature--one of the
boosters; the EKV's infrared seeker; and the fire control software of
the battle management component. MDA expects to demonstrate the
maturity of 3 other technologies--two EKV technologies and the battle
management component--by the third quarter of fiscal year 2004. Three
critical technologies--a second booster, the sea-based X-Band Radar,
and the early warning radar at Beale Air Force Base, California--are
expected to be fully mature by December 2005. It is not clear if the
final technology--the upgraded Cobra Dane radar--will reach maturity by
September 2004.
Although MDA is developing two boosters, only one booster--known as
OSC--will have reached maturity prior to the initial capability in
September 2004. While the OSC booster was tested successfully in August
2003, the other booster--known as BV+--has experienced continual delays
in flight and booster tests, indicating development problems. GMD's
three radar components, needed to detect and track enemy missiles, are
the least mature. Software for the Beale radar is still under
development. Although the planned sea-based X-Band Radar uses existing
technology, it has not been demonstrated in its new environment, a
platform located in the ocean. Finally, it is unclear if the Cobra Dane
Upgrade--GMD's primary radar when first fielded--will reach full
maturity prior to September 2004 because MDA does not plan to
demonstrate its capability in integrated flight tests. The anticipated
launch of foreign test missiles might serve as a test of the radar, but
testing in this manner might not provide all of the needed information,
since MDA will not control the configuration of the target or the
flight environment.
Design Maturity:
The GMD program has released about 89 percent of all engineering
drawings needed to produce an initial capability, indicating design
stability. The ongoing effort to mature critical technologies, however,
may lead to more design changes.
Production Maturity:
We did not assess the production maturity of GMD because process
control data was unavailable. The program plans to deliver five
interceptors to meet the initial capability target in September 2004,
with 15 additional interceptors to be delivered by December 2005,
splitting booster production between two manufacturers. It remains
unclear whether GMD can meet this schedule and program officials admit
that the interceptor production schedule is high risk. The contractors
have not yet proven that they can manufacture the EKV at the planned
rate or that they can accelerate production of the OSC booster quickly
enough to manufacture all five boosters needed for the initial
capability. Finally, due to an explosion at a subcontractor facility
and questions related to its development, the BV+ booster is at risk of
not meeting its production goals for the December 2005 capability.
Other Program Issues:
Approximately $3.4 billion in funding that MDA expects to use to
accomplish activities in fiscal years 2004 and 2005 contribute directly
to the development of Block 2004, but were budgeted as future block
activities. While funding has not been moved between blocks, the actual
estimated cost of Block 2004 will be higher than the amount reflected
in budget estimates. In addition, using contractor cost performance
data, we independently estimate that the contract will overrun its
budget by between $237 million and $467 million at its completion in
2007, with the interceptors accounting for approximately 84 percent of
this overrun.
Program Office Comments:
In commenting on a draft of this assessment, the program office
acknowledged that a portion of the funding budgeted for Block 2006--the
next increment--directly supports Block 2004 efforts. Program officials
expressed concern that our assessment could give the incorrect
impression that Block 2004 has incurred a $3.4 billion cost overrun or
that funding is intentionally being moved to complete Block 2004. The
program office also noted that the prime contractor is reporting no
cost overrun at the completion of the contract. Although the contractor
estimates that the interceptor will have an overrun of approximately
$135 million at its completion, it will be offset by underruns in other
program areas.
[End of section]
Joint Air-to-Surface Standoff Missile (JASSM):
JASSM is a joint Air Force and Navy missile system designed to attack
surface targets outside of the range of area defenses. JASSM will be
delivered by a variety of aircraft including the F-16 C/D, the B-52H,
the F/A-18E/F, the B-2, and the B-1B. The system includes the missile,
software, and software interfaces with the host aircraft and mission
planning system. We assessed all components.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin:
Program office: Fort Walton Beach, Fla.
Funding needed to complete:
R&D: $262.7 million:
Procurement: $2,458.3 million:
Total funding: $2,721.0 million:
Procurement quantity: 4,164:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The JASSM program entered production in December 2001 without ensuring
that production processes were in control. However, program officials
indicated that they have demonstrated the production processes by
sampling statistical data at the subsystem level. The program used
mature technology, and the design was stable at the design review.
Although there were some developmental and operational test failures,
program officials incorporated fixes that subsequent tests demonstrated
to be successful. The contractor has been able to produce at the rates
required for the initial production.
[See PDF for image]
[End of figure]
JASSM Program:
Technology Maturity:
The JASSM program used existing technologies and the level of
technology maturity is high. Although none of the subsystems are based
on new technologies, three critical technologies are new applications
of existing technologies. These three technologies are the global
positioning system anti-spoofing receiver module, the low observable
technology, and the composite materials. These technologies are mature.
Design Maturity:
The contractor has released 100 percent of the drawings to
manufacturing and has completed developmental and operational tests.
The full-rate production decision is scheduled for March 2004, pending
an analysis of these tests. Developmental tests were completed in March
2003. Fourteen developmental flight tests were performed, with 3 tests
failing to meet the test objectives. Program officials stated that they
identified the issues involved and incorporated fixes. The fixes were
successfully tested in later developmental tests. Eleven operational
tests were also performed from June 2002 to September 2003. The Air
Force Operational Test and Evaluation Command evaluated the results of
these tests and rated the JASSM as effective and potentially suitable
and recommended for full-rate production.
Production Maturity:
Program officials do not collect production process control data at the
system level. However, they stated that all production processes had
been demonstrated and that statistical data is collected at the
subsystem level and is sampled as required. Program officials indicated
that the contractor has produced at the rates required for the low-rate
initial production buy of 176 missiles and that it will be able to
produce at the full-rate production level of 250 missiles per year.
Program officials believe that none of the manufacturing processes that
affect critical system characteristics are a problem, although there
are key production processes that have cost implications, such as
bonding for the low observable materials and the painting/coating
application.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the JASSM program development phase concluded during 2003. All
developmental test vehicles were delivered and successfully tested
during this period. This year also included the required deliveries
from the first low-rate initial production of 76 missiles, with the
second low-rate initial production contract of 100 missiles ongoing,
and the third contract for 200 missiles awarded in December 2003.
Additionally, the contractor built 3 more operational test missiles
than planned during this time period. Lastly, the program office
expects to award a contract for an extended range JASSM in early 2004.
[End of section]
Joint Helmet Mounted Cueing System (JHMCS):
JHMCS is a joint Air Force and Navy program, led by the Air Force. The
system is designed to cue radars and weapons at a target based on where
the pilot is looking. This avoids having to line up the aircraft with
the intended target. The system works with the Navy and Air Force AIM-
9X missile on the F-18, F-15, and F-16 aircraft. JHMCS also provides
situational awareness by displaying information about the aircraft and
weapons. Development is jointly funded by the services, and procurement
is funded by the aircraft platforms.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: $0.0 million:
Procurement: $161.2 million:
Total funding: $161.2 million:
Procurement quantity: 924:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Although JHMCS has been in production for 3 years, data has not been
collected on whether its production processes are in control. The
program has experienced design defects and quality control problems.
Operational testing, completed in August 2002, found that while the
system was operationally effective, it was not suitable to be fielded
due to low reliability and maintainability. JHMCS is in its fourth low-
rate initial production effort, and a decision for full-rate production
is anticipated in February 2004. We did not assess the maturity of
critical technologies at development start.
[See PDF for image]
[End of figure]
JHMCS Program:
Technology Maturity:
All six of the JHMCS program's critical technologies are mature and
have been demonstrated in an operational environment using production
representative hardware. We did not assess critical technology maturity
at development start because the program is well into production.
Design Maturity:
The JHMCS design appears complete. Operational testing, completed in
June 2002, found that while the system was operationally effective, it
was not suitable to be fielded due to low reliability and
maintainability caused by design defects and poor quality control. The
recently released Beyond Low Rate Initial Production Report indicates
some improvement in these areas. However, JHMCS is still not compatible
with pilot night vision systems or laser eye protection, a finding that
partly led to the conclusion that the system was operationally not
suitable. Resolving these issues could result in design changes.
Production Maturity:
Production maturity could not be determined because the contractor does
not use statistical process controls to ensure that production
processes are stable. To date, approximately 218 systems have been
delivered to the Air Force and 124 systems to the Navy. The program is
in its fourth low-rate initial production buy and the full-rate
production decision is likely to be made early next year.
Other Program Issues:
The Air Force and the Navy purchased 35 percent of the total quantities
of the system under low-rate initial production, despite reliability
and maintainability issues identified in testing.
The full capability of the JHMCS program will not be available until it
is deployed with the AIM-9X missile. However, almost 85 percent of the
total JHMCS quantities for the F-16 will be under contract before the
AIM-9X missile is fielded on the aircraft. In addition, the Air Force
deferred indefinitely incorporation of JHMCS onto the F-22. Current
plans call for a separate development effort for a helmet mounted
cueing system for the aircraft.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the JHMCS capability is not dependent on AIM-9X deployment, nor is
the JHMCS utility limited to the air-to-air arena. Users are finding
the increased situational awareness, while using JHMCS air-to-ground,
is outstanding. JHMCS is being installed in aircraft at logical, cost-
effective times, principally in planned aircraft modification and
production lines, and is not necessarily linked to other weapons or
avionics upgrades.
JHMCS has basic production maturity. Companies producing the bulk of
system hardware are ISO-9000 certified. Statistical process controls
are used, but data is not reported to the program office.
JHMCS reliability has more than doubled in the last year via system
improvements and increased user proficiency.
A separate program is underway to integrate night vision devices with
JHMCS. An interim solution to laser eye protection has been identified.
Further improvements will require a new laser eye protection program.
[End of section]
Joint Common Missile:
The Joint Common Missile is an air-launched and potentially ground-
launched missile designed to target tanks; light armored vehicles;
missile launchers; command, control, and communications vehicles;
bunkers; and buildings. It will be a joint Army and Navy program with
Marine Corps participation and United Kingdom involvement. It will
provide line-of-sight and beyond line-of-sight capabilities. It can be
employed in a fire-and-forget mode--providing maximum survivability--
or a precision attack mode, providing the greatest accuracy.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon/Boeing/ Lockheed Martin:
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: TBD:
Procurement: TBD:
Total funding: TBD:
Procurement quantity: TBD:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The Joint Common Missile is scheduled to enter system development of
the air-launched version before any of its critical technologies are
fully mature. Program officials currently project that the critical
technologies will reach maturity 3 months after design review, about
half way through product development.
[See PDF for image]
[End of figure]
Joint Common Missile Program:
Technology Maturity:
None of the Joint Common Missile's three critical technologies have
demonstrated full maturity according to best practices. These
technologies include a multi-mode seeker for increased countermeasure
resistance, boost-sustain propulsion for increased standoff range, and
a multi-purpose warhead for increased lethality capability. Program
officials noted that many of the components of these technologies are
in production on other missile systems, but they have not been fully
integrated into a single missile. While backup technologies exist for
each of the critical technologies, substituting any of them would
result in degraded performance or increased costs.
Design Maturity:
Program officials project that full integration of the subsystems into
the Joint Common Missile will occur by June 2005 and that the system
will reach maturity by December 2005, over 1-1/2 years after the start
of system development and demonstration.
Program officials believe that the program's modular design will reduce
life-cycle costs, including demilitarization, and will enable
continuous technology insertion to ensure improvements against
advancing threats.
Other Program Issues:
Current cost estimates are likely to increase because the program has
yet to incorporate the full Army and Navy quantities. The Army's
previous estimate of 54,290 was based on the AH-64D Apache and the
Comanche. The current estimate does not include the Comanche. The
Navy's previous estimate of 23,000 increased because of additional
requirements.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that during the fourth quarter of 2003 the Army recommended, and DOD
approved, the restructure of the Joint Common Missile system
development and demonstration program from a 36-month spiral
development to a 48-month two-phase program to reduce risk. A risk
reduction phase of 12 to 14 months will allow full integration of the
subsystems in a missile prior to the initiation of system
demonstration. Program officials stated that they demonstrated the
technology maturity required by DOD acquisition system policy via tower
tests, captive flight tests and the development and submittal to the
government for verification of an integrated flight simulation using
the tactical seeker software. Joint Common Missile development will be
demonstrated in an operational environment in December 2005. A system
integration and demonstration phase of 36 months will lead to a low-
rate initial production decision in April 2008. Beginning in fiscal
year 2009 and running through fiscal year 2012, additional
capabilities, such as man-in-the-loop target update and antiradiation
homing variant, will be added. This portion of the program does not
currently have a DOD approved acquisition strategy.
[End of section]
Joint Strike Fighter (JSF):
The JSF program goals are to develop and field a family of stealthy,
strike fighter aircraft for the Navy, the Air Force, the Marine Corps,
and U.S. allies, with maximum commonality to minimize life-cycle costs.
The carrier suitable version will complement the Navy F/A-18 E/F. The
Air Force version will primarily be an air-to-ground replacement for
the F-16 and the A-10, and complement the F/A-22. The short take-off
and vertical landing version will replace the Marine Corps F/A-18 and
AV-8B. Significant foreign military purchases are expected.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin Aeronautics:
Program office: Arlington, Va.
Funding needed to complete:
R&D: $26,080.6 million:
Procurement: $128,860.8 million:
Total funding: $155,173.9 million:
Procurement quantity: 2,443:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The JSF program entered system development without demonstrating the
maturity of its eight critical technologies. The JSF program no longer
focuses on those technology areas; instead it uses a different method
of integration and risk management that tracks 28 program level risks.
We were unable to assess the new risk areas, but program data indicates
that 5 are high, 20 are moderate, and 3 are low risk. We obtained no
data that indicates that the technological maturity has changed.
Contractor efforts since the start of product development have focused
on the design and producibility of technology, not on further
demonstrating technology maturity. By its design review in 2005, the
program expects to have 100 percent of its critical drawings (referred
to as build-to-packages) completed for the Air Force and Marine Corps
versions and 80 percent completed for the Navy version.
[See PDF for image]
[End of figure]
JSF Program:
Technology Maturity:
During its concept development phase, JSF had eight critical
technologies: short take-off vertical landing/integrated flight
propulsion control, prognostic and health management, integrated
support systems, subsystems technology, integrated core processor,
radar, mission systems integration, and manufacturing. We reported in
May 2000, and again in October 2001, that low levels of maturity in
these technologies could increase the likelihood of cost and schedule
growth.
An independent review performed by DOD in 2001, using a different
method than technology readiness levels, concluded that the overall
technology maturity of the JSF program was sufficient to enter into
system development. Contractor efforts since that time have focused on
the design and producibility of the technology elements, not on
furthering the technology beyond that already demonstrated at the start
of the current phase. We obtained no data that indicates that the
technological maturity has changed. The program now uses Lockheed
Martin's Key System Development Integration approach to monitor overall
technology and design integration. Further, the program currently
tracks 28 program level risk areas and has assessed 5 as high, 20 as
moderate, and 3 as low risk. This represents an increase in risk from
last year when only 23 overall program risks were identified with 2
high, 18 moderate, and 3 low risk areas. We did not evaluate the
current JSF technique for assessing risks.
Design Maturity:
The program office has not provided information on the number of
drawings completed for any of the JSF versions. The preliminary design
review in March 2003 revealed significant issues related to airframe
design immaturity and other areas. At that time, estimates were about
5,000 pounds above targets.
While much of this overage has been reduced through better estimating,
design changes, and improved structural efficiency, the program will
still require reductions in aircraft specifications to meet
requirements for the Air Force version. Further estimating and weight
reduction assessments are being performed to determine the impact on
the Navy and Marine Corps versions. In addition, Lockheed Martin's
detailed design efforts for the Air Force version have been delayed by
2 months due to immature design tools, required structural analysis,
design team training, and redesigns because of overweight items.
Consequently, the program has a current $103 million unfavorable
schedule variance and the first flight for all three versions could be
delayed by 3 months.
Other Program Issues:
The Director, Operational Test and Evaluation, expects numerous test
challenges for the program, including the integration of highly
advanced sensors with the avionics systems, vertical thrust capability
for the Marine Corps version, and performance and maintenance
requirements of the low observable capabilities. According to program
documentation, vulnerability assessments for live fire test and
evaluation indicate that the current design will not meet requirements.
In July 2003, we recommended increased program oversight to adequately
plan for incorporation of foreign suppliers to protect sensitive U.S.
technology and meet program goals.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that DOD conducted an independent review in 2001 and concluded that the
technology maturity was sufficient to proceed into system development.
For this phase, JSF has adopted Lockheed Martin's approach of Program
Risk mitigation and Key System Development Integration (KSDI) plans to
monitor overall technology development and design integration as a best
practice. The program continues to address the 8 critical technology
categories through the this process. All 8 categories are mapped to the
KSDI, while 4 of the 8 are also mapped to Program Risk plans.
Furthering technology maturity is inherent in the development and risk
management process. Teams have traveled throughout the world looking
for better technologies to fit requirements. Also, existing Small
Business and Innovative Research and Science and Technology efforts
across the Navy, Air Force, and partner countries are focused on this
area. JSF is addressing technical issues primarily focused on weight.
Maturity of the original eight technology categories is not related to
current weight issues.
[End of section]
Joint Standoff Weapon (JSOW):
JSOW is a joint Air Force and Navy guided bomb to attack targets from
outside the range of most enemy air defenses. A dispenser variant (JSOW
A) carries submunitions to attack soft targets. In 2002, the Joint
Requirements Oversight Council deferred production of an antiarmor JSOW
variant (JSOW B). The unitary variant (JSOW C) uses a seeker,
autonomous targeting acquisition software, and a single warhead to
attack targets. All the variants use a common air vehicle. We assessed
the unitary variant and the common air vehicle.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon Missile Systems:
Program office: Patuxent River, Md.
Funding needed to complete:
R&D: $0.0 million:
Procurement: $784.6 million:
Total funding: $784.6 million:
Procurement quantity: 2,915:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The JSOW program began low-rate production in June 2003 without knowing
whether production processes were in control. However, the contractor
has begun studies to determine the feasibility of using statistical
process controls for production. The program relies on an after-
production process of inspection to discover defects. An operational
assessment is complete, but operational evaluation will not start
before award of the low-rate production contract.
[See PDF for image]
[End of figure]
JSOW Program:
Technology Maturity:
The JSOW unitary variant's technology appears mature. The program
office identified the imaging infrared seeker with the autonomous
acquisition software as the only critical technology for the system.
The seeker was not mature at the start of development, but it did
demonstrate maturity in October 2001--about three-fourths through
development--when it was flown aboard an aircraft in a captive flight
test. Program officials stated that in seven developmental tests, three
free-flight tests with the seeker only and four combined seeker/warhead
tests, the seeker's performance substantially exceeded requirements.
Design Maturity:
The JSOW unitary variant's basic design appears complete. At the system
design review in May 2002, the program office had completed 99 percent
of the drawings. The Navy completed 10 developmental tests (adding one
combined seeker/warhead test in 2003) in its development program--three
sled tests with the warhead, three free-flights with the seeker, and
four combined warhead/seeker tests. However, the Navy delayed the
beginning of operational evaluation to resolve a problem with the fuze.
In the third warhead test, the charge penetrated the target, but the
follow-through charge failed to detonate. The program office identified
the cause, incorporated a change, and confirmed the change through
additional testing.
Production Maturity:
JSOW production maturity could not be determined because the contractor
does not use statistical process controls. Rather, the contractor uses
a process of post-production inspection to control production quality.
Raytheon is investigating a defect reduction program and is evaluating
the use of statistical process controls where feasible. According to
program officials, 20 percent of their suppliers already use
statistical process controls. Program officials report that the
contractor has met the production schedule for more than 2 years for
the JSOW baseline variant and that the scrap and rework rates remain
low.
Program Office Comments:
In commenting on a draft of this report, the program office said that
Raytheon Missile Systems, the JSOW prime contractor, is responsible for
final assembly of the missile and that the assembly process does not
lend itself to a heavy statistical process control program. However,
components of the process, such as the circuit card assembly, have a
robust statistical process control program, and many of Raytheon's key
subcontractors have active statistical process control programs where
processes are closely monitored and controlled. Further, Raytheon's
Supplier Management Teams that manage first-and second-tier suppliers,
meet monthly, at a minimum, or more often if necessary, to address
issues.
[End of section]
Joint Tactical Radio System (JTRS):
The JTRS program is developing software-defined radios that will
interoperate with existing radios and significantly increase
communications capabilities. A joint service program office is
responsible for developing the JTRS architecture and waveforms, while
service-led program offices will develop and procure radio hardware for
platforms with similar requirements. We assessed Cluster 1, led by the
Army, which is developing radios for ground vehicles and helicopters.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Fort Monmouth, N.J.
Funding needed to complete:
R&D: $641.8 million:
Procurement: $7,453.4 million:
Total funding: $8,095.3 million:
Procurement quantity: 108,097:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The JTRS Cluster 1 program's demonstrated knowledge is difficult to
characterize. Almost all engineering drawings have been completed, and
key production processes are in control, suggesting design stability
and production maturity. However, until the technologies are
demonstrated, the potential for change remains. Officials do not expect
to achieve technology maturity until late 2004, when prototype radios
will be tested. In December 2003, the program attained design stability
for the Cluster 1 radio, though the program projects the need for
additional design drawings for various installation packages to install
on different platforms. The program claims to have production processes
in statistical control at this point; however, as development
transitions to low-rate production, the program expects this may change
as a result of design enhancements and technology insertion.
[See PDF for image]
[End of figure]
JTRS Program:
Technology Maturity:
None of the JTRS Cluster 1 program's 20 critical hardware and software
technologies are mature according to best practice standards. Many of
these critical technologies have been used in other radio applications,
but cannot be assessed as mature because they have not been integrated
into a Cluster 1 radio set. Mature backup technologies exist for some
critical technologies, but program officials have cautioned that
substituting them could complicate integration or result in degraded
performance. The program recently experienced a 4-month schedule slip
that officials attribute to short-term technology deviations affecting
size, weight, and power requirements of Cluster 1 radio sets. Program
officials do not expect these issues to be resolved until the system is
in full-rate production.
Design Maturity:
The program reports achieving design stability for the basic Cluster 1
radio design. The program recently completed its design review after a
delay of 5 months. The program's design consists of two major
components--the B kit, which is the basic Cluster 1 radio, and the A
kit, which is the installation components to integrate the radio with
the host platforms. The B-kit design is complete. The A-kit design
drawings are expected to increase as the platforms to be equipped with
Cluster 1 radios are better defined. The program does not attribute
this expected increase to the design of the Cluster 1 radio itself,
which it considers stable, but rather to the uncertainty about the
design of the A-kit, which involves mounting fixtures, cables,
antennas, and other such components required for integration of the
radio with host platforms. As more platforms are identified for Cluster
1 sets, more A-kit design drawings will be required. The undefined
design centers largely on the Army's FCS components.
Production Maturity:
The program reports that most production processes to be utilized in
manufacturing the JTRS radios are mature and in statistical control.
The program office, however, expects the number of processes to change
due to anticipated design enhancements and/or technology insertion.
Other Program Issues:
The JTRS Cluster 1 program has made considerable progress, but it faces
several challenges that could affect a successful outcome. The program
entered product development with an ambitious schedule that program
officials recognized as high risk. In particular, the program has a
software development plan with insufficient schedule reserve to
incorporate knowledge gained from initial development increments and a
compressed test and evaluation phase that leaves little room for
rework. The JTRS Cluster 1 information security certification approach
is also unprecedented, and the radios must go through a certification
process that is outside the program office's control. Further technical
challenges that could affect the program include platform integration,
networking, and spectrum certification.
Program Office Comments:
The program office generally concurred with our assessment and noted
that the number of slots in the Cluster 1 radio design decreased from
six to five mainly as a result of heat dissipation issues encountered
during the critical design review completed in December 2003. In
response to further questions, officials stated that they do not expect
the change to reduce the number of channels the radio will run, but
acknowledged that some of the channels will need to be mounted in
external vehicle mounts rather than in the radio itself. Officials
added that they do not anticipate this change having any additional
impact on performance.
[End of section]
Littoral Combat Ship (LCS):
The Navy's LCS will be a fast, maneuverable, shallow draft ship for
littoral warfare. It will use innovative hull designs to create a self-
deploying and self-sustaining ship. LCS will utilize interchangeable
mission modules to address three mission areas: mine, antisubmarine,
and small boat surface warfare. This review focuses on the technology
maturity of the mission modules for the two ships that comprise the
initial acquisition. Because competition for the hull is continuing, we
did not assess maturity of the sea frame itself.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: General Dynamics, Lockheed Martin, Raytheon:
Program office: Washington, D.C.
Funding needed to complete:
R&D: $632.9 million:
Procurement: $208.2 million:
Total funding: $841.1 million:
Procurement quantity: 2:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Data represents a program office estimate for the cost of the first two
ships. In the 2004 President's Budget, the funding for the initial two
ships and future ships are combined.
The LCS program has 22 mission related critical technologies, and is
scheduled to enter system development with 10 of those technologies
fully mature. Nine of the remaining 11 technologies will be close to
reaching full maturity by the start of system development. The
technologies that have not reached maturity affect all 3 of the
littoral warfare missions--mine warfare, antisubmarine warfare, and
surface warfare.
[See PDF for image]
[End of figure]
LCS Program:
Technology Maturity:
Ten of the 22 critical technologies on LCS will be fully mature at the
start of system development. Six technologies are not expected to
mature until after the design review.
Four critical technologies act as platforms, which employ other
technologies as payloads. These platforms will support operations
across the three littoral warfare missions--mine warfare, antisubmarine
warfare, and surface warfare. Three of these technologies--the MH-60R,
MH-60S, and Vertical Takeoff and Landing Tactical Uninhabited Aerial
Vehicle--have reached acceptable levels of technology maturity or they
will do so by program development. One technology, the Spartan
uninhabited surface vehicle, is not expected to be fully mature until
the lead ship award date.
The MH-60R is a helicopter capable of operating as an antisubmarine
warfare platform as well as a surface warfare combatant. At the time of
our review, the MH-60R and its critical technologies were undergoing
technical and operational evaluation for operations in both mission
areas.
The MH-60S is a helicopter that will be used in mine warfare and
surface warfare missions. At the time of our review the MH-60S in its
mine warfare configuration had reached maturity. The technologies used
for mine detection have reached maturity, but the technologies used for
mine neutralization are not expected to reach maturity by system
development. To operate in the surface warfare role, the MH-60S
requires structural changes. While the technologies planned for use by
the MH-60S in this mission are mature, the MH-60S itself will lack full
maturity by system development.
The Vertical Takeoff and Landing Tactical Uninhabited Aerial Vehicle is
an uninhabited helicopter originally developed for the role of
reconnaissance. For operations with LCS, the vehicle will be integrated
with a number of different technologies for operations in littoral
warfare missions. To operate as a mine warfare platform, it will
utilize the Coastal Battlefield Reconnaissance and Analysis System, a
system that is not expected to be fully mature by system development.
At the time of our review, no systems had been chosen for operations in
the antisubmarine or surface warfare roles.
In contrast to the three aerial platforms, the Spartan is an
uninhabited surface vessel. While it was first developed to support
reconnaissance, Spartan will be used in all three littoral warfare
missions. A prototype of this technology is being tested on deployed
naval assets for reconnaissance and force protection functions. These
tests should be completed in fiscal year 2004. The technologies that
will support other littoral warfare mission will be mature or near
maturity at the start of system development, but there is some
uncertainty about the complexity of integration with Spartan.
Several additional systems will operate independent of these platforms.
These include three uninhabited undersea vehicles for mine warfare that
have been used on other naval vessels. Also in development are two
distributed sensing systems that will not be mature by system
development. A final technology under consideration is Netfires, a
missile system being developed by the Army for FCS. This system will
not be fully mature by system development. No fallback technologies for
any systems have been identified due primarily to the redundant
capabilities among the mission modules.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the first two LCS ships would employ mission modules composed of
existing technologies including, but not limited to, those discussed in
this report. Future LCS vessels will utilize newly developed mission
module packages and will leverage lessons learned from the initial two
vessels, including risk mitigation for new technologies such as
advanced materials and nontraditional hull types.
The program office also stated that an important aspect of the LCS
program is the development of open interfaces between the ship and the
mission modules. LCS modular mission payloads will plug into an open
modular architecture through a set of standard systems interfaces. This
will mitigate the potential that a single mission package system could
negatively affect ship design viability and allow for rapid
introduction of new capabilities to the Fleet.
[End of section]
Long-term Mine Reconnaissance System (LMRS):
The Navy's LMRS is a mine reconnaissance system that employs unmanned
undersea vehicles. These vehicles are launched and recovered from
submarine torpedo tubes. LMRS is designed for autonomous operation to
survey potential minefields in support of amphibious and other battle
group operations. The Navy plans to obtain 12 operational systems and 1
development system. Each system consists of two unmanned vehicles,
ship-deployed command, control and recovery equipment, and shore-based
maintenance equipment.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Washington D.C.
Funding needed to complete:
R&D: $41.6 million:
Procurement: $324.0 million:
Total funding: $365.7 million:
Procurement quantity: 12:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The LMRS program began system development with neither of its two
critical technologies mature. While progress has been made in the past
7 years, program officials do not expect to achieve maturity on the
technologies until July 2004, at the earliest. While the design is
currently mature, only about two-thirds of the drawings were complete
at the time of the design review. According to program officials,
issues with sonar and software development delayed the test program for
LMRS. The impact of these delays is being evaluated by the program
office.
[See PDF for image]
[End of figure]
LMRS Program:
Technology Maturity:
Neither of LMRS' critical technologies, the sonar suite and the lithium
energy system, are fully mature. Program officials expect both
technologies to be fully mature by April 2004 and July 2004,
respectively. The program began product development in 1996 with both
technologies in conceptual form only.
Full technology maturity for the lithium energy system is contingent on
receiving a U.S. Navy safety certification so the technology can be
tested in an operational environment. Program officials stated that
lithium batteries aboard submarines can pose a deadly safety hazard but
that it would take a catastrophic incident to release lithium battery
byproducts. Program officials indicated that they are taking
appropriate actions to reduce this risk to an acceptable level. The
lithium energy system is particularly critical as no other technology
exists to meet LMRS' endurance requirements.
Design Maturity:
The LMRS program's design is mature, with approximately 95 percent of
the drawings currently releasable. However, the design was not fully
mature at the design review, with only two-thirds of the drawings
releasable to manufacturing. Program officials did note that LMRS had
developed computer-aided design models by that time in order to assess
system agreement with the design.
Program officials told us two significant issues facing the program are
sonar and software development. When the program started, sonar
development received lower priority and fewer resources compared to
other program areas because sonar development was deemed a medium to
lower risk. Similarly, at program start, the program office decided not
to purchase a software development test set that would have allowed for
earlier testing of LMRS software. Although a report summarizing test
results is not yet available, program officials informed us that recent
tests identified problems with sonar and software development. The
program office is evaluating the affect of resulting delays on the
program.
Other Program Issues:
Future costs are expected to increase. According to program officials,
three preplanned product improvements that will enhance LMRS'
resolution, range, and identification of mines are scheduled for
incorporation into six LMRS units. The cost of these improvements was
not included in the latest cost figures from the program office. In
addition, delays in sonar development will likely affect the cost and
schedule of these improvements.
The initial operating capability of LMRS was delayed by approximately 1
year. Work on the program was suspended for 1 year due to a funding
mismatch between government funds and contractor requirements.
Additionally, cost overruns resulted from unrealistic program
projections.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the program started as an acquisition streamlining initiative and
began with a panel of experts performing a risk assessment. The panel
identified five moderate risk areas, including acoustic sensors and
energy systems, and no high risk areas. Program officials indicated
that risk mitigation plans for the moderate risks were implemented. In
the acoustic sensor area, program officials stated that electrical
noise and interference problems have delayed development; the effect of
this delay is being assessed. Officials indicated that lithium energy
batteries are on track to achieve certification to support system test
schedules and that all risk areas are undergoing evaluation and are
expected to be mitigated to an acceptable level. Program officials also
stated that the program cost estimate and schedule for the preplanned
product improvements are being assessed in conjunction with the
acoustic sensor issue. Program officials emphasized that the program is
progressing and will deliver a much needed capability to the
warfighter.
[End of section]
Minuteman III Guidance Replacement Program (MM III GRP):
The Air Force's Minuteman III is an intercontinental ballistic missile
that can be launched from nuclear-hardened silos located throughout the
United States. First deployed in 1970, the system includes the missile,
the launch facilities, and the communications network. We assessed the
program that is replacing the aging guidance system. This and other
life-extension programs are designed to ensure the reliability and
supportability of the weapon system through 2020.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Mission Systems:
Program office: Hill Air Force Base, Utah:
Funding needed to complete:
R&D: $0.0 million:
Procurement: $794.4 million:
Total funding: $794.4 million:
Procurement quantity: 257:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The technology and design of the GRP appear fully mature, and the
production appears fairly mature. The program entered full-rate
production in November 1999, and it is currently eight sets ahead of
its original delivery schedule. This level of maturity follows several
years of difficult development. The program's low-rate decision was
deferred to 1998 as a result of two restructuring decisions: one in
1995 to reduce program risk from concurrency between program
development and production and another in 1997 to analyze design
functionality.
[See PDF for image]
[End of figure]
MM III GRP Program:
Technology Maturity:
Although we did not assess technology maturity in detail, the GRP
upgrades and extends the life of the 1960s era mature technology used
in the Minuteman III guidance system with electronic components that
were successfully demonstrated in the commercial sector in the 1990s.
The electronics in the guidance system require replacement because
current electronic components continue to degrade and are becoming
unreliable and unsupportable.
Design Maturity:
The GRP's design is mature because the program has released only 25
additional drawings out of 1,600 since production began. Four hardware
fixes to the configuration baseline have been implemented in
production. All previously produced guidance sets will be brought up to
the latest configuration with no impact on the production schedule or
cost.
Production Maturity:
The program's production processes appear to be fairly mature. Three
major production processes use statistical process control measures. Of
the eight key subprocesses that are used to monitor these three major
production processes, seven use statistical process control data. Five
of those seven are meeting the best practice standard. Other production
metrics that are used to assess production processes, such as cost of
quality (rework), are meeting expectations. As of July 2003, the GRP's
production was eight sets ahead of its original fiscal year 1998
baseline delivery schedule. The sets have a performance requirement of
15,000 hours between failures and are averaging 17,000 hours after
about 2.3 million hours of operation.
Other Program Issues:
Because the main navigation unit--the gyrostabilized platform--was
designed and built in the 1960s, the demand for parts required to
support the platform has decreased and vendors no longer make the
parts. Currently, the repair depot has been using parts from
decommissioned Minuteman II and Peacekeeper guidance sets to maintain
both the old and new Minuteman III sets. It is important that this
problem be resolved since the guidance system needs to stay viable
through 2020. However, the latest estimates available indicate that
parts may only be available through fiscal year 2008 or 2009. To
address this problem, the Air Force will need to identify qualified
vendors and provide funding prior to and in sufficient time to avoid
any interruptions in parts availability.
Program Office Comments:
In commenting on a draft of this assessment, program officials noted
that total program and unit costs only increased 7.8 percent when
compared with the latest approved fiscal year 1999 acquisition program
baseline. Program officials also noted that the program is executing
its third, fourth, and fifth production options and is currently 1
month ahead of schedule. In addition, the GRP utilizes statistical
process control data to measure production processes when possible, but
there is also extensive work that does not fit a classical statistical
process control format, e.g., cable harness, gyrostabilized platform,
and missile guidance set assembly, which are final component
assemblies. The program employs learning curve and cost of quality
metrics to address these assemblies. The GRP has approved two
engineering changes to further streamline the production process and
reduce costs. Demonstrated field performance is excellent, according to
program officials.
GAO Comments:
While the program has established a new cost and performance baseline
since the August 1993 decision to begin development, the comparison
presented provides an accurate picture of change since that major
decision. While DOD may subsequently update its baseline for management
purposes, our goal is to provide an aggregate or overall picture of a
program's history.
[End of section]
Minuteman III Propulsion Replacement Program (MM III PRP):
The Air Force's Minuteman III is an intercontinental ballistic missile
that can be launched from nuclear-hardened silos located throughout the
United States. First deployed in 1970, the system includes the missile,
the launch facilities, and the communications network. We assessed the
program's remanufacturing of the missile's three-stage solid-
propellant rocket motors. This and other life-extension programs are
designed to ensure the reliability and supportability of the weapon
system through 2020.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Mission Systems:
Program office: Hill Air Force Base, Utah:
Funding needed to complete:
R&D: $0.0 million:
Procurement: $1,101.3 million:
Total funding: $1,101.3 million:
Procurement quantity: 378:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The program currently has three-fourths of its critical production
processes under statistical control. Production maturity has
deteriorated from the 100 percent that was in control at the September
2001 full-rate production decision due to recent explosions at a
subcontractor facility, where the stage 2 and 3 motors are
manufactured. Although a new vendor has been requalified for the
production of these stages, the program office does not know if the
final procurement schedule for fiscal year 2007 can be met. Technology
and design of the solid-propellant rocket motors program appear fully
mature.
[See PDF for image]
[End of figure]
MM III PRP Program:
Technology Maturity:
The PRP technologies appear mature because the program is using
existing commercial technology previously used on the Minuteman III
motors. The upgrade involves chemicals that are compliant with current
environmental standards.
Design Maturity:
The PRP's design is mature because the program released 100 percent of
the drawings to manufacturing at the design review in July 1998. Since
that time, obsolete production methods, materials, or components have
resulted in minor engineering changes to the design. Further stability
has been demonstrated by the successful firing of 20 remanufactured
motors and the operational launching of 5 Minuteman III missiles using
remanufactured motors.
Production Maturity:
According to the program office, 75 percent of critical manufacturing
processes are in control. In September 2001, 100 percent were in
control, but due to recent problems at a subcontractor facility, this
number has declined.
Other Program Issues:
The August and September 2003 explosions are the latest in a series of
incidents at the stage 2 and 3 motor remanufacturing facility,
including a December 2002 incident in which small lead pellets from a
cracked dead blow mallet were found in 12 stage 2 motors and 9 stage 3
motors. In response to the December 2002 incident, a joint independent
team of government and industry experts addressed many problems.
According to the program office, these problems included a lack of
adherence to procedures and a lack of commitment to producing quality
products. In response to other incidents, the program office began
withholding progress payments until the subcontractor provided a
recovery plan to address the problems. Moreover, the program office is
now addressing issues through management reviews of the production
facility. However, program officials stated that they do not yet know
how the latest incidents will affect the critical path of all Minuteman
III life extension programs needed to make the weapon system
operational through 2020.
Program Office Comments:
In commenting on a draft of this assessment, the program office
generally agreed with the information in this report. Program officials
noted that the PRP requalified a new vendor for stage 2 and 3 rocket
motor production. The new vendor successfully met its first major
milestone involving casting trials. Transitioning the tooling and
material from the previous vendor to the new one is on track. The new
vendor is expected to reach full-rate production in July 2004. The PRP
is mitigating risk to national security by augmenting the remaining
production line with spare motor assets to keep rocket motor production
moving.
[End of section]
Mobile User Objective System (MUOS):
The Navy's MUOS, a satellite communication system, is expected to
provide low data rate voice and data communications capable of
penetrating most weather, foliage, and manmade structures. It is being
developed to replace the Ultra High Frequency (UHF) Follow-On satellite
system currently in operation and is required to support worldwide,
multiservice, mobile and fixed-site terminal users. MUOS consists of a
network of advanced UHF satellites and multiple ground segments. We
assessed both the space and ground segments.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin Space Systems or Raytheon Company:
Program office: San Diego, Calif.
Funding needed to complete:
R&D: $1,421.0 million:
Procurement: $4,052.0 million:
Total funding: $5,651.0 million:
Procurement quantity: 4:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The MUOS satellite program plans to enter the development phase in
February 2004 with five of eight critical technologies mature. The
remaining three technologies are projected to be mature by March 2006
in time for the critical design review. Mature backup technologies are
available should the new technology fail to mature; however, use of
backup technologies could degrade system performance in some key areas.
The product development period will likely require concurrent
technology maturation and product development activities to maintain
schedule.
[See PDF for image]
[End of figure]
MUOS Program:
Technology Maturity:
None of MUOS' eight critical technologies have demonstrated full
maturity, although program officials expect five of the eight critical
technologies to be mature by the start of the development program in
February 2004. The remaining three technologies are expected to be
mature by the time the program reaches its critical design review, in
March 2006. The eight critical technologies have mature backup
technologies in the event that they fail to mature. However, the use of
backup technologies could cause MUOS performance to fall below its
minimum requirements in some key areas.
The two contractors currently competing for the MOUS program are
developing their own unique designs that could be based on different
technologies. Therefore, it is possible that a technology now expected
to be immature at the start of the development program will not be
included in the winning contractor's design. However, due to source
selection sensitivity, no specific information regarding either
contractors' design or associated program cost could be disclosed.
Design Maturity:
The program's acquisition strategy requiring concurrent technology
maturation and product development could affect the timely achievement
of a stable design. The critical design review is scheduled for March
2006. Until a development contractor is selected, design maturity
information is considered source selection sensitive.
Program Office Comments:
In commenting on a draft of this assessment, the program office
generally agreed with our characterization of the MUOS program. In
response to our concern about concurrent technology maturation and
product development, it noted that the initial operational capability
was moved out 1 year, which it believes will allow the MUOS contractor
more time to mature the necessary technology and finalize the system
design.
In addition to the comments noted above, technical comments were
provided and appropriate changes were made to the assessment.
[End of section]
National Polar-Orbiting Operational Environmental Satellite System
(NPOESS):
NPOESS is a triagency National Oceanic and Atmospheric Administration
(NOAA), DOD, and National Aeronautics and Space Administration (NASA)
satellite program designed to monitor the weather and environment.
Current NOAA and DOD satellites will be merged into a single national
system with projected savings of at least $1.3 billion. The program
consists of five segments: space; command, control, and communications;
interface data processing; launch; and system integration. We assessed
all segments.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Space Technology:
Program office: Silver Spring, Md.
Funding needed to complete:
R&D: $3,087.6 million:
Procurement: $1,272.2 million:
Total funding: $4,846.2 million:
Procurement quantity: 4:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The NPOESS program entered system development in August 2002 with 12 of
its 14 critical technologies mature. While the total number of design
drawings has yet to be determined, the program has completed half the
currently identified engineering drawings, well in advance of the
design review. Over 5 years ago, program officials considered the
program to have several high-risk areas, but since then, officials have
taken steps to reduce program risk. One significant step being taken is
to demonstrate three critical sensors on a demonstrator satellite to
assess how well those sensors work within the context of the overall
system. Recently, however, the sensor schedule has slipped and combined
with new funding challenges, both design review and first satellite
launch have slipped to 2006 and 2009, respectively.
[See PDF for image]
[End of figure]
NPOESS Program:
Technology Maturity:
Twelve of the NPOESS 14 critical technologies were fully mature at the
start of development in August 2002. The two technologies that are not
mature are needed for two key sensors--the cross-track infrared sounder
and the conical microwave imager/sounder. The program projects that all
technologies will reach full maturity by the time of the design review
in 2006.
The NPOESS program plans to demonstrate three critical sensors in an
operational environment through a demonstration satellite that is to be
launched in 2006. The sensors to be tested include the visible/infrared
imager radiometer suite, the cross-track infrared sounder, and the
advanced technology microwave sounder. The program will use the
demonstration to provide data processing centers with an early
opportunity to work with sensors, ground controls, and data processing
systems, thereby incorporating lessons learned into the NPOESS
satellites. By July 2003, however, development schedules for these
sensors were extended due to performance problems.
Program officials indicated that they achieved maturity on other
technologies by concentrating on the early development of key
individual sensors. The acquisition strategy focused on maturing key
sensor technologies, using individual development contracts structured
to demonstrate the maturity of each sensor through a component-level
design review prior to the system-level design review.
Design Maturity:
Program officials indicated that at least 50 percent of the 6,971
currently identified drawings have been completed and released to
manufacturing; however, the total number of engineering drawings has
yet to be determined. Program officials project that all currently
identifiable drawings will be complete by the system design review in
2006.
Contract Management:
In late 2002, DOD extended the launch date of one of its legacy
meteorological satellites to 2010, delaying the need for the NPOESS
replacement satellites. In view of this, DOD and NOAA reduced their
funding for the NPOESS program by about $130 million. Program officials
also extended the deployment of the first NPOESS satellite launch about
21 months to November 2009.
The recent funding reductions prompted officials to restructure the
NPOESS program. A revised plan was completed in December 2003. Program
officials stated that the revised plan will necessitate few design
changes for the NPOESS satellites and that any changes will be
executable within the current 5-year budget.
Program Office Comments:
The NPOESS integrated program office concurred with this assessment and
provided information on updated project milestones and the
restructuring plan, which have been incorporated.
[End of section]
Guided Missile System Air Defense (Patriot) PAC-3 Program:
The Army's Patriot system is a long-range, high-medium altitude air and
missile defense system. The PAC-3 program is designed to enhance the
Patriot's ability to detect and identify missiles and other targets,
increase system computer capabilities, increase the number of missiles
in each launcher, improve communications, and incorporate a new hit-to-
kill missile. The PAC-3 system has two primary components, the fire
unit and the missile. We assessed both components.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon (prime), Lockheed Martin (missile):
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: $379.2 million:
Procurement: $4,049.2 million:
Total funding: $4,428.4 million:
Procurement quantity: 1,281:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The PAC-3 program continues to report that only a limited number of
critical production processes are under control, causing production and
testing problems. The technologies and design are stable on the
existing system. The Army will award a new contract in 2004 for an
additional 135 missiles. These missiles incorporate three alternative
technologies that will either reduce the missile's production cost or
increase its capability. These technologies have not yet reached full
maturity.
[See PDF for image]
[End of figure]
Patriot PAC-3 Program:
Technology Maturity:
Although the PAC-3's critical technologies appear mature, the program
office plans to incorporate three alternative technologies into the
missile under the fiscal year 2004 production contract. These
technologies are not yet fully mature. The advanced master frequency
generator and the simplified inertial measurement unit are intended to
offer lower cost than current components through the use of common
commercial off-the-shelf components. The multi-band radio frequency
down link will provide added capability for the missile at a higher
unit cost. Each of these alternative technologies is scheduled for
environment qualification and missile level ground testing between
March and April 2004. Flight-testing is scheduled between April and
September 2004. Should these new technologies fail to mature, program
officials said they could stay with existing technology.
Design Maturity:
The PAC-3's basic design is complete, with 100 percent of the drawings
released to manufacturing. However, as a result of the technology
insertion program, funded under the fiscal year 2004 production
contract, there will be an additional 103 drawings. Of the 103 total
drawings, 75 have been released thus far. The remaining drawings will
be released between March and April 2004.
Production Maturity:
The program has 23 percent of the key manufacturing processes used to
assemble the missile and the seeker under control. Significant
improvement in bringing additional processes under control has not
occurred, and production and testing problems remain. However, program
officials noted that rework needed before the seeker passes inspection
has decreased from an average of about three times to less than two
times within the past year.
Proposals are being considered from Lockheed to convert the fiscal year
2002 and 2003 contracts from fixed price incentive to firm fixed price.
A new contract for fiscal year 2004 production is projected to be
awarded by December 2003.
Other Program Issues:
On July 29, 2003, the PAC-3 and Medium Extended Air Defense System
(MEADS) programs were combined. MEADS, which will use the PAC-3
missile, is designed to be more mobile on the battlefield. The combined
system is intended to provide a more robust capability against theater
ballistic and cruise missiles, unmanned aerial vehicles, and rotary-and
fixed-wing threats. MEADS is scheduled to be deployed in 2012.
Program Office Comments:
The Patriot PAC-3 program office concurred with this assessment.
[End of section]
MQ-9 Predator B:
The Air Force's MQ-9 Predator B is a multirole, medium-to-high altitude
endurance unmanned aerial vehicle system capable of flying at higher
speeds and higher altitudes than its predecessor, the MQ-1 Predator A.
The Predator B is designed to provide a ground attack capability and
will employ fused multispectral sensors to find and track small ground
mobile or fixed targets. As envisioned, each Predator B system will
consist of four aircraft, a ground control station, and a satellite
communications suite operated by 55 military personnel.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: General Atomics Aeronautical Systems Incorporated:
Program office: Dayton, Ohio:
Funding needed to complete:
R&D: TBD:
Procurement: TBD:
Total funding: TBD:
Procurement quantity: 49:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The Predator B is scheduled to enter system development with three of
its four critical technologies mature. The fourth technology is
comprised of several off-the-shelf components and is expected to be
mature by July 2004. Unlike the other technologies, no backup is
available in the event this critical technology fails to mature as
expected.
[See PDF for image]
[End of figure]
Predator B Program:
Technology Maturity:
Three of the Predator B's four critical technologies, the synthetic
aperture radar, the multispectral targeting system, and the air
vehicle, are fully mature. The one immature technology is the stores
management system. This system, which is an avionics subsystem designed
to integrate and store data necessary to launch munitions, is currently
being evaluated in a laboratory environment. Program officials expect
this technology will be ready by July 2004. They believe there is low
risk associated with this technology since it is comprised of off-the-
shelf components. However, they did acknowledge that no backup
technology is available at this time.
Design Maturity:
By the start of system development, the program office expects about 22
percent of its engineering drawings, which reflect the aircraft's
baseline configuration, will be released. Further, it projects 91
percent of the drawings will be complete and released to manufacturing
by the September 2005 critical design review. The program office
believes the current design benefits from incorporating several common
components from the Predator A aircraft and the current design and
development of two prototype Predator B aircraft.
Production Maturity:
According to program officials, the contractor does not plan to use
statistical process control techniques. Instead, the contractor plans
to use other quality control techniques such as scrap, rework, and
repair to track and measure the quality of its manufacturing processes.
We have found this approach reactive versus prospective and may result
in cost and schedule increases.
Other Program Issues:
Recent changes to the Predator B acquisition strategy may create
additional program risks. In July 2003, at the direction of Air Force
headquarters, the Predator B acquisition approach was changed to
standardize the development process. This, along with recent budget
cuts, caused program officials to consider how best to restructure the
program. The Air Force had planned to procure 62 aircraft through 2009.
Program officials are now considering a plan to procure the 62 aircraft
through 2014, 5 years longer than the original plan. Program risk
assessments are underway to prioritize and match user requirements with
program resources. No final decisions will be made until early 2004.
Because of altitude limitations, the Army's Hellfire laser-guided
missile is no longer the weapon of choice for the Predator B. The Air
Force is considering other lightweight munitions.
Program Office Comments:
In commenting on a draft of this assessment, the program office
acknowledged it did not contractually require collection of statistical
process control data on critical manufacturing processes. Program
officials stated that their program strategy to demonstrate
manufacturing process maturity includes building, testing, and
evaluating production representative aircraft; conducting multiple
readiness reviews; and utilizing low-rate initial production to test
production processes. The contractor is also performing statistical
data analyses on nonconformance items, defects, unscheduled depot
returns, and supplier performance. Program officials also stated the
system user is reassessing the Predator B system profile. The Predator
B's modular, open-ended design has resulted in operational improvements
and may no longer require a system of four aircraft. Thus, the actual
number of aircraft, supporting equipment, and personnel needed to
support a Predator B system has yet to be determined.
[End of section]
Space Based Infrared System (SBIRS) High:
The Air Force's SBIRS High program is a satellite system intended to
provide missile warning information and to support the missile defense,
technical intelligence, and battlespace characterization missions. It
is intended to replace the Defense Support Program and will consist of
four satellites (plus one spare) in geosynchronous earth orbit (GEO),
two sensors on host satellites in highly elliptical orbit (HEO), and
associated ground stations. Our assessment discusses the sensors and
satellites only.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin Space Systems Company:
Program office: El Segundo, Calif.
Funding needed to complete:
R&D: $2,677.0 million:
Procurement: $1,358.3 million:
Total funding: $4,603.0 million:
Procurement quantity: 3:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The SBIRS High program's critical technologies have demonstrated
acceptable levels of maturity after many years of difficult
development. The level of design stability is unknown since the
contractor was unable to provide information on the total number of
releasable drawings. Similarly, production maturity could not be
determined because the contractor does not collect statistical control
data. In August 2002, the program underwent a major restructuring after
program costs increased to the point of triggering a departmental-level
review. Though corrective measures have been taken, the program is
still beset with technical problems and scheduling delays.
[See PDF for image]
[End of figure]
SBIRS High Program:
Technology Maturity:
The SBIRS High program's three critical technologies--the infrared
sensor, thermal management, and onboard processor--are mature. Program
officials indicated that the hardware was built and tested in a thermal
vacuum chamber under expected flight conditions. These technologies
were not mature at the start of development.
Design Maturity:
The SBIRS High design was immature at the time of the design review.
Less that 50 percent of the current drawings had been released at that
time. We could not assess the program's current design stability
because program officials do not know how many total design drawings
are expected for the program.
Design stability has been an issue for SBIRS High. The delivery of the
first HEO sensor has been delayed over 12 months since the program was
restructured in August 2002, due to excessive electromagnetic
interference (radio waves emitted by the sensor's electronics that
interfere with the host satellite). The first HEO sensor is now
scheduled for delivery in February 2004.
The program office has reported that it is applying the knowledge
gained from the design problems on this sensor to the second HEO sensor
which is now due for delivery in June 2004--a 5-month delay from the
restructured schedule.
Production Maturity:
We could not assess the production maturity of SBIRS High because the
contractor does not collect statistical process control data. However,
the program office does track and assess production maturity through
detailed monthly manufacturing and test data and monthly updates on
flight hardware qualifications. According to the program office, these
updates continue to reveal acceptable results.
Other Program Issues:
The delayed delivery of the first of two HEO sensors will likely have
long-term consequences for the remainder of the program. For example,
resources needed for the second HEO sensor and GEO satellites were
pulled and used on the first HEO sensor. As a result, the program will
likely encounter additional delays.
The Air Force has decided to purchase two additional HEO sensors for
constellation replenishment but has yet to fund them. Its current
acquisition strategy is to procure them separately at an estimated cost
of $314 million for the third HEO sensor and $237 million for the
fourth. In addition, the Air Force had considered accelerating the
schedules for the last three GEO satellites after concerns were
expressed by Congress over plans to delay these acquisitions. The Air
Force has now determined not to accelerate the GEO production schedule
and that the right time to begin procurement of these satellites is in
fiscal year 2006 (it plans to include $1.3 billion for this purpose in
fiscal years 2006 and 2007 budget requests). The Air Force believes
this schedule provides the optimal balance among concurrency,
operational needs, and industrial base sustainment.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that as part of the restructure activities, the program office
instituted incentive fees for cost performance, rigorous management
mechanisms to improve program stability and executability, and
increased senior-level oversight. It also continues to focus on
minimizing the downstream effects resulting from the initial program
shortcomings.
Additionally, program officials agreed that the difficulties
encountered on the HEO sensor have added pressure to the overall SBIRS
High schedule, but they noted that the program office is committed to
stabilizing requirements by following disciplined processes and
continues to assess and mitigate, when possible, cost and schedule
risks. Despite these changes, program officials project that SBIRS High
will continue to face the consequences associated with earlier program
decisions for several more years, but they asserted that the program
remains postured to identify and respond to them within the current
budget.
[End of section]
Small Diameter Bomb (SDB):
The Air Force's SDB is a small autonomous, conventional, air-to-ground,
precision bomb able to strike fixed and stationary targets. The weapon
will be installed on the F-15E aircraft and is designed to accommodate
integration with other aircraft, such as the F/A-22. Potential follow-
on capabilities, such as precision strike against moving targets, are
being considered.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing:
Program office: Fort Walton Beach, Fla.
Funding to complete through 2009:
R&D: $260.8 million:
Procurement: $379.0 million:
Total funding: $639.9 million:
Procurement quantity: 24,000:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The program office assessed all eight critical technologies for the SDB
as mature. The program office held the design review prior to starting
system development and, although data was not collected, the program
maintains that the contractor released over 90 percent of the
production drawings prior to system development. Beginning in 2004, the
program will begin its seamless verification test program, which
combines developmental, live fire, and operational testing, in an
effort to decrease time spent in system development. This concurrent
approach may increase program risks.
[See PDF for image]
[End of figure]
SDB Program:
Technology Maturity:
The program office assessed all eight critical technologies for the SDB
as mature. Program officials stated that many of the critical
technologies have been demonstrated in a free-flight environment. They
also stated that they have flight tested the system with the properly
sized components.
Design Maturity:
The program office held the design review prior to the start of system
development. Also, although data was not accumulated, the program
office maintains that Boeing released over 90 percent of the production
drawings prior to system development. According to the program office,
although the contractor has ultimate responsibility for the weapon
system and has given the government a 20-year warranty, the program
office has insight into the contractor's configuration control board
process and all changes are coordinated with the government.
The SDB program plans to combine developmental, live fire, and
operational testing beginning in 2004, and early test objectives will
be primarily defined by the contractor. It believes this combined
testing will eliminate or reduce redundant testing. This process could
expose the program to additional risk, as there may be more concurrency
between system developmental and operational tests.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the SDB program concluded a highly successful source selection
with a preproduction weapon that met all requirements. Program
officials also noted that the 2-year competition achieved the
following: design reviews completed; early live fire tests conducted;
over 80 percent production representative hardware flown; and Boeing
conducted six SDB free flights. This maturity resulted in a budgeted
average unit production price below the program objective goal with
significant savings to the government. A seamless verification test
program was designed to involve the operational community earlier in
the test process, reduce the test schedule and assets, and meet
requirements. SDB is on track to meet its production decision, 18
months after system development, and meet its 2006 fielding date.
[End of section]
RQ-7A Shadow 200 Unmanned Aerial Vehicle System (Shadow 200):
The Army's Shadow system is intended to be a ground commander's
reconnaissance, surveillance, target acquisition, and battle damage
assessment system. The system is comprised of four air vehicles,
payloads, ground control stations, launch and recovery equipment, and
communications equipment. The small, lightweight air vehicle is
intended to provide up to 4 hours of operations at 50 kilometers from
the launch and recovery site. The program entered product development
and limited production simultaneously in December 1999.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: AAI Corporation:
Program office: Huntsville, Ala.
Funding needed to complete:
R&D: $71.0 million:
Procurement: $266.2 million:
Total funding: $344.6 million:
Procurement quantity: 19:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The Shadow program's technology is mature and the basic design is
complete. However, the program began production in December 1999 before
achieving design stability or production maturity. Because the design
was not mature, testing revealed product reliability problems, delaying
operational testing and the full-rate production decision. The
contractor only recently started to capture statistical control data on
its manufacturing processes. Despite resultant cost increases and
operational shortfalls, the Army was still able to quickly deliver a
needed capability to the warfighter that has been used during recent
operations.
[See PDF for image]
[End of figure]
Shadow 200 Program:
Technology Maturity:
All of the Shadow's critical technologies are mature because they have
been demonstrated using actual hardware in realistic conditions. At the
limited production decision, which coincided with product development
start, four of the five technologies critical to the system's
performance were considered mature. The one immature technology, task
automation, is now considered mature. Prior to limited production, a
representative air vehicle was flown and evaluated to demonstrate
feasibility before a commitment to limited production was made. It was
not until about 3 years later that the last technology reached
maturity.
Design Maturity:
The basic design of the Shadow is now complete. However, the design was
not considered stable when it entered low-rate production. At that
time, the program had completed 67 percent of the drawings. Subsequent
testing revealed examples of design immaturity, especially relating to
the reliability of the system. Early testing revealed significant
problems. For example, testing revealed problems with the air vehicle
alternator and fuel bladders that resulted in restrictions on the
endurance and altitudes that could be flown. An immature design and
testing delays caused the Army to postpone its decision to enter full-
rate production by about 6 months from that planned at the low-rate
production decision.
Production Maturity:
According to the program office, the contractor only recently started
to track statistical control data for its critical manufacturing
processes. As a result, the program entered full-rate production in
September 2002 without ensuring that manufacturing processes were
mature. The program did conduct a production readiness review that
identified some low-to moderate-risk areas but concluded the contractor
could successfully execute the full-rate production contract.
The delay in achieving design maturity affected attainment of
production knowledge and delayed operational testing. Problems
encountered during early tests forced the program to delay the
completion of operational testing by about 1 year. The Director of
Operational Test and Evaluation reported in December 2002 that the
Shadow was not operationally suitable, survivable, and may not be
affordable. During operational tests, the system did not meet its
reliability or maintainability requirements. The Army decided to field
the system as is, rather than meeting 100 percent of the operational
requirements. Since the beginning of the program, it has been
recognized that deficiencies would exist and would be corrected through
subsequent block upgrades. However, the lack of funding has deferred
some of these improvements. As of December 2003, 12 systems (48 air
vehicles) had been fielded, and according to Army leadership, the
Shadow has provided critical intelligence during operations in Iraq.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the Army's requirement was to field an unmanned aerial vehicle
system as quickly as possible. It was understood the system would be
modified in production to achieve a time-phased incorporation of
objective and growth capabilities. The program entered engineering and
manufacturing development and low-rate initial production in December
1999. The program received a successful full-rate production decision
in September 2002. The successful full-rate production decision is a
first for any DOD unmanned aerial vehicle program and was accomplished
in only 33 months. To date, 12 systems have been fielded, including 4
to Operation Iraqi Freedom, which are operating at five to six times
their peacetime operational tempo. The systems are receiving
outstanding feedback from the field, and commanders are requesting that
fieldings be expedited. The Army considers the program and its
acquisition strategy successful.
[End of section]
Space Tracking and Surveillance System (STSS):
STSS is being developed in incremental, capability-based blocks
designed to track missiles throughout their flight. The initial
increment is composed of two demonstration satellites built under the
Space-Based Infrared System-low (SBIRS-low) program. MDA plans to
launch these satellites in 2007 to assess how well they work within the
context of the missile defense program. MDA may also develop a new
constellation of satellites and plans to launch the first of these in
2011. We assessed the two demonstration satellites.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Northrop Grumman Space Technology:
Program office: El Segundo, Calif.
Funding to complete through 2009:
R&D: $3,970.4 million:
Procurement: $0.0 million:
Total funding: $3,970.4 million:
Procurement quantity: 0:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Latest cost includes all costs from the program's inception through
fiscal year 2009. Procurement funding and quantities have yet to be
determined. NA = not applicable:
Only two of the initial STSS increment's six critical technologies have
reached an acceptable level of maturity, two are expected to reach
maturity in September 2004, and the remaining two are not expected to
reach full maturity until June 2006--1 year prior to launch. We could
not assess the design or production maturity, as there was no data
available from the program office. The initial STSS increment
demonstration satellites were partially built under the previous SBIRS-
low effort and put into storage 5 years ago. SBIRS-low was stopped
after the Air Force encountered significant cost and scheduling
increases and spent nearly $1.7 billion without launching a single
satellite. Prior to launch, the program must complete testing on the
satellite components and perform assembly, integration, and system
level testing activities.
[See PDF for image]
[End of figure]
STSS Program:
Technology Maturity:
Only two of six critical technologies--satellite communication cross-
links and on-board processor--are nearing maturity. Of the remaining
four technologies, the acquisition sensor and the tracking sensor are
expected to reach maturity by September 2004 and the single-stage
cryocooler and the two-stage cryocooler are not expected to achieve
maturity until June 2006, about 1 year before the satellites are to be
launched.
Design Maturity:
We did not assess the design maturity of the STSS demonstrator
satellites because drawing release data was not available. The program
currently has prototypes for the two mature technologies.
To launch the satellites, the STSS program must address certain risk
areas. Some of these areas include assessing the working condition of
the satellite hardware and software; dealing with insufficient time to
complete the ground segment, payload, and infrared software development
and testing; analyzing critical tests for acceptable performance prior
to launch; making modifications to the tracking sensor; and handling
issues related to parts obsolescence. The program faces other
challenges to get the satellites ready for launch within budget and on
schedule. A number of space segment design activities are still needed
for the existing satellite hardware and are proving to be more complex
or require more effort than originally planned. In addition, the
payload subcontractor has had a number of program management and
quality process problems that have led to delays in developing the
software and the upgrades to improve tracking sensor performance.
Other Program Issues:
Neither the prime contractor nor the payload subcontractor has
demonstrated a consistent ability to identify and correct problems
without strong program office involvement. The program office stated
that it has frequently taken steps to ensure the quality control of
this program.
The STSS program also includes plans for developing a new constellation
of missile tracking satellites in support of the ballistic missile
defense system. The new satellites could be different and more capable
than the ones to be launched in 2007. This part of the program is still
in a conceptual stage. MDA plans to start work on the new constellation
of satellites in 2005 or 2006 and launch a demonstrator satellite in
2011. The satellites are to serve as a baseline for follow-on
satellites that will comprise the STSS constellation.
We reported in May 2003, that by pursuing efforts to get the existing
satellites ready for launch in 2007, MDA may be missing an opportunity
to spend more time and money developing technologies needed for the new
constellation of satellites. Further, by focusing on the newer
constellation of satellites, MDA could launch the first new satellite
earlier than 2011 as now planned.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the remaining work to mature the four technologies is to integrate
the various components into the satellite end-item system. Program
officials further noted that most of the design difficulties relate to
improving tracking sensor performance and accommodating the launch of
the two satellites on a single booster. With the completion of a system
critical design review in November 2003, they believe these design
issues are behind them and should not cause any further significant
variances. Additionally, program officials noted that they and the
prime contractor have had a relationship that has not required any more
program office intervention than originally envisioned. Finally, they
stated that MDA had considered other alternatives to launching the
existing satellites but found them not to be prudent in the context of
the overall ballistic missile defense system.
GAO Comments:
Our prior work has shown that MDA's assessment of alternatives to
launching the demonstration satellites did not fully consider the
option of focusing solely on development of new technology, which could
offer operational capability sooner.
[End of section]
Theater High Altitude Area Defense (THAAD):
MDA's THAAD element is being developed in incremental, capability-based
blocks to provide a ground-based missile defense system. This system is
designed to protect forward-deployed military forces, population
centers, and civilian assets from short-and medium-range ballistic
missile attacks. THAAD will include missiles, launcher, X-band radar,
and a command and control battle management system. We assessed the
Block 2008 initial capability expected to be available in 2009.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin:
Program office: Huntsville, Ala.
Funding to complete through 2009:
R&D: $3,853.1 million:
Procurement: $0.0 million:
Total funding: $3,853.1 million:
Procurement quantity: 0:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
Latest cost includes all costs from the program's inception through
fiscal year 2009. Procurement funding and quantities have yet to be
determined. NA = not applicable:
THAAD's technologies are mature, and the design is stable. The
capability of the design will be demonstrated in flight tests that are
scheduled to begin in 2004. THAAD's initial deployment could occur
sooner than planned if early flight tests are successful. The current
THAAD acquisition strategy, as demonstrated by its extensive test
program, shows a strong emphasis on attaining knowledge and using that
knowledge to make acquisition decisions.
[See PDF for image]
[End of figure]
THAAD Program:
Technology Maturity:
The THAAD program office assessed all of its 50 critical technologies
as mature. These technologies are included in four major components:
command and control battle management and communications (C2BMC);
interceptor; launcher; and radar. A new component, a primary power
unit, will be added in the next few years, but this unit will most
likely be purchased as a commercial off-the-shelf item.
Despite early test failures, the THAAD development program of the 1990s
made progress in maturing critical technologies. Early flight-test
failures were caused primarily by the program's compressed schedule and
missile quality control problems. After these failures, program
officials placed more emphasis on risk reduction efforts, which
included using technology readiness levels to assess the maturity of
critical technologies.
Design Maturity:
The basic design of THAAD is essentially complete because the program
has released approximately 100 percent of its engineering drawings. The
program office successfully conducted the design review in December
2003.
THAAD's design is expected to change little between the design review
and initial capability in 2009, when MDA plans to incorporate the
element into the Ballistic Missile Defense System. However, if problems
are identified during flight-testing, scheduled from 2004 to 2008,
design changes could occur.
Other Program Issues:
THAAD program officials stated their principal objective for the
current block is the demonstration of a missile defense capability
through flight-testing, enabling an initial defensive capability in
2009. However, achieving this capability will require approval to
fabricate equipment for fielding and approval to redirect funds for
this purpose. MDA is examining opportunities to deploy an earlier THAAD
capability. For example, if early flight-testing is successful, MDA may
consider reallocating funds to deliver a THAAD capability in 2006 or
2007. MDA officials are also examining whether THAAD's radar can serve
as a forward-deployed radar for the Ballistic Missile Defense System.
Further development, customization, and testing of the radar have begun
in an effort to provide this capability in the next 2 years.
According to the program manager, the contractor has completed
approximately 50 percent of the work under the existing THAAD contract
and is performing work slightly ahead of schedule and under cost. Our
analysis of contractor data confirms this assessment. The contract is
being modified to align the program with MDA's block approach.
Program Office Comments:
In commenting on a draft of this assessment, MDA generally agreed with
the information provided in this report. Program officials also
provided technical comments, which were incorporated where appropriate.
[End of section]
Tactical Tomahawk Missile:
The Navy's Tactical Tomahawk (Block IV) is a major upgrade to the
Tomahawk Land Attack Missile (Block III). The Tactical Tomahawk missile
will provide ships and submarines with enhanced capability to attack
targets on land. New features include improved antijamming global
positioning system, in-flight retargeting, and the ability to transmit
battle damage imagery. The system includes the missile, the weapon
control system, and the mission planning system. We assessed only the
missile.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Raytheon Missile Systems:
Program office: Patuxent River, Md.
Funding needed to complete:
R&D: $19.6 million:
Procurement: $1,920.3 million:
Total funding: $1,939.9 million:
Procurement quantity: 2,194:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The Tactical Tomahawk missile entered low-rate production without
ensuring that production processes were in control. Although program
officials have identified critical processes and have procedures to
capture statistical process control data, only preliminary data,
gathered from the assembly of low-rate missiles, will be available by
the full-rate decision in June 2004. Trend analysis is not expected
until after the first complete low-rate delivery, scheduled for
November 2004. Not until this time does the program expect to have
tested sufficient missile quantities and have obtained adequate
knowledge to determine whether the chosen process control metrics are
valid and viable. The technology and design have reached full maturity.
[See PDF for image]
[End of figure]
Tomahawk Program:
Technology Maturity:
We did not assess the technology readiness levels of the key
technologies for the Tactical Tomahawk missile because at the time of
our review, critical technologies were already mature. According to the
program office, the critical technologies for the key subsystems--
antijamming global positioning system, digital scene matching area
correlator, and cruise engine--were modified derivatives from other
programs or upgrades to existing Tomahawk subsystems.
Design Maturity:
The design of the Tactical Tomahawk missile is complete. At the time of
the design review in June 2000, approximately 47 percent of the
drawings had been released to manufacturing. By the end of technical
evaluation in October 2003, 100 percent of the drawings had been
released. Technical evaluation was successfully completed and the
program entered operational evaluation in December 2003. Operational
evaluation is scheduled to be completed in March 2004.
Production Maturity:
Raytheon concluded that processes and controls are in place to
successfully enter full-rate production. Officials have begun
collecting statistical control data from the assembly of components for
the first low-rate production cycle. Initial data in support of
verifying critical process compliance is expected in March 2004.
Program officials plan to establish preliminary boundaries for upper
and lower control limits by the full-rate production decision in June
2004, but metrics are not expected to be fully stable until completion
of the low-rate deliveries in November 2004. Full-rate production is
planned as a multiyear procurement, from fiscal 2004 through fiscal
year 2009.
Other Program Issues:
Additional funding is expected from the Iraq Freedom Fund to accelerate
replenishment of missiles expended in Operation Iraqi Freedom. The
funding is expected to support a third low-rate production lot or an
increase in full-rate quantities by an estimated 183 missiles. At the
time of our review, negotiations had not been completed nor had the
Navy acquisition strategy been approved.
Program Office Comments:
In commenting on a draft of this assessment, the program office noted
that the Tactical Tomahawk missile successfully completed the technical
evaluation test phase with an unprecedented eight for eight flight-test
record. Program officials maintain that the design is sufficiently
mature to enter full-rate production based on the completion of design
reviews, technical evaluation, and the manufacture of 15 flight-test/
qualification missiles prior to low-rate deliveries. The missile
utilizes proven technologies from the Block 3 Tomahawk program and
other currently fielded military programs. Key technologies utilized
have been successfully demonstrated during development verification
testing. Low-rate production has validated critical manufacturing
processes and assured that critical design parameters are maintained.
The program is currently meeting all fleet performance requirements and
remains within acquisition program baseline cost, schedule, and
performance thresholds.
[End of section]
V-22 Joint Services Advanced Vertical Lift Aircraft (V-22):
The V-22 Osprey is a tilt rotor, vertical takeoff and landing aircraft
being developed by the Navy for joint service application. It is
designed to meet the amphibious and vertical assault needs of the
Marine Corps, the strike rescue needs of the Navy, and the special
operations needs of the Air Force and Special Operations Command. The
MV-22 version will replace the CH-46E and CH-53D helicopters of the
Marine Corps. We assessed the MV-22 Block A, which has been undergoing
changes to make it safe and operational.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Bell-Boeing JPO:
Program office: Patuxent River, Md.
Funding needed to complete:
R&D: $1,106.8 million:
Procurement: $28,533.9 million:
Total funding: $29,671.5 million:
Procurement quantity: 397:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
MV-22 Block A technologies are considered mature and the design is
considered stable. Significant modification and redesign have taken
place to address aircraft deficiencies that surfaced after fatal
mishaps in 2000. The program's production effort has had parts
shortages and quality issues with excessive scrap and rework.
Corrections are in place and are being monitored to verify a positive
and permanent fix. The program is using a new spiral development
approach. Operational assessment of Block A is scheduled for January
2005 to April 2005 to support a recommendation regarding fleet
introduction. However, the Marine Corps considers Block B the preferred
configuration for operational deployment. Block B will have capability,
reliability, and maintainability improvements. Operational assessment
of the Block B configuration will not be completed until 2006.
[See PDF for image]
[End of figure]
V-22 Program:
Technology Maturity:
Although we did not specifically assess the MV-22's technology
maturity, the program office states that based on DOD criteria, that
the Block A technologies are considered mature.
Design Maturity:
Design of Block A is essentially stable. Additional development tests
directed after two fatal mishaps in 2000 resulted in redesigning the
hydraulic and electrical lines. This increased the total number of
drawings by 31 percent. Currently, 100 percent of drawings have been
completed and released to manufacturing.
Production Maturity:
The program office was not able to provide statistical process control
data for measuring critical manufacturing processes. Contractors have
recently begun to measure production maturity using Six Sigma, process
certification, and process surveillance programs. Parts shortages and
excessive scrap and rework, which have caused inefficiencies in
assembly operations and cost growth, have been a production issue.
However, corrective actions have been taken and a positive trend has
emerged.
Other Program Issues:
The V-22's $74 million unit cost is 28 percent greater than the $58
million unit cost the contractors believe is needed to generate V-22
sales. About a third of more than 100 identified cost reduction
initiatives will be implemented using $58 million budgeted through
fiscal year 2003. An August 8, 2003, program acquisition decision
memorandum decreased program risk by limiting production. The savings
from this adjustment will be used for interoperability improvements and
further cost reduction initiatives to reduce production costs.
Concerns have been raised about the V-22's ability to operate safely
while performing evasive maneuvers, especially in high workload and
stressful situations. Also, while not a requirement, the aircraft
cannot safely perform auto rotation while in helicopter mode.
Operational effectiveness and suitability of Block A is scheduled to
begin in January 2005. A number of key performance parameters--which
are capabilities that if not met can be cause for program reevaluation,
reassessment, or termination--were removed from the operational
requirements document in October 2001 and redesignated so that they are
no longer absolute requirements.
The Marine Corps states that the Block B aircraft is the preferred
configuration for operational deployment. Block B development tests are
scheduled for August 2003 to December 2005. Operational assessment of
Block B is scheduled to begin in January 2006. Current plans are to
shift MV-22 initial operational capability from September 2004 to
fiscal year 2007.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that the program successfully implemented significant design changes
that resulted from two catastrophic mishaps, as well as rebaselined the
program. These changes have been implemented into delivered V-22s. An
extremely comprehensive, event driven flight test program, reinitiated
in May of 2002, accomplished 1,000 flight test hours on 9 test aircraft
without mishap. A May 23, 2003, program acquisition decision memorandum
stated that the program is proceeding well and that the V-22 has
demonstrated safe and reliable operations in the flight envelope,
combat maneuverability superior to helicopters, effective formation
flying, acceptable handling qualities in low-speed flight with
crosswinds, and other areas. The V-22 is meeting requirements for all
its key performance parameters and reliability and maintainability
metrics.
[End of section]
Wideband Gapfiller Satellites (WGS):
WGS is a joint Air Force and Army program intended to provide
communications to the U.S. warfighters, allies, and coalition partners
during all levels of conflict short of nuclear war. It is the next
generation wideband component in DOD's future Military Satellite
Communications architecture.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Boeing Satellite Systems:
Program office: Los Angeles Air Force Base, Calif.
Funding needed to complete:
R&D: $61.7 million:
Procurement: $664.6 million:
Total funding: $726.3 million:
Procurement quantity: 2:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The WGS program technology, design, and production are mature. However,
integration issues and manufacturing problems have contributed to a
delay in the launch of the first WGS satellite by over a year. The
integration issues have since been rectified, but the manufacturing
problems remain unresolved. A decision to delay the procurement of the
fourth and fifth satellites is expected to increase program costs.
[See PDF for image]
[End of figure]
WGS Program:
Technology Maturity:
WGS' two key technologies were mature when the program entered
production in November 2000. According to program officials, one of
these technologies has been demonstrated successfully in the commercial
sector.
Design Maturity:
The WGS design is essentially complete, as the program office has
released over 97 percent of the expected drawings to manufacturing.
However, the contractor has experienced problems in integrating the
phased array antenna into the satellite. The contractor assumed the
antenna would be easily integrated because of similarity with portions
of another commercial program. However, subsequent efforts invalidated
this assumption and WGS experienced unanticipated design changes.
Though the problems with integrating the antenna have since been
resolved, they have contributed to a delay in the launch of the first
satellite by over a year.
Production Maturity:
According to program officials, the contractor has two key
manufacturing processes, the automated wire-bonding and epoxy attach,
both of which are under control. However, the automated wire-bonding
process was not in control at the start of production due to the
quality of the materiel supplied by the subcontractor. While this
quality issue has been rectified, other manufacturing problems continue
to delay the launch of the first satellite.
The manufacturing processes employed for the phased array antenna and
the digital channelizer are relatively new. The contractor was relying
on experiences gained in manufacturing these technologies in the
commercial sector, but anticipated commercial orders for these
technologies did not materialize and the manufacturing processes did
not mature as expected. As a result, the contractor has experienced
manufacturing problems with both technologies. The problems with
manufacturing the digital channelizer have been resolved, but the
contractor is still having difficulty manufacturing components for the
phased array antenna at the rate required to meet the program schedule,
further delaying the program.
Other Program Issues:
DOD directed that launches for satellites four and five be delayed to
fiscal years 2009 and 2010, respectively. However, these dates are
outside the allowable dates of the WGS contract option clauses and will
likely cause a production gap. A decision to delay the procurement of
the fourth and fifth satellites will increase program costs; however,
the actual program increase will not be known until negotiations with
the contractor are completed.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that manufacturing problems with components on the phased array antenna
continue to cause schedule delays. While the WGS Program Office and the
contractor are trying to resolve the problems, the contractor has
requested a delay in the launch of the first WGS satellite to December
2005. The 2-year gap in production caused by delaying the procurement
of satellites four and five will result in higher costs for those
satellites. The higher costs are the result of parts obsolescence, loss
of manufacturing expertise, and greater costs to produce the first
three satellites than the government or contractor originally
predicted. The program office is assessing the expected cost increase
to identify funding needs and will address it in the fiscal year 2006
President's Budget.
[End of section]
Warfighter Information Network-Tactical (WIN-T):
WIN-T is the Army's high-speed and high-capacity backbone
communications network. It will provide reliable, secure, and seamless
video, data, imagery, and voice services, allowing users to communicate
simultaneously at various levels of security. The network will have the
ability to be initialized and modified based upon unit task
organization. WIN-T is being fielded in blocks, and we assessed the
first block.
[See PDF for image]
[End of figure]
Program Essentials:
Prime contractor: Lockheed Martin, General Dynamics:
Program office: Fort Monmouth, N.J.
Funding needed to complete:
R&D: $661.1 million:
Procurement: $9,290.7 million:
Total funding: $9,951.8 million:
Procurement quantity: 1:
Program Performance (fiscal year 2004 dollars in millions):
[See PDF for image]
[End of table]
The WIN-T program entered system development with 3 of its 12 critical
technologies close to reaching full maturity. None of these
technologies are expected to be fully mature until after design review
in March 2005. Eight have mature backup technologies available.
However, use of these technologies would degrade system overall
reliability, security, and performance. Because of the significant
interdependencies among critical technologies, and the fact that some
describe network functionality, it may not be possible to fully mature
these technologies until after production begins. Design and production
maturity could not be assessed because the program office does not
track the number of releasable drawings or the number of production
processes in control as metrics. WIN-T is primarily an information
technology system integration effort rather than a manufacturing
effort.
[See PDF for image]
[End of figure]
WIN-T Program:
Technology Maturity:
WIN-T entered system development with 3 of its 12 critical technologies
close to reaching full maturity. While program officials do not expect
these technologies to reach full maturity until the network is built
and can be demonstrated in an operational environment, they do expect
the technologies to have been demonstrated in a simulated operational
environment by the time design review is held in March 2005. A
technology readiness assessment determined that WIN-T would enter
system development prior to full definition of the first block's design
and specific technology-based components, systems or subsystems. WIN-T
will include technologies such as switching/routing and subscriber
access nodes; handheld terminal; information assurance; information
dissemination; transmission systems; and network management, some of
which are expected to undergo continuous maturation up until the design
review.
Design Maturity:
Design maturity could not be assessed because the program office does
not plan to track the number of releasable drawings as a design metric.
According to the program office, WIN-T is not a manufacturing effort,
but primarily an information technology system integration effort.
Consequently, the government does not obtain releasable design drawings
for many WIN-T components, particularly commercial components. The WIN-
T design will evolve using performance-based specifications and open
systems design and is to conform to DOD's Joint Technical Architecture.
Production Maturity:
Production maturity for the entire system could not be assessed because
the program does not plan to track manufacturing metrics for all WIN-T
components. According to the program office, WIN-T is not a
manufacturing effort, but primarily an information technology system
integration effort. Consequently, the government does not collect
information on the manufacturing statistical process control for many
WIN-T components, including commercial components. To ensure industrial
capabilities are reasonably available, a production readiness review
will be conducted prior to the end of system development.
Other Program Issues:
Additional areas that will require close attention by the program
office include the interdependence of WIN-T with FCS and JTRS programs;
the interdependence between WIN-T, FCS, and Global Information Grid
requirements; the scalability of WIN-T; the system-of-systems challenge
of linking all nodes and networks; the coordination of unmanned relay
programs with FCS; tracking external factors that will affect WIN-T
such as the DOD Net-Centric Data Strategy, U.S. Strategic Command's
oversight of Command, Control, Communications, Computers,
Intelligence, Surveillance, and Reconnaissance, Network Operations and
others; and coordination of Technology Transition Agreements. WIN-T
deployment will be essential for FCS deployment. As each system
evolves, integration demonstrations will need to be performed to ensure
WIN-T and FCS interoperability.
Program Office Comments:
In commenting on a draft of this assessment, the program office stated
that it is managing risks related to technology, design, and production
maturity by requiring contractors to develop critical technology
maturation plans and to demonstrate technology maturity prior to or
during the developmental testing/operational testing event scheduled
soon after the March 2005 design review. The program office is also
monitoring the maturity of form, fit, and function of prototype
equipment to be demonstrated in the testing event relative to the
production design.
Agency Comments:
DOD did not provide general comments on a draft of this report, but did
provide technical comments on individual assessments. These comments,
along with program office comments, are included with each individual
assessment as appropriate. (See app. I for a copy of DOD's response.):
Scope of Our Review:
For the 51 programs, each assessment provides the historical and
current program status and offers the opportunity to take early
corrective action when a program's projected attainment of knowledge
diverges significantly from the best practices. The assessments also
identify programs that are employing practices worthy of emulation by
other programs. If a program is attaining the desired levels of
knowledge, it has less risk--but not zero risk--of future problems.
Likewise, if a program shows a gap between demonstrated knowledge and
best practices, it indicates an increased risk--not a guarantee--of
future problems. The real value of the assessments is recognizing gaps
early, which provides opportunities for constructive intervention--
such as adjustments to schedule, trade-offs in requirements, and
additional funding--before cost and schedule consequences mount.
We selected programs for the assessments based on several factors,
including (1) high dollar value, (2) stage in acquisition, and (3)
congressional interest. The majority of the 51 programs covered in this
report are considered major defense acquisition programs by DOD. A
program is defined as major if its estimated research and development
costs exceed $365 million or its procurement exceeds $2.19 billion in
fiscal year 2000 constant dollars.
We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretaries of the Army,
Navy, and Air Force; and the Director, Office of Management and Budget.
We will also make copies available to others upon request. In addition,
the report will be available at no charge on the GAO Web site at
[Hyperlink, http://www.gao.gov.
If you have any questions on this report, please contact me at (202)
512-4841 or Paul Francis at (202) 512-4841. Major contributors to this
report are listed in appendix IV.
Signed by:
Katherine V. Schinasi:
Managing Director:
Acquisition and Sourcing Management:
List of Congressional Committees:
The Honorable John W. Warner:
Chairman:
The Honorable Carl Levin:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Ted Stevens:
Chairman:
The Honorable Daniel K. Inouye:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Duncan Hunter:
Chairman:
The Honorable Ike Skelton:
Ranking Minority Member:
Committee on Armed Services:
House of Representatives:
The Honorable Jerry Lewis:
Chairman:
The Honorable John P. Murtha:
Ranking Minority Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendixes:
Appendix I: Comments from the Department of Defense:
OFFICE OF THE UNDER SECRETARY OF DEFENSE:
3000 DEFENSE PENTAGON
WASHINGTON, DC 20301-3000:
ACQUISITION, TECHNOLOGY AND LOGISTICS:
MAR 17 2004:
Mr. Paul Francis:
Director, Acquisition and Sourcing Management
U.S. General Accounting Office:
441 G Street, N.W.
Washington, D.C. 20548:
Dear Mr. Francis:
This is the Department of Defense response to the GAO draft report,
Defense Acquisitions: Assessments of Major Weapon Programs, dated
February 17, 2004 (GAO Code 120272/GAO-04-248). We have enclosed
technical comments to ensure accuracy. These comments should be
reflected in the final report and in the individual program summaries.
My point of contact is Mr. Skip Hawthorne, (703) 692-9556, or e-mail:
skip. hawthorne@osd.mil.
Sincerely,
Signed by:
Deidre A . Lee:
Director,
Defense Procurement and Acquisition Policy:
Enclosure: As stated:
[End of section]
Appendix II: Scope and Methodology:
In conducting our work, we evaluated performance and risk data from
each of the programs included in this report. We summarized our
assessments of each individual program in two components--a system
profile and a product knowledge assessment. We did not validate or
verify the data provided by the Department of Defense (DOD). However,
we took several steps to address data quality. Specifically, we
reviewed the data and performed various quality checks, which revealed
some discrepancies in the data. We discussed these discrepancies with
program officials and adjusted the data accordingly.
System Profile Assessment:
In the past 3 years, DOD revised its policies governing weapon system
acquisitions and changed the terminology used for major acquisition
events. To make DOD's acquisition terminology more consistent across
the 51 program assessments, we standardized the terminology for key
program events. In the individual program assessments, program start
refers to the initiation of a program; DOD usually refers to program
start as milestone I or milestone A, which begins the concept and
technology development phase. Similarly, development start refers to
the commitment to system development that coincides with either
milestone II or milestone B, which begins DOD's system development and
demonstration phase. The production decision generally refers to the
decision to enter the production and deployment phase, typically with
low-rate initial production. Initial capability refers to the initial
operational capability, sometimes also called first unit equipped or
required asset availability.
The information presented on the funding needed to complete from fiscal
2004 through completion, unless otherwise noted, draws on information
from Selected Acquisition Reports or on data from the program office.
In some instances this data was not available, and we annotate this by
the term "to be determined" (TBD). The program cost comparisons are the
latest estimates provided by the individual programs. The quantities
listed refer to total quantities, including both procurement and
development quantities.
To assess the cost, schedule, and quantity changes of each program, we
reviewed DOD's Selected Acquisition Reports or obtained data directly
from the program offices. In general, we compared the latest available
Selected Acquisition Report information with a baseline for each
program. For systems that have started system development--those that
are beyond milestone II or B--we compared the latest available Selected
Acquisition Report to the development estimate from the first Selected
Acquisition Report issued after the program was approved to enter
development. For systems that have not yet started system development,
we compared the latest available data to the planning estimate issued
after milestone I or A. For systems not included in Selected
Acquisition Reports, we attempted to obtain comparable baseline and
current data from the individual program offices.
All cost information is presented in base year 2004 dollars, unless
otherwise noted, using Office of the Secretary of Defense approved
deflators to eliminate the effects of inflation. We have depicted only
the programs' main elements of acquisition cost--research and
development, and procurement; however, the total program costs also
include military construction and acquisition operation and maintenance
costs. Because of rounding and these additional costs, in some
situations the total cost may not match the exact sum of the research
and development and procurement costs. The program unit costs are
calculated by dividing the total program cost by the total quantities
planned. These costs are often referred to as program acquisition unit
costs. In some instances, the data was not applicable and we annotate
this by using the term "NA.":
The schedule assessment is based on acquisition cycle time, defined as
the number of months between the program start, usually milestone I or
A, and the achievement of initial operational capability or an
equivalent fielding date. In some instances, the data was not available
or classified, and we annotate this by using the term TBD.
The intent of these comparisons is to provide an aggregate or overall
picture of a program's history. These assessments represent the sum
total of the federal government's actions on a program, not just those
of the program manager and the contractor. DOD does a number of
detailed analyses of changes that attempt to link specific changes with
triggering events or causes. Our analysis does not attempt to make such
detailed distinctions.
Product Knowledge Assessment:
To assess the product development knowledge of each program at key
points in development, we submitted a data collection instrument to
each program office. The results are graphically depicted in each 2-
page assessment. We also reviewed pertinent program documentation, such
as the operational requirements document, the acquisition program
baseline, test reports, and major program reviews.
To assess technology maturity, we asked program officials to apply a
tool, referred to as technology readiness levels, for our analysis. The
National Aeronautics and Space Administration originally developed
technology readiness levels, and the Army and Air Force Science and
Technology research organizations use them to determine when
technologies are ready to be handed off from science and technology
managers to product developers. Technology readiness levels are
measured on a scale of one to nine, beginning with paper studies of a
technology's feasibility and culminating with a technology fully
integrated into a completed product. (See appendix III for the
definitions of technology readiness levels.) Our best practices work
has shown that a technology readiness level of 7--demonstration of a
technology in an operational environment--is the level of technology
maturity that constitutes a low risk for starting a product development
program. In our assessment, the technologies that have reached
technology readiness level 7, a prototype demonstrated in an
operational environment, are considered mature and those that have
reached technology readiness level 6, a prototype demonstrated in a
relevant environment, are assessed as attaining 50 percent of the
desired level of knowledge. Satellite technologies that have achieved
technology readiness level 6 are assessed as fully mature due to the
difficulty of demonstrating maturity in an operational environment--
space.
In most cases, we did not validate the program offices' selection of
critical technologies or the determination of the demonstrated level of
maturity. We sought to clarify the technology readiness levels in those
cases where information existed that raised concerns. If we were to
conduct a detailed review, we might adjust the critical technologies
assessed, the readiness level demonstrated, or both. It was not always
possible to reconstruct the technological maturity of a weapon system
at key decision points after the passage of many years.
To assess design maturity, we asked program officials to provide the
percentage of engineering drawings completed or projected for
completion by the design review, the production decision, and as of our
current assessment. Completed engineering drawings were defined as the
number of drawings released or deemed releasable to manufacturing that
can be considered the "build to" drawings.
To assess production maturity, we asked program officials to identify
the number of critical manufacturing processes and, where available, to
quantify the extent of statistical control achieved for those
processes. We used a standard called the Process Capability Index,
which is a process performance measurement that quantifies how closely
a process is running to its specification limits.[Footnote 3] The index
can be translated into an expected product defect rate, and we have
found it to be a best practice. We sought other data, such as scrap and
rework trends in those cases where quantifiable statistical control
data was unavailable.
Although the knowledge points provide excellent indicators of potential
risks, by themselves, they do not cover all elements of risk that a
program encounters during development, such as funding instability. Our
detailed reviews on individual systems normally provide for a fuller
treatment of risk elements.
[End of section]
Appendix III: Technology Readiness Levels:
Technology readiness level: 1. Basic principles observed and reported;
Description: Lowest level of technology readiness. Scientific research
begins to be translated into applied research and development. Examples
might include paper studies of a technology's basic properties;
Hardware software: None (Paper studies and analysis);
Demonstration environment: None.
Technology readiness level: 2. Technology concept and/or application
formulated;
Description: Invention begins. Once basic principles are observed,
practical applications can be invented. The application is speculative
and there is no proof or detailed analysis to support the assumption.
Examples are still limited to paper studies;
Hardware software: None (Paper studies and analysis);
Demonstration environment: None.
Technology readiness level: 3. Analytical and experimental critical
function and/or characteristic proof of concept;
Description: Active research and development is initiated. This
includes analytical studies and laboratory studies to physically
validate analytical predictions of separate elements of the technology.
Examples include components that are not yet integrated or
representative;
Hardware software: Analytical studies and demonstration of nonscale
individual components (pieces of subsystem);
Demonstration environment: Lab.
Technology readiness level: 4. Component and/or breadboard. Validation
in laboratory environment;
Description: Basic technological components are integrated to establish
that the pieces will work together. This is relatively "low fidelity"
compared to the eventual system. Examples include integration of "ad
hoc" hardware in a laboratory;
Hardware software: Low fidelity breadboard. Integration of nonscale
components to show pieces will work together. Not fully functional or
form or fit but representative of technically feasible approach
suitable for flight articles;
Demonstration environment: Lab.
Technology readiness level: 5. Component and/or breadboard validation
in relevant environment;
Description: Fidelity of breadboard technology increases
significantly. The basic technological components are integrated with
reasonably realistic supporting elements so that the technology can be
tested in a simulated environment. Examples include "high fidelity"
laboratory integration of components;
Hardware software: High fidelity breadboard. Functionally equivalent
but not necessarily form and/or fit (size weight, materials, etc.).
Should be approaching appropriate scale. May include integration of
several components with reasonably realistic support elements/
subsystems to demonstrate functionality;
Demonstration environment: Lab demonstrating functionality but not form
and fit. May include flight demonstrating breadboard in surrogate
aircraft. Technology ready for detailed design studies.
Technology readiness level: 6. System/subsystem model or prototype
demonstration in a relevant environment;
Description: Representative model or prototype system, which is well
beyond the breadboard tested for TRL 5, is tested in a relevant
environment. Represents a major step up in a technology's demonstrated
readiness. Examples include testing a prototype in a high fidelity
laboratory environment or in simulated operational environment;
Hardware software: Prototype--Should be very close to form, fit and
function. Probably includes the integration of many new components and
realistic supporting elements/subsystems if needed to demonstrate full
functionality of the subsystem;
Demonstration environment: High-fidelity lab demonstration or limited/
restricted flight demonstration for a relevant environment. Integration
of technology is well defined.
Technology readiness level: 7. System prototype demonstration in an
operational environment;
Description: Prototype near or at planned operational system.
Represents a major step up from TRL 6, requiring the demonstration of
an actual system prototype in an operational environment, such as in an
aircraft, vehicle or space. Examples include testing the prototype in a
test bed aircraft;
Hardware software: Prototype. Should be form, fit and function
integrated with other key supporting elements/subsystems to demonstrate
full functionality of subsystem;
Demonstration environment: Flight demonstration in representative
operational environment such as flying test bed or demonstrator
aircraft. Technology is well substantiated with test data.
Technology readiness level: 8. Actual system completed and "flight
qualified" through test and demonstration;
Description: Technology has been proven to work in its final form and
under expected conditions. In almost all cases, this TRL represents the
end of true system development. Examples include developmental test and
evaluation of the system in its intended weapon system to determine if
it meets design specifications;
Hardware software: Flight qualified hardware;
Demonstration environment: DT&E in the actual system application.
Technology readiness level: 9. Actual system "flight proven" through
successful mission operations;
Description: Actual application of the technology in its final form and
under mission conditions, such as those encountered in operational test
and evaluation. In almost all cases, this is the end of the last "bug
fixing" aspects of true system development. Examples include using the
system under operational mission conditions;
Hardware software: Actual system in final form;
Demonstration environment: OT&E in operational mission conditions.
Source: GAO and its analysis of National Aeronautics and Space
Administration data.
[End of table]
[End of section]
Appendix IV: GAO Contact and Acknowledgments:
GAO Contact:
Paul Francis (202) 512-4841:
Acknowledgments:
David B. Best, Leslie M. Hickey, and James L. Morrison made key
contributions to this report. Other key contributors included Robert L.
Ackley, Dora C. Baltzell, Cristina T. Chaplain, Lily J. Chin, Thomas J.
Denomme, James A. Elgas, Arthur Gallegos, William R. Graveline, David
J. Hand, Barbara H. Haynes, Sigrid L. McGinty, John E. Oppenheim,
Maria-Alaina I. Rambus, Rae Ann H. Sapp, Ronald E. Schwenn, Wendy P.
Smythe, Michael J. Sullivan, Robert S. Swierczek, Adam Vodraska, and
Karen S. Zuckerstein. The following staff were responsible for
individual programs:
System: Airborne Laser (ABL);
Primary staff: Marcus C. Ferguson/ Tana M. Davis.
System: Aegis Ballistic Missile Defense (Aegis BMD);
Primary staff: Tana M. Davis/ Richard A. Cederholm.
System: Advanced Extremely High Frequency Satellite (AEHF);
Primary staff: Bradley L. Terry/ Brian W. Eddington.
System: Active Electronically Scanned Array Radar (AESA);
Primary staff: Jerry W. Clark/ Gaines R. Hensley/ Bonita P. Oden.
System: Advanced Precision Kill Weapon System (APKWS);
Primary staff: John S. Warren Jr./ Wendy P. Smythe.
System: Advanced SEAL Delivery System (ASDS);
Primary staff: Mary K. Quinlan.
System: Advanced Threat Infrared Countermeasure/Common Missile Warning
System (ATIRCM/CMWS);
Primary staff: Jonathan E. Watkins/ Danny G. Owens.
System: Advanced Wideband Satellite/Transformational Satellite (AWS/
TSat);
Primary staff: Matthew R. Mongin/ David G. Hubbell/ Travis J. Masters.
System: B-2 Radar Modernization Program (B-2);
Primary staff: Don M. Springman/ Arthur L. Cobb.
System: C-130 Avionics Modernization Program (C-130 AMP);
Primary staff: Katrina D. Taylor/ Christopher A. Deperro.
System: C-5 Avionics Modernization Program (C-5 AMP);
Primary staff: Roger S. Corrado/ Sameena S. Nooruddin.
System: C-5 Reliability Enhancement and Reengining Program (C-5 RERP);
Primary staff: Sameena S. Nooruddin/ Roger S. Corrado.
System: Cooperative Engagement Capability (CEC);
Primary staff: Johana R. Ayers/ Leslie M. Hickey.
System: CH-47F Improved Cargo Helicopter (CH-47);
Primary staff: Leon S. Gill/ Wendy P. Smythe.
System: Comanche Reconnaissance Attack Helicopter (RAH-66);
Primary staff: Wendy P. Smythe/ Leon S. Gill.
System: Future Aircraft Carrier CVN-21;
Primary staff: J. Kristopher Keener/ Tedra Cannella.
System: DD(X) Destroyer;
Primary staff: J. Kristopher Keener/ Chris Durbin.
System: E-10A Multi-Sensor Command and Control Aircraft (E-10A);
Primary staff: Joseph E. Dewechter/ Jerry W. Clark/ Bonita P. Oden.
System: E-2 Advanced Hawkeye (E-2 AHE);
Primary staff: Bruce H. Thomas/ Gary L. Middleton.
System: EA-18G Growler (EA-18G);
Primary staff: Christopher R. Miller/ Brian T. Mullins/ Lillian I.
Slodkowski.
System: Evolved Expendable Launch Vehicle (EELV);
Primary staff: Maria A. Durant.
System: Expeditionary Fighting Vehicle (EFV);
Primary staff: Chad R. Holmes/ Dayna L. Foster.
System: Extended Range Guided Munition (ERGM);
Primary staff: Ronald E. Schwenn/ Shelby S. Oakley/ Carmen T. Donohue.
System: Excalibur Precision Guided Extended Range Artillery Projectile;
Primary staff: Lawrence D. Gaston Jr./ John P. Swain.
System: F/A-22 Raptor;
Primary staff: Marvin E. Bonner/ Edward R. Browning.
System: Future Combat Systems (FCS);
Primary staff: John P. Swain/ Lawrence D. Gaston Jr.
System: Global Hawk Unmanned Aerial Vehicle;
Primary staff: Bruce D. Fairbairn/ Matthew B. Lea.
System: Ground-Based Midcourse Defense (GMD);
Primary staff: Diana L. Dinkelacker/ Randolph S. Zounes.
System: Joint Air-to-Surface Standoff Missile (JASSM);
Primary staff: Beverly A. Breen/ LaTonya D. Miller.
System: Joint Common Missile;
Primary staff: Danny G. Owens/ Jonathan E. Watkins.
System: Joint Helmet Mounted Cueing System (JHMCS);
Primary staff: Dayna L. Foster/ Michael W. Aiken.
System: Joint Strike Fighter (JSF);
Primary staff: Brian T. Mullins/ Brendan S. Culley.
System: Joint Standoff Weapon (JSOW);
Primary staff: Carol T. Mebane/ Ivy G. Hubler.
System: Joint Tactical Radio System (JTRS);
Primary staff: Joel C. Christenson/ James P. Tallon.
System: Littoral Combat Ship (LCS);
Primary staff: J. Kristopher Keener/ Tedra Cannella.
System: Long-term Mine Reconnaissance System (LMRS);
Primary staff: Ian A. Ferguson/ Ricardo A. Marquez/ Gaines R. Hensley.
System: Minuteman III Guided Replacement Program (MM III GRP);
Primary staff: Brian W. Eddington/ Arturo Holguin Jr.
System: Minuteman III Propulsion Replacement Program (MM III PRP);
Primary staff: Arturo Holguin Jr./ Brian W. Eddington.
System: Mobile User Objective System (MUOS);
Primary staff: Travis J. Masters/ Matthew R. Mongin.
System: National Polar-Orbiting Operational Environmental Satellite
System (NPOESS);
Primary staff: Yvonne J. Vigil/ Bruce H. Thomas.
System: Guided Missile System Air Defense (Patriot) PAC-3 Program;
Primary staff: James A. Elgas/ William S. Lipscomb.
System: MQ-9 Predator B;
Primary staff: Steven M. Hunter/ Cheryl K. Andrew.
System: Space Based Infrared System High (SBIRS High);
Primary staff: Nancy Rothlisberger/ Maricela Cherveny.
System: Small Diameter Bomb (SDB);
Primary staff: LaTonya D. Miller/ Beverly A. Breen.
System: RQ-7A Shadow 200 Unmanned Aerial Vehicle System (Shadow 200);
Primary staff: Matt B. Lea.
System: Space Tracking & Surveillance System (STSS);
Primary staff: Sigrid L. McGinty/ Richard Y. Horiuchi.
System: Theater High Altitude Area Defense (THAAD);
Primary staff: Carrie R. Wilson/ Tana M. Davis.
System: Tactical Tomahawk Missile;
Primary staff: Ivy G. Hubler/ Carol T. Mebane.
System: V-22 Joint Services Advanced Vertical Lift Aircraft (V-22);
Primary staff: Jerry W. Clark/ Joseph E. Dewechter/ Bonita P. Oden.
System: Wideband Gapfiller Satellites (WGS);
Primary staff: Tony A. Beckham/ Arthur Gallegos.
System: Warfighter Information Network-Tactical (WIN-T);
Primary staff: James P. Tallon/ Joel C. Christenson.
Source: GAO.
[End of table]
[End of section]
Related GAO Products:
Best Practices: Setting Requirements Differently Could Reduce Weapon
Systems' Total Ownership Costs. [Hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-03-57] GAO-03-57. Washington, D.C.: February 11, 2003.
Defense Acquisitions: DOD's Revised Policy Emphasizes Best Practices,
but More Controls Are Needed. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO-04-53] GAO-04-53. Washington, D.C.: November 10, 2003.
Best Practices: Capturing Design and Manufacturing Knowledge Early
Improves Acquisition Outcomes. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO-02-701] GAO-02-701. Washington, D.C.: July 15, 2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best
Practices. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-02-469T]
GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to
Better Weapon System Outcomes. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO-01-288] GAO-01-288. Washington, D.C.: March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better
Weapon System Outcomes. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO/NSIAD-00-199] GAO/NSIAD-00-199. Washington, D.C.: July 31,
2000.
Defense Acquisition: Employing Best Practices Can Shape Better Weapon
System Decisions. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-
NSIAD-00-137] GAO/T-NSIAD-00-137. Washington, D.C.: April 26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon System Program
Implement Best Practices. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO/NSIAD-99-206] GAO/NSIAD-99-206. Washington, D.C.: August
16, 1999.
Best Practices: Better Management of Technology Development Can Improve
Weapon System Outcomes. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO/NSIAD-99-162] GAO/NSIAD-99-162. Washington, D.C.: July 30,
1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program
Outcomes. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-NSIAD-
99-116] GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.
Defense Acquisition: Improved Program Outcomes Are Possible.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/T-NSIAD-98-123] GAO/
T-NSIAD-98-123. Washington, D.C.: March 18, 1998.
Best Practices: DOD Can Help Suppliers Contribute More to Weapon System
Programs. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/NSIAD-98-
87] GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires
Changes in DOD's Environment. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO/NSIAD-98-56] GAO/NSIAD-98-56. Washington, D.C.: February
24, 1998.
Major Acquisitions: Significant Changes Underway in DOD's Earned Value
Management Process. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/
NSIAD-97-108] GAO/NSIAD-97-108. Washington, D.C.: May 5, 1997.
Best Practices: Commercial Quality Assurance Practices Offer
Improvements for DOD. [Hyperlink, http://www.gao.gov/cgi-bin/
getrpt?GAO/NSIAD-96-162] GAO/NSIAD-96-162. Washington, D.C.: August
26, 1996.
(120272):
FOOTNOTES:
[1] Technology maturity is attained when a technology demonstrates that
it works in an operational environment. See appendix III for
definitions of technology readiness levels.
[2] We used a standard called the Process Capability Index, which is a
process performance measurement that quantifies how closely a process
is running to its specification limits. (See app. II for further
information.):
[3] Process Capability Index provides assurance that production
processes are under 100 percent statistical control. A high index value
equates to fewer defects per part based on statistical process control
data. The general rule of thumb used by the manufacturing industry
states that if the index value for a process is less than 1.33, then
the process is not capable of producing a part with acceptable
consistency.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office:
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548: