Tactical Aircraft
DOD Should Reconsider Decision to Increase F/A-22 Production Rates While Development Risks Continue Gao ID: GAO-03-431 March 14, 2003The Air Force is developing the F/A-22 aircraft to fly at higher speeds for longer distances, be less detectable, and improve the pilot's awareness of the surrounding situation. The F/A-22 will replace the Air Force's existing fleet of F-15 aircraft. Over the past several years the program has experienced significant cost overruns and schedule delays. Congress mandated that GAO assess the development program and determine whether the Air Force is meeting key performance, schedule, and cost goals. GAO also assessed the implications of the progress of the development program on production.
The F/A-22 development program did not meet key performance, schedule, and cost goals in fiscal year 2002, and delays in the flight test program have led to an increase in the development cost estimate of $876 million. In response to this increase, DOD restructured the development program and reduced production aircraft by 27. If additional delays occur, further changes may be required. The program also continues to address technical problems that have limited the performance of test aircraft, including violent movement or "buffeting" of the vertical fins, overheating in portions of the aircraft, weakening of materials in the horizontal tail, and instability of avionics software. Air Force officials cannot predict when they will resolve these problems. These technical problems, along with the late delivery of aircraft to the flight test center, have delayed the development program. Based on F/A-22 flight test accomplishment data and current flight test plans, we believe that operational testing will likely be delayed several months beyond the planned August 2003 start date. The F/A-22 program is in its final stages of development, and low-rate initial production has begun. Since fiscal year 1997, funds have been appropriated to acquire production aircraft, and the F/A-22 acquisition plan calls for steadily increasing annual production rates. However, GAO considers the Air Force's acquisition strategy at high risk for increases in production costs. In past reports, GAO has reported that acquiring aircraft while significant technical challenges remain does not allow for adequate testing of the aircraft. The uncertainties regarding performance capabilities of the F/A-22 aircraft and its development schedule will persist until technical problems have been addressed, including testing of modifications or fixes necessary to potentially alleviate these problems. In light of those uncertainties, steadily increasing annual production rates could result in the Air Force having to modify a larger quantity of aircraft after they are built.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-431, Tactical Aircraft: DOD Should Reconsider Decision to Increase F/A-22 Production Rates While Development Risks Continue
This is the accessible text file for GAO report number GAO-03-431
entitled 'Tactical Aircraft: DOD Should Reconsider Decision to Increase
F/A-22 Production Rates While Development Risks Continue' which was
released on March 14, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to Congressional Committees:
United States General Accounting Office:
GAO:
March 2003:
TACTICAL AIRCRAFT:
DOD Should Reconsider Decision to Increase F/A-22 Production Rates
While Development Risks Continue:
F/A-22 Aircraft:
GAO-03-431:
GAO Highlights:
Highlights of GAO-03-431, a report to Congressional Committees
Why GAO Did This Study:
The Air Force is developing the F/A-22 aircraft to fly at higher
speeds for longer distances, be less detectable, and improve the
pilot‘s awareness of the surrounding situation. The F/A-22 will
replace the Air Force‘s existing fleet of F-15 aircraft. Over the
past several years the program has experienced significant cost
overruns and schedule delays. Congress mandated that GAO assess the
development program and determine whether the Air Force is meeting key
performance, schedule, and cost goals. GAO also assessed the
implications of the progress of the development program on production.
What GAO Found:
The F/A-22 development program did not meet key performance, schedule,
and cost goals in fiscal year 2002, and delays in the flight test
program have led to an increase in the development cost estimate of
$876 million. In response to this increase, DOD restructured the
development program and reduced production aircraft by 27. If
additional delays occur, further changes may be required. The program
also continues to address technical problems that have limited the
performance of test aircraft, including violent movement or ’buffeting“
of the vertical fins, overheating in portions of the aircraft,
weakening of materials in the horizontal tail, and instability of
avionics software. Air Force officials cannot predict when they will
resolve these problems. These technical problems, along with the late
delivery of aircraft to the flight test center, have delayed the
development program. Based on F/A-22 flight test accomplishment data
and current flight test plans, we believe that operational testing
will likely be delayed several months beyond the planned August 2003
start date.
The F/A-22 program is in its final stages of development, and low-rate
initial production has begun. Since fiscal year 1997, funds have been
appropriated to acquire production aircraft, and the F/A-22
acquisition plan calls for steadily increasing annual production
rates. However, GAO considers the Air Force‘s acquisition strategy at
high risk for increases in production costs. In past reports, GAO has
reported that acquiring aircraft while significant technical
challenges remain does not allow for adequate testing of the aircraft.
The uncertainties regarding performance capabilities of the F/A-22
aircraft and its development schedule will persist until technical
problems have been addressed, including testing of modifications or
fixes necessary to potentially alleviate these problems. In light of
those uncertainties, steadily increasing annual production rates could
result in the Air Force having to modify a larger quantity of aircraft
after they are built.
What GAO Recommends:
To help minimize the risks of producing large quantities of aircraft
that may require costly modifications, GAO recommends that the
Secretary of Defense (1) reconsider the decision to increase the
annual production rate beyond 16 aircraft until greater knowledge on
any need for modifications is established through completion of
operational testing and (2) update the 2002 risk assessment and
certification with sufficient detail to allow verification of the
conclusions. In comments on a draft of this report, the Department of
Defense (DOD) stated that it agreed, for the most part, with our
description of the current state of the F/A-22 program‘s content,
schedule and cost. However, DOD did not concur with our recommendation.
www.gao.gov/cgi-bin/getrpt?GAO-03-431.
To view the full report, including the scope and methodology, click on
the link above. For more information, contact Allen Li at (202)
512-4841 or lia@gao.gov.
March 2003
Contents:
Letter:
Results in Brief:
Background:
F/A-22 Technical Problems Continue to Affect Performance:
Flight Test Schedules Have Been Extended and May Slip Further:
Development and Modernization Costs Have Increased:
Risks in the F/A-22 Acquisition Plan:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Scope and Methodology:
Appendix I: Estimates of Performance for Key Parameters:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Staff Acknowledgments:
Related GAO Products:
Table:
Table 1: Schedule Changes for Key F/A-22 Test Program Events:
Figures:
Figure 1: F/A-22 Flight Test Schedule Changes:
Figure 2: Planned Modernization Funding Increases, President‘s Budgets
for Fiscal Years 2001-2004:
Figure 3: Number of Production Aircraft on Contract Prior to Completion
of Operational Testing:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO‘s product.
Abbreviations:
DIOT&E: Dedicated Initial Operational Test and Evaluation:
DOD: Department of Defense:
OSD: Office of the Secretary of Defense:
United States General Accounting Office:
Washington, DC 20548:
March 14, 2003:
Congressional Committees:
The Air Force is developing the F/A-22[Footnote 1] aircraft to replace
its fleet of F-15 air superiority aircraft. The F/A-22 is designed to
be superior to the F-15 because it is capable of flying at higher
speeds for longer distances, more difficult to detect, and able to
provide the pilot with substantially improved awareness of the
surrounding situation. The Air Force began the F/A-22 development
program in 1991. During the past several years, the program has
experienced repeated and significant cost overruns and schedule delays.
Congressional concern about the aircraft‘s development program cost and
progress is long-standing, and it continues.
The National Defense Authorization Act for Fiscal Year 1998[Footnote 2]
requires us to assess the Air Force‘s F/A-22 development program
annually and determine whether the Air Force is meeting key
performance, schedule, and cost goals.[Footnote 3] This is our fifth
report. Specifically, we determined (1) the progress of F/A-22
development in terms of performance, schedule, and cost and
(2) implications of this progress on the Department of Defense‘s (DOD)
acquisition plans.
Results in Brief:
The F/A-22 development program did not meet its key performance,
schedule, and cost goals for fiscal year 2002. The program continues to
address technical problems that have limited the performance of test
aircraft. These problems include unexpected shutdowns of the aviation
electronics (avionics) and excessive movement of the vertical tails.
Air Force officials stated they do not yet understand the problems
associated with the avionics instability well enough to predict when
they would be able to resolve them. Aircraft also have been unable to
meet maintenance requirements and are spending more time than planned
on the ground undergoing maintenance.
In addition, the program has experienced schedule delays. These delays
are the result of technical problems and the late delivery of
developmental aircraft to the flight test center. Many tasks originally
scheduled for 2002 have been rescheduled for 2003, and the Air Force
now plans to conduct more developmental flight testing concurrent with
operational testing.[Footnote 4] Moreover, we believe it is unlikely
the Air Force will complete all necessary flight testing prior to the
planned start of operational testing. Therefore, the start of
operational testing may need to be delayed several months beyond the
planned August 2003 start date.
Delays in the flight test program have significant consequences. Most
recently, they have led to an increase of $876 million in the
development cost estimate. In December 2002, in response to this
increase, DOD restructured the program using funds from production and
modernization upgrades to cover the cost increases. As a result, DOD
reduced the number of production aircraft by 27, which decreased the
total number of aircraft to be acquired from 303 to 276.[Footnote 5] If
additional delays occur, further changes may be required.
Despite continuing development problems and challenges, the Air Force
plans to continue to acquire aircraft during low-rate production at
increasing yearly rates. For example, the Air Force plans to acquire
20 aircraft in 2003, rather than the maximum of 16 Congress allowed
without DOD submittal of a risk assessment and certification. However,
as we have previously reported, acquiring aircraft before adequate
testing is a high-risk strategy that could serve to further increase
production costs. The performance capabilities of the F/A-22 and the
aircraft‘s development schedule will remain uncertain until technical
problems have been addressed, including testing of modifications or
fixes necessary to potentially alleviate these problems.
We are providing recommendations aimed at reducing the risk of
increasing the production rate of F/A-22 aircraft before technical
challenges have been addressed through operational testing. In its
comments on a draft of this report, DOD indicated it did not concur
with our recommendation that it not exceed an annual production rate of
16 aircraft until operational testing is complete. DOD stated that the
acquisition of more than 16 aircraft in fiscal year 2003 involves lower
risk and lower total program cost than staying at 16.
Background:
The F/A-22 is to be an air superiority and ground attack aircraft with
advanced features to make it less detectable to adversaries (stealth
characteristics) and capable of high speeds for long ranges.[Footnote
6] It is designed to have integrated avionics that greatly improve
pilots‘ awareness of the situation surrounding them. The objectives of
the F/A-22 development program are to (1) design, fabricate, test, and
deliver nine F/A-22 development test aircraft, two non-flying
structural test aircraft, six production representative test aircraft,
and 37 flight-qualified engines; (2) design, fabricate, integrate,
and test the avionics; and (3) design, develop, and test the support
and training systems. The F/A-22 is being developed under contracts
with Lockheed Martin Corporation, the prime contractor (for the
aircraft), and Pratt & Whitney Corporation (for the engine).
Following a history of increasing cost estimates to complete the
development phase of the F/A-22 program, the National Defense
Authorization Act for Fiscal Year 1998 established a cost limitation
for both the development and production.[Footnote 7] Subsequently, the
National Defense Authorization Act of 2002 eliminated the cost
limitation for the development, but left the cost limit for production
cost in place.[Footnote 8] The production program is now limited to
$36.8 billion.[Footnote 9] The current cost estimate of the development
program is $21.9 billion.
Currently, the F/A-22 program is in both development and production.
Development is in its final stages, and low rate initial production has
begun. Since fiscal year 1997, funds have been appropriated to acquire
production aircraft, and the F/A-22 acquisition plan calls for steadily
increasing annual production rates.
The aircraft‘s development problems and schedule delays have caused
congressional concerns, particularly in light of DOD‘s planned increase
in production rates. The National Defense Appropriations Act for Fiscal
Year 2003[Footnote 10] prohibited the obligation of funds for the
acquisition of more than 16 production aircraft in fiscal year 2003,
until the Under Secretary of Defense for Acquisition, Technology, and
Logistics submits the following to the congressional defense
committees: (1) a formal risk assessment that identifies and
characterizes the potential cost, technical, schedule, or other
significant risks resulting from increasing the F/A-22 production
quantities prior to the Dedicated Initial Operational Test and
Evaluation (DIOT&E)[Footnote 11] of the aircraft and (2) either a
certification that increasing the F/A-22 production quantity for fiscal
year 2003 beyond 16 aircraft involves lower risk and lower total
program cost than staying at that quantity or implementing a revised
production plan, funding, and test schedule. In December 2002, DOD
submitted the risk assessment and certification to Congress.
F/A-22 Technical Problems Continue to Affect Performance:
The F/A-22 developmental program did not meet key performance goals
established for fiscal year 2002 and continues to confront numerous
technical challenges. Major technical problems include instability of
the avionics software, violent movement, or ’buffeting,“ of vertical
fins, overheating in portions of the aircraft, weakening of materials
in the horizontal tail, and the inability to meet airlift support and
maintenance requirements. Modifications are being made to some test
aircraft to address some of these problems in preparation for
operational testing. Nevertheless, these problems continue to restrict
the performance and testing of the F/A-22.
Avionics Instability:
Software instability has hampered efforts to integrate advanced
avionics capabilities into the F/A-22 system. Avionics control and
integrated airborne electronics and sensors provide an increased
awareness of the situation around the pilot. The Air Force told us that
the avionics have failed or shut down during numerous tests of F/A-22
aircraft due to software problems. The shutdowns occur when the pilot
attempts to use the radar, communication, navigation, identification,
and electronic warfare systems concurrently. Although the plane can
still be flown after the avionics have failed, the pilot is unable to
successfully demonstrate the performance of the avionics. Therefore,
the Air Force has had to extend the test program schedule.
The Air Force recognized that the avionics problems pose a high
technical risk to the F/A-22 program, and in June 2002 the Air Force
convened a special team to address the problem. According to the team,
the unpredictable nature of the shutdowns was not surprising
considering the complexity of the avionics system. The team recommended
that the software be stabilized in the laboratory before releasing it
to flight testing. The team further recommended conducting a stress
test on the software system architecture to reduce problems and ensure
that it is operating properly. The Air Force implemented these
recommendations. Further, the Air Force extended the avionics schedule
to accommodate avionics stability testing and now plans to complete
avionics testing in the first quarter of 2005. However, Air Force
officials stated that they do not yet understand the problems
associated with the instability of the avionics software well enough to
predict when they will be able to resolve this problem.
Vertical Fin Buffeting:
Under some circumstances, the F/A-22 experiences violent movement, or
buffeting, of the vertical fins in the tail section of the aircraft.
This occurs as air, moving first over the body and the wings of the
aircraft, places unequal pressures on the vertical fins and rudders.
Unless the violent movement is resolved or the fins strengthened, the
vertical fins will break over time because the pressures experienced
exceed the strength limits of the fins. In addition, the buffeting
problem has restricted the testing of aerial maneuvers of the aircraft.
Lockheed Martin has developed several modifications to strengthen the
vertical fins and has performed an analysis to test the structural
strength of the aircraft. It concluded that no flight restrictions
above 10,000 feet are necessary as a result of buffeting. Currently,
the Air Force has not begun testing to verify flight operations at or
below 10,000 feet; operational limitations at altitudes below
10,000 feet remain in effect, with testing scheduled to begin in
June 2003.
Overheating Concerns:
Overheating in the rear portions of the aircraft has significantly
restricted the duration of high-speed flight testing. As the F/A-22
flies, heat builds up inside several areas in of the rear of the
aircraft. Continued exposure to high temperatures would weaken these
parts of the aircraft. For example, a portion of the airframe that sits
between the engines‘ exhausts[Footnote 12] experiences the highest
temperatures. This intense heat could weaken or damage the airframe. To
prevent this heat buildup during flight testing, the aircraft is
restricted to flying just over 500 miles per hour, about the same speed
as a modern jet liner, and significantly below the supercruise[Footnote
13] requirement. Currently, the F/A-22 flies with temperature sensors
in those areas of the aircraft, and it slows down whenever the
temperature approaches a certain level. The Air Force may add copper
sheets to the rear of the aircraft to alleviate the problem. The Air
Force began these modifications in January 2003 and plans to complete
them by July 2003.
Horizontal Tail Material Separations:
F/A-22 aircraft have experienced separations of materials in the
horizontal tail and the shaft, which allows the tail to pivot. Because
the separations reduce tail strength, the Air Force restricted flight
testing of some aircraft until it determined that this problem would
not affect flight safety during testing. The Air Force and the
contractor initially believed that improvements to the aircraft‘s
manufacturing process would solve this problem. However, the Air Force
has determined that it could only solve this problem by redesigning the
tail of the aircraft. The Air Force plans to conduct flight testing of
the redesigned tail between February 2004 and April 2004.
Meeting Airlift Support Requirements:
The Air Force estimates it will not meet the F/A-22 airlift support
requirement despite last year‘s estimate that it would meet all
identified key performance parameters.[Footnote 14] (Appendix I
contains a list of key performance parameters.) The airlift support
requirement is that 8 C-141 aircraft or their equivalents would be
sufficient to deploy a squadron of 24 F/A-22s for thirty days without
resupply. Today the Air Force estimates that 8.8 C-141 equivalents will
be necessary.
Impact of Maintenance Needs on Performance:
The F/A-22‘s performance may also be affected by maintenance needs
that exceed established objectives. The Air Force estimates that the F/
A-22 should, at this point in its development, be able to complete
1.67 flying hours between maintenance actions and 1.95 flying hours by
the end of development. However, aircraft are requiring five times the
maintenance actions expected at this point in development. As of
November 2002, the development test aircraft have been completing
only .29 flying hours between maintenance actions. Therefore, the
development test aircraft are spending more time than planned on the
ground undergoing maintenance.
In addition, the F/A-22 program has not completed the testing required
to prove the aircraft can be maintained worldwide without unique
support equipment. For example, the Air Force planned to fly the F/A-22
a minimum of 650 hours prior to the start of operational testing to
establish that special support equipment is not necessary to maintain
the materials on the exterior of the aircraft. These materials are
critical to the aircraft‘s low observable, or stealthy, nature.
However, as of December 2002, the program has only accomplished
191.6 hours. According to the Air Force, the program will not complete
testing for this requirement until the completion of the development
program, currently planned for July 2004.
Modifications to Improve Performance:
In 2002, the F/A-22 development program implemented several
modifications to development aircraft to improve performance. The
majority of modifications were related to installing the necessary
upgrades to complete operational testing. The last three development
test aircraft have required an average of 63 modifications. The first
two production aircraft have required an average of 50 of
these upgrades.
In addition, the program repaired problems in the aircraft‘s arresting
gear system that were discovered during development testing. Further,
the Air Force has scheduled modifications to address the previously
cited problems found with the vertical tail of the aircraft (fin-
buffeting). The Air Force included these repairs in its 2002
modification schedule, but did not begin them in 2002. The
modifications will begin during fiscal year 2003.
Flight Test Schedules Have Been Extended and May Slip Further:
Progress in F/A-22 flight testing was slower than expected in 2002 in
all test areas, according to Office of the Secretary of Defense (OSD)
testing officials. Consequently, the Air Force extended flight test
schedules and reduced the number of flight tests. Many tasks originally
planned for 2002 were rescheduled for 2003. Further, the Air Force now
plans to conduct more developmental flight testing concurrently with
operational testing.
Continuing technical problems were the primary reasons for the delays
in flight testing. In addition, late delivery of development aircraft
to the flight test center was a contributing problem; three
developmental aircraft were delivered from 9 to 12 months late. Late
deliveries were due not only to technical problems, but also to
continuing problems associated with the manufacture and assembly of
development aircraft by the prime contractor.
With the new schedule, the Air Force delayed the beginning of
operational testing for 4 months, until the portion of developmental
testing required to begin operational testing could be completed.
Operational testing is now planned to begin in August 2003. Figure 1
and table 1 show the changes in the FA/-22 flight test schedules.
Figure 1: F/A-22 Flight Test Schedule Changes:
[See PDF for image]
[End of figure]
Table 1: Schedule Changes for Key F/A-22 Test Program Events:
Key Events: Completion of development flight testing necessary prior to
operational testing; Prior schedule: April 2003; Revised schedule:
August 2003; Change in months: 4.
Key Events: Start of operational testing; Prior schedule: April 2003;
Revised schedule: August 2003; Change in months: 4.
Key Events: Completion of operational testing; Prior schedule: December
2003; Revised schedule: July 2004; Change in months: 7.
Key Events: High-rate production decision; Prior schedule: March 2004;
Revised schedule: March 2004; Change in months: 0.
Source: U.S. Air Force.
[End of table]
However, according to OSD officials involved in operational testing,
there is a high risk of not completing an adequate amount of
development flight testing before operational testing is scheduled to
begin. Indeed, we believe that it is unlikely that the Air Force will
be able to complete all necessary avionics flight testing prior to the
planned start of operational testing. Based on F/A-22 flight test
accomplishment data and current flight test plans, we project that the
start of operational testing might be delayed until January 2004. As a
result, operational testing could be delayed by several months beyond
the current planned date of August 2003.
Development and Modernization Costs Have Increased:
In December 2002, the Air Force estimated that development costs had
increased by $876 million, bringing total development costs to
$21.9 billion. This increase was due to the technical problems and
schedule delays discussed earlier.
In addition, since fiscal year 2001, there have been dramatic increases
in planned funding for modernization upgrades[Footnote 15] that enhance
the operational capabilities of the F/A-22, as shown in figure 2.
Currently, the Air Force has almost $3.0 billion in funding for
modernization projects, which it plans to spend through fiscal year
2009. Most of the recent increase in modernization funding is necessary
to provide increased ground attack capability. Other modernization
projects include upgrading avionics software, adding an improved short-
range missile capability, upgrading instrumentation for testing, and
incorporating a classified project.
Figure 2: Planned Modernization Funding Increases, President‘s Budgets
for Fiscal Years 2001-2004:
[See PDF for image]
[End of figure]
In December 2002, in response to the increase in development costs, the
Under Secretary of Defense, Comptroller, approved the restructuring of
the F/A-22 program. According to the Comptroller, the cost increase
will not require increased funds from Congress. Rather, the estimated
$876 million increase for development will be met by a $763 million
decrease in production funding and a transfer of $113 million from
modernization funds. This restructure eliminates 27 aircraft from the
current production program, reducing the total number of aircraft to be
acquired from 303 to 276.[Footnote 16]
Risks in the F/A-22 Acquisition Plan:
Despite continuing development problems and challenges, the Air Force
plans to continue acquiring production aircraft at increasing annual
rates. This is a very risky strategy, because, as we have previously
reported, the Air Force may encounter higher production costs as a
result of acquiring significant quantities of aircraft before adequate
testing. Late testing could identify problems that require costly
modifications in order to achieve satisfactory performance.
For example, as shown in figure 3, the Air Force plans to acquire
20 aircraft during 2003, rather than the maximum of 16 Congress allowed
without DOD‘s submittal of a risk assessment and certification. DOD
justified this strategy in the December 2002 risk assessment and
certification it submitted to Congress.[Footnote 17] In this document,
DOD certified that acquiring more than 16 aircraft involved lower risk
and lower total program cost than acquiring only 16. DOD identified the
costs associated with acquiring more than 16 aircraft per year as
between $7 million and $221 million, depending on the number of
aircraft in excess of 16. DOD concluded that this additional cost would
be less than the potential cost of modifying production aircraft once
operational testing has been completed. Figure 3 shows the Air Force‘s
acquisition plan.
Figure 3: Number of Production Aircraft on Contract Prior to Completion
of Operational Testing:
[See PDF for image]
[End of figure]
[A] Operational testing is scheduled to be completed in July 2004.
However, DOD‘s risk assessment may be overly optimistic because it
is grounded in the conclusion that there is a low risk that remaining
development and operational testing will identify needs for expensive
modifications. The performance capabilities of the F/A-22 and its
schedule will remain uncertain until technical problems have been
addressed, including testing of modifications or fixes necessary to
potentially alleviate these problems. Furthermore, we believe that the
amount of development and operational testing and the remaining
uncertainties increase the possibility that modifications considered
unlikely in DOD‘s analysis will, indeed, need to be made. For example,
the Air Force has still not completely defined the fin-buffet problem
described earlier in this report. The remaining 15 percent of flight
testing to help characterize the problem is not scheduled to begin
until June 2003. Consequently, there is still the possibility that
additional modifications and costs may be necessary to correct this
problem on production aircraft. DOD‘s risk assessment acknowledges that
additional fin buffet testing is needed, but concludes that
modifications are not expected.
The optimism of DOD‘s risk assessment is reflected in the Air Force‘s
general acquisition strategy. As also shown by figure 3, the Air Force
is currently committed to acquiring 73 production aircraft
(26 percent) before operational and development testing is complete. We
believe that--like the fiscal year 2003 decision to acquire more than
16 aircraft--this is an overly optimistic strategy given the remaining
F/A-22 technical problems and the current status of testing. As we have
noted, acquiring aircraft before completing adequate testing to resolve
significant technical problems increases the risk of costly
modifications later. If F/A-22 testing schedules slip further--as we
believe is likely--even more aircraft will be acquired before
development and operational testing is complete, and the risk of costly
modifications will increase still more.
Conclusions:
Continuing the acquisition of aircraft in increasing quantities when
significant development testing and technical problems remain is an
acquisition strategy that relies on overly optimistic assumptions
regarding the outcome and timing of the remaining testing events. By
employing such a strategy, major problems are more likely to be
discovered after production has begun when it is either too late or
very costly to correct them. At the very least, key decisions are being
made without adequate information about the weapon system‘s
demonstrated operational test results. In its certification, DOD
quantified the estimated costs associated with a higher production
rate. However, the potential advantage was predicated on the assumption
that the risks of modifications are low. As we stated last year, by
limiting F/A-22 production quantities and completing development
testing, the Air Force could gain information that would reduce
uncertainties and the risks of increased costs and delays before
committing to additional production aircraft. As we discussed earlier
in this report, DOD recently decided to reduce production quantities as
part of a program restructure to address F/A-22 development problems
and associated cost increases. Based on uncertainties about the
resolution of problems found in the past year, we continue to maintain
the position that production quantities should be limited.
Recommendations for Executive Action:
In light of continued uncertainties regarding the resolution of
problems found in the past year and notwithstanding the
December 2, 2002 certification provided by DOD, we recommend that the
Secretary of Defense:
* reconsider the Department‘s decision to increase the annual
production rate beyond 16 aircraft until greater knowledge on any
need for modifications is established through completion of
operational testing, and:
* update the 2002 risk assessment and certification with sufficient
detail to allow for verification of the conclusions following the
completion of operational testing.
Agency Comments and Our Evaluation:
In written comments on a draft of this report, DOD stated that it
agreed, for the most part, with our description of the current state of
the F/A-22 program‘s content, schedule, and cost. However, DOD did not
concur with our recommendation that it not increase its production rate
beyond the maximum of 16 aircraft Congress allowed without DOD
submitting a risk assessment and certification. DOD said that our
recommendation does not sufficiently account for the costs of
termination associated with the approval given to funding long-lead
items, the manufacturing inefficiencies associated with a reduction in
aircraft quantities, or the effects of inflation on the cost of
acquiring aircraft at a lower rate. DOD also noted that we had not
provided a quantitative assessment to justify limiting production, and
it reiterated its reliance on the risk assessment and certification it
submitted to Congress in December 2002. DOD also asserted, incorrectly,
that our report concludes that minimal cost risk would be realized by
slowing production.
Following review of DOD‘s comments, we clarified the recommendation
in our draft report by establishing two recommendations. These
recommendations are based on the current state of the program--
including the challenges and risks it faces--and on our examination of
DOD‘s risk assessment and certification. DOD acknowledges the
challenges faced by the program but believes the risk of modification
is low. As we discussed in this report, until testing has been
completed and technical problems have been addressed, the performance
capabilities of the F/A-22 and its schedule will remain uncertain;
thus, it is not possible to predict that expensive modifications will
not be required. For example, as we stated earlier in this report,
DOD‘s risk assessment concludes that significant costs associated with
a more extensive modification to resolve the fin buffet problem may be
required, but the probability is low. DOD arrives at this conclusion
even though the last phase of testing to help characterize the fin
buffet problem has not yet begun. Furthermore, we continue to believe
there is still significant risk that the F/A-22 program will not be
able to begin operational testing as scheduled in August 2003.
Subsequent to our providing the draft of this report to DOD for
comment, OSD‘s operational test and evaluation office issued a report
stating that F/A-22 technical and schedule risk are still high, as is
the risk that operational testing will be further delayed.
While DOD‘s December 2002 risk assessment and certification did provide
an indication that manufacturing inefficiencies and inflation as a
result of lower production rates would increase costs, sufficient
detail was not provided in its risk assessment for us to verify DOD‘s
conclusion. We requested additional detailed information to help us
evaluate and verify the conclusions. However, the information provided
to us was not adequate to verify the conclusions contained in the risk
assessment. Regardless, even with such verification, still needing to
be resolved are the uncertainties to date regarding when development
problems can be fixed and the possibility of finding additional
problems prior to the completion of operational testing. As a result,
we have little confidence that existing problems can be quickly
resolved and will not result in further delays. Our work has shown that
continuing the acquisition of aircraft in increasing quantities when
significant development testing and technical problems remain is risky.
By employing such a strategy, major problems are more likely to be
discovered after the program has begun production when it is either too
late or very costly to correct them.
DOD also provided various technical comments, which we have
incorporated as appropriate. One of these comments related to the total
number of production aircraft to be acquired. The projected number of
production aircraft the Air Force plans to or can actually acquire has
historically been fluid and elusive. For example, the President‘s
budget for fiscal year 2003 reflected plans to acquire 333 production
aircraft, even though the approved program at the time called for
acquiring 295 production aircraft. In its technical comments, DOD
stated that the approved program plan is to acquire 295 aircraft. As a
result of the recent F/A-22 restructuring to cover development cost
increases, the Air Force says that it now plans to acquire 276
aircraft. However, DOD estimates that the cost of production to acquire
these 276 aircraft will be $42.2 billion, which exceeds the current
production cost limit by $5.4 billion.[Footnote 18] Consequently,
unless the production cost limit is raised or substantial cost
reduction plans are achieved, it appears that the number of aircraft
that can actually be purchased will have to be lowered from the 276
planned. This is particularly true if production or development costs-
-or both--continue to rise and no additional funds are provided by
the Congress.
Last month, we recommended in another report that DOD provide Congress
with documentation reflecting the quantity of aircraft that DOD
believes can be procured within the existing production cost
limit.[Footnote 19] DOD‘s explanation in its technical comments to a
draft of this report identifies the likelihood that F/A-22 aircraft
quantities will continue to fluctuate. This makes our recent
recommendation that much more compelling.
Scope and Methodology:
To determine whether the development program is likely to meet
performance goals, we analyzed information on the status of key
performance parameters. We compared performance goals established by
the Under Secretary of Defense for Acquisition, Technology, and
Logistics with the Air Force‘s estimates of performance for completion
of development made in December 2002.
To identify the status of F/A-22 modifications, we collected updated
information on the status of existing aircraft structural problems that
have required aircraft modifications. To determine whether the program
is expected to meet schedule goals, we reviewed program and avionics
schedules and discussed potential changes to these schedules with F/
A-22 program officials. We tracked progress in the flight test program
and evaluated schedule variances in the contractors‘ performance
management system and compared planned milestone accomplishment dates
with actual dates. We tracked technical problems in manufacturing and
assembling the development test aircraft.
To determine whether the program is likely to meet the cost goal, we
examined (1) the extent to which the development program is likely to
be completed within the current cost estimate, (2) the Air Force‘s
plans to fund the program for fiscal year 2003, and (3) the program‘s
funding plan compared to the current cost estimate.
In examining DOD‘s risk assessment, we discussed the various DOD
assumptions and approaches used in the assessment with a program
official who conducted the assessment. We then analyzed the various DOD
assumptions and approaches used to make the assessment conclusions.
In making these determinations, assessments, and identifications, we
required access to current information about test results, performance
estimates, schedule achievements and revisions, costs being incurred,
aircraft modifications, and the program‘s plans for continued
development and initial production. The Air Force and contractors gave
us access to sufficient information to make informed judgments on the
matters covered in this report.
In performing our work, we obtained information or interviewed
officials from the Office of the Secretary of Defense, Washington,
D.C., and the F/A-22 System Program Office, Wright-Patterson Air Force
Base, Ohio. We performed our work from September 2002 through December
2002 in accordance with generally accepted government auditing
standards.
We are sending copies of this report to interested congressional
committees; the Secretary of Defense; the Secretary of the Air Force;
and the Director, Office of Management and Budget. We will also make
copies available to others upon request. In addition, the report will
be available at no charge on the GAO Web site at http://www.gao.gov.
Please contact me at (202) 512-4841 or Catherine Baltzell at (202) 512-
8001 if you or your staff have any questions concerning this report.
Major contributors to this report are listed in appendix III.
Allen Li
Director, Acquisition and Sourcing Management:
Signed by Allen Li
List of Congressional Committees:
The Honorable John Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate:
The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate:
The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives:
The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives:
[End of section]
Appendix I: Estimates of Performance for Key Parameters:
Key performance parameter: Radar cross section; Requirement:
Classified; Current Estimate: Favorable; Margin: Favorable.
Key performance parameter: Supercruise; Requirement: 1.5 Mach; Current
Estimate: 1.68 Mach; Margin: 12% favorable.
Key performance parameter: Acceleration (
