Joint Strike Fighter
Restructuring Places Program on Firmer Footing, but Progress Still Lags Gao ID: GAO-11-325 April 7, 2011The F-35 Lightning II, also known as the Joint Strike Fighter (JSF), is the Department of Defense's (DOD) most costly and ambitious aircraft acquisition, seeking to simultaneously develop and field three aircraft variants for the Air Force, Navy, Marine Corps, and eight international partners. The JSF is critical for recapitalizing tactical air forces and will require a long-term commitment to very large annual funding outlays. The current estimated investment is $382 billion to develop and procure 2,457 aircraft. This report, prepared in response to a congressional mandate in the National Defense Authorization Act for Fiscal Year 2010, discusses (1) program cost and schedule changes and their implications on affordability; (2) progress made during 2010; (3) design and manufacturing maturity; and (4) test plans and progress. GAO's work included analyses of a wide range of program documents and interviews with defense and contractor officials.
DOD continues to substantially restructure the JSF program, taking positive actions that should lead to more achievable and predictable outcomes. Restructuring has consequences--higher up-front development costs, fewer aircraft in the near term, training delays, and extended times for testing and delivering capabilities to warfighters. Total development funding is now $56.4 billion to complete in 2018, a 26 percent increase in cost and a 5-year slip in schedule compared to the current baseline. DOD also reduced procurement quantities by 246 aircraft through 2016, but has not calculated the net effects of restructuring on total procurement costs nor approved a new baseline. Affordability for the U.S. and partners is challenged by a near doubling in average unit prices since program start and higher estimated life-cycle costs. Going forward, the JSF requires unprecedented funding levels in a period of more austere defense budgets. The program had mixed success in 2010, achieving 6 of 12 major goals it established and making varying degrees of progress on the others. Successes included the first flight of the carrier variant, award of a fixed-price aircraft procurement contract, and an accelerated pace in development flight tests that accomplished three times as many flights in 2010 as the previous 3 years combined. However, the program did not deliver as many aircraft to test and training sites as planned and made only a partial release of software capabilities. The short take off and landing variant (STOVL) experienced significant technical problems and did not meet flight test expectations. The Secretary of Defense directed a 2-year period to evaluate and engineer STOVL solutions. After more than 9 years in development and 4 in production, the JSF program has not fully demonstrated that the aircraft design is stable, manufacturing processes are mature, and the system is reliable. Engineering drawings are still being released to the manufacturing floor and design changes continue at higher rates than desired. More changes are expected as testing accelerates. Test and production aircraft cost more and are taking longer to deliver than expected. Manufacturers are improving operations and implemented 8 of 20 recommendations from an expert panel, but have not yet demonstrated a capacity to efficiently produce at higher production rates. Substantial improvements in factory throughput and the global supply chain are needed. Development testing is still early in demonstrating that aircraft will work as intended and meet warfighter requirements. Only about 4 percent of JSF capabilities have been completely verified by flight tests, lab results, or both. Only 3 of the extensive network of 32 ground test labs and simulation models are fully accredited to ensure the fidelity of results. Software development--essential for achieving about 80 percent of the JSF functionality--is significantly behind schedule as it enters its most challenging phase. To sustain a focus on accountability and facilitate tradeoffs within the JSF program, GAO recommends that DOD (1) maintain annual funding levels at current budgeted amounts; (2) establish criteria for evaluating the STOVL's progress and make independent reviews, allowing each variant to proceed at its own pace; and (3) conduct an independent review of the software development and lab accreditation processes. DOD concurred.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Michael J. Sullivan Team: Government Accountability Office: Acquisition and Sourcing Management Phone: (937) 258-7915GAO-11-325, Joint Strike Fighter: Restructuring Places Program on Firmer Footing, but Progress Still Lags
This is the accessible text file for GAO report number GAO-11-325
entitled 'Joint Strike Fighter: Restructuring Places Program on Firmer
Footing, but Progress Still Lags' which was released on April 7, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Committees:
April 2011:
Joint Strike Fighter:
Restructuring Places Program on Firmer Footing, but Progress Still
Lags:
GAO-11-325:
GAO Highlights:
Highlights of GAO-11-325, a report to congressional committees.
Why GAO Did This Study:
The F-35 Lightning II, also known as the Joint Strike Fighter (JSF),
is the Department of Defense‘s (DOD) most costly and ambitious
aircraft acquisition, seeking to simultaneously develop and field
three aircraft variants for the Air Force, Navy, Marine Corps, and
eight international partners. The JSF is critical for recapitalizing
tactical air forces and will require a long-term commitment to very
large annual funding outlays. The current estimated investment is $382
billion to develop and procure 2,457 aircraft.
This report, prepared in response to a congressional mandate in the
National Defense Authorization Act for Fiscal Year 2010, discusses (1)
program cost and schedule changes and their implications on
affordability; (2) progress made during 2010; (3) design and
manufacturing maturity; and (4) test plans and progress. GAO‘s work
included analyses of a wide range of program documents and interviews
with defense and contractor officials.
What GAO Found:
DOD continues to substantially restructure the JSF program, taking
positive actions that should lead to more achievable and predictable
outcomes. Restructuring has consequences”higher up-front development
costs, fewer aircraft in the near term, training delays, and extended
times for testing and delivering capabilities to warfighters. Total
development funding is now $56.4 billion to complete in 2018, a 26
percent increase in cost and a 5-year slip in schedule compared to the
current baseline. DOD also reduced procurement quantities by 246
aircraft through 2016, but has not calculated the net effects of
restructuring on total procurement costs nor approved a new baseline.
Affordability for the U.S. and partners is challenged by a near
doubling in average unit prices since program start and higher
estimated life-cycle costs. Going forward, the JSF requires
unprecedented funding levels in a period of more austere defense
budgets.
The program had mixed success in 2010, achieving 6 of 12 major goals
it established and making varying degrees of progress on the others.
Successes included the first flight of the carrier variant, award of a
fixed-price aircraft procurement contract, and an accelerated pace in
development flight tests that accomplished three times as many flights
in 2010 as the previous 3 years combined. However, the program did not
deliver as many aircraft to test and training sites as planned and
made only a partial release of software capabilities. The short take
off and landing variant (STOVL) experienced significant technical
problems and did not meet flight test expectations. The Secretary of
Defense directed a 2-year period to evaluate and engineer STOVL
solutions.
After more than 9 years in development and 4 in production, the JSF
program has not fully demonstrated that the aircraft design is stable,
manufacturing processes are mature, and the system is reliable.
Engineering drawings are still being released to the manufacturing
floor and design changes continue at higher rates than desired. More
changes are expected as testing accelerates. Test and production
aircraft cost more and are taking longer to deliver than expected.
Manufacturers are improving operations and implemented 8 of 20
recommendations from an expert panel, but have not yet demonstrated a
capacity to efficiently produce at higher production rates.
Substantial improvements in factory throughput and the global supply
chain are needed.
Development testing is still early in demonstrating that aircraft will
work as intended and meet warfighter requirements. Only about 4
percent of JSF capabilities have been completely verified by flight
tests, lab results, or both. Only 3 of the extensive network of 32
ground test labs and simulation models are fully accredited to ensure
the fidelity of results. Software development-”essential for achieving
about 80 percent of the JSF functionality”-is significantly behind
schedule as it enters its most challenging phase.
What GAO Recommends:
To sustain a focus on accountability and facilitate tradeoffs within
the JSF program, GAO recommends that DOD (1) maintain annual funding
levels at current budgeted amounts; (2) establish criteria for
evaluating the STOVL‘s progress and make independent reviews, allowing
each variant to proceed at its own pace; and (3) conduct an
independent review of the software development and lab accreditation
processes. DOD concurred.
View [hyperlink, http://www.gao.gov/products/GAO-11-325] or key
components. For more information, contact Michael J. Sullivan at (202)
512-4841 or sullivanm@gao.gov.
[End of section]
Contents:
Letter:
Background:
Affordability Expectations Are Challenged as JSF Acquisition Costs
Rise and Schedules Slip:
Progress in Achieving the JSF Program's 2010 Goals Was Mixed:
Program Has Still Not Fully Demonstrated a Stable Design and Mature
Manufacturing Processes as It Enters Its Fifth Year of Production:
Testing Has Been Slow and Has Not Demonstrated That the Aircraft Will
Work in Its Intended Environment:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Prior GAO Reports on JSF and DOD Responses and Subsequent
Actions:
Appendix III: Comments from the Department of Defense:
Appendix IV: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: Changes in Reported JSF Program Cost, Quantities, and
Deliveries:
Table 2: Budgeted Development and Procurement Funding and Quantities
for Fiscal Years 2012-2016:
Table 3: JSF Progress on Stated Goals for 2010:
Table 4: Flight Test Performance in 2010:
Figures:
Figure 1: JSF Annual Development and Procurement Funding Requirements
(April 2010 Estimate):
Figure 2: Monthly Design Changes for JSF Aircraft:
Figure 3: JSF Labor Hours for Manufacturing Test Aircraft:
Figure 4: JSF Mean Times between Failure Demonstrated to Date:
Figure 5: Actual JSF Flight Tests Completed through 2010 Compared to
the 2007 Plan:
Figure 6: Test Point Distribution by Variant (2010 Test Plan):
Figure 7: Slips in Delivering Software to Flight Test:
Abbreviations:
CAPE: Cost Assessment and Program Evaluation:
CTOL: conventional takeoff and landing:
CV: carrier variant:
DCMA: Defense Contract Management Agency:
DOD: Department of Defense:
IMRT: Independent Manufacturing Review Team:
JAT: Joint Assessment Team:
JET: Joint Estimating Team:
JSF: Joint Strike Fighter:
OSD: Office of the Secretary of Defense:
STOVL: short takeoff and vertical landing:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
April 7, 2011:
Congressional Committees:
The F-35 Lightning II, also known as the Joint Strike Fighter (JSF) is
the Department of Defense's (DOD) most costly and ambitious aircraft
acquisition, seeking to simultaneously develop and field three
aircraft variants for the Air Force, Navy, Marine Corps, and eight
international partners. The JSF is the core of DOD's long term
tactical aircraft recapitalization plans as it is intended to replace
hundreds of legacy aircraft. Total U.S. planned investment in the JSF
is about $385 billion to develop and acquire 2,457 aircraft through
2035. With that many dollars at stake amidst pressing warfighter
requirements for this new 5th generation capability, the Department
has lately recognized numerous technical, financial, and management
shortcomings and continues to significantly restructure the program,
adding more time and money and making other changes that we support.
GAO has reported on JSF issues for a number of years. Appendix 2
summarizes key findings and recommendations from that body of work.
One recurring theme has been the "single step," or revolutionary,
acquisition strategy the JSF program adopted to develop and acquire
full combat capabilities on a very aggressive, risky schedule with
substantial concurrency, or overlap among development, testing, and
production activities.[Footnote 1] That strategy, coupled with a
management environment that was slow to acknowledge and address
problems with needed changes, are prime contributors to the relatively
poor cost, schedule, and performance outcomes experienced to date. Our
March 2010 report[Footnote 2] discussed additional cost and schedule
pressures, unsatisfactory performance in manufacturing and delivering
aircraft, and concerns about not meeting warfighter requirements on
time and in quantity. We recommended that (1) DOD complete an
independent, comprehensive cost and schedule estimate and establish it
as the official program of record for planning, budgeting, and
congressional reporting purposes and (2) reassess warfighter
requirements to determine the minimum needs (both capabilities and
capacity) that can be achieved within realistic and reasonable
timeframes and, if necessary, defer some capabilities to future
increments. In addition, we suggested that Congress consider requiring
DOD to establish a "system maturity matrix," a management tool to
better measure the program's annual progress toward key objectives to
improve oversight and better inform future budget decisions.
This is our second report required by law[Footnote 3] in which we (1)
evaluate program cost and schedule changes and their implications on
affordability; (2) identify progress made in 2010 against established
goals; (3) assess elements of design stability and manufacturing
maturity and review production results; and (4) report status of
development testing and technical challenges facing the program. To
conduct this work, we evaluated DOD's restructuring actions and
impacts on the program, tracked cost and schedule changes, and
determined factors driving the changes. We reviewed program status
reports, manufacturing data, test plans, and internal DOD analyses. We
discussed results to date and future plans to complete JSF development
and move further into procurement with DOD, JSF, and contractor
officials including members of the independent review teams. We toured
aircraft and engine manufacturing plants, obtained production and
supply performance indicators, and discussed improvements underway
with contractors. We conducted this performance audit from May 2010 to
March 2011 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
The F-35 program is a joint, multinational acquisition to develop and
field an affordable, highly common family of stealthy, next-generation
strike fighter aircraft for the United States Air Force, Marine Corps,
Navy, and eight international partners. The JSF is a single-seat,
single-engine aircraft incorporating low-observable (stealth)
technologies, defensive avionics, advanced sensor fusion,[Footnote 4]
internal and external weapons, and advanced prognostic maintenance
capability. There are three variants. The F-35A conventional takeoff
and landing (CTOL) variant will provide air-to-ground attack
capabilities to replace the Air Force's F-16 Fighting Falcon and the A-
10 Thunderbolt II aircraft, and will complement the F-22A Raptor. The
F-35B short takeoff and vertical landing (STOVL) aircraft will be a
multi-role strike fighter to replace the Marine Corps' F/A-18C/D
Hornet and AV-8B Harrier aircraft. The F-35C carrier-suitable variant
(CV) will provide the Navy and Marine Corps a multi-role, stealthy
strike aircraft to complement the F/A-18E/F Super Hornet. The JSF is
DOD's largest cooperative program.[Footnote 5] Our international
partners are providing about $5.1 billion toward development, and
foreign firms are part of the industrial base producing aircraft.
DOD's funding requirements for the JSF assume economic benefits from
partner purchases in reducing unit costs for U.S. aircraft.
JSF concept development began in November 1996 with a 5-year
competition between contractors to determine the most capable and
affordable preliminary aircraft design. Lockheed Martin won the
competition and the JSF program entered system development and
demonstration in October 2001. Pratt and Whitney is the primary engine
manufacturer, while General Electric has been developing a potential
second source for the engine. System integration efforts and a
preliminary design review then revealed significant airframe weight
problems impacting key performance requirements. In March 2004, DOD
rebaselined the program, adding time and money for development and
delaying key milestones. The Navy and Marine Corps also reduced their
planned procurement by 409 jets, reducing the total U.S. buy to the
current 2,457 quantity. The program was again rebaselined in March
2007 to reflect additional cost increases and schedule slips and the
procurement period was extended by 7 years to 2034 with reduction in
annual quantities.
Because of continuing problems and poor outcomes, the Secretary of
Defense announced another comprehensive restructuring of the JSF
program in February 2010. The restructuring followed an extensive
Department-wide review initiated in 2009 and considered the findings
and recommendations from three independent groups chartered to assess
the program: the Joint Estimating Team (JET) evaluated program
execution and resource requirements; the Independent Manufacturing
Review Team (IMRT) assessed contractor capabilities and plans for
ramping-up and sustaining production at maximum rates; and the Joint
Assessment Team (JAT) reviewed engine costs and affordability
initiatives. Key restructuring actions included adding $2.8 billion
for development, extending flight testing by 13 months, adding flight
test resources (one new test jet and use of 3 production jets),
reduced near-term procurement by 122 aircraft, and review of the
military services' capability need dates. The Under Secretary of
Defense for Acquisition, Technology and Logistics stated that the
department-wide review would continue under new program management and
cited 2010 as a critical year for assessing progress against the new
plans and the expected delivery of all test aircraft, completion of
hundreds of test flights, and meeting other key milestones. We
supported these actions in our March 2010 report and subsequent
testimonies.[Footnote 6] We noted the likelihood of additional cost
growth and schedule extensions as the restructuring continues.
In March 2010, the Department declared that the program experienced a
breach of the critical cost growth statutory thresholds.[Footnote 7]
The Department subsequently certified to Congress in June 2010 that
the JSF program should continue.[Footnote 8] Table 1 summarizes the
evolution of JSF cost and schedule estimates at key junctures in its
acquisition history through the current Nunn-McCurdy certification.
Since then, in January 2011, the Secretary of Defense announced
additional development cost increases and further changes consequent
to the ongoing restructure, but has not yet established a new approved
acquisition program baseline.
Table 1: Changes in Reported JSF Program Cost, Quantities, and
Deliveries:
Expected quantities:
Development quantities;
October 2001 (system development start): 14;
December 2003 (2004 replan): 14;
March 2007 (approved baseline): 15;
April 2010 (initial program restructure): 14;
June 2010 (Nunn-McCurdy): 14.
Procurement quantities (U.S. only);
October 2001 (system development start): 2,852;
December 2003 (2004 replan): 2,443;
March 2007 (approved baseline): 2,443;
April 2010 (initial program restructure): 2,443;
June 2010 (Nunn-McCurdy): 2,443.
Total quantities;
October 2001 (system development start): 2,866;
December 2003 (2004 replan): 2,457;
March 2007 (approved baseline): 2,458;
April 2010 (initial program restructure): 2,457;
June 2010 (Nunn-McCurdy): 2,457.
Cost estimates (then-year dollars in billions):
Development;
October 2001 (system development start): $34.4;
December 2003 (2004 replan): $44.8;
March 2007 (approved baseline): $44.8;
April 2010 (initial program restructure): $50.2;
June 2010 (Nunn-McCurdy): $51.8.
Procurement;
October 2001 (system development start): $196.6;
December 2003 (2004 replan): $199.8;
March 2007 (approved baseline): $231.7;
April 2010 (initial program restructure): $277.5;
June 2010 (Nunn-McCurdy): $325.1.
Military construction;
October 2001 (system development start): $2.0;
December 2003 (2004 replan): $0.2;
March 2007 (approved baseline): $2.0;
April 2010 (initial program restructure): $0.6;
June 2010 (Nunn-McCurdy): $5.6.
Total program acquisition;
October 2001 (system development start): $233.0;
December 2003 (2004 replan): $244.8;
March 2007 (approved baseline): $278.5;
April 2010 (initial program restructure): $328.3;
June 2010 (Nunn-McCurdy): $382.5.
Unit cost estimates (then-year dollars in millions):
Program acquisition;
October 2001 (system development start): $81;
December 2003 (2004 replan): $100;
March 2007 (approved baseline): $113;
April 2010 (initial program restructure): $134;
June 2010 (Nunn-McCurdy): $156.
Average procurement;
October 2001 (system development start): $69;
December 2003 (2004 replan): $82;
March 2007 (approved baseline): $95;
April 2010 (initial program restructure): $114;
June 2010 (Nunn-McCurdy): $133.
Estimated delivery and production dates:
First operational aircraft delivery;
October 2001 (system development start): 2008;
December 2003 (2004 replan): 2009;
March 2007 (approved baseline): 2010;
April 2010 (initial program restructure): 2010;
June 2010 (Nunn-McCurdy): 2010.
Initial operational capability;
October 2001 (system development start): 2010-2012;
December 2003 (2004 replan): 2012-2013;
March 2007 (approved baseline): 2012-2015;
April 2010 (initial program restructure): 2012-2016;
June 2010 (Nunn-McCurdy): TBD.
Full-rate production;
October 2001 (system development start): 2012;
December 2003 (2004 replan): 2013;
March 2007 (approved baseline): 2013;
April 2010 (initial program restructure): 2016;
June 2010 (Nunn-McCurdy): 2016.
Source: GAO analysis and DOD data.
Note: Does not reflect cost and schedule effects from additional
restructuring actions announced after June 2010.
[End of table]
Affordability Expectations Are Challenged as JSF Acquisition Costs
Rise and Schedules Slip:
Ongoing JSF restructuring continues to add more cost and time for
developing, testing, and delivering aircraft to the warfighter. These
actions, effectively implemented, should result in more predictable
and achievable program outcomes, but restructuring comes with
consequences--higher upfront development costs, fewer aircraft
received in the near term, training delays, and extended times for
testing and delivering the capabilities required by the warfighter.
Affordability for the U.S. and our allies is challenged because unit
prices are about double what they were at program start and with new
forecasts that the aircraft may cost substantially more to operate and
maintain over the life cycle than the legacy aircraft they replace.
Going forward, the program requires unprecedented levels of funding in
a period of more austere defense funding. Defense leaders stated that
the JSF program lost its focus on affordability and that restoring the
focus is paramount to improving program outcomes.
Additional Cost Increases and Schedule Delays Are Expected as Program
Restructuring Continues:
Defense leadership continued to restructure the JSF program following
the Nunn-McCurdy certification. In January 2011, the Secretary of
Defense directed additional changes, stemming in large part from the
results of a comprehensive technical baseline review under new
government and contractor management. Key program changes (1) added
$4.6 billion to the development program through completion for a total
development program estimate of $56.4 billion (an increase of 26
percent against the current baseline and 64 percent from the original
baseline at program start); (2) extended the development test period
to 2016 (a 4-year slip from the current baseline); and (3) reduced
near-term procurement quantities by 124 aircraft in addition to the
122 aircraft cut announced in February 2010; and (4) lowered the
annual rate of increase for boosting future production.
Because of the lingering technical issues on the STOVL, the most
complex variant, the Secretary decoupled STOVL flight tests from the
combined test plan and scaled back STOVL production to only 3 in
fiscal year 2011 and to 6 per year for fiscal years 2012 and 2013.
This represents a total cut of 37 STOVL aircraft during this 3-year
period compared to the fiscal year 2011 budget plans. In announcing
these changes, the Secretary also noted the STOVL's significant
testing problems which include lift fan engine deficiencies, and poor
durability test results, which could require redesigns and add weight
to aircraft's structure and propulsion system. While the Secretary
decoupled STOVL from the flight test program, STOVL was not further
separated from the rest of the JSF program for management and
reporting activities. It remains a part of the combined JSF program
for milestone decisions and cost, schedule, and performance reporting.
Resolving STOVL problems and moving forward at an affordable cost is
essential to the Marine Corps' future plans, which depend upon
acquiring the STOVL in quantity to directly accompany, protect, and
provide firepower to its ground expeditionary forces.
The recently submitted fiscal year 2012 Defense Budget reflects the
financial impacts from restructuring actions through 2016. Compared to
estimates in the fiscal year 2010 future years defense program for the
same 5-year period, the Department increased development funding by
$7.7 billion and decreased procurement funding by $8.4 billion
reflecting plans to buy fewer aircraft. Table 2 summarizes the revised
development and procurement funding requirements and annual quantities
following the Secretary's reductions. Even after decreasing near-term
quantities and lowering the ramp rate, JSF procurement still rapidly
increases. Annual funding levels more than double and quantities more
than triple during this period. These numbers do not include the
additional orders expected from the international partners.
Table 2: Budgeted Development and Procurement Funding and Quantities
for Fiscal Years 2012-2016:
(Dollars in billions).
Development Funding: Air Force (CTOL);
2012: $1.4;
2013: $1.2;
2014: $0.9;
2015: $0.6;
2016: $0.4;
Total: $4.5.
Development Funding: Navy (CV);
2012: $0.7;
2013: $0.7;
2014: $0.7;
2015: $0.6;
2016: $0.5;
Total: $3.2.
Development Funding: Marine Corps (STOVL);
2012: $0.7;
2013: $0.7;
2014: $0.7;
2015: $0.6;
2016: $0.5;
Total: $3.2.
Development Funding: U.S. total development:
2012: $2.7
2013: $2.7
2014: $2.3
2015: $1.8
2016: $1.3
Total: $10.8.
Procurement Funding: Air Force (CTOL);
2012: $3.8;
2013: $4.1;
2014: $5.6;
2015: $6.5;
2016: $8.5;
Total: $28.5.
Procurement Funding: Navy (CV);
2012: $1.8;
2013: $2.5;
2014: $2.8;
2015: $3.3;
2016: $2.9;
Total: $13.2.
Procurement Funding: Marine Corps (STOVL);
2012: $1.3;
2013: $1.3;
2014: $1.4;
2015: $2.0;
2016: $2.9;
Total: $9.0.
Procurement Funding: U.S. total procurement;
2012: 6.9;
2013: $7.9;
2014: $9.8;
2015: $11.8;
2016: $14.3;
Total: $50.7.
Procurement Quantities: Air Force (CTOL);
2012: 19;
2013: 24;
2014: 40;
2015: 50;
2016: 70;
Total: 203.
Procurement Quantities: Navy (CV);
2012: 7;
2013: 12;
2014: 14;
2015: 19;
2016: 20;
Total: 72.
Procurement Quantities: Marine Corps (STOVL);
2012: 6;
2013: 6;
2014: 8;
2015: 12;
2016: 18;
Total: 50.
Procurement Quantities: U.S. total quantity;
2012: 32;
2013: 42;
2014: 62;
2015: 81;
2016: 108;
Total: 325.
Source: GAO analysis of fiscal year 2012 President's Budget.
Note: Numbers may not add due to rounding.
[End of table]
Additional changes to cost and schedule are likely as restructuring
continues. At the time of this report, the Secretary had not yet
granted new milestone B approval nor approved a new acquisition
program baseline. Originally planned for November 2010, program
officials now expect the new acquisition program baseline in late
2011. Also, cost analysts are still revising procurement funding
requirements for the period fiscal year 2017 through completion of
procurement in 2035. Accordingly, the net effect of reducing near-term
procurement quantities and deferring these aircraft to future years is
uncertain and depends upon the assumptions made about future unit
prices, annual quantities, and inflation. We expect total procurement
costs will be somewhat higher than the estimate submitted in the Nunn-
McCurdy certification (refer to table 1). Reduced quantities and use
of production aircraft in testing will also limit training activities
for the near-term and delay deliveries of new capabilities to the
warfighters. Officials now forecast that the completion of system
development, completion of initial operational testing, and the full
rate production decision will extend into 2018. This represents slips
of about 5 years in these important milestones against the current
program baseline approved in 2007.
The military services are evaluating the impacts from restructuring on
their initial operational capability (IOC) milestones, the critical
need dates when the warfighter must have in place the first increment
of operational forces available for combat. In response to the initial
set of restructuring actions, the Air Force and Navy tentatively
extended these milestones to 2016, but the Marine Corps slightly
adjusted its IOC date by 9 months to December 2012. It is all but
certain that the Marine Corps will be delaying its IOC date in the
wake of the Secretary's STOVL actions. Air Force and Navy dates may
also be adjusted to reflect the newest developments.
Program Requires Unprecedented Funding Levels Well into the Future:
Affordability--both in terms of the investment costs to acquire the
JSF and the continuing costs to operate and maintain it over the life-
cycle--is at risk. A key tenet of the JSF program from its inception
has been to deliver an affordable, highly common fifth generation
[Footnote 9] aircraft that could be acquired by the warfighters in
large numbers. Rising aircraft prices erode buying power and make it
difficult for the U.S. and its allies to buy as many aircraft as
planned. Quantity reductions could drive additional price increases
for future aircraft. Further, while the Department is still refining
cost projections for operating and supporting future JSF fleets, cost
forecasts have increased as the program matures and more data becomes
available. Current JSF life-cycle cost estimates are considerably
higher than the legacy aircraft it will replace; this has major
implications for future demands on military operating and support
budgets and plans for recapitalizing fighter forces. Defense
leadership stated that the JSF program lost focus on affordability and
that restoring and maintaining that focus is paramount to improving
program outcomes.
In light of continued cost growth, the program places unprecedented
demands for funding in the defense budget--an annual average of almost
$11 billion for the next two decades. (This and other data in this
paragraph reflect the fiscal year 2011 budget submission.) During the
peak years of production, the average annual requirement is about $13
billion. The JSF will have to annually compete with other defense and
nondefense priorities for the shrinking discretionary federal dollar
amid continued concerns about the national debt and long term fiscal
pressures. The JSF program has received more than $56 billion through
fiscal year 2010. To complete the acquisition program as currently
planned, another $272 billion will be required from 2011 through 2035.
Figure 1 illustrates the annual funding requirements outlined in the
program's Selected Acquisition Report released in April 2010. These
funding levels do not reflect the additional funding increases in the
Nunn-McCurdy certification and the Secretary's recent actions. DOD is
in the process of establishing a new acquisition program baseline
which will likely project even higher funding requirements.
Figure 1: JSF Annual Development and Procurement Funding Requirements
(April 2010 Estimate):
[Refer to PDF for image: vertical bar graph]
Fiscal year: 1994;
Funding requirement: $0.03 billion.
Fiscal year: 1995;
Funding requirement: $0.18 billion.
Fiscal year: 1996;
Funding requirement: $0.20 billion.
Fiscal year: 1997;
Funding requirement: $0.63 billion.
Fiscal year: 1998;
Funding requirement: $0.99 billion.
Fiscal year: 1999;
Funding requirement: $0.98 billion.
Fiscal year: 2000;
Funding requirement: $0.52 billion.
Fiscal year: 2001;
Funding requirement: $0.68 billion.
Fiscal year: 2002;
Funding requirement: $1.74 billion.
Fiscal year: 2003;
Funding requirement: $3.68 billion.
Fiscal year: 2004;
Funding requirement: $4.66 billion.
Fiscal year: 2005;
Funding requirement: $4.93 billion.
Fiscal year: 2006;
Funding requirement: $5.37 billion.
Fiscal year: 2007;
Funding requirement: $5.60 billion.
Fiscal year: 2008;
Funding requirement: $7.14 billion.
Fiscal year: 2009;
Funding requirement: $7.31 billion.
Fiscal year: 2010;
Funding requirement: $11.41 billion.
Through FY 2010: $56.1 billion.
Fiscal year: 2011;
Funding requirement: $11.90 billion.
Fiscal year: 2012;
Funding requirement: $12.11 billion.
Fiscal year: 2013;
Funding requirement: $14.12 billion.
Fiscal year: 2014;
Funding requirement: $14.33 billion.
Fiscal year: 2015;
Funding requirement: $14.66 billion.
Fiscal year: 2016;
Funding requirement: $13.52 billion.
Fiscal year: 2017;
Funding requirement: $13.46 billion.
Fiscal year: 2018;
Funding requirement: $13.03 billion.
Fiscal year: 2019;
Funding requirement: $12.91 billion.
Fiscal year: 2020;
Funding requirement: $12.92 billion.
Fiscal year: 2021;
Funding requirement: $13.02 billion.
Fiscal year: 2022;
Funding requirement: $13.05 billion.
Fiscal year: 2023;
Funding requirement: $12.87 billion.
Fiscal year: 2024;
Funding requirement: $11.67 billion.
Fiscal year: 2025;
Funding requirement: $10.51 billion.
Fiscal year: 2026;
Funding requirement: $10.16 billion.
Fiscal year: 2027;
Funding requirement: $8.21 billion.
Fiscal year: 2028;
Funding requirement: $8.33 billion.
Fiscal year: 2029;
Funding requirement: $8.33 billion.
Fiscal year: 2030;
Funding requirement: $8.20 billion.
Fiscal year: 2031;
Funding requirement: $8.25 billion.
Fiscal year: 2032;
Funding requirement: $8.36 billion.
Fiscal year: 2033;
Funding requirement: $8.46 billion.
Fiscal year: 2034;
Funding requirement: $7.85 billion.
Fiscal year: 2035;
Funding requirement: $1.94 billion.
FY 2011 through FY 2035: $272.2 billion.
Source: GAO analysis of DOD data.
[End of figure]
The JSF is the linchpin in DOD's tactical aircraft recapitalization
plans, replacing hundreds of legacy aircraft. Because of its sheer
size and high priority within the Department, even relatively modest
cost growth on the JSF can require the sourcing of billions of
additional funds, largely from other programs in DOD's acquisition
portfolio. On the other hand, slips in JSF schedules, cuts in annual
procurement quantities, and deferred delivery of operational aircraft
can require additional monies be spent on legacy aircraft, postponing
planned retirements and sustaining fleets for longer periods of time.
To mitigate projected shortfalls in tactical aircraft inventories due
to JSF perturbations, the Navy recently procured additional F/A-18E/F
Super Hornets and both the Navy and Air Force are funding service life
extension programs and adding new capabilities to legacy aircraft.
Furthermore, international partners' participation in the JSF program
is very important to maintaining affordability for all buyers. DOD
budget plans expect the partners to buy 223 aircraft costing $24.1
billion during the fiscal year 2011-2016 period. However, JSF cost
increases, schedule delays, and internal issues may result in reduced
or deferred foreign buys. Some partners have already signaled plans to
buy fewer aircraft, a different mix of aircraft, or defer purchases to
later years. On the positive side, other countries have expressed
interest in acquiring the JSF. Decisions made by the international
community and its impact on JSF affordability are largely beyond the
program's direct control. However, improving JSF program outcomes to
lower costs and reassure buyers is within DOD's and the contractors'
control.
The eight international partners have important stakes in the JSF
program, having provided about $5 billion in development funding,
being counted upon to procure hundreds of aircraft, and expecting
their industries to receive a significant portion of JSF manufacturing
and supply business. DOD's procurement cost estimates provided to the
Congress have long assumed that the eight partners will buy at least
730 JSF aircraft. Unit prices for U.S. quantities assume the economic
benefit of these purchases. If fewer are sold overseas, the Air Force,
Navy and Marine Corps (and the American taxpayer) may have to pay
more. Unit costs can be expected to increase with smaller purchases
due to diminished manufacturing economies of scale and because fixed
costs have to be spread over fewer aircraft.
Maintaining a strong focus on affordability necessitates having
reliable and complete cost data that provides accurate accounting
reports, identifies potential cost and schedule problems early, and
produces sound estimates of the cost to complete work. The JSF program
has been hampered in this regard because, for at least the past three
years, the prime contractor has not had an adequate and disciplined
earned value management (EVM) system in place to effectively track
costs and control schedule. The prime contractor was found deficient
in meeting 19 of 32 required guidelines, calling into question its
ability to manage the escalating costs and complex scheduling of the
JSF program. In October 2010, the Defense Contract Management Agency
(DCMA) withdrew the determination of compliance for the prime
contractor's EVM system due to longstanding non-compliance issues with
specific guidelines that underpin a sound system.[Footnote 10] To
address these shortcomings, the contractor is developing new
processes, tools, training, and enforcement in order to achieve a
fully integrated and automated EVM system. Officials will reassess the
earned value system by March 2012--more than four years after these
problems were first discovered to see if modifications needed have
been made.
EVM is an important, established tool that can provide objective
product status reports. DOD requires its use by major defense
suppliers to facilitate good insight and oversight of the expenditure
of government dollars, thereby improving both affordability and
accountability. JSF is DOD's largest acquisition ever, so it is
particularly critical to improve and certify the contractor's EVM
system as expeditiously as possible. If not improved, inaccurate
performance reports and late notice of cost overruns will likely
continue to hinder timely decision making and corrective actions.
Strong leadership and a shared vision among stakeholders are critical
to implementing EVM effectively.
Progress in Achieving the JSF Program's 2010 Goals Was Mixed:
The JSF program established 12 clearly-stated goals in testing,
contracting, and manufacturing for completion in calendar year 2010.
It had mixed success, achieving 6 goals and making varying degrees of
progress on the other 6. For example, the program exceeded its goal
for the number of development flight tests but did not deliver as many
test and production aircraft as planned. Also, the program awarded its
first fixed-price contract on its fourth lot of aircraft production,
but did not award the fixed-price engine contract in 2010 as planned.
Table 3 summarizes JSF goals and accomplishments for 2010.
Table 3: JSF Progress on Stated Goals for 2010:
Key event: Complete 400 development flight tests;
Achieved in 2010: Yes;
Status: Completed 410 test flights.
Key event: First vertical landing of STOVL variant;
Achieved in 2010: Yes;
Status: Achieved March 2010.
Key event: Carrier variant first flight;
Achieved in 2010: Yes;
Status: Achieved June 2010.
Key event: Autonomic logistic information system is operational;
Achieved in 2010: Yes;
Status: Began limited operations July 2010.
Key event: Training for 125 maintenance personnel completed;
Achieved in 2010: Yes;
Status: Trained 138 maintenance personnel.
Key event: Award contract for fourth aircraft production lot;
Achieved in 2010: Yes;
Status: Awarded contract November 2010.
Key event: Eleven test aircraft delivered to test sites;
Achieved in 2010: No;
Status: Delivered eight aircraft.
Key event: Flight test rate of 12 flights per aircraft per month
demonstrated;
Achieved in 2010: No;
Status: Achieved flight test rate of 2 to 8 per month.
Key event: At least 3 aircraft delivered to Eglin Air Force Base;
Achieved in 2010: No;
Status: None delivered, expected mid-2011.
Key event: Begin flight training operations at Eglin Air Force Base;
Achieved in 2010: No;
Status: Expected September 2011.
Key event: Block 1.0 software delivered to flight test;
Achieved in 2010: No;
Status: Delivered limited capability November 2010 with full
capability expected June 2011.
Key event: Award contract for fourth engine production lot;
Achieved in 2010: No;
Status: Expected May 2011.
Source: GAO analysis of DOD data.
[End of table]
The development flight test program significantly ramped up operations
in 2010, accomplishing three times as many test flights as the
previous 3 years combined. Table 4 summarizes actual flights, hours,
and test points[Footnote 11] flown by each variant compared to the
2010 plan.
Table 4: Flight Test Performance in 2010:
Flight tests: Actual;
Conventional takeoff and landing variant: 171;
Short takeoff and vertical landing variant: 212;
Carrier variant: 27;
Total: 410.
Flight tests: Planned;
Conventional takeoff and landing variant: 112;
Short takeoff and vertical landing variant: 251;
Carrier variant: 31;
Total: 394.
Flight tests: Difference;
Conventional takeoff and landing variant: 59;
Short takeoff and vertical landing variant: (39);
Carrier variant: (4);
Total: 16.
Flight test hours: Actual;
Conventional takeoff and landing variant: 290;
Short takeoff and vertical landing variant: 286;
Carrier variant: 41;
Total: 617.
Flight test hours: Planned;
Conventional takeoff and landing variant: 202;
Short takeoff and vertical landing variant: 409;
Carrier variant: 56;
Total: 667.
Flight test hours: Difference;
Conventional takeoff and landing variant: 88;
Short takeoff and vertical landing variant: (123);
Carrier variant: (15);
Total: (50).
Flight test points flown: Actual;
Conventional takeoff and landing variant: 1373;
Short takeoff and vertical landing variant: 1924;
Carrier variant: 496;
Total: 3793.
Flight test points flown: Planned;
Conventional takeoff and landing variant: 1064;
Short takeoff and vertical landing variant: 2438;
Carrier variant: 270;
Total: 3772.
Flight test points flown: Difference;
Conventional takeoff and landing variant: 309;
Short takeoff and vertical landing variant: (514);
Carrier variant: 226;
Total: 21.
Source: GAO analysis of DOD data.
[End of table]
Although still hampered as in prior years by the late delivery of test
aircraft, flight tests substantially increased in volume and pace at
the two main government test sites--Edwards Air Force Base,
California, for CTOL tests and Patuxent River Naval Air Station for
STOVL and CV testing. The CTOL variant significantly exceeded plans
while initial testing of the carrier variant was judged satisfactory,
below plans for the number and hours of flight but ahead on test
points flown. The STOVL, however, substantially under-performed in
flight tests and experienced significant technical issues unique to
this variant that could add to its weight and cost. The STOVL's test
problems were a major factor in the heightened scrutiny and two-year
probation period directed by the Secretary to engineer solutions,
assess impacts, and inform a future decision as to whether and how to
proceed with this variant.
Evaluating annual performance against stated goals can be an effective
tool that facilitates oversight by the Congress and defense leadership
and useful for informing future budget decisions. In our 2010 report,
we suggested that Congress consider requiring DOD to establish a
system maturity matrix to better measure the program's progress in
maturing the weapon system and providing evidence to support budget
decisions.[Footnote 12] The Ike Skelton National Defense Authorization
Act for Fiscal Year 2011[Footnote 13] established this requirement and
we understand the Department is working on its implementation. We
believe this tool and process will improve oversight and budgeting,
holding people accountable for meeting interim objectives and, for
objectives not met, providing criteria and a forum for evaluating
reasons why and what should be done.
Program Has Still Not Fully Demonstrated a Stable Design and Mature
Manufacturing Processes as It Enters Its Fifth Year of Production:
After completing 9 years of system development and 4 years of
overlapping production activities, the JSF program has been slow to
gain adequate knowledge that its design and manufacturing process are
fully mature and ready for greater levels of annual production. The
JSF program still lags in achieving critical indicators of success
expected from well-performing acquisition programs. Specifically, the
program has not yet stabilized aircraft designs--engineering changes
continue at higher than expected rates long after critical design
reviews and well into procurement, and more changes are expected as
testing accelerates. Also, the aircraft and engine manufacturing
processes are not yet mature enough to support efficient production at
higher annual rates and substantial improvements in the global supply
network are needed. Further, the growth in aircraft reliability--
crucial for managing life-cycle costs--has not been demonstrated to
the extent planned by this time.
The Program Has Not Yet Stabilized Aircraft Designs:
Engineering drawings released since design reviews and the number and
rate of design changes are excessive compared to plans and best
practices. Critical design reviews were completed on the three
aircraft variants in 2006 and 2007 and the designs declared mature,
but the program continues to experience numerous changes. Since 2007,
the program has produced 20,000 more engineering drawings, a 50-
percent increase in total drawings and about 5 times more than best
practices suggest. In addition, changes to drawings have not decreased
and leveled off as planned. Figure 2 tracks and compares monthly
design changes and future forecasts against contractor plans in 2007.
The monthly rate in 2009 and 2010 was higher than expected and the
program now anticipates more changes over a longer period of time--
about 10,000 more changes through January 2016. We expect this number
to go up given new forecasts for additional testing and extension of
system development until 2018.
Figure 2: Monthly Design Changes for JSF Aircraft:
[Refer to PDF for image: combination vertical bar and line graph]
2006:
January;
Anticipated design changes (2007 plan): 264;
Design changes (actual): 251.
February;
Anticipated design changes (2007 plan): 323;
Design changes (actual): 283.
March;
Anticipated design changes (2007 plan): 341;
Design changes (actual): 362.
April;
Anticipated design changes (2007 plan): 374;
Design changes (actual): 281.
May;
Anticipated design changes (2007 plan): 403;
Design changes (actual): 373.
June;
Anticipated design changes (2007 plan): 378;
Design changes (actual): 447.
July;
Anticipated design changes (2007 plan): 411;
Design changes (actual): 347.
August;
Anticipated design changes (2007 plan): 513;
Design changes (actual): 520.
September;
Anticipated design changes (2007 plan): 575;
Design changes (actual): 385.
October;
Anticipated design changes (2007 plan): 490;
Design changes (actual): 515.
November;
Anticipated design changes (2007 plan): 495;
Design changes (actual): 505.
December;
Anticipated design changes (2007 plan): 547;
Design changes (actual): 378.
2007:
January;
Anticipated design changes (2007 plan): 653;
Design changes (actual): 620.
February;
Anticipated design changes (2007 plan): 579;
Design changes (actual): 567.
March;
Anticipated design changes (2007 plan): 576;
Design changes (actual): 598.
April;
Anticipated design changes (2007 plan): 595;
Design changes (actual): 647.
May;
Anticipated design changes (2007 plan): 663;
Design changes (actual): 697.
June;
Anticipated design changes (2007 plan): 702;
Design changes (actual): 634.
July;
Anticipated design changes (2007 plan): 747;
Design changes (actual): 681.
August;
Anticipated design changes (2007 plan): 797;
Design changes (actual): 739.
September;
Anticipated design changes (2007 plan): 775;
Design changes (actual): 744.
October;
Anticipated design changes (2007 plan): 730;
Design changes (actual): 876.
November;
Anticipated design changes (2007 plan): 761;
Design changes (actual): 678.
December;
Anticipated design changes (2007 plan): 814;
Design changes (actual): 578.
2008:
January;
Anticipated design changes (2007 plan): 853;
Design changes (actual): 606.
February;
Anticipated design changes (2007 plan): 828;
Design changes (actual): 594.
March;
Anticipated design changes (2007 plan): 759;
Design changes (actual): 645.
April;
Anticipated design changes (2007 plan): 675;
Design changes (actual): 560.
May;
Anticipated design changes (2007 plan): 624;
Design changes (actual): 387.
June;
Anticipated design changes (2007 plan): 599;
Design changes (actual): 459.
July;
Anticipated design changes (2007 plan): 606;
Design changes (actual): 605.
August;
Anticipated design changes (2007 plan): 578;
Design changes (actual): 590.
September;
Anticipated design changes (2007 plan): 477;
Design changes (actual): 590.
October;
Anticipated design changes (2007 plan): 428;
Design changes (actual): 621.
November;
Anticipated design changes (2007 plan): 422;
Design changes (actual): 527.
December;
Anticipated design changes (2007 plan): 421;
Design changes (actual): 508.
2009:
January;
Anticipated design changes (2007 plan): 389;
Design changes (actual): 573.
February;
Anticipated design changes (2007 plan): 394;
Design changes (actual): 655.
March;
Anticipated design changes (2007 plan): 328;
Design changes (actual): 716.
April;
Anticipated design changes (2007 plan): 346;
Design changes (actual): 629.
May;
Anticipated design changes (2007 plan): 330;
Design changes (actual): 522.
June;
Anticipated design changes (2007 plan): 306;
Design changes (actual): 568.
July;
Anticipated design changes (2007 plan): 288;
Design changes (actual): 493.
August;
Anticipated design changes (2007 plan): 291;
Design changes (actual): 492.
September;
Anticipated design changes (2007 plan): 301;
Design changes (actual): 482.
October;
Anticipated design changes (2007 plan): 282;
Design changes (actual): 480.
November;
Anticipated design changes (2007 plan): 266;
Design changes (actual): 415.
December;
Anticipated design changes (2007 plan): 259;
Design changes (actual): 302.
2010:
January;
Anticipated design changes (2007 plan): 260;
Design changes (actual): 342.
February;
Anticipated design changes (2007 plan): 259;
Design changes (actual): 323.
March;
Anticipated design changes (2007 plan): 270;
Design changes (actual): 363.
April;
Anticipated design changes (2007 plan): 258;
Design changes (actual): 376.
May;
Anticipated design changes (2007 plan): 253;
Design changes (actual): 353.
June;
Anticipated design changes (2007 plan): 254;
Design changes (actual): 356.
July;
Anticipated design changes (2007 plan): 257;
Design changes (actual): 314.
Program now anticipates 10,000 more design changes than anticipated in
2007.
August;
Anticipated design changes (2007 plan): 242;
Design changes (actual): 302;
Anticipated future design changes (2010 plan): 233.
September;
Anticipated design changes (2007 plan): 244;
Anticipated future design changes (2010 plan): 230.
October;
Anticipated design changes (2007 plan): 226;
Anticipated future design changes (2010 plan): 235.
November;
Anticipated design changes (2007 plan): 216;
Anticipated future design changes (2010 plan): 233.
December;
Anticipated design changes (2007 plan): 210;
Anticipated future design changes (2010 plan): 229.
2011:
January;
Anticipated design changes (2007 plan): 202;
Anticipated future design changes (2010 plan): 231.
February;
Anticipated design changes (2007 plan): 193;
Anticipated future design changes (2010 plan): 245.
March;
Anticipated design changes (2007 plan): 192;
Anticipated future design changes (2010 plan): 245.
April;
Anticipated design changes (2007 plan): 178;
Anticipated future design changes (2010 plan): 241.
May;
Anticipated design changes (2007 plan): 168;
Anticipated future design changes (2010 plan): 254.
June;
Anticipated design changes (2007 plan): 155;
Anticipated future design changes (2010 plan): 249.
July;
Anticipated design changes (2007 plan): 152;
Anticipated future design changes (2010 plan): 241.
August;
Anticipated design changes (2007 plan): 145;
Anticipated future design changes (2010 plan): 256.
September;
Anticipated design changes (2007 plan): 142;
Anticipated future design changes (2010 plan): 241.
October;
Anticipated design changes (2007 plan): 142;
Anticipated future design changes (2010 plan): 253.
November;
Anticipated design changes (2007 plan): 138;
Anticipated future design changes (2010 plan): 247.
December;
Anticipated design changes (2007 plan): 138;
Anticipated future design changes (2010 plan): 238.
2012:
January;
Anticipated design changes (2007 plan): 144;
Anticipated future design changes (2010 plan): 245.
February;
Anticipated design changes (2007 plan): 151;
Anticipated future design changes (2010 plan): 242.
March;
Anticipated design changes (2007 plan): 137;
Anticipated future design changes (2010 plan): 245.
April;
Anticipated design changes (2007 plan): 140;
Anticipated future design changes (2010 plan): 243.
May;
Anticipated design changes (2007 plan): 137;
Anticipated future design changes (2010 plan): 245.
June;
Anticipated design changes (2007 plan): 138;
Anticipated future design changes (2010 plan): 240.
July;
Anticipated design changes (2007 plan): 140;
Anticipated future design changes (2010 plan): 240.
August;
Anticipated design changes (2007 plan): 137;
Anticipated future design changes (2010 plan): 241.
September;
Anticipated design changes (2007 plan): 129;
Anticipated future design changes (2010 plan): 260.
October;
Anticipated design changes (2007 plan): 124;
Anticipated future design changes (2010 plan): 244.
November;
Anticipated design changes (2007 plan): 121;
Anticipated future design changes (2010 plan): 257.
December;
Anticipated design changes (2007 plan): 105;
Anticipated future design changes (2010 plan): 237.
2013:
January;
Anticipated design changes (2007 plan): 72;
Anticipated future design changes (2010 plan): 235.
February;
Anticipated design changes (2007 plan): 63;
Anticipated future design changes (2010 plan): 235.
March;
Anticipated design changes (2007 plan): 55;
Anticipated future design changes (2010 plan): 231.
April;
Anticipated design changes (2007 plan): 38;
Anticipated future design changes (2010 plan): 233.
May;
Anticipated design changes (2007 plan): 39;
Anticipated future design changes (2010 plan): 213.
June;
Anticipated design changes (2007 plan): 38;
Anticipated future design changes (2010 plan): 229.
July;
Anticipated design changes (2007 plan): 30;
Anticipated future design changes (2010 plan): 227.
August;
Anticipated design changes (2007 plan): 30;
Anticipated future design changes (2010 plan): 251.
September;
Anticipated design changes (2007 plan): 30;
Anticipated future design changes (2010 plan): 242.
October;
Anticipated design changes (2007 plan): 30;
Anticipated future design changes (2010 plan): 262.
November;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 260.
December;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 268.
2014:
January;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 271.
February;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 251.
March;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 249.
April;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 251.
May;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 247.
June;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 224.
July;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 221.
August;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 217.
September;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 204.
October;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 197.
November;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 182.
December;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 169.
2015:
January;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 162.
February;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 162.
March;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 142.
April;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 131.
May;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 114.
June;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 94.
July;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 80.
August;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 66.
September;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 48.
October;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 33.
November;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 20.
December;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 8.
2016:
January;
Anticipated design changes (2007 plan): 0;
Anticipated future design changes (2010 plan): 0.
Source: GAO analysis of DOD data.
[End of figure]
A key indicator of a product's maturity is the stability of its
design. The number of engineering drawings released and subsequent
changes provide indicators of the maturity of the design. Engineering
drawings are critical because they communicate to the manufacturer and
suppliers how the part functions, what it looks like, and what
materials and critical processes are used to build the product. Best
practices suggest 90 percent of a product's engineering drawings be
released by the critical design review. Late engineering drawings and
high levels of changes often indicate a lack of understanding about
the design, and can cause part shortages and inefficient manufacturing
processes as work is performed out of sequence. Some level of design
change is expected during the production cycle of any new and highly
technical product, but excessive changes raise questions about the
JSF's design maturity and its readiness for higher rates of production.
With most of development testing still ahead for the JSF, the risk and
impact from required design changes are significant. Acquisition
programs typically encounter higher and more substantive changes as a
result of discovery and rework during development flight and ground
testing. Future changes may require alterations to the manufacturing
process, changes to the supply base, and costly retrofitting of
aircraft already produced and fielded. A key cost driver for the
program has been the higher than expected effort needed to address
design related issues. The contractor has not been able to reduce
engineering staff as fast as expected. DOD's restructuring actions
recognize these issues and added time to development, more flight
testing, and reduced procurement. Additional changes are likely as
development flight testing continues.
Some emerging concerns may drive additional and substantive design
changes:
* JSF Lift System Development and Integration. Essential to STOVL
operations, the lift fan continues to be a prime risk area. The
program is working to mature lift fan and drive shaft technologies and
a required redesign expected in spring 2011.
* Fatigue Cracks in STOVL Test Article. During a recent durability
ground test, fatigue cracks were discovered in a major bulkhead of the
STOVL test article. Cracks were discovered after 1,500 hours of
durability testing, less than one-tenth of the hours planned for
fatigue tests to certify that the STOVL airframe meets its design life
requirement. Officials reported that stress data had been under-
estimated during initial design. Inspections of aircraft and other
test articles did not identify cracks at the same site. Decisions
about potential redesign and re-manufacture are still to be determined.
* Wing Tip Vortex. Prime contractor officials identified wing tip
vortices as a potential risk to the program. Wing tip vortices are
tubes of circulating air which are left behind the aircraft's wing as
it generates lift. The cores of the vortices are sometimes visible
because of water condensation. If these are visible during daytime
flights they could negatively impact the aircraft's stealth
capabilities.
* Outer Mold Lines. Defense Contract Management Agency officials noted
difficulties in manufacturing outer mold lines, resulting from tight
tolerance specifications and multiple manufacturing methodologies
among the different JSF parts suppliers. The manufacturing processes
are new and different from legacy practices. Inability to meet the
outer mold line requirements could have major impacts on cost as well
as stealth requirements and capabilities. This problem is not expected
to be resolved until the June 2015 time frame after which a large
number of aircraft will have been built and would need to be
retrofitted for any design changes. Program officials stated some
redesign activities have begun and will take into the 2013 timeframe
to begin developing the changes, their costs, and implementation. The
effects of these changes could extend out into 2015, but will be
prioritized to reduce performance and cost impacts.
Manufacturing Processes are Not Yet Mature Enough for Efficient
Production at Increased Rates:
Manufacturing and delivering test jets took much more time and money
than planned and the full contingent of test aircraft is still not
available at military testing sites, years later than promised.
Projected costs to complete the first three production lots for
aircraft and engines also exceed the negotiated amounts at contract
award and aircraft will be delivered late. The production impacts of
restructuring actions that reduced quantities, lowered the ramp rate,
and delayed the full-rate production decision have not been fully
determined. We found that the aircraft and engine manufacturers are
making good faith efforts to implement the IMRT and JAT
recommendations and to make other improvements with performance
measures indicating some success.
As in prior years, lingering management inefficiencies, including
substantial out-of-station work[Footnote 14] and part shortages,
continued to increase the labor needed to manufacture test aircraft.
Figure 3 depicts forecasted and actual labor hour requirements for
building 12 production-representative test jets. Total labor hours
required to produce the test aircraft increased over time. The 2010
actual labor hours exceeded the 2007 budgeted hours by more than 1.5
million hours, a 75 percent increase.
Figure 3: JSF Labor Hours for Manufacturing Test Aircraft:
[Refer to PDF for image: multiple line graph]
Aircraft: BF1;
2007 Budget: 342,647;
2009 Budget: 361,047;
2010 Budget: 392,006.
Aircraft: BF2;
2007 Budget: 233,319;
2009 Budget: 296,681;
2010 Budget: 356,814.
Aircraft: BF3;
2007 Budget: 179,892;
2009 Budget: 254,394;
2010 Budget: 311,830.
Aircraft: BF4;
2007 Budget: 155,420;
2009 Budget: 223,391;
2010 Budget: 354,329.
Aircraft: AF1;
2007 Budget: 187,781;
2009 Budget: 262,694;
2010 Budget: 337,244.
Aircraft: AF2;
2007 Budget: 145,607;
2009 Budget: 222,396;
2010 Budget: 275,728.
Aircraft: AF3;
2007 Budget: 130,955;
2009 Budget: 209,861;
2010 Budget: 289,143.
Aircraft: CF1;
2007 Budget: 239,351;
2009 Budget: 263,064;
2010 Budget: 310,099.
Aircraft: CF2;
2007 Budget: 125,602;
2009 Budget: 218,194;
2010 Budget: 236,538.
Aircraft: CF3;
2007 Budget: 118,153;
2009 Budget: 204,908;
2010 Budget: 240,438.
Aircraft: BF5;
2007 Budget: 102,505;
2009 Budget: 191,410;
2010 Budget: 262,069.
Aircraft: AF4;
2007 Budget: 91,218;
2009 Budget: 183,480;
2010 Budget: 222,580.
Source: GAO analysis of DOD data.
[End of figure]
Manufacturing production aircraft is different from building test
aircraft, and some gains in learning as more aircraft are built can be
expected to, over time, reduce labor hour costs. However, the
experience to date on the test aircraft and initial production
aircraft suggests that future costs for building production aircraft
may be higher than currently budgeted. The costs on the first three
low-rate production contracts have increased from amounts negotiated
at contract award and the completion dates for delivering aircraft
have been extended over nine months on average. We are encouraged by
DOD's award of a fixed price incentive fee contract for lot 4
production and the prospects for the cost study to inform lot 5
negotiations, but we have not examined contract specifications.
DOD began procuring production jets in 2007 and has now ordered 58
aircraft on the first four low-rate initial production lots. JSF
contracts anticipated the delivery of 14 production jets through 2010,
but none have been delivered. Delivery of the first two production
jets (both CTOLs) has been delayed several times since the contract
was signed and is now expected in April 2011. In addition, DOD expects
to procure 32 more aircraft in fiscal year 2011. Building a large
backlog of jets on order but undelivered is not an efficient use of
federal funds, tying up millions of dollars in obligations ahead of
the ability of the manufacturing process to produce. We note that the
Secretary used a similar line of reasoning to reduce STOVL production.
DOD does not yet know the full effect that restructuring actions will
have on future annual procurement funding requirements. Cost analysts
are still calculating the impacts from deferring procurement of 246
aircraft from the near-term to future years, lowering the ramp rate,
and extending the full-rate production decision. Future funding
requirements could be even higher than projected and the quantities
considered affordable by the U.S. and allies could be reduced, further
driving up unit costs.
The Secretary's decisions to reduce near-term procurement quantities
and adopt a less-steep ramp up in future production were based on IMRT
findings. The Secretary chartered the IMRT to comprehensively review
JSF manufacturing capacity to assess the contractor's ability to
achieve planned production ramp-up and to sustain the predicted
maximum production rates. The IMRT's October 2009 report made 20
specific recommendations for corrective actions. As of September 2010,
officials considered eight of the recommendations complete and three
others on track. Implementation of the remaining nine recommendations
was incomplete or behind schedule.
The most significant incomplete recommendation is improving global
supply chain management. The JSF already has an extensive number of
suppliers worldwide and those numbers will increase with future
workload shared among numerous domestic and foreign firms. The IMRT
cites the global supply chain as the critical manufacturing challenge
facing the program, requiring significant improvement in delivery
performance and responsiveness in order to achieve the program's
eventual production rate goal of 20 aircraft per month. According to
the prime contractor, the global supply chain remains on the critical
path and progress has been made, but the global transportation plan
and supply chain risk management plan are incomplete. Another IMRT
recommendation that still needs to be addressed is the performance of
a comprehensive schedule risk assessment, now expected to begin in
spring 2011. We recommended this in our March 2009 report.[Footnote
15] Schedule risk assessments can provide keen insight into critical
path activities, cost and schedule interrelationships, and emerging
risks.
The primary F135 engine contractor faces similar challenges as it
moves deeper into production. All development engines and initial
production units have been delivered, but the costs to complete each
of the first three engine production contracts increased and
deliveries slipped since contract awards. Officials said these delays
have not been especially troublesome to date because aircraft
deliveries were even later. The contractor achieved the initial
service release for the CTOL and CV engine, meaning the engine
configuration is qualified and ready to go into production, but the
STOVL's initial release was delayed until December 2010 due to
qualification testing. The JAT reviewed F135 program performance,
identified cost drivers, and made affordability projections. JAT
officials said the contractor's cost reduction efforts were credible
but largely dependent on receiving more government funding for
affordability initiatives and alternative sourcing arrangements.
Our past work in best practices found that successful product
development programs reach a point at which they know that
manufacturing processes will efficiently produce a new product
conforming to cost, quality, and schedule targets before they begin
producing a system. Reaching this point means more than knowing that
the product can be built; it means that critical manufacturing
processes are under control, such that the quality, volume, and cost
are proven acceptable. By these criteria, the JSF contractors'
abilities to ramp-up to greater rates of production have not yet been
demonstrated. The aircraft and engine manufacturers now have
significantly more items in production flow compared to prior years,
but throughput capacity to complete all work and deliver end items is
constrained. We determined that the aircraft and engine contractors
are making good faith efforts to implement the recommendations of the
IMRT and JAT and to make other improvements to production capacity and
flow. The aircraft manufacturer is reporting a decrease in out of
station work, more efficient work stations, improved quality,
increased parts availability, and reduced span times. Until
improvements are fully implemented and demonstrated, the restructuring
actions to reduce near term procurement quantities and establish a
more achievable ramp rate was appropriate and will provide more time
to fully mature manufacturing and supply processes and catch up with
aircraft backlogs. Improving factory throughput and controlling costs--
driving down unit costs and delivering on time--are essential for
efficient manufacturing and timely delivery to the warfighter at the
increased production rates planned for the future.
Aircraft Are Not Meeting Early Reliability Growth Plans:
STOVL and CTOL aircraft are behind reliability growth plans aimed at
demonstrating that the aircraft will meet warfighter support and
availability requirements. The carrier variant is in early stages of
flight testing and sufficient reliability data was not available.
Reliability is a function of the specific elements of a product's
design; a system is reliable when it can perform over a specified
period of time without failure, degradation, or need of repair.
Improvements over time occur through design changes or manufacturing
process improvements. A key reliability metric is mean flying hours
between failure, defined as the number of flying hours achieved
divided by the number of failures incurred. Reliability growth plans
called for the STOVL to have achieved at least 1.9 flying hours
between failures and for the CTOL 2.9 flying hours between failures by
this point in the test program. However, the STOVL aircraft is
significantly behind plans, achieving about 0.4 hours between
failures, or about 20 percent of what was expected by this time. The
CTOL variant was also behind plans achieving 1.8 hours between
failures, approximately 60 percent of what was expected. Figure 4
depicts progress of each variant in demonstrating mean flying hours
between failures, as of September 2010.
Figure 4: JSF Mean Times between Failure Demonstrated to Date:
[Refer to PDF for image: vertical bar graph]
Mean flying hours between failure:
JSF variant: CTOL;
Demonstrated: 1.8;
Planned to date: 2.9;
Goal at maturity: 6.
JSF variant: STOVL;
Demonstrated: 0.4;
Planned to date: 1.9;
Goal at maturity: 4.
JSF variant: CV;
Demonstrated: not applicable;
Planned to date: not applicable;
Goal at maturity: 4.
Source: GAO analysis of DoD data.
[End of figure]
Improving reliability rates are essential to control future operating
costs and ensure aircraft are available as needed by the warfighter.
Compared to the up-front costs of acquiring aircraft, the long-term
costs for operating, maintaining, and sustaining JSF fleets over an
aircraft's useful life represent the much larger portion of total
ownership costs. We have reported in the past that it is important to
demonstrate that system reliability is on track to meet goals before
production begins as changes after production commences can be
inefficient and costly.[Footnote 16]
Testing Has Been Slow and Has Not Demonstrated That the Aircraft Will
Work in Its Intended Environment:
The JSF program is still very early in demonstrating aircraft design
and testing to verify it works as intended. As of December 2010, about
four percent of JSF capabilities have been completely verified by
flight tests, lab results, or both. Initial tests of a fully
integrated aircraft to demonstrate full mission systems capabilities
and weapons delivery is now not expected until 2015, three years later
than planned. The program demonstrated measurable progress in
development flight testing during 2010, but still lags earlier
expectations, and the STOVL problems have constrained overall
progress. Only 3 of 32 ground test labs and simulation models critical
to complement and, in some cases, substitute for flight tests, are
accredited to verify and ensure the fidelity of results. Software
development--essential for achieving about 80 percent of the JSF
functionality--is significantly behind schedule as it enters its most
challenging phase. Software delivery to the test program that is
essential to demonstrating full system capability is now expected in
late 2014, a 3-year delay.
Our work in best practices suggests that a key indicator of a
product's maturity and readiness for production is when a fully
integrated, capable system has been demonstrated to work in its
intended environment. A fully integrated, capable system would include
the integration of all the hardware, including mission avionics
systems, and software needed to provide the system its full mission
capabilities. Many past DOD weapons programs have failed to
demonstrate that the system works as intended before entering
production, discovering costly design problems late in development
when the more complex software and advanced capabilities are
integrated and tested.
Development Flight Testing Is Progressing but Behind Plans:
Development flight testing was much more active in 2010 than prior
years and had some notable successes, but overall still lagged behind
expectations. The continuing effects from late delivery of test
aircraft and an inability to achieve the planned flying rates per
aircraft substantially reduced the amount and pace of testing planned
previously. Consequently, even though the flight test program
accelerated its pace last year, the total number of flights
accomplished during the first four years of the test program
significantly lagged expectations when the program's 2007 baseline was
established. Figure 5 shows that the cumulative number of flights
accomplished by the end of 2010 was only about one-fifth the number
forecast by this time in the 2007 test plan.
Figure 5: Actual JSF Flight Tests Completed through 2010 Compared to
the 2007 Plan:
[Refer to PDF for image: multiple line graph]
Date: January 2010:
2007 planned : 1,155;
2010 actual flights: 144.
Date: February 2010;
2007 planned flights: 1,291;
2010 actual flights: 157.
Date: March 2010;
2007 planned flights: 1,420;
2010 actual flights: 170.
Date: April 2010;
2007 planned flights: 1,546;
2010 actual flights: 197.
Date: May 2010;
2007 planned flights: 1,681;
2010 actual flights: 230.
Date: June 2010;
2007 planned flights: 1,812;
2010 actual flights: 273.
Date: July 2010;
2007 planned flights: 1,953;
2010 actual flights: 322.
Date: August 2010;
2007 planned flights: 2,079;
2010 actual flights: 370.
Date: September 2010;
2007 planned flights: 2,250;
2010 actual flights: 406.
Date: October 2010;
2007 planned flights: 2,393;
2010 actual flights: 458.
Date: November 2010;
2007 planned flights: 2,519;
2010 actual flights: 518.
Date: December 2010;
2007 planned flights: 2,649;
2010 actual flights: 547.
Source: GAO analysis of DOD data.
[End of figure]
Program officials reported that 13 test aircraft are now out of
production.[Footnote 17] Ten test aircraft have been ferried to test
sites and others are in varying stages of final check-out. The program
has accomplished first flights for all three variants. Officials had
hoped aircraft could achieve a rate of 12 flights per month. However,
the average flight rate for 2010 ranged from over 2 to almost 8 per
month.
By the end of 2010, about 10 percent of more than fifty thousand
planned test points have been completed. According to program
officials, completion of a test point means that the test point has
been flown and that flight engineers ruled that the point has met the
need. Further analysis may be necessary for the test point to be
closed out. The majority of the points were earned on airworthiness
tests (basic airframe handling characteristics) and in ferrying the
planes to test sites. According to a senior level DOD test official,
airworthiness and ferry test points should be relatively easy to
accomplish. Remaining test points include more complex and stringent
requirements, such as mission systems, ship suitability, and weapons
integration that have yet to be demonstrated.
As discussed earlier, STOVL flight performance lagged plans during
2010, while the CTOL variant exceeded and the CV variant generally met
plans. Officials reported that design and manufacturing defects and
excessive component failures caused prolonged maintenance periods that
drove the low fly rates. For instance, in the July to August 2010
period, STOVL test aircraft were down for unscheduled maintenance more
than half the time. Further test delays will likely cause the program
to miss critical future milestones. STOVL initial at-sea testing will
not start until October 2011 because of delays in clearing the
vertical-landing envelope. STOVL-related delays are also causing
Marine Corps leadership to reassess its requirements and will likely
extend the date for achieving initial operational capabilities,
currently set in December 2012.
Concerned that STOVL testing problems were negatively affecting the
other variants, the Department moved to decouple the STOVL testing and
placed the variant on a two-year probation period to work out problems
and get back on track. The Secretary's actions will require a new test
plan since current flight test plans rely substantially on the STOVL
to fly and demonstrate test points in common with other variants. The
current plan has the STOVL responsible for completing about 43 percent
of the total test points.
Figure 6: Test Point Distribution by Variant (2010 Test Plan):
[Refer to PDF for image: pie-chart]
CV: 27%;
CTOL: 30%;
STOVL: 43%.
Source: GAO analysis of DoD data.
[End of figure]
JSF restructuring actions are positive and support a more robust and
achievable test plan. Officials added more resources for development
testing, extended the flight test schedule, and reduced the overlap
with initial operational testing. More recently, officials revised the
test plan increasing the total number of test flights from 5,856 to
7,727, about one-third more. To increase capacity, the restructure
added one carrier variant test aircraft, an additional software
integration line, and allowed the program to utilize up to three
production aircraft for development testing. Compared to the previous
test plan, officials assumed more ground time for aircraft maintenance
and planned modifications, as well as a more measured ramp-up in the
rate of flights per test aircraft. The restructuring largely reverses
the program's earlier Mid-Course Risk Reduction plan that reduced test
resources. Our March 2008 report[Footnote 18] criticized DOD's mid-
course plan, particularly the cuts made in flight test assets and the
number of flight tests, as well as the program's failure to address
root causes of cost growth, the very reasons why officials felt the
mid-course plan was needed. Since that report was issued, JSF cost and
schedule continued to deteriorate and officials recognized a need to
increase test assets and add more flight testing.
Most Ground Test Labs and Simulation Models Are Not Accredited or
Verified:
The JSF test program relies much more heavily than previous weapon
systems on its modeling and simulation labs to test and verify
aircraft design and subsystem performance. However, only 3 of 32 labs
and models have been fully accredited to date; the program had planned
to accredit 11 labs and models by now. Accreditation is essential to
ensure the fidelity of results validate that the models accurately
reflect aircraft performance. Accreditation is a lengthy and involved
technical evaluation using flight test data to verify lab results.
Much work remains before the program can fully utilize the models and
simulation capabilities needed to verify results and to demonstrate
that ground testing can substitute for flight testing. However, the
ability to substitute is unproven and progress in reducing program
risk is difficult to assess. Contracting officials told us that early
results are providing good correlation between ground and flight tests.
The Director of Operational Test and Evaluation reported that 50
percent of the models will be accredited during the final year of
flight testing, a highly risky approach. Delays in accreditation add
risks to not completing future software blocks on time and for
discovering defects late. More flight testing may be needed to cover
lab shortcomings, but is generally more expensive, and could lead to
more delays in completing development and operational testing. It
could also require more production aircraft for a longer period to
supplement test assets, resulting in fewer systems at training sites
and operational bases.
Contractor utilization of labs has increased markedly and the number
and integration of labs is impressive, but capacity may be
constrained. Because of development concurrency, there is overlap in
scheduling the new blocks and resources must be shared between blocks
when rework on an earlier block is required. If integration and test
is delayed due to capacity or conflict with an earlier block, lab
officials said that expected capabilities may not be delivered on time
to meet flight test and training dates. Mitigating strategies include
adding people, lab capacity, software test lines, and shifting
capabilities to later blocks. The 2010 restructuring added $250
million to increase integration lab capacity. According to program
officials, the greater number of labs allows engineers to work
simultaneously on different development blocks, reducing bottlenecks
that may occur in testing. Program and contractor officials believe
that the up-front investment of $5 billion in simulation labs will pay
off in early risk reduction, reduce flights, control costs, and are
essential to meet key milestones in JSF's aggressive test plan.
Software Development Is behind Schedule with Most Complex Work Still
Ahead:
Software providing essential JSF capability is not mature and releases
to the test program are behind schedule. Officials underestimated the
time and effort needed to develop and integrate the software,
substantially contributing to the program's overall cost and schedule
problems and testing delays, while requiring the retention of
engineers for longer periods. Significant learning and development
work remains before the program can demonstrate the mature software
capabilities needed to meet warfighter requirements. Good progress has
been made in the writing of software code--about three-fourths of the
software has been written and integrated, but testing is behind
schedule and the most complex work is still ahead. Program
restructuring added a second software integration line which should
improve throughput.
The JSF software development effort is one of the largest and most
complex in DOD history, providing 80 percent of JSF's functionality
essential to capabilities such as sensor fusion, weapons and fire
control, maintenance diagnostics, and propulsion. JSF has about 8
times more on-board software lines of code than the F/A-18E/F Super
Hornet and 4 times more than the F-22A Raptor. Also, the amount of
code needed will likely increase as integration and testing efforts
intensify. In 2009, officials reported that about 40 percent of the
software had completed integration and testing. They did not provide
us a progress report through 2010. Integration and test is a lengthy
effort and is typically the most challenging phase of software
development requiring specialized skills and integration test lines.
The program has experienced a growth of 40 percent in total software
lines of code since preliminary design review and 13 percent growth
since the critical design review. Other recent defense acquisitions
have experienced 30 to 100 percent growth in software over time.
Software capabilities are developed, tested, and delivered in 5
blocks, or increments. Several blocks have grown in size and taken
longer to complete than planned. Software defects, low productivity,
and concurrent development of successive blocks created
inefficiencies, taking longer to fix defects and delaying the
demonstration of critical capabilities. In addition, program and prime
contractor officials acknowledge they do not include integration as a
key tracking metric and have been unable to agree on how to track it.
This has made it hard for the program to analyze integration trends
and take action to remedy the situation. Instead the program office
and prime contractor have made several adjustments to the software
development schedule, each time lengthening the time needed to
complete work, as shown in figure 7.
Figure 7: Slips in Delivering Software to Flight Test:
[Refer to PDF for image: illustrated table]
Block 0.1: Flight sciences;
Initial estimate (2006): Mid-2006;
Current estimate (2011): Early 2007.
Block 0.5: Initial mission systems architecture;
Initial estimate (2006): Mid-2008;
Current estimate (2011): Mid-2010.
Block 1.0: Initial training capability;
Initial estimate (2006): Late 2008;
Current estimate (2011): Mid-2011.
Block 2.0: Initial warfighting capability;
Initial estimate (2006): Mid-2010;
Current estimate (2011): Late 2013.
Block 3.0: Full warfighting capability;
Initial estimate (2006): Late 2011;
Current estimate (2011): Late 2014.
Source: GAO analysis of DOD data.
[End of figure]
Delays in developing, integrating, and releasing software to the test
program have cascading effects hampering flight tests, training, and
lab accreditation. While progress is being made, a substantial amount
of software work remains before the program can demonstrate full
warfighting capability. The program released block 0.5 for flight test
nearly 2 years later than planned in the 2006 plan, largely due to
integration problems. Each of the remaining three blocks--providing
full mission systems and warfighting capabilities--are now projected
to slip between 2 to 3 years compared to the 2006 plan. Defects and
workload bottlenecks delayed the release of full block 1 capabilities;
the initial limited release of block 1 software was flown for the
first time in November 2010. Software defects increased throughout
2010, but fixing defects did not keep pace. Some capabilities were
moved to future blocks in attempts to meet schedule and mitigate
risks. For example, full data fusion mission systems[Footnote 19] were
deferred from block 2 to 3. Further trades and deferrals may be
needed. Rather than working all blocks concurrently, focusing efforts
on a more measured evolutionary approach could result in more timely
release of incremental capabilities to the testing, training, and
warfighter communities. Development and integration of the most
advanced capabilities could be deferred to future increments and
delivered to the warfighter at a later date.
The recent technical baseline review identified software as a
significant challenge, slowing system development and requiring more
time and money. Although officials are confident that such risks can
be addressed, the scale and complexity of what is involved remains a
technically challenging and lengthy effort. Uncertainties pertaining
to critical technologies, including the helmet-mounted display and
advanced data links, add to challenges. Deficiencies in the helmet
mounted display, especially latency in transmitting sensor data, are
causing officials to develop a second helmet while trying to fix the
first model. Resolution could result in a major redesign or changes in
the JSF's concept of operations by placing limitations on the
operational environment, according to program officials.
Conclusions:
The JSF program is at a critical juncture--9 years in development and
4 years in limited production, but still early in testing and
verifying aircraft performance. If effectively implemented and
sustained, the Department's restructuring should place the JSF program
on firmer footing and lead to more achievable and predictable
outcomes. However, restructuring comes with a price tag--higher up-
front development costs, fewer aircraft received in the near term,
training delays, and prolonged times for testing and delivering the
capabilities required by the warfighter. Reducing near-term
procurement quantities lessens concurrency, but the overlap among
development, testing, and production activities is still substantial
and risky. Development and testing activities will now overlap 11
years of production based on the latest extension in key milestones.
Flight testing and production activity are increasing and contractors
are improving supply and manufacturing processes, but deliveries are
still lagging. The challenge in front of the aircraft and engine
contractors is improving the global supply chain and accelerating
manufacturing throughput to produce quality products in economic
quantities and on time. Slowed deliveries have built a growing backlog
of jets on order but not delivered; this is not a good use of federal
funds, tying up millions of obligated dollars much ahead of the
ability of the manufacturing process to produce. The Secretary of
Defense used similar reasoning in significantly reducing STOVL
procurement until technical issues are resolved and the manufacturing
process able to deliver efficiently and on time.
The JSF acquisition demands an unprecedented share of the Department's
future investment funding. The program's size and priority is such
that its cost overruns and extended schedules are either borne by
funding cuts to other programs or else drive increases in the top line
of defense spending, the latter not an attractive option in a period
of more austere budgets. Up until now, JSF problems have been
addressed either with more time and money or by deferring aircraft
procurement to be borne by future years' budgets. It is past time to
place some boundaries on the program such that future difficulties can
be managed within a finite budget by facilitating trades within the
JSF program and thereby minimizing impacts on other defense programs
and priorities. Also, Department actions to limit STOVL procurement,
decouple it from development testing, and concentrate efforts to
resolve deficiencies are appropriate. Given its criticality to the
Marine Corp's future tactical aviation plans, additional steps may be
needed to set the framework and criteria for the "probation period"
and to sustain management focus on STOVL in order to better ascertain
its progress and inform future decisions. Focused individual attention
on STOVL apart from the other two variants could allow each variant to
proceed through development and testing at its own pace. Furthermore,
development testing is hampered both by the late delivery of software
increments and the lagging schedule for accrediting ground labs and
simulation models. A comprehensive independent review of the software
development process and lab accreditation issues could enhance
management insight and identify opportunities for improvement in these
critical areas. We note that the previous independent teams
established by the Department significantly improved the
manufacturing, engine, and cost estimating processes.
We agree with defense leadership that a renewed and sustained focus on
affordability by contractors and the Government is critical to moving
this important program forward and enabling our military services and
our allies to acquire and sustain JSF forces in needed quantities.
Maintaining senior leadership's increased focus on program results,
holding government and contractors accountable for improving
performance, and bringing a more assertive, aggressive management
approach for the JSF to "live within its means" could help effectively
manage growth in the program and limit the consequences on other
programs in the portfolio. Controlling JSF future cost growth would
minimize funding disruption and help stabilize the defense acquisition
portfolio by providing more certainty to financial projections and by
facilitating the allocation of remaining budget authority to other
defense modernization programs.
Recommendations for Executive Action:
Given the other priorities that DOD must address in a finite budget, a
renewed and sustained focus on affordability by contractors and the
Government is critical for successfully moving the JSF program
forward. DOD must plan ahead for a way to address and manage JSF
challenges and risks in the future. To facilitate making tradeoff
decisions with respect to the JSF program that limit impacts to other
DOD programs and priorities and to improve key management processes,
we recommend that the Secretary of Defense take the following actions
to reinforce and strengthen program cost controls and oversight:
1. The JSF program should maintain total annual funding levels for
development and procurement at the current budgeted amounts in the
fiscal year 2012-2016 future years defense plan (modified, if
warranted, by the new acquisition program baseline expected this
year). It should facilitate trades among cost, schedule, requirements,
and quantities to control cost growth. Having gone through the
Technical Baseline Review (TBR) and budget approval process, it is
reasonable to expect the program to execute against the future years
defense plan going forward. Only in instances of major and unforeseen
circumstances, should the Department consider spending more money on
the program. Even then, we would expect changes to be few and adopted
only after close scrutiny by defense leadership. Approved changes
should be well supported, adequately documented, and reported to the
congressional defense committees.
2. Establish criteria for the STOVL probation period and take
additional steps to sustain individual attention on STOVL-specific
issues, including independent F-35B/STOVL Progress Reviews with Senior
Leadership to ensure cost and schedule milestones are achieved to
deliver required warfighter capabilities. The intent is to allow each
JSF variant to proceed and demonstrate success at its own pace and
could result in separate full-rate production decisions.
3. The Department should conduct an independent review of the
contractor's software development, integration, and test processes--
similar to its review of manufacturing operations--and look for
opportunities to streamline software efforts. This review should
include an evaluation of the ground lab and simulation model
accreditation process to ensure it is properly structured and robustly
resourced to support software test and verification requirements.
Agency Comments and Our Evaluation:
DOD provided us with written comments on a draft of this report. The
comments are reprinted in appendix III. We worked collaboratively with
defense officials to hone our draft recommendations, making them more
targeted. DOD concurred with the recommendations as amended. We also
incorporated technical comments as appropriate.
We are sending copies of this report to the Secretary of Defense; the
Secretaries of the Air Force and Navy; the Commandant of the Marine
Corps; and the Director of the Office of Management and Budget. The
report also is available at no charge on the GAO Web site at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841 or sullivanm@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. Staff members making key
contributions to this report are listed in appendix IV.
Signed by:
Michael J. Sullivan:
Director:
Acquisition and Sourcing Management:
List of Congressional Committees:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable Daniel K. Inouye:
Chairman:
The Honorable Thad Cochran:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Howard P. McKeon:
Chairman:
The Honorable Adam Smith:
Ranking Member:
Committee on Armed Services:
House of Representatives:
The Honorable C.W. Bill Young:
Chairman:
The Honorable Norman D. Dicks:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
To determine the Joint Strike Fighter (JSF) program's progress in
meeting cost, schedule, and performance goals, we received briefings
by program and contractor officials and reviewed financial management
reports, budget documents, annual Selected Acquisition Reports,
monthly status reports, performance indicators, and other data. We
identified changes in cost and schedule, and obtained officials'
reasons for these changes. We interviewed officials from the JSF
program, contractors, and the Department of Defense (DOD) to obtain
their views on progress, ongoing concerns and actions taken to address
them, and future plans to complete JSF development and accelerate
procurement. At the time of our review, the most recent Selected
Acquisition Report available was dated December 31, 2009 and released
in April 2010. At the time of our review, DOD was preparing a new
acquisition program baseline for the program which would reflect
updated cost and schedule projections.
In assessing program cost estimates, we compared the official program
cost estimate in the 2009 Selected Acquisition Report and subsequent
cost estimate developed after the Nunn-McCurdy breach to estimates
developed by the JSF program and Defense Contract Management Agency
(DCMA) reports. We interviewed program office officials and members of
the DOD Cost Analysis and Program Evaluation Office (CAPE), and DCMA
to understand their methodology, data, and approach in developing cost
estimates. To assess the validity and reliability of contractors' cost
estimates, we reviewed audit reports prepared by DCMA and cost
performance reports prepared by the contractor.
To access the program's plans and risk in manufacturing and its
capacity to accelerate production, we analyzed manufacturing cost and
work performance data to assess progress against plans. We compared
budgeted program labor hours to actual labor hours and identified
growth trends. We reviewed data and briefings provided by the program,
DCMA, and CAPE to assess supplier performance and ability to support
accelerated production in the near term. We also determined reasons
for manufacturing delays, discussed program and contractor plans to
improve, and projected the impact on development and operational
tests. We interviewed Naval Air Systems Command and contractor
officials to discuss Earned Value Management System issues but we did
not conduct any analysis since the data was deemed unreliable by DCMA.
To assess plans, progress, and risks in test activities, we examined
program documents and interviewed DOD, program office, and contractor
officials about current test plans and progress. To assess progress
towards test plans, we compared the number of flight tests conducted
as of December 2010 to the original test plan established in 2007. We
also reviewed documents and interviewed prime contractors about flight
testing, the integrated airborne test bed, and ground testing. To
assess the ground labs and test bed, we interviewed officials and
toured the testing labs at the Lockheed Martin facilities in Fort
Worth, Texas. We also reviewed the independent assessments conducted
by the JET and NAVAIR to obtain their perspective on the program's
progress in test activities.
In performing our work, we obtained information and interviewed
officials from the JSF Joint Program office, Arlington, Virginia;
Naval Air Systems Command, Patuxent River, Maryland; Defense Contract
Management Agency, Fort Worth, Texas; Lockheed Martin Aeronautics,
Fort Worth, Texas; Defense Contract Management Agency, Middletown,
Connecticut; and Pratt & Whitney, Middletown, Connecticut. We also met
with and obtained data from the following offices of the Secretary of
Defense in Washington, D.C.: Director, Operational Test and
Evaluation; Cost Analysis and Program Evaluation Office; and Systems
and Software Engineering. We assessed the reliability of DOD and JSF
contractor data by (1) performing electronic testing of required data
elements, (2) reviewing existing information about the data, and (3)
interviewing agency officials knowledgeable about the data. We
determined that the data were sufficiently reliable for the purposes
of this report. We conducted this performance audit from May 2010 to
February 2011 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
[End of section]
Appendix II: Prior GAO Reports on JSF and DOD Responses and Subsequent
Actions:
GAO report: 2001; GAO-02-39;
Estimated development costs: $34.4 Billion;
Estimated development length: 10 years;
Average procurement unit cost: $69 Million;
Key program event: Start of system development and demonstration
approved;
Primary GAO message: Critical technologies needed for key aircraft
performance elements are not mature. Program should delay start of
system development until critical technologies are mature to
acceptable levels;
DOD response and actions: DOD did not delay start of system
development and demonstration stating technologies were at acceptable
maturity levels and will manage risks in development.
GAO report: 2005; GAO-05-271;
Estimated development costs: $44.8 Billion;
Estimated development length: 12 years;
Average procurement unit cost: $82 Million;
Key program event: The program undergoes re-plan to address higher
than expected design weight, which added $7 billion and 18 months to
development schedule;
Primary GAO message: We recommend that the program reduce risks and
establish executable business case that is knowledge-based with an
evolutionary acquisition strategy;
DOD response and actions: DOD partially concurred but does not adjust
strategy, believing that their approach is balanced between cost,
schedule and technical risk.
GAO report: 2006; GAO-06-356;
Estimated development costs: $45.7 Billion;
Estimated development length: 12 years;
Average procurement unit cost: $86 Million;
Key program event: Program sets in motion plan to enter production in
2007 shortly after first flight of the non-production representative
aircraft;
Primary GAO message: The program plans to enter production with less
than 1 percent of testing complete. We recommend program delay
investing in production until flight testing shows that JSF performs
as expected;
DOD response and actions: DOD partially concurred but did not delay
start of production because they believe the risk level was
appropriate.
GAO report: 2007;
GAO-07-360;
Estimated development costs: $44.5 Billion;
Estimated development length: 12 years;
Average procurement unit cost: $104 Million;
Key program event: Congress reduced funding for first two low-rate
production buys thereby slowing the ramp up of production;
Primary GAO message: Progress is being made but concerns remain about
undue overlap in testing and production. We recommend limits to annual
production quantities to 24 a year until flying quantities are
demonstrated;
DOD response and actions: DOD non-concurred and felt that the program
had an acceptable level of concurrency and an appropriate acquisition
strategy.
GAO report: 2008;
GAO-08-388;
Estimated development costs: $44.2 Billion;
Estimated development length: 12 years;
Average procurement unit cost: $104 Million;
Key program event: DOD implemented a Mid-Course Risk Reduction Plan to
replenish management reserves from about $400 million to about $1
billion by reducing test resources;
Primary GAO message: We believe new plan actually increases risks and
that DOD should revise the plan to address concerns about testing, use
of management reserves, and manufacturing. We determine that the cost
estimate is not reliable and that a new cost estimate and schedule
risk assessment is needed;
DOD response and actions: DOD did not revise risk plan nor restore
testing resources, stating that they will monitor the new plan and
adjust it if necessary. Consistent with a report recommendation, a new
cost estimate was eventually prepared, but DOD refused to do a risk
and uncertainty analysis that we felt was important to provide a range
estimate of potential outcomes.
GAO report: 2009;
GAO-09-303;
Estimated development costs: $44.4 Billion;
Estimated development length: 13 years;
Average procurement unit cost: $104 Million;
Key program event: The program increased the cost estimate and adds a
year to development but accelerated the production ramp up.
Independent DOD cost estimate (JET I) projects even higher costs and
further delays;
Primary GAO message: Because of development problems, we stated that
moving forward with an accelerated procurement plan and use of cost
reimbursement contracts is very risky. We recommended the program
report on the risks and mitigation strategy for this approach;
DOD response and actions: DOD agreed to report its contracting
strategy and plans to Congress. In response to our report
recommendation, DOD subsequently agreed to do a schedule risk
analysis, but still had not done so as of February 2011. In February
2010, the Department announced a major restructuring of the JSF
program, including reduced procurement and a planned move to fixed-
price contracts.
GAO report: 2010;
GAO-10-382;
Estimated development costs: $49.3 Billion;
Estimated development length: 15 years;
Average procurement unit cost: $112 Million;
Key program event: The program was restructured to reflect findings of
recent independent cost team (JET II) and independent manufacturing
review team. As a result, development funds increased, test aircraft
were added, the schedule was extended, and the early production rate
decreased;
Primary GAO message: Because of additional costs and schedule delays,
the program's ability to meet warfighter requirements on time is at
risk. We recommend the program complete a full comprehensive cost
estimate and assess warfighter and IOC requirements. We suggest that
Congress require DOD to prepare a "system maturity matrix"--a tool for
tying annual procurement requests to demonstrated progress;
DOD response and actions: DOD continued restructuring actions and
announced plans to increase test resources and lower the production
rate. Independent review teams evaluated aircraft and engine
manufacturing processes. As we projected in this report, cost
increases later resulted in a Nunn-McCurdy breach. Military services
are currently reviewing capability requirements as we recommended. The
Department and Congress are working on a "system maturity matrix"
tool, which we suggested to Congress for consideration, to improve
oversight and inform budget deliberations.
Source: DOD data and GAO analysis.
[End of table]
[End of section]
Appendix III: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
Acquisition, Technology and Logistics:
3000 Defense Pentagon:
Washington, DC 20301-3000
April 5, 2011:
Mr. Michael Sullivan:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, NW:
Washington. DC 20548:
Dear Mr. Sullivan:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-11-325. "Joint Strike Fighter: Restructuring Places
Program on Firmer Footing, but Progress is Still Lagging Overall,"
dated February 28, 2011 (GAO Code 120918). Detailed comments on the
report recommendations are enclosed.
The DoD concurs with all three recommendations. The rationale and
actions taken by DoD are included in the enclosure.
We appreciate the opportunity to comment on the draft report. My point
of contact is Colonel Jason Denney, U.S. Air Force.
Jason.Denney@%osd.mil, 703-697-3619.
Sincerely,
Signed by:
David G. Ahern:
Deputy Assistant Secretary of Defense:
Portfolio Systems Acquisition:
Enclosure: As stated:
[End of letter]
GAO Draft Report Dated February 28, 2011:
GAO-11-325 (GAO Code 120918):
"JOINT STRIKE FIGHTER: RESTRUCTURING PLACES PROGRAM ON FIRMER FOOTING,
BUT PROGRESS IS STILL LAGGING OVERALL"
The Department remains committed to the F-35 Joint Strike Fighter
(JSF) program. The Fiscal Year (FY) 2012 President's Budget (PB)
demonstrates this commitment to the F-35 as the backbone of the future
tactical aircraft inventory for the Air Force, Navy, Marine Corps, as
well as our International Partners.
The Department and the F-35 prime contractor are committed to
delivering F-35 aircraft that meet the Services' requirements as
specified in the Joint Operational Requirements Document (JORD) in
addition to controlling and reducing costs wherever and whenever
possible, with the goal of providing the Services' an affordable
tactical aviation capability.
Following a Critical Nunn-McCurdy Breach, a rescission of the original
October 26, 2001 Milestone B (MS B) decision, and program
recertification in June 2010, the Department set out to establish a
firmer foundation for the program. The F-35 Program Executive Officer
(PEO) executed an extensive bottoms-up Technical Baseline Review (TBR)
involving over 120 tactical aircraft experts from both the Services
and the Department to evaluate every aspect of the System Development
and Demonstration (SDD) phase of the program. In addition to the TBR,
the PEO will execute a schedule risk assessment this summer and
finalize the program's Integrated Master Schedule this fall following
an Integrated Baseline Review.
The Department will hold a Defense Acquisition Board (DAB) in May 2011
to review the restructured SDD phase of the program and update the
Acquisition Program Baseline.
Department Of Defense Comments To The GAO Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense to
maintain total annual funding levels for development and procurement
at the current budgeted amounts in the fiscal year 2012-2016 future
years defense plan (modified, if warranted, by the new acquisition
program baseline expected this year). It should facilitate trades
among cost, schedule, requirements, and quantities to control cost
growth. Having gone through the Technical Baseline Review (TBR) review
and budget approval process, it is reasonable to expect the program to
execute against the FYDP budget going forward. Only in instances of
major and unforeseen circumstances, should the Department consider
spending more money on the program. Even then, we would expect changes
to be few and adopted only after close scrutiny by defense leadership.
Approved changes should be well supported, adequately documented, and
reported to the congressional defense committees.
DOD Response: Concur The Department is confident that the F-35 budget
request for FY 2012-2016 provides the appropriate level of funding for
the restructured development program and revised procurement profile.
The Department undertakes a very thorough and in-depth process to
develop the annual Defense Budget. The goal is to prepare a budget
that does not change significantly from year to year and provides
acquisition programs and the Services a stable funding and procurement
profile to plan against. In this instance, the JSF Program Office
conducted an extensive TBR involving over 120 tactical aircraft
experts from both the Services and the Department to evaluate every
aspect of the SDD phase of the program. The resultant schedule and
cost adjustments made following the TBR, and reflected in the FY2012
Budget, afforded the Department a more realistic basis to manage the
program. The approval of a new MS B and APB will reflect these budget
decisions. Any significant proposed changes to the F-35 budget in
future years will be thoroughly reviewed as part of the annual Defense
Acquisition Executive Program Reviews and approved as part of the
budget review and submission process.
Recommendation 2: The GAO recommends that the Secretary of Defense to
establish criteria for the short take off and landing variant (STOVL)
probation period and take additional steps to sustain individual
attention to STOVL-specific issues, including independent F-35B/STOVL
Progress Reviews with Senior Leadership to ensure cost and schedule
milestones are achieved to deliver required warfighter capabilities.
The intent is to allow each JSF variant to proceed and demonstrate
success at its own pace and could result in separate full-rate
production decisions.
DOD Response: Concur. The Service Acquisition Executives (SAEs) for the
Department of the Navy and Air Force have established a battle rhythm
of monthly SAE reviews with the F-35 PEO to assess the overall F-35
program, with additional emphasis on F-35B Short Take Off and Vertical
Landing (STOVL) variant. These monthly SAE Reviews examine the F-35
airframe and propulsion SDD, Production, and Sustainment programs with
particular emphasis on Cost/Affordability, Risk, and Schedule. In
addition, the Commandant of the Marine Corps has established a monthly
F-35 review focused solely on the F-35B variant. This review is also
led by Department of the Navy (DoN) Senior Acquisition Executives. The
topics and metrics that are assessed include, but are not limited to,
the following:
* Cost/Affordability/Earned Value Management (EVM): A review of
Acquisition Procurement Unit Cost, Program Acquisition Unit Cost,
Operations and Support costs, and EVM cost/schedule indices.
* Risk: Monthly assessment of Program Risk' with explanations about
each risk item, their interactions, and risk burn-down plans.
Assessments include a review of the assumptions and environment used
to determine the risk evaluations.
* F-35B weight/weight growth: Weight assessments track each pound
added to the airframe with an understanding of the underlying reasons
for the growth. If there are trades that need to be made to mitigate
weight growth, DoN senior leadership/warfighters are to be consulted.
* Key Performance Parameters (KPPs): Review of F-35B KPPs with a
tracking/trending methodology and monthly discussions of CONOPS
considerations that might help facilitate achievement of F-35 KPPs and
program goals.
* Airframe Technical Performance Measurements: Assessments of
reliability, maintainability, combat radius, and gross weight (with
metrics that indicate the desired value(s), the current status,
margin, and trends).
* F-35B Flight Test: Review of F-35B flight test data, to include,
planned/scheduled test points to be flown versus achieved test points
flown; scheduled test flights flown versus actual test flights flown
(delineated by STOVL variant); and Clean-Wing Flight Envelope coverage
(to assess the progress on the envelope cleared for flight as a result
of Developmental Test and alignment with software delivery).
As these monthly reviews mature, the DoN will refine key F-35B metrics
to ensure this essential capability is delivered to the Marine Corps
Warfighters and to enable the Department to make a decision on the F-
35B STOVL variant probation status.
Recommendation 3: The GAO recommends that the Secretary of Defense to
conduct an independent review of the contractor's software
development, integration, and test processes”similar to its review of
manufacturing operations”and look for opportunities to streamline
software efforts. This review should include an evaluation of the
ground lab and simulation model accreditation process to ensure it is
properly structured and robustly resourced to support software test
and verification requirements.
DOD Response: Concur The Department believes that the newly structured
program, which delivers a JORD-compliant Block 3 capable aircraft at
the end of the SDD phase, provides the Warfighter capability needed.
The Department acknowledges that delivery of that capability has taken
longer and will cost more than planned, in part due to poor
performance to date in software development. The recent TBR ensured
that full Block 3 capability was adequately costed and scheduled with
the appropriate amount of acceptable risk, to include the risk in the
software development portion of the program. In order to maximize our
developmental investment, it would be prudent to independently verify
the contractor has appropriate processes to develop and field software
and is diligently following them accordingly.
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Michael Sullivan (202) 512 4841 or sullivanm@gao.gov:
Acknowledgments:
In addition to the contact name above, the following staff members
made key contributions to this report: Bruce Fairbairn, Assistant
Director; Charlie Shivers; Julie Hadley; Matt Lea; Jason Lee; Sean
Merrill; LeAnna Parkey; Karen Richey; Dr. W. Kendal Roberts; and
Robert Swierczek.
[End of section]
Related GAO Products:
Joint Strike Fighter: Restructuring Should Improve Outcomes, but
Progress Is Still Lagging Overall. [hyperlink,
http://www.gao.gov/products/GAO-11-450T]. Washington, D.C.: March 15,
2011.
Tactical Aircraft: Air Force Fighter Force Structure Reports Generally
Addressed Congressional Mandates, but Reflected Dated Plans and
Guidance, and Limited Analyses. [hyperlink,
http://www.gao.gov/products/GAO-11-323R]. Washington, D.C.: February
24, 2011.
Defense Management: DOD Needs to Monitor and Assess Corrective Actions
Resulting from Its Corrosion Study of the F-35 Joint Strike Fighter.
[hyperlink, http://www.gao.gov/products/GAO-11-171R]. Washington D.C.:
December 16, 2010.
Joint Strike Fighter: Assessment of DOD's Funding Projection for the
F136 Alternate Engine. [hyperlink,
http://www.gao.gov/products/GAO-10-1020R]. Washington, D.C.: September
15, 2010.
Tactical Aircraft: DOD's Ability to Meet Future Requirements is
Uncertain, with Key Analyses Needed to Inform Upcoming Investment
Decisions. [hyperlink, http://www.gao.gov/products/GAO-10-789].
Washington, D.C.: July 29, 2010.
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/products/GAO-10-388SP]. Washington,
D.C.: March 30, 2010.
Joint Strike Fighter: Significant Challenges and Decisions Ahead.
[hyperlink, http://www.gao.gov/products/GAO-10-478T]. Washington,
D.C.: March 24, 2010.
Joint Strike Fighter: Additional Costs and Delays Risk Not Meeting
Warfighter Requirements on Time. [hyperlink,
http://www.gao.gov/products/GAO-10-382]. Washington, D.C.: March 19,
2010.
Joint Strike Fighter: Significant Challenges Remain as DOD
Restructures Program. [hyperlink,
http://www.gao.gov/products/GAO-10-520T]. Washington, D.C.: March 11,
2010.
Joint Strike Fighter: Strong Risk Management Essential as Program
Enters Most Challenging Phase. [hyperlink,
http://www.gao.gov/products/GAO-09-711T]. Washington, D.C.: May 20,
2009.
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/products/GAO-09-326SP]. Washington,
D.C.: March 30, 2009.
Joint Strike Fighter: Accelerating Procurement before Completing
Development Increases the Government's Financial Risk. [hyperlink,
http://www.gao.gov/products/GAO-09-303]. Washington D.C.: March 12,
2009.
Defense Acquisitions: Better Weapon Program Outcomes Require
Discipline, Accountability, and Fundamental Changes in the Acquisition
Environment. [hyperlink, http://www.gao.gov/products/GAO-08-782T].
Washington, D.C.: June 3, 2008.
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/products/GAO-08-467SP]. Washington,
D.C.: March 31, 2008.
Joint Strike Fighter: Impact of Recent Decisions on Program Risks.
[hyperlink, http://www.gao.gov/products/GAO-08-569T]. Washington,
D.C.: March 11, 2008.
Joint Strike Fighter: Recent Decisions by DOD Add to Program Risks.
[hyperlink, http://www.gao.gov/products/GAO-08-388]. Washington, D.C.:
March 11, 2008.
Tactical Aircraft: DOD Needs a Joint and Integrated Investment
Strategy. [hyperlink, http://www.gao.gov/products/GAO-07-415].
Washington, D.C.: April 2, 2007.
Defense Acquisitions: Assessments of Selected Weapon Programs.
[hyperlink, http://www.gao.gov/products/GAO-07-406SP]. Washington,
D.C.: March 30, 2007.
Defense Acquisitions: Analysis of Costs for the Joint Strike Fighter
Engine Program. [hyperlink, http://www.gao.gov/products/GAO-07-656T].
Washington, D.C.: March 22, 2007.
Joint Strike Fighter: Progress Made and Challenges Remain. [hyperlink,
http://www.gao.gov/products/GAO-07-360]. Washington, D.C.: March 15,
2007.
Tactical Aircraft: DOD's Cancellation of the Joint Strike Fighter
Alternate Engine Program Was Not Based on a Comprehensive Analysis.
[hyperlink, http://www.gao.gov/products/GAO-06-717R]. Washington,
D.C.: May 22, 2006.
Defense Acquisitions: Major Weapon Systems Continue to Experience Cost
and Schedule Problems under DOD's Revised Policy. [hyperlink,
http://www.gao.gov/products/GAO-06-368]. Washington, D.C.: April 13,
2006.
Defense Acquisitions: Actions Needed to Get Better Results on Weapons
Systems Investments. [hyperlink,
http://www.gao.gov/products/GAO-06-585T]. Washington, D.C.: April 5,
2006.
Tactical Aircraft: Recapitalization Goals Are Not Supported by
Knowledge-Based F-22A and JSF Business Cases. [hyperlink,
http://www.gao.gov/products/GAO-06-487T]. Washington, D.C.: March 16,
2006.
Joint Strike Fighter: DOD Plans to Enter Production before Testing
Demonstrates Acceptable Performance. [hyperlink,
http://www.gao.gov/products/GAO-06-356]. Washington, D.C.: March 15,
2006.
Joint Strike Fighter: Management of the Technology Transfer Process.
GAO-06-364. Washington, D.C.: March 14, 2006.
Tactical Aircraft: F/A-22 and JSF Acquisition Plans and Implications
for Tactical Aircraft Modernization. [hyperlink,
http://www.gao.gov/products/GAO-05-519T]. Washington, D.C: April 6,
2005.
Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike
Fighter Program with Different Acquisition Strategy. [hyperlink,
http://www.gao.gov/products/GAO-05-271]. Washington, D.C.: March 15,
2005.
[End of section]
Footnotes:
[1] Rather than a single step approach, best practices and current DOD
acquisition guidance recommend that complex weapon system programs
instead adopt a more evolutionary acquisition strategy, developing and
procuring new systems incrementally to help achieve better program
outcomes and deliver new capabilities to the warfighters sooner.
[2] GAO, Joint Strike Fighter: Additional Costs and Delays Risk Not
Meeting Warfighter Requirements on Time, [hyperlink,
http://www.gao.gov/products/GAO-10-382] (Washington, D.C.: Mar. 19,
2010). Refer to the related products section for a list of prior GAO
reports and testimonies.
[3] National Defense Authorization Act for Fiscal Year 2010, Pub. L.
No. 111-84 § 244 (2009).
[4] Sensor fusion is the ability to take information from both
multiple onboard and off board aircraft sensors and display the
information in an easy-to-use format for the single pilot.
[5] The international partners are the United Kingdom, Italy, the
Netherlands, Turkey, Canada, Australia, Denmark, and Norway. These
nations are contributing funds for system development and plan to
procure more than 700 aircraft.
[6] [hyperlink, http://www.gao.gov/products/GAO-10-382]. Also GAO,
Joint Strike Fighter: Significant Challenges and Decision Ahead,
[hyperlink, http://www.gao.gov/products/GAO-10-478T] (Washington,
D.C.: Mar. 24, 2010) and GAO, Joint Strike Fighter: Significant
Challenges Ahead as DOD Restructures Program, [hyperlink,
http://www.gao.gov/products/GAO-10-520T] (Washington, D.C.: Mar. 11,
2010).
[7] Commonly referred to as Nunn-McCurdy, 10 U.S.C. § 2433 establishes
the requirement for DOD to submit unit cost reports on major defense
acquisition programs or designated major subprograms. Two measures are
tracked against the current and original baseline estimates for a
program: procurement unit cost (total procurement funds divided by the
quantity of systems procured) and program acquisition unit cost (total
funds for development, procurement, and system-specific military
construction divided by the quantity of systems procured). If a
program's procurement unit cost or acquisition unit cost increases by
at least 25 percent over the current baseline estimate or at least 50
percent over the original baseline estimate, it constitutes a breach
of the critical cost growth threshold. Programs are required to notify
Congress if a Nunn-McCurdy breach is experienced.
[8] When a program experiences a Nunn-McCurdy breach of the critical
cost growth threshold, DOD is required to take a number of steps
including reassessing the program and submitting a certification to
Congress in order to continue the program, in accordance with 10
U.S.C. § 2433a.
[9] Fifth generation aircraft include the F-22A and JSF and
incorporate stealth characteristics, fused sensor data, and advanced
radars.
[10] American National Standards Institute/Electronics Industries
Alliance-748 is a collection of 32 earned value management system
guidelines that incorporate business best practices for program
management systems proven to provide strong benefits for program or
enterprise planning and control. The processes include integration of
program scope, schedule, and cost objectives, establishment of a
baseline plan for accomplishment of program objectives, and use of
earned value techniques for performance measurement during the
execution of a program. The system provides a sound basis for problem
identification, corrective actions, and management replanning as
required.
[11] Flight test points are specific, quantifiable objectives in
flight plans that are needed to verify aircraft design and performance.
[12] [hyperlink, http://www.gao.gov/products/GAO-10-382].
[13] Pub. L. No. 111-383, § 122.
[14] Out of station work occurs when manufacturing steps are not
completed at its designated work station and must be finished
elsewhere later in production. This is highly inefficient, increasing
labor hours, causing delays, and sometimes quality problems.
[15] GAO, Joint Strike Fighter: Accelerating Procurement before
Completing Development Increases the Government's Financial Risk,
[hyperlink, http://www.gao.gov/products/GAO-09-303] (Washington D.C.:
Mar. 12, 2009).
[16] GAO, Best Practices: Capturing Design and Manufacturing Knowledge
Early Improves Acquisition Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: July 15,
2002).
[17] This includes 12 test aircraft and the non-production
representative model that achieved much of the test flights prior to
2010. A 14th test aircraft, the test carrier variant added in the
recent restructuring is expected to be delivered in 2012.
[18] GAO, Joint Strike Fighter: Recent Decisions by DOD Add to Program
Risks, [hyperlink, http://www.gao.gov/products/GAO-08-388]
(Washington, D.C.: Mar. 11, 2008).
[19] Mission systems are critical to realizing increased warfighter
capability in combat effectiveness through next generation sensors
with fused information from on-board and off-board systems (i.e.
electronic warfare, communication navigation identification, electro-
optical target system, electro-optical distributed aperture system,
radar, and data links).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
