Defense Acquisitions

Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals Gao ID: GAO-08-448 March 14, 2008

By law, GAO annually assesses the Missile Defense Agency's (MDA) progress in developing and fielding a Ballistic Missile Defense System (BMDS). Funded at $8 billion to nearly $10 billion per year, it is the largest research and development program in the Department of Defense (DOD). The program has been managed in 2-year increments, known as blocks. Block 2006, the second BMDS block, was completed in December 2007. GAO assessed MDA's progress in (1) meeting Block 2006 goals for fielding assets, completing work within estimated cost, conducting tests, and demonstrating the performance of the overall system in the field, and (2) making managerial improvements to transparency, accountability, and oversight. In conducting the assessment, GAO reviewed the assets fielded; contractor cost, schedule, and performance; and tests completed during 2007. GAO also reviewed pertinent sections of the U.S. Code, acquisition policy, and the charter of a new missile defense board.

MDA made progress in developing and fielding the BMDS during Block 2006 but fell short of meeting its original goals. Specifically, it fielded additional assets such as land-based interceptors and sea-based missiles and upgraded other assets, including Aegis BMD-equipped ships. It also met most test objectives, with a number of successful tests conducted. As a result, fielded capability has increased. On the other hand, it is difficult to assess how well BMDS is progressing relative to the funds it has received because fewer assets were fielded than originally planned, the cost of the block increased by at least $1 billion, some flight tests were deferred, and the performance of the fielded system remains unverified. In particular, GAO could not determine the full cost of Block 2006 because MDA continued to defer budgeted work into the future, where it is no longer counted as a Block 2006 cost. Also making cost difficult to assess is a work planning method--referred to as level of effort--used by contractors that does not link time and money with what is produced. When not appropriately used, level-of-effort planning can obscure work accomplished, portending additional cost in the future. MDA is working to minimize the use of this planning method--a needed step as contractors overran their fiscal year 2007 budgets. Performance of the fielded system is as yet not verifiable because too few tests have been conducted to validate the models and simulations that predict BMDS performance. Moreover, the tests that are done do not provide enough information for DOD's independent test organization to fully assess the BMDS' suitability and effectiveness. GAO has previously reported that MDA has been given unprecedented funding and decision-making flexibility. While this flexibility has expedited BMDS fielding, it has also made MDA less accountable and transparent in its decisions than other major programs, making oversight more challenging. MDA, with direction from Congress, has taken several steps to address these concerns. MDA implemented a new way of defining blocks--its construct for developing and fielding BMDS increments--that should make costs more transparent. For example, under the newly-defined blocks, MDA will no longer defer work from one block to another. Accountability should also be improved as MDA will, for the first time, estimate unit costs for selected assets and report variances from those estimates. DOD also chartered a new board with more BMDS oversight responsibility than its predecessor, although it does not have approval authority for some key decisions made by MDA. Finally, MDA will begin buying certain assets with procurement funds like other programs. This will benefit transparency and accountability, because procurement funding generally requires that assets be fully paid for in the year they are bought. Previously, MDA, with Congressional authorization, was able to pay for assets incrementally over several years. Additional steps could be taken to further improve oversight. For example, MDA has not yet estimated the total cost of a block, and therefore, cannot have its costs independently verified--actions required of other programs to inform decisions about affordability and investment choices. However, MDA does plan to estimate block costs and have them verified at some future date.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: Team: Phone:


GAO-08-448, Defense Acquisitions: Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals This is the accessible text file for GAO report number GAO-08-448 entitled 'Defense Acquisitions: Progress Made in Fielding Missile Defense but Program is Short of Meeting Goals' which was released on March 17, 2008. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to Congressional Committees: United States Government Accountability Office: GAO: March 2008: Defense Acquisitions: Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals: Missile Defense: GAO-08-448: GAO Highlights: Highlights of GAO-08-448, a report to congressional committees March. Why GAO Did This Study: By law, GAO annually assesses the Missile Defense Agency‘s (MDA) progress in developing and fielding a Ballistic Missile Defense System (BMDS). Funded at $8 billion to nearly $10 billion per year, it is the largest research and development program in the Department of Defense (DOD). The program has been managed in 2-year increments, known as blocks. Block 2006, the second BMDS block, was completed in December 2007. GAO assessed MDA‘s progress in (1) meeting Block 2006 goals for fielding assets, completing work within estimated cost, conducting tests, and demonstrating the performance of the overall system in the field, and (2) making managerial improvements to transparency, accountability, and oversight. In conducting the assessment, GAO reviewed the assets fielded; contractor cost, schedule, and performance; and tests completed during 2007. GAO also reviewed pertinent sections of the U.S. Code, acquisition policy, and the charter of a new missile defense board. What GAO Found: MDA made progress in developing and fielding the BMDS during Block 2006 but fell short of meeting its original goals. Specifically, it fielded additional assets such as land-based interceptors and sea-based missiles and upgraded other assets, including Aegis BMD-equipped ships. It also met most test objectives, with a number of successful tests conducted. As a result, fielded capability has increased. On the other hand, it is difficult to assess how well BMDS is progressing relative to the funds it has received because fewer assets were fielded than originally planned, the cost of the block increased by at least $1 billion, some flight tests were deferred, and the performance of the fielded system remains unverified. In particular, GAO could not determine the full cost of Block 2006 because MDA continued to defer budgeted work into the future, where it is no longer counted as a Block 2006 cost. Also making cost difficult to assess is a work planning method”referred to as level of effort”used by contractors that does not link time and money with what is produced. When not appropriately used, level-of-effort planning can obscure work accomplished, portending additional cost in the future. MDA is working to minimize the use of this planning method”a needed step as contractors overran their fiscal year 2007 budgets. Performance of the fielded system is as yet not verifiable because too few tests have been conducted to validate the models and simulations that predict BMDS performance. Moreover, the tests that are done do not provide enough information for DOD‘s independent test organization to fully assess the BMDS‘ suitability and effectiveness. GAO has previously reported that MDA has been given unprecedented funding and decision-making flexibility. While this flexibility has expedited BMDS fielding, it has also made MDA less accountable and transparent in its decisions than other major programs, making oversight more challenging. MDA, with direction from Congress, has taken several steps to address these concerns. MDA implemented a new way of defining blocks”its construct for developing and fielding BMDS increments”that should make costs more transparent. For example, under the newly-defined blocks, MDA will no longer defer work from one block to another. Accountability should also be improved as MDA will, for the first time, estimate unit costs for selected assets and report variances from those estimates. DOD also chartered a new board with more BMDS oversight responsibility than its predecessor, although it does not have approval authority for some key decisions made by MDA. Finally, MDA will begin buying certain assets with procurement funds like other programs. This will benefit transparency and accountability, because procurement funding generally requires that assets be fully paid for in the year they are bought. Previously, MDA, with Congressional authorization, was able to pay for assets incrementally over several years. Additional steps could be taken to further improve oversight. For example, MDA has not yet estimated the total cost of a block, and therefore, cannot have its costs independently verified”actions required of other programs to inform decisions about affordability and investment choices. However, MDA does plan to estimate block costs and have them verified at some future date. What GAO Recommends: GAO makes several recommendations that include (1) adding sufficient scope to tests to enable an assessment of the BMDS‘ suitability and effectiveness and (2) developing a cost estimate for each block and requesting an independent verification of that cost. DOD agreed with our recommendation to estimate the full cost of a block, but only partially agreed with adding scope to developmental tests. To view the full product, including the scope and methodology, click on [hyperlink, http://www.GAO-08-448]. For more information, contact Paul Francis, (202) 512-4841, francisp@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: More Capability Fielded but Less than Planned and at Higher Cost: Efforts Underway to Improve BMDS Management and Oversight: Conclusions: Recommendations for Executive Action: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: MDA Contracts: Appendix III: Incremental Funding: Appendix IV: Comments from the Department of Defense: Appendix V: GAO Contact and Staff Acknowledgments: Tables: Table 1: Description of BMDS Elements: Table 2: BMDS Block 2006 Deliveries and Total Fielded Assets: Table 3: Prime Contractor Fiscal Year 2007 and Cumulative Cost and Schedule Performance: Table 4: Level of Effort Percentages for BMDS Prime Contracts: Table 5: MDA New Block Construct: Table 6: MDEB Standing Committee Functions: Table 7: Incremental Funding Costs versus Full funding Costs for THAAD and Aegis BMD Fielded Assets: Table 8: MDA's General Process for Determining Award Fee Amounts: Table 9: Fee Awarded during Fiscal Year 2007: Table 10: Amount of Fees Rolled over to Future Award Fee Periods During Fiscal Year 2007: Table 11: MDA's Block 2006 Unpriced Changes and Unpriced Task Orders from January 1, 2007 to September 26, 2007: Table 12: MDA Support Contractor Job Functions: Table 13: Program Office Full-time Equivalent Staffing Levels: Table 14: MDA's Incremental Funding Plan for THAAD and Aegis BMD Assets: Table 15: Full Funding Alternative for THAAD Fire Units versus MDA Incremental Funding Plan: Table 16: Full Funding Alternative for Aegis BMD Standard Missile-3s versus MDA Incremental Funding Plan: Table 17: Full Funding Alternative for Aegis BMD Shipsets versus MDA Incremental Funding Plan: Figures: Figure 1: Deployed BMDS Assets as of December 31, 2007: Figure 2: Percentages of LOE as of December 2007: Figure 3: Aegis AWS Fiscal Year 2007 Cost and Schedule Performance: Figure 4: Aegis BMD SM-3 Fiscal Year 2007 Cost and Schedule Performance: Figure 5: ABL Fiscal Year 2007 Cost and Schedule Performance: Figure 6: C2BMC Fiscal Year 2007 Cost and Schedule Performance: Figure 7: GMD Fiscal Year 2007 Cost and Schedule Performance: Figure 8: KEI Fiscal Year 2007 Cost and Schedule Performance: Figure 9: MKV Task Order 5 Fiscal Year 2007 Cost and Schedule Performance: Figure 10: MKV Task Order 6 Fiscal Year 2007 Cost and Schedule Performance: Figure 11: Sensors Fiscal Year 2007 Cost and Schedule Performance: Figure 12: STSS Fiscal Year 2007 Cost and Schedule Performance: Figure 13: THAAD Fiscal Year 2007 Cost and Schedule Performance: Abbreviations: ABL: Airborne Laser: AT&L: Acquisition, Technology, and Logistics: BMDS: Ballistic Missile Defense System: C2BMC: Command Control, Battle Management, and Communications: DFARS: Defense Federal Acquisition Regulation Supplement: DOD: Department of Defense: DOT&E: Director, Operational Test and Evaluation: EVM: Earned Value Management: EVMS: Earned Value Management System: FAR: Federal Acquisition Regulation: FBX-T: Forward-Based X-Band Transportable: FFRDC: Federally Funded Research and Development Center: FYDP: Future Years Defense Plan: GMD: Ground-based Midcourse Defense: ICBM: Intercontinental Ballistic Missile: KEI: Kinetic Energy Interceptor: LOE: Level of Effort: LRS&T: Long-range Surveillance and Tracking: MDA: Missile Defense Agency: MDEB: Missile Defense Executive Board: MDSG: Missile Defense Support Group: MKV: Multiple Kill Vehicle: SM-3: Standard Missile-3: STSS: Space Tracking and Surveillance System: THAAD: Terminal High Altitude Area Defense: United States Government Accountability Office: Washington, DC 20548: March 14, 2008: Congressional Committees: In 2002, the President of the United States directed the Department of Defense (DOD) to begin fielding an initial Ballistic Missile Defense System (BMDS) capable of defending the U.S. homeland, deployed troops, friends, and allies against ballistic missiles of all ranges in all phases of flight. Since the 1980s, DOD has spent more that $100 billion on the development and early fielding of this system and it estimates that continued development will require an additional $50 billion between fiscal years 2008 and 2013. The Missile Defense Agency (MDA) has been tasked to carry out the President's direction and is developing and fielding the BMDS, DOD's largest research and development program. MDA placed an initial set of missile defense components in the field by December 2005. These components are collectively referred to as Block 2004. Recently, MDA delivered its second increment of capability--Block 2006--which includes additional components as well as performance enhancements. Current plans call for the continuation of BMDS development for many years, with the system eventually including a diverse collection of land-, air-, sea-, and space-based assets located around the globe. In its fiscal year 2002 and 2006 National Defense Authorization Acts, Congress directed GAO to assess the cost, schedule, testing, and performance progress that MDA is making in developing the BMDS.[Footnote 1] We have delivered assessments covering fiscal years 2003, 2004, 2005, and 2006.[Footnote 2] This report assesses Block 2006 and gives special attention to the progress made during fiscal year 2007 toward Block 2006 goals. It also follows up on BMDS program transparency, accountability, and oversight issues addressed in our March 2007 report. To assess progress during Block 2006, we examined the accomplishments of all nine BMDS elements that MDA is developing and fielding. These elements include the Airborne Laser (ABL); Aegis Ballistic Missile Defense (Aegis BMD); BMDS Sensors; Command, Control, Battle Management, and Communications (C2BMC); Ground-based Midcourse Defense (GMD); Kinetic Energy Interceptors (KEI); Multiple Kill Vehicles (MKV); Space Tracking and Surveillance System (STSS); and Terminal High Altitude Area Defense (THAAD).[Footnote 3] These elements collectively account for about 77 percent of MDA's research and development budget. In assessing BMDS progress, we examined documents such as Program Execution Reviews, test plans and reports, production plans, and Contract Performance Reports. We also interviewed officials within program offices and within MDA functional directorates, such as the Directorate for System Engineering. In addition, we discussed each element's test program and its results with DOD's Office of the Director, Operational Test and Evaluation. In following up on transparency, accountability, and oversight issues raised in our March 2007 report, we held discussions with officials in MDA's Directorate of Business Operations to determine whether its new block structure improved accountability and transparency of the BMDS. In addition, we reviewed pertinent sections of the U.S. Code to compare MDA's current level of accountability with federal acquisition laws. We also interviewed officials from the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics and DOD's Joint Staff to discuss the oversight role of the new Missile Defense Executive Board. Additionally, we reviewed the Board's charter to identify its oversight responsibility. Our scope and methodology is discussed in more detail in appendix IV. We conducted this performance audit from May 2007 to March 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Results in Brief: During Block 2006, MDA fielded additional and new assets, enhanced the capability of some assets, and demonstrated fielded elements in intercept tests. Although the fielded BMDS capability was increased, MDA did not meet the goals it originally set for the block. Ultimately, MDA fielded fewer assets, increased costs by about $1 billion, and conducted fewer tests. For example, MDA fielded one less Ground-based interceptor and seven fewer SM-3 missiles than planned. After MDA reported the $1 billion cost increase, it had to defer work to keep Block 2006 costs from increasing further, as some contractors overran their fiscal year 2007 budgets. Deferring work obscures the cost of the block because such work is no longer counted as part of Block 2006. The cost of the block may have been further obscured by the way that several contractors plan work. In planning work as level of effort, contractors can recognize work as complete when the time planned for the effort has been expended even if the work does not yield the intended product. To the extent that more work has to be done to complete the product, MDA could incur additional costs that are not yet recognized. While most test objectives were achieved, test delays were experienced by five of nine BMDS elements. In addition to tests of individual elements, MDA also sets goals for determining the overall performance of the BMDS. Similar to other DOD programs, MDA uses models and simulations to predict BMDS performance. However, data from flight tests are needed to prove the accuracy of the models and simulations. Although tests have been conducted that provide some assurance that the BMDS will perform as designed, further tests are needed to be highly confident that the models and simulations can make accurate predictions. Thus, we were unable to assess whether MDA met this overall performance goal. Moreover, the tests done to date have been developmental in nature, and do not provide sufficient realism for DOD's test and evaluation director to fully determine whether the BMDS is suitable and effective for battle. We previously reported that MDA has been given unprecedented funding and decision-making flexibility that has expedited the fielding of assets. This flexibility has also made oversight more challenging than for other programs. Several actions are underway that stand to improve oversight, but do not completely resolve concerns. First, MDA has adopted a new block approach that offers several improvements--unit costs for selected assets will now be tracked and work will no longer be deferred from one block to another. However, although MDA plans to do so, it has not yet estimated the total cost of any block of BMDS capability nor had that cost independently verified, as is done for other major programs. Second, DOD has established a Missile Defense Executive Board to review and make recommendations on MDA's acquisition strategy, plans, and funding. This board promises to be more substantive than its predecessor but will not provide the oversight that a Defense Acquisition Board provides on other major programs. For example, MDA does not need the Executive Board's approval for decisions regarding blocks. In addition, until MDA develops total cost estimates for each block, the Executive Board will not have access to such estimates. Neither will the board have a complete independent assessment of BMDS performance based on realistic testing. A unique challenge for the Executive Board is how to evaluate MDA's technology development efforts that range from $2 billion to $5 billion a year. Due to their experimental nature, such efforts normally do not have a firm cost, schedule, and performance baseline, but the scale of these investments warrant some basis for overseeing progress and cost. Third, Congress directed that MDA begin using procurement funds to purchase certain operational assets. This benefits transparency and accountability because procurement funding generally requires assets be fully paid for in the year they are bought. Previously, using research and development funding, MDA could spread the cost of individual assets over several years, making it more difficult to determine their cost. We are making several recommendations to build on the actions already taken to further improve the transparency of block costs and oversight of the BMDS program. These include having MDA (1) develop a full cost estimate for each block of BMDS capability with verification of that estimate, and (2) examine ways to develop a baseline or some other standard against which the progress of technology programs may be assessed. We are also recommending that MDA and the Director of Operational Test and Evaluation agree on criteria and incorporate corresponding scope into developmental tests that will allow a determination of whether a block of BMDS capability is suitable and effective for fielding. DOD concurred with three of our five recommendations, including having MDA develop block cost estimates and obtain independent verification of those estimates. DOD partially concurred with our recommendation that MDA examine ways to measure the progress of technology programs. DOD stated that MDA already uses key knowledge points, technology levels, and engineering readiness levels to assess the progress of technology programs and that it will continue to investigate other ways of making such assessment. Although we recognize the value of MDA's assessment methods, we continue to believe that DOD and MDA would benefit from understanding the remaining cost and time needed to complete a technology program, important information that MDA's methods do not yet provide. DOD also partially concurred with our recommendation on adding scope to developmental tests. DOD noted that MDA's mission is to work with the warfighter, rather than Director of Operational Test and Evaluation, to determine that the BMDS is ready for fielding. However, DOD stated that MDA will continue to work with operational testers to strengthen the testing of BMDS suitability and effectiveness. While we agree that the Director of Operational Test and Evaluation is not responsible for fielding decisions, it is relied upon to ensure that weapon systems are realistically and adequately tested so that production decisions can be based on accurate evaluations of operational effectiveness, suitability, and survivability. BMDS tests conducted to date only partially support such an evaluation. Because it is important that only reliable and effective systems be placed in the hands of the warfighter, we continue to believe that MDA should take specific steps to ensure that testing supports as full an evaluation of operational effectiveness, suitability, and survivability as possible. Background: The Missile Defense Agency's mission is to develop an integrated and layered BMDS to defend the United States, its deployed forces, allies, and friends. The BMDS is expected to be capable of engaging all ranges of enemy ballistic missiles in all phases of flight. This is a challenging expectation, requiring a complex combination of defensive components--space-based sensors, surveillance and tracking radars, advanced interceptors, and a battle management, command, control, and communications component--that work together as an integrated system. A typical scenario to engage an intercontinental ballistic missile (ICBM) would unfold as follows: * Infrared sensors aboard early-warning satellites detect the hot plume of a missile launch and alert the command authority of a possible attack. * Upon receiving the alert, land-or sea-based radars are directed to track the various objects released from the missile and, if so designed, to identify the warhead from among spent rocket motors, decoys, and debris. * When the trajectory of the missile's warhead has been adequately established, an interceptor--consisting of a kill vehicle mounted atop a booster--is launched to engage the threat. The interceptor boosts itself toward a predicted intercept point and releases the kill vehicle. * The kill vehicle uses its onboard sensors and divert thrusters to detect, identify, and steer itself into the warhead. With a combined closing speed on the order of 10 kilometers per second (22,000 miles per hour), the warhead is destroyed above the atmosphere through a "hit to kill" collision with the kill vehicle. To develop a system capable of carrying out such an engagement, MDA, until December 2007, executed an acquisition strategy in which the development of missile defense capabilities was organized in 2-year increments known as blocks. Each block was intended to provide the BMDS with capabilities that enhanced the development and overall performance of the system. The first 2-year block--Block 2004--fielded a limited initial capability that included early versions of the GMD, Aegis BMD, Patriot Advanced Capability-3, and C2BMC elements. The agency's second 2-year block--Block 2006--culminated on December 31, 2007, and fielded additional BMDS assets. Block 2006 also continued the evolution of Block 2004 by providing improved GMD interceptors, enhanced Aegis BMD missiles, upgraded Aegis BMD ships, a Forward-Based X-Band- Transportable radar, and enhancements to C2BMC software. On December 7, 2007, MDA's Director approved a new block construct that will be the basis for all future development and fielding.[Footnote 4] Table 1 provides a brief description of all elements currently being developed by MDA. Table 1: Description of BMDS Elements: Element: Ground-based Midcourse Defense; Missile defense role: GMD is a ground-based missile defense system designed to destroy ICBMs during the midcourse phase of their flight. Its mission is to protect the U.S. homeland against ballistic missile attacks from North Korea and the Middle East. GMD is part of the initial capability fielded in 2004-2005 and to date 24 interceptors have been emplaced for operational use. MDA plans to field about 20 additional interceptors in Alaska and California through 2011. Element: Aegis Ballistic Missile Defense; Missile defense role: Aegis BMD is a ship-based missile defense system designed to destroy short-to intermediate-range ballistic missiles during the midcourse phase of their flight. Its mission is twofold: to protect deployed U.S. forces, allies, and friends against ballistic missile attacks and to serve as a forward-deployed BMDS sensor, especially in support of the GMD mission. MDA is planning to procure 147 Aegis BMD missiles--the Standard Missile 3--from calendar year 2004 through 2013 and to upgrade 18 ships for the BMD mission by the end of 2008. MDA also requested funding in its fiscal year 2008 budget request to make Aegis BMD capable of defeating targets during the terminal phase of their flight. Element: Command, Control, Battle Management and Communications; Missile defense role: C2BMC is the integrating and controlling element of the BMDS. Its role is to provide deliberate planning, situational awareness, sensor management--including control of the Forward-Based X- Band-Transportable (FBX-T) radar--and battle management for the integrated BMDS. Element: BMDS Sensors; Missile defense role: MDA is developing various stand-alone radars for fielding. In particular, MDA leveraged the hardware design for the THAAD radar and modified existing software to develop the FBX-T. MDA placed the first FBX-T in Japan to augment existing BMD surveillance and tracking capabilities. The program has produced two FBX-T radars and expects to produce two more during the 2008-2009 timeframe. Element: Airborne Laser; Missile defense role: ABL is an air-based missile defense system designed to destroy all classes of ballistic missiles during the boost phase of their flight. ABL employs a high- energy chemical laser to rupture a missile's motor casing, causing the missile to lose thrust or flight control. MDA plans to demonstrate proof of concept in a system demonstration in 2009. An operational ABL capability is expected to be demonstrated in the 2016-2017 timeframe. Element: Kinetic Energy Interceptors; Missile defense role: KEI is a mobile land-based missile defense system designed to destroy medium, intermediate, and intercontinental ballistic missiles during the boost and midcourse phases of their flight. The agency expects to demonstrate defensive capability through flight testing during 2012-2015. This capability could be expanded to sea-basing in subsequent blocks. Element: Space Tracking and Surveillance System; Missile defense role: The Block 2006 STSS consists of two demonstration satellites. MDA intends to use these satellites for testing missile surveillance and tracking capabilities in the 2008-2010 timeframe. If the demonstration satellites perform successfully, MDA plans an operational capability of next-generation satellites. Element: Terminal High Altitude Area Defense; Missile defense role: THAAD is a ground-based missile defense system designed to destroy short-and medium-range ballistic missiles during the late-midcourse and terminal phases of flight. Its mission is to defend deployed U.S. forces and population centers. MDA plans to field a fire unit, which includes 24 missiles, in 2010 and a second unit in 2011. Element: Multiple Kill Vehicle; Missile defense role: The MKV is being designed as an optional warhead for all midcourse interceptors. The concept mitigates the need to pinpoint a single lethal object in a threat cluster by using numerous kill vehicles to engage all objects that might be lethal. The concept under development consists of a carrier vehicle housing a number of smaller kill vehicles, which would primarily benefit the Ground-based and Kinetic Energy interceptors. However, to mitigate risk, MDA has initiated a parallel acquisition with a second contractor that could result in an alternative payload for the Ground-based and Kinetic Energy interceptors as well as the Aegis BMD Standard Missile -3. Because MKV is a technology program, it does not project an initial capability date, but the program expects that the capability could be available by 2017. Source: MDA data. Note: The Patriot Advanced Capability-3 system is also part of the BMDS, but it is not included in the table because management responsibility for this element has been transferred to the Army. [End of table] More Capability Fielded but Less than Planned and at Higher Cost: MDA made progress in developing and fielding the BMDS during 2007. Additional assets were fielded and/or upgraded, several tests met planned objectives, and other development activities were conducted. On the other hand, fewer assets were fielded than originally planned, the cost of the block increased, some flight tests were deferred, and the performance of fielded assets could not be fully evaluated. Block 2006 Improves BMDS Capability: During Block 2006, MDA increased its inventory of BMDS assets while enhancing the system's performance. The agency fielded 14 additional Ground-based interceptors, 12 Aegis BMD missiles designed to engage more advanced threats, 4 new Aegis BMD destroyers, 1 new Aegis BMD cruiser, as well as 8 C2BMC Web browsers and 1 C2BMC suite. In addition, MDA upgraded half of its Aegis BMD ship fleet, successfully conducted four Aegis BMD and two GMD intercept tests, and completed a number of ground tests to demonstrate the capability of BMDS components. Considering assets fielded during Blocks 2004 and 2006, MDA, by December 31, 2007, had cumulatively fielded a total of 24 Ground-based interceptors, 2 upgraded early-warning radars, an upgraded Cobra Dane surveillance radar, 1 Sea-based X-band radar, 2 Forward- Based X-Band Transportable radars, 21 Aegis BMD missiles, 14 Aegis BMD destroyers, and 3 Aegis BMD cruisers. In addition, MDA had fielded 6 C2BMC suites; 46 warfighter enterprise workstations with situational awareness; BMDS planner and sensor management capabilities; 31 C2BMC Web browsers, 13 with laptop planners; and redundant communications node equipment to connect BMDS elements worldwide. MDA Does Not Meet Original Goals, but Meets or Exceeds Most Revised Goals: In March 2005, MDA submitted to Congress the number of assets it planned to field during Block 2006. However, increasing costs, technical challenges, and schedule delays prompted the agency to reduce the quantity of planned assets. Consequently, in March 2006, shortly after submitting its fiscal year 2007 budget, MDA notified Congress that it was revising its Block 2006 Fielded Configuration Baseline. Although MDA did not meet its original block fielding goals, it was able in nearly all instances to meet or exceed its revised goals. Of the four elements delivering assets during Block 2006, one--Sensors-- was able to meet its original goal. However, two elements--GMD and C2BMC--were able to exceed their revised fielding goals. Table 2 depicts the goals and the number of assets fielded. Table 2: BMDS Block 2006 Deliveries and Total Fielded Assets: BMDS element: GMD; Original goal as of March 2005: Up to 15 interceptors; Thule Interim Upgrade Early Warning Radar; Goal as of March 2006: Up to 12 interceptors; Deferred to Block 2008; Block 2006 assets as of December 31, 2007: 14 interceptors; Total assets available (cumulative total for Block 2004and Block 2006): 24 interceptors. BMDS element: Sensors; Original goal as of March 2005: 1 Forward-Based; X-Band-Transportable; Radar (FBX-T); Goal as of March 2006: 1 Forward- Based; X-Band-Transportable; Radar; Block 2006 assets as of December 31, 2007: 1 Forward-Based; X-Band- Transportable; Radar; Total assets available (cumulative total for Block 2004and Block 2006): 2 Forward- Based; X-Band-Transportable; Radars[A]. BMDS element: Aegis BMD; Original goal as of March 2005: 19 SM-3 missiles; 4 new destroyers; long-range surveillance and tracking (LRS&T) only; 8 upgraded destroyers for the engagement mission; 1 new cruiser; Goal as of March 2006: 15 SM-3 missiles; 4 new destroyers (LRS&T-only); 7 upgraded destroyers for the engagement mission; 1 new cruiser; Block 2006 assets as of December 31, 2007: 12 SM-3 missiles; 4 new destroyers; 7 upgraded destroyers; all perform the engagement mission; 1 new cruiser; that engages and performs LRS&T; Total assets available (cumulative total for Block 2004and Block 2006): 21SM-3 missiles. 14 destroyers; 7 perform LRS&T only and 7 engage and perform LRS&T; 3 cruisers; all engage and perform LRS&T. BMDS element: C2BMC; Original goal as of March 2005: 3 suites; Goal as of March 2006: Suites deferred; replaced with 3 Web browsers; Block 2006 assets as of December 31, 2007: 1 suite; 8 Web browsers/planners; Total assets available (cumulative total for Block 2004and Block 2006): 6 suites; 31 Web browsers; 46 enterprise workstations. Source: MDA data. [A] The second FBX-T radar is currently located at Vandenberg Air Force Base. It is available for use, but no decision has been made as to where it will be located. [End of table] Although GMD did not meet its original goal of fielding up to 15 interceptors and partially upgrading the Thule early warning radar, the element was able to surpass its revised goal of fielding 12 interceptors. By December 31, 2007, the GMD element fielded 14 interceptors--2 more than planned. To achieve its revised goal, the element's prime contractor added a manufacturing shift during 2007 and extended the number of hours that certain shifts' personnel worked. These actions allowed the contractor to more than double its interceptor emplacement rate. Last year, we reported that MDA delayed the partial upgrade of the Thule early-warning radar--one of GMD's original goals--until a full upgrade could be accomplished.[Footnote 5] According to DOD, the full upgrade of Thule is the most economical option and it meets DOD's desire to retain a single configuration of upgraded early warning radars. The Thule early warning radar upgrade is being accomplished by two separate contract awards. Raytheon was awarded a contract in April 2006 to develop and install prime mission equipment; while Boeing was expected to receive a contract in January 2008 to integrate the equipment into the BMDS ground communication network. In March 2005, MDA included three C2BMC suites as part of its fielding goal for Block 2006. These suites were to be fielded at U.S. European Command, U.S. Central Command, and another location that was to be identified later. Faced with a $30 million reduction in C2BMC's fiscal year 2006 budget, MDA in March 2006 revised this goal to replace the 3 suites with 3 less expensive Web browsers. However, by the end of Block 2006, MDA found an innovative way to increase combatant commands' situational awareness and planning capability. In 2005, the C2BMC program conducted a network load analysis and concluded that situational awareness and planning capability--equivalent to that provided by a suite--could be gained by combining Web browsers and planners. To prove that this approach would work, MDA fielded 4 Web browsers and one planner at the U.S. European Command. MDA learned that this combination of hardware, fielded in the quantities needed to meet a command's needs and connected to an existing server, provided the situational awareness and planning capability of a suite at less cost. MDA extended this approach by fielding one Web browser and one planner at four other locations--U.S. Forces Japan; U.S. Forces Korea; the Commander of U.S. Strategic Command; and the Commander of the Space and Missile Defense Command. In addition, MDA fielded one suite at U.S. Pacific Command. The Aegis BMD element was able to meet its revised block goals for only one of its two components. The program upgraded all planned ships, but fielded three fewer Aegis BMD Standard Missile-3s (SM-3) than planned. The program did not meet its revised missile goal because three U.S missiles were delayed into 2008 to accommodate an unanticipated requirement to deliver three missiles to Japan. Figure 1 below depicts the location of current BMDS assets. Figure 1: Deployed BMDS Assets as of December 31, 2007: This figure is a map showing deployed BMDS assets as of December 31, 2007. [See PDF for image] Source: MDA (data); GAO (presentation), map (Map Resources). [End of figure] Block Costs Have Increased, but Full Cost Is Not Known: MDA's Block 2006 program of work culminated with higher than anticipated costs. In March 2007, we reported that MDA's cost goal for Block 2006 increased by approximately $1 billion because of greater than expected GMD operations and sustainment costs and technical problems. During fiscal year 2007, some prime contractors performing work for the BMDS overran their budgeted costs. To stay within its revised budget, MDA was forced to reduce the amount of work it expected to accomplish during the block. The full cost of the block cannot be determined because of the deferral of work from one block to another. In addition, some MDA prime contractors too often employ a planning methodology that has the potential to obscure the time and money that will be needed to produce the outcomes intended. If the work does not yield the intended results, MDA could incur additional future costs. While MDA struggled to contain costs during Block 2006, the agency awarded two contractors a large percentage of available fee for performance in cost and/or program management although the contractor- reported data showed declining cost and schedule performance. Both award fee plans for these contractors direct that cost and schedule performance be considered as factors in making the evaluation. While these factors are important, MDA's award fee plans provide for the consideration of many other factors in making award fee determinations. Five Contractors Overran 2007 Budgets: To determine if contractors are executing the work planned within the funds and time budgeted, each BMDS program office requires its prime contractor to provide monthly Earned Value Management reports detailing cost and schedule performance.[Footnote 6] If more work was completed than scheduled and the cost of the work performed was less than budgeted, the contractor reports a positive schedule and cost variance. However, if the contractor was unable to complete all of the work scheduled and needed more funds to complete the work than budgeted, the contractor reports a negative schedule and cost variance. Of course, the results can be mixed. That is, the contractor may have completed more work than scheduled but at a cost that exceeded the budget. As shown in table 3 below, the contractors for the nine BMDS elements collectively overran their fiscal year 2007 budgets by approximately $166 million.[Footnote 7] We estimate that at completion, the cumulative overrun in the contracts could be between about $1.3 billion and $1.9 billion. Our predictions of final contract costs were developed using formulas accepted within the cost community and were based on the assumption that the contractor will continue to perform in the future as it has in the past. It should also be noted that some contracts include more than Block 2006 work. For example, the STSS contract includes work being accomplished in anticipation of future blocks. Our analysis is presented in table 3 below. Appendix II provides further details on the cost and schedule performance of the contractors outlined in the table. Table 3: Prime Contractor Fiscal Year 2007 and Cumulative Cost and Schedule Performance: (Dollars in millions). BMDS element: ABL; FY07 Cost variance[A]: $3.7; FY07 Schedule variance[A]: $24.2; Cumulative cost variance: ($74.2); Cumulative schedule variance: ($25.8); Percent of contract completed: 83%; Estimated contract underrun/overrun at completion[B]: Overrun of $95.4 to $202.5. BMDS element: Aegis BMD Weapon System; FY07 Cost variance[A]: $7.7; FY07 Schedule variance[A]: ($3.8); Cumulative cost variance: $7.0; Cumulative schedule variance: ($3.3); Percent of contract completed: 67%; Estimated contract underrun/overrun at completion[B]: Underrun of $8.8 to $17.7. BMDS element: Aegis BMD SM-3 (through March 2007)[C]; FY07 Cost variance[A]: $4.1; FY07 Schedule variance[A]: $8.4; Cumulative cost variance: $7.2; Cumulative schedule variance: ($.5); Percent of contract completed: 98%; Estimated contract underrun/overrun at completion[B]: Underrun of $7.2. BMDS element: Aegis BMD SM-3 (February 2007 through September 2007)[D]; FY07 Cost variance[A]: $6.2; FY07 Schedule variance[A]: ($4.0); Cumulative cost variance: $6.2; Cumulative schedule variance: ($4.0); Percent of contract completed: 66%; Estimated contract underrun/overrun at completion[B]: Underrun of $7.4 to $11.1. BMDS element: C2BMC; FY07 Cost variance[A]: ($11.1); FY07 Schedule variance[A]: ($1.5); Cumulative cost variance: ($14.5); Cumulative schedule variance: ($3.5); Percent of contract completed: 90%; Estimated contract underrun/overrun at completion[B]: Overrun of $9.9 [E]. BMDS element: GMD; FY07 Cost variance[A]: ($22.1); FY07 Schedule variance[A]: $84.9; Cumulative cost variance: ($1,081.8); Cumulative schedule variance: ($52.9); Percent of contract completed: 77%; Estimated contract underrun/overrun at completion[B]: Overrun of $1,055.9 to $1,422.3. BMDS element: KEI; FY07 Cost variance[A]: $2.1; FY07 Schedule variance[A]: ($7.5); Cumulative cost variance: $5.7; Cumulative schedule variance: ($12.8); Percent of contract completed: 10%; Estimated contract underrun/overrun at completion[B]: N/A[F]. BMDS element: MKV (Design and test of prototype propulsion system); FY07 Cost variance[A]: ($2.3); FY07 Schedule variance[A]: $0.0; Cumulative cost variance: ($2.7); Cumulative schedule variance: $0.0; Percent of contract completed: 93%; Estimated contract underrun/overrun at completion[B]: Overrun of $2.6 to $2.9. BMDS element: MKV (Prototype Carrier Vehicle Seeker)[G]; FY07 Cost variance[A]: $0.3; FY07 Schedule variance[A]: $0.9; Cumulative cost variance: $0.3; Cumulative schedule variance: $0.9; Percent of contract completed: 37%; Estimated contract underrun/overrun at completion[B]: Underrun of $0.8 to $2.5. BMDS element: Sensors; FY07 Cost variance[A]: $3.9; FY07 Schedule variance[A]: ($8.8); Cumulative cost variance: $24.1; Cumulative schedule variance: $17.8; Percent of contract completed: 65%; Estimated contract underrun/ overrun at completion[B]: Underrun of $22.0 to $46.8. BMDS element: STSS[H]; FY07 Cost variance[A]: ($67.7); FY07 Schedule variance[A]: $84.7; Cumulative cost variance: ($231.4); Cumulative schedule variance: ($19.7); Percent of contract completed: 49%; Estimated contract underrun/overrun at completion[B]: N/A. BMDS element: THAAD[I]; FY07 Cost variance[A]: ($91.1); FY07 Schedule variance[A]: $19.0; Cumulative cost variance: ($195.2); Cumulative schedule variance: ($9.1); Percent of contract completed: 86%; Estimated contract underrun/overrun at completion[B]: Overrun of $227.2 to $325.8. Total; FY07 Cost variance[A]: ($166.4); FY07 Schedule variance[A]: $196.5; Cumulative cost variance: ($1,549.3); Cumulative schedule variance: ($112.6); Percent of contract completed: [Empty]; Estimated contract underrun/overrun at completion[B]: Overrun of $1,344.8 to $1,878.1. Source: Contract Performance Reports (data); GAO (analysis). [A] A cost variance is defined as the difference between the budget for the work performed and the actual cost of work performed; while a schedule variance is the difference between the budgeted cost of planned work and the budgeted cost of work performed. Negative cost variances (budget overruns) and negative schedule variances (less work performed than planned) are shown with parentheses around the dollar amounts. [B] Contracts may include some work that is not related to Block 2006. [C] The Aegis BMD SM-3 contractor concluded work in March 2007 on three contract line items that directed SM-3 development, flight tests, and delivery of 12 Block 1A missiles. However, the contractor continues to flight test the SM-3 Block 1A missile and develop additional SM-3 capability. [D] The Aegis BMD SM-3 contractor began work in February 2007 on a third contract line item that directed the acquisition of 20 additional SM-3 Block 1A missiles. [E] C2BMC Part 4 work was completed in December 2007. The actual overrun for that work is reported as of March 4, 2008. [F] We could not estimate the likely outcome of the KEI contract at completion because a trend cannot be predicted until 15 percent of the planned work is completed. [G] All MKV work is performed under task orders issued as part of an Indefinite Delivery/Indefinite Quantity contract performed by one contractor. The contractor provided the first cost and schedule performance data for the prototype seeker in March 2007. [H] We did not estimate the cost of the STSS contract at completion. The STSS contract includes not only the effort to develop and launch two demonstration satellites (the Block 2006 capability) but also effort that will benefit future blocks. Block 2006 work is about 86 percent complete, while work on future blocks is about 16 percent complete. [I] Earned Value data for the THAAD contract is reported under two contract line item numbers, 1 and 10. We report only the contractor's cost and schedule performance for contract line item 1 because it represents the majority of the total work performed under the contract. Contract line item 10 provides for Patriot Common Launcher initiatives funded by the Army's Lower Tier Program Office. [End of figure] Technical problems and software issues caused several BMDS elements to overrun their fiscal year 2007 budgeted costs. In addition, 4 of the 10 contracts we reviewed contained some kind of replanning activity during fiscal year 2007 and the ABL contract was partially rebaselined. Contractors may replan when they conclude that the current plan for completing the effort remaining on the contract is unrealistic. A replan can include reallocating the remaining budget over the rest of the work, realigning the schedule within the contractually defined milestones, and setting either cost or schedule variances to zero or setting both to zero. A rebaseline is similar, but it may also add additional time and/or funding for the remaining work. The ABL contractor was overrunning both its fiscal year 2007 budget and schedule early in the year. Although by year's end it appears that the contractor recovered, the contractor would have continued to overrun both its budget and its schedule if most of the contract had not been rebaselined. The contractor realized cost and schedule growth as it worked to solve software integration problems in the Beam Control/Fire Control component and dealt with a low-power laser needed for flight tests that was not putting enough energy on the target. After encountering these problems, the ABL contractor did not have sufficient schedule or budget to complete the remaining contract work. Therefore, in May 2007, the program allowed the contractor to rebaseline all of the remaining work devoted to developing, integrating, flight testing, and delivering the ABL prototype. The rebaselining effort added about $253 million to the contract and extended the contract's period of performance by more than a year. The THAAD prime contractor's cost overrun of $91.1 million was primarily caused by technical problems related to the element's missile, launcher, radar, and test components. Missile component cost overruns were caused by higher than anticipated costs in hardware fabrication, assembly, and support touch labor for structures, propulsion, and other subassembly components. Additionally, design issues with the launcher's missile round pallet and the electronics assembly that controls the launcher caused the contractor to experience higher than anticipated labor and material costs. The radar component ended the fiscal year with a negative cost variance as more staff was required than planned to resolve hardware design issues in the radar's prime power unit. The contractor also experienced negative cost variances with the system test component because the Launch and Test Support Equipment required additional set-up time at the flight test range. The STSS contractor's $67.7 million fiscal year 2007 cost variance is primarily attributed to problems that occurred during thermal vacuum testing of the first satellite. Since the satellites are legacy hardware built under a former program, there are no spares available for testing. As a result, the contractor needed to handle the parts carefully to avoid damage to the hardware, increasing the time devoted to the test. Further test delays occurred when a number of interface issues surfaced during testing and when the cause of component problems could not be easily traced to their source. The program office believes that the cost variance would have been less if design engineers had been available during testing. Because engineers were not present to quickly identify the cause of component problems, a time-consuming analysis of each problem was needed. Total Block 2006 Costs Cannot be Determined: In March 2007, we reported that a full accounting of Block 2006 costs was not possible because MDA has the flexibility to redefine block outcomes. That is, MDA can delay the delivery of assets or other work activities from block to block and count the work as a cost of the block during which the work is performed, even though the work does not benefit that block. For example, MDA deferred some Block 2004 work until Block 2006 so that it could use the funds appropriated for that work to cover unexpected cost increases caused by technical problems recognized during development, testing, and production. With the deferral of the work, its cost was no longer counted as a Block 2004 cost, but as a Block 2006 cost. As a result, Block 2004's cost was understated and Block 2006's cost is overstated. Because MDA did not track the cost of the deferred work, the agency could not make an adjustment that would have matched the cost with the correct block. The cost of Block 2006 was further blurred as MDA found it necessary to defer some Block 2006 work until a future block. For example, when the STSS contractor overran its fiscal year 2007 budget because of testing problems, the program did not have sufficient funds to launch the demonstration satellites in 2007 as planned. The work is now scheduled for 2008. The consequence of deferring Block 2004 work to Block 2006 and Block 2006 to 2008 is that the full cost of Block 2006 cannot be determined. Contractors' Planning Methodology Has the Potential to Obscure Block Cost: Some MDA prime contractors too often employ a planning methodology that has the potential to obscure the time and money that will be needed to produce the outcomes intended. Contractors typically divide the total work of a contract into small efforts in order to define them more clearly and to ensure proper oversight. Work may be planned in categories including (1) level of effort (LOE) --work that contains tasks of a general or supportive nature and do not produce a definite end product--or (2) discrete work--work that has a definable end product or event. Level of effort work assumes that if the staff assigned to the effort spend the planned length of time, they will attain the outcome expected. According to earned value experts and the National Defense Industrial Association, while it is appropriate to plan such tasks as supervision or contract administration as LOE, it is not appropriate to plan tasks that are intended to result in a product, such as a study or a software build, as LOE because contractors do not report schedule variances for LOE work.Therefore, when contractors incorrectly plan discrete work as LOE, reports that are meant to allow the government to assess contractor cost and schedule performance may be positive, but the government may not have full insight into the contractor's progress. The greater the percentage of LOE, the weaker the link between inputs (time and money) and outcomes (end products), which is the essence of earned value analysis. Essentially, depending on the magnitude of LOE, schedule variances at the bottom line can be understated. The significant amount of BMDS work being tracked by LOE may have limited our assessment of the contractors' performance. That is, the contractor's performance may appear to be more positive than it would be if work had been correctly planned. In such cases, the government may have to expend additional time and money to achieve the outcomes desired. MDA Earned Value Management officials agreed that some BMDS prime contractors incorrectly planned discrete work as LOE, but the agency is taking steps to remedy this situation so that they can better monitor the contractors' performance. While it is not possible to state with certainty how much work a contractor should plan as LOE, experts within the government cost community, such as Defense Contract Management Agency officials, agree that LOE levels over 20 percent warrant investigation. According to MDA, many of its prime contractors plan a much larger percentage than 20 percent of their work as LOE. Table 4 presents the percentage of work in each BMDS prime contract that is categorized as LOE. Table 4: Level of Effort Percentages for BMDS Prime Contracts: BMDS contract element: Aegis BMD SM-3 (Development, flight test, and delivery of 12 Block 1A missiles)[A]; Level of effort percentage: 73. BMDS contract element: MKV; Level of effort percentage: 63[B]. BMDS contract element: ABL; Level of effort percentage: 57. BMDS contract element: C2BMC; Level of effort percentage: 52. BMDS contract element: KEI; Level of effort percentage: 40. BMDS contract element: Aegis BMD Weapon System; Level of effort percentage: 40. BMDS contract element: THAAD; Level of effort percentage: 30. BMDS contract element: GMD; Level of effort percentage: 28. BMDS contract element: Sensors (FBX-T); Level of effort percentage: 26. BMDS contract element: STSS; Level of effort percentage: 25. BMDS contract element: Aegis BMD SM-3 (Production of 20 additional Block 1A missiles)[C]; Level of effort percentage: 18. Source: MDA data. Note: Percentages For Aegis BMD SM-3 (Production Of 20 Additional Block 1a Missiles), ABL, C2BMC, And MKV Were Updated In January 2008. The Percentage of LOE Forall Other Elements Is The Amount Reported By MDA as Of December 2007.7. [A] Aegis BMD concluded work on SM-3 development, flight tests, and delivery of 12 Block 1A missiles in March 2007. [B] The percentage LOE reported for MKV is a weighted average for Task Orders 4 through 8. For more details on the amount of LOE included in individual tasks directed by the contract, refer to appendix II. [C] Aegis BMD began work on 20 additional SM-3 Block 1A missiles in February 2007. [End table] The Aegis BMD SM-3, MKV, ABL, and C2BMC contractors planned more than half of certain work as LOE.[Footnote 8] In several instances, MDA Earned Value Management officials and program office reviewers agreed that some of the LOE work could be redefined into discrete work packages. For example, from January through December 2007, the C2BMC contractor planned 73 percent of its work as LOE. This included activities such as software development and integration and test activities that result in two definable products--software packages and tests. At the direction of the C2BMC Program Office, the C2BMC contractor redefined some contract work, including software development and integration and test activities, as discrete, reducing the amount of LOE on the contract to 52 percent. The Aegis BMD element also reported a high percentage of LOE for its Standard Missile-3 contract, particularly considering that its products--individual missiles--are quite discrete. In August 2007, the element reported that the contractor had planned 73 percent of the contract work as LOE. The portion of the work that contained this amount of LOE was completed in March 2007 with an underrun of $7.2 million. Although the contractor reported an underrun for this work upon its completion, the high percentage of LOE may have, over the contract period, distorted the contractor's actual cost and schedule performance. Similarly, it is important to note that the amount of LOE for the SM-3 work that is currently ongoing is considerably less. Program officials told us that prior to the commencement of this segment of work, the MDA Earned Value Management Group and program officials recommended that the program minimize the amount of LOE on its contracts. Currently, only 18 percent of the SM-3 contract is considered LOE. Two Contractors Awarded Fee That Does Not Reflect Earned Value Performance: MDA uses award fees to encourage its contractors to perform in an innovative, efficient, and effective way in areas considered important to the development of the BMDS. Because award fees are intended to motivate contractor performance for work that is neither feasible nor effective to measure objectively, award fee criteria and evaluations tend to be subjective. Each element's contract has an award fee plan that identifies the performance areas to be evaluated and the methodology by which those areas will be assessed. An award fee evaluation board--made up of MDA personnel, program officials, and officials from key organizations knowledgeable about the award fee evaluation areas--judges the contractor's performance against specified criteria in the award fee plan. The board then recommends to a fee determining official the amount of fee to be paid. MDA's Director is the fee-determining official for all BMDS prime contracts that we assessed. During fiscal year 2007, MDA awarded approximately 95 percent, or $606 million, of available award fee to its prime contractors. While the cost, schedule, and technical performance of several contractors appeared to be aligned with their award fee, two contractors were rated as performing very well in the cost and/or program management elements and received commensurate fees even though earned value management data showed that their cost and schedule performance was declining. On the other hand, MDA did not award any fee to the THAAD contractor for its management of contract cost during a time when earned value data showed steadily increasing costs. Although DOD guidance discourages the use of earned value performance metrics in award fee criteria, MDA includes this as a factor in several of its award fee plans. The agency considers many factors in rating contractors' performance and making award fee determinations, including consideration of earned value data that shows cost, schedule, and technical trends. In addition, MDA has begun to revise its award fee policy to align agency practices more closely with DOD's current policy that better links performance with award fees. The ABL and Aegis BMD weapon system contractors received a large percentage of the 2007 award fee available to them for the cost and/or program management element. MDA rated the ABL contractor's performance in cost and program management elements as "very good," awarding the contractor 88 percent of the fee available in these performance areas. According to the award fee plan, one of several factors that is considered in rating the contractor's performance as very good is whether earned value data indicates that there are few unfavorable cost, schedule, and/or technical variances or trends. During the February 2006 to January 2007 award fee period, earned value data shows that the contractor overran its budget by more than $57 million and did not complete $11 million of planned work. Similarly, the Aegis BMD weapon system contractor was to be rated as to how effectively it managed its contract's cost. The award fee plan for this contractor also directs that earned value be one of the factors considered in making such an evaluation. During the fee period that ran from October 2006 through March 2007, MDA rated the contractor's cost management performance as outstanding and awarded 100 percent of the available fee. Earned value data during this time period indicates that the contractor overran its budget by more than $6 million. MDA did not provide us with more detailed information as to other factors that may have influenced its decision as to the amount of fee awarded to the ABL and Aegis BMD Weapon System contractors. MDA recognizes that there is not always a good link between the agency's intentions for award fees and the amount of fee being earned by its contractors. In an effort to rectify this problem, the agency released a revised award fee policy in February 2007 to ensure its compliance with recent DOD policies that are intended to address award fee issues throughout the Department. Specifically, MDA's policy directs that every contract's award fee plan include: * Criteria for each element of the award fee that is specific enough to enable the agency to evaluate contractor performance and to determine how much fee the contractor can earn for that element. The criteria is to clearly define the performance that the government expects from the contractor for the applicable award fee period and the criteria for any one element must be distinguishable from criteria for other elements of the award fee; * An emphasis on rewarding results rather than effort or activity; and: * An incentive to meet or exceed agency requirements. Additionally, MDA's policy calls for using the Award Fee Advisory Board to not only make award fee recommendations to the fee determining official, but to also biannually report to MDA's Director as to whether award fee recommendations are consistent with DOD's Contractor Performance Assessment Report--a report that provides a record, both positive and negative, on a given contract for a specific period of time. Appendix II of this report provides additional information on BMDS prime contracts and award fees. Most Test Objectives Achieved Despite Schedule Delays: During 2007, several BMDS programs experienced setbacks in their test schedules. The Aegis BMD, THAAD, ABL, STSS, and C2BMC elements experienced test delays, but all were able to achieve their primary test objectives. GMD, on the other hand, experienced a schedule delay caused by an in-flight target anomaly that prevented full accomplishment of one major 2007 test objective. The remaining three elements--MKV, KEI, and Sensors--were able to execute all scheduled activities as planned. Several Elements Met Key Test Objectives: The Aegis BMD, THAAD, C2BMC, ABL, and STSS elements continued to achieve important test objectives in 2007, although some tests were delayed. Aegis BMD proved its capability against more advanced threats, while THAAD proved that it could intercept both inside and outside of the atmosphere. C2BMC completed a number of software and system-level tests. The ABL and STSS programs saw delays in important ground tests, but ABL was able to begin flight testing its beam control/fire control component using a low-power laser in 2007 and STSS completed thermal vacuum testing of both satellites by the end of the year. However, the delays in the ABL and STSS programs may hold up their incorporation into the BMDS during future blocks. Although the Aegis BMD program encountered some test delays, it was able to achieve all fiscal year 2007 test objectives. In December 2006, the program stopped a test after a crew member changed the ship's doctrine parameters just prior to target launch, preventing the ship's fire control system from conducting the planned engagement. During this test event, the weapon system failed to recognize the test target as a threat, which prevented the SM-3 missile from launching. Also, according to program officials, the system did not provide a warning message which contributed to the mission being aborted prematurely and prevented the Aegis BMD program from meeting its test objectives. However, 4 months later, the same flight test event was successfully completed and all test objectives were met. During that event, the program was able to demonstrate that the Aegis BMD could simultaneously track and intercept a ballistic missile and an anti-ship cruise missile. In June 2007, the program successfully completed its first flight test utilizing an Aegis BMD destroyer to intercept a separating target, and in November, the program conducted its first test that engaged two ballistic missile targets simultaneously. During the last test, Aegis missiles onboard an Aegis BMD cruiser successfully intercepted two short-range non-separating targets and achieved all primary test objectives outlined for this event. The THAAD program expected to complete four flight tests prior to the end of fiscal year 2007 but was only able to complete three. Two tests successfully resulted in intercepts of short-range ballistic missiles at different levels of the atmosphere. The third test successfully demonstrated component capability in a high-pressure environment and was the lowest altitude interceptor verification test to date. However, the fourth test was delayed, initially due to target availability driven by late modifications to the target hardware configuration. Additionally, during pre-flight testing, the contractor found debris in the interceptor. This caused the interceptor to be returned to the factory for problem investigation. While the problem was corrected and the interceptor was returned to the test range in only 7 days, the test was rescheduled because the test range was not available before the end of fiscal year 2007. During fiscal year 2007, the C2BMC program completed BMDS-level ground and flight tests, successfully achieving its test objectives of verifying the capabilities and readiness of a new software configuration. The software is designed to provide the BMDS with improved defense planning capability, including better accuracy and speed; a new operational network; and additional user displays. Because of the integral nature of the C2BMC product, problems encountered in some elements' test schedules have a cascading effect on C2BMC's test schedule. Even though this limited C2BMC testing, a review of the integrated and distributed ground test data resulted in the decision to field the software in December 2007. ABL achieved most of its test objectives during fiscal year 2007, but experienced delays during Block 2006 that deferred future BMDS program decisions. The program experienced a number of technical problems during fiscal year 2006 that pushed some planned activities into fiscal year 2007. One such activity was the execution of the program's first of four key knowledge points--a ground test to demonstrate ABL's ability to acquire and track a target while performing atmospheric compensation.[Footnote 9] The test was conducted in December 2006, 3 months later than planned. At the culmination of the test, program officials noted two problems. First, the system's beam control/fire control software was not integrated as anticipated. In addition, the energy that the low-power laser placed on the target during the test was not optimal. According to program officials, both of these issues were resolved before the system began flight testing the full beam control/fire control component in February 2007. However, the delays caused the program to further postpone a key lethality demonstration-- a demonstration in which the ABL will attempt to shoot down a short- range ballistic missile--until last quarter of fiscal year 2009. This demonstration is important to the program because it is the point at which MDA will decide the program's future. Although the ABL program experienced some setbacks with its first key knowledge point, it was able to meet all objectives for each subsequent knowledge point. In addition to the first knowledge point, the program planned to demonstrate three additional knowledge points during fiscal year 2007. The second knowledge point was contingent upon completion of the first. To demonstrate the achievement of the two knowledge points, the contractor performed a flight test that showed the low-power laser was integrated and the beam control/fire control functioned sufficiently to perform target tracking and atmospheric compensation against an airborne target board. The third knowledge point was completed three months ahead of the planned 2007 schedule and demonstrated that ABL's optical subsystem was adequate to support its high-power laser system. The fourth knowledge point-the completion of a series of flight tests to demonstrate the performance of the low-power laser system in flight--was completed in August 2007. Delays in the STSS test program, along with funding shortages, postponed the planned 2007 launch of the program's demonstration satellites. The STSS program is integrating two demonstration satellites with sensor payloads from legacy hardware developed under a former program. The use of legacy hardware has complicated the test program because spares needed for testing are not available. In order to preserve the condition of the legacy components, the program must exercise caution in handling the components to prevent damage, which has caused delays in testing. Additionally, a thermal vacuum test on the first space vehicle, to assess the ability of the satellite to operate in the cold vacuum of space, took twice as long as scheduled, due to a number of interface issues.[Footnote 10] Although the program was able to complete the integration and test of both demonstration satellites in 2007--major objectives for the program--funds were not available to launch the satellites as planned. Program officials believe that the satellites could be launched as early as April 2008 and as late as July 2008, 1 year later than originally scheduled. According to the program office, there is no margin in the 2008 budget, so any unexpected issues could put the 2008 launch date at risk. The delays in launching the STSS demonstration satellites do not impact MDA's Block 2006 fielding plans as the satellites are intended to demonstrate a surveillance and tracking capability and do not provide any operational capability during the block. However, the delay in launching the demonstration satellites is causing a delay in MDA's ability to initiate development of an operational constellation, which may delay a BMDS global midcourse tracking capability. Despite delays in hardware and software testing and integration, other parts of the STSS program have proceeded according to schedule. Lessons learned from the thermal vacuum test for the first satellite's sensor payload facilitated the completion of thermal vacuum testing of the second satellite's payload in November 2007. Additionally, command and control capabilities of the ground segment were demonstrated and the second part of the acceptance test of STSS ground components was completed in September 2007. Test Delays Prevent GMD from Achieving All 2007 Objectives: A target anomaly prevented the GMD element from achieving all 2007 objectives. The GMD program planned to conduct three flight tests-two intercept attempts and one radar characterization test--but was only able to conduct the radar test and one intercept test. The radar characterization test was conducted in March 2007. The target was launched from Vandenberg Air Force Base and was successfully tracked by the SBX radar and the radar of two Aegis BMD ships. During the test, officials indicated the SBX exhibited some anomalous behavior, yet was able to collect target tracking data and successfully transmit the information to the C2BMC element and the GMD fire control system at DOD's Missile Defense Integration and Operations Center. No live interceptor was launched. However, an intercept solution was generated and simulated interceptor missiles were "launched" from Fort Greely, Alaska. To address anomalous behavior, MDA adjusted software and performance parameters of the SBX radar. In May 2007, the program attempted an intercept test, but a key component of the target malfunctioned. For that reason, the weapon system did not release the Ground-based interceptor and program officials declared the flight test a "no test" event. To date, program officials have not determined the root cause of the malfunction. In September 2007, the program successfully conducted a re-test and achieved an intercept of the target using target tracking data provided by the Beale upgraded early warning radar. MDA test officials told us that aging target inventory could have contributed to the target anomaly. The officials explained that some targets in MDA's inventory are more than 40 years old and their reliability is relatively low. Target officials told us that they are taking preventive actions to avoid similar anomalies in the future. The time needed to complete the first 2007 intercept delayed GMD's second planned intercept attempt until at least the second quarter of fiscal year 2008. The delayed test was to have determined whether the SBX radar could provide data in "real time" that could be used by the GMD fire control component to develop a weapon task plan. Although the weapon task plan was not developed in real time during 2007, GMD was able to demonstrate that the SBX radar could plan an engagement when the target was live but the interceptor was simulated. Three BMDS Elements Completed Activities as Planned: During 2007, the KEI program redefined its development efforts and focused on near-term objectives. Also, the MKV program redefined its strategy to acquire multiple kill capability. Once redefined, these programs conducted all planned activities as scheduled and each was able to meet all planned objectives. In addition, the Sensors program successfully completed all planned tests. In June 2007, MDA directed the KEI program to focus on two near-term objectives--the development of its booster and its 2008 booster flight test. Some work, such as development of the fire control and communications and mobile launcher, was deferred into the future. During fiscal year 2007, the KEI program conducted all planned test activities, including booster static fire tests that demonstrated the rocket motor's performance in induced environments and wind tunnel tests that gathered data to validate aerodynamic models for the booster flight controls. MKV officials redefined their acquisition strategy by employing a parallel path to develop multiple kill vehicles for the GMD and KEI interceptors and the Aegis BMD SM-3 missile. MDA initiated the MKV program in 2004 with Lockheed Martin. In 2007, the MKV program added Raytheon as a second payload provider. According to program officials, the two payload providers may use different technologies and design approaches, but both adhere to the agency's goal of delivering common, modular MKV payloads for integration with all BMDS midcourse interceptors. In fiscal year 2007, Lockheed Martin successfully conducted static fire tests of its Divert Attitude Control System as planned. Additionally, Raytheon, funded with excess KEI funds made available when that program was replanned, began concept development. Raytheon did not have any major test activities scheduled for the fiscal year. During 2007, the Sensors program focused on testing FBX-T radars that were permanently emplaced and newly produced. After the first FBX-T was moved from its temporary location in Japan to its permanent location in Shariki, Japan, various ground tests and simulations were conducted to ensure its interoperability with the BMDS. The program also delivered a second FBX-T to Vandenberg Air Force Base, where its tracking capability is being tested against targets of opportunity. According to program officials, a decision has not been made as to where the second FBX-T radar will be permanently located. BMDS Achievement of Performance Goals Remains Uncertain: As we reported in March 2007, MDA altered its original Block 2006 performance goals commensurate with the agency's reductions in the delivery of fielded assets.[Footnote 11] However, insufficient data exists to fully assess whether MDA achieved its revised performance goals. The performance of some fielded assets is also questionable because parts have not yet been replaced that were identified by auditors in MDA's Office of Quality, Safety, and Mission Assurance as less reliable or inappropriate for use in space. In addition, tests of the GMD element have not included target suite dynamic features and intercept geometries representative of the operational environment in which GMD will perform its mission and BMDS tests only allow a partial assessment of the system's effectiveness, suitability, and survivability. MDA uses a combination of simulations and flight tests to determine whether performance goals are met. Models and simulations are needed to predict performance because the cost of tests prevents the agency from conducting sufficient testing to compute statistical probabilities of performance. The models and simulations that project BMDS capability against intercontinental ballistic missiles present several problems. First, the models and simulations that predict performance of the GMD element have not been accredited by an independent agency. According to the Office of the Director, Operational Test and Evaluation without accredited models, GMD's performance cannot be predicted with respect to (1) variations in threat parameters that lie within the bounds of intelligence estimates, (2) stressing ground-based interceptor fly- outs and exoatmospheric kill vehicle engagements, and (3) variations in natural environments that lie within meteorological norms. Second, too few flight tests have been completed to ensure the accuracy of the models' and simulations' predictions. Since 2002, MDA has only completed two end-to-end tests of engagement sequences that the GMD element might carry out. While these tests provide some evidence that the element can work as intended, MDA must test other engagement sequences, which would include other GMD assets that have not yet participated in an end-to-end flight test. For example, MDA has not yet used the Sea-based X-band radar as the primary sensor in an end-to-end test. Additionally, officials in the Office of the Director, Operational Test and Evaluation told us that MDA needs more flight tests to have a high level of confidence that GMD can repeatedly intercept incoming ICBMs. Further testing is also needed to demonstrate that Aegis BMD can provide real-time, long-range surveillance and tracking data for the GMD element. In March 2006, we reported that the cancellation of a GMD flight test prevented MDA from exercising Aegis BMD's long-range surveillance and tracking capability in a manner consistent with an actual defensive mission.[Footnote 12] Program officials informed us that the Aegis BMD is capable of performing this function and has demonstrated its ability to surveil and track ICBMs in several exercises. However, MDA has not yet shown that Aegis BMD can communicate this data to GMD during a live intercept engagement and that GMD can use the data to prepare a weapon task plan for actual-- rather than simulated--interceptors. Officials in the Office of the Director for Operational Test and Evaluation told us that having Aegis BMD perform long-range surveillance and tracking during a live engagement would provide the data needed to more accurately gauge performance. Similarly, MDA has not yet proved that the FBX-T radar can provide real- time, long-range surveillance and tracking data for the GMD element. On several occasions, MDA has shown that the FBX-T can acquire and track targets of opportunity, but the radar's data has not yet been used to develop a weapon system task plan for a GMD intercept engagement. Because the radar's permanent location in Japan does not allow MDA to conduct tests in which the FBX-T is GMD's primary fire control radar, the Director, Operational Test and Evaluation, in 2006 recommended that prior to emplacing a second FBX-T at its permanent location that MDA test the radar's capability to act as GMD's primary sensor in an intercept test. Confidence in the performance of the BMDS is also reduced because of unresolved GMD technical and quality issues. The GMD element has experienced the same anomaly during each of its flight tests since 2001. This anomaly has not yet prevented the program from achieving any of its primary test objectives, but to date neither its source nor solution has been clearly identified or defined. Program officials plan to continue their assessment of test data to identify the anomaly's root cause and have implemented design changes to mitigate the effects and reduce risks associated with the anomaly. The reliability of emplaced GMD interceptors raises further questions about the performance of the BMDS. Quality issues discovered by auditors in MDA's Office of Quality, Safety, and Mission Assurance nearly 3 years ago have not yet been rectified in all fielded interceptors. According to the auditors, inadequate mission assurance and quality control procedures may have allowed less reliable parts or parts inappropriate for use in space to be incorporated into the manufacturing process, thereby limiting the reliability and performance of some fielded assets. The program has strengthened its quality control processes and is taking several steps to mitigate similar risks in the future. These steps include component analysis of failed items, implementing corrective action with vendors, and analyzing system operational data to determine which parts are affecting weapon system availability. MDA has begun to replace the questionable parts in the manufacturing process and to purchase the parts that it plans to replace in fielded interceptors. However, it will not complete the retrofit effort until 2012. Additionally, test officials told us that although the end-to-end GMD test conducted during 2007 demonstrated that for a single engagement sequence military operators could successfully engage a target, the target represented a relatively unsophisticated threat because it lacked specific target suite dynamic features and intercept geometry. Other aspects of the test were more realistic--such as closing velocity and fly-out range--but these were relatively unchallenging. While the test parameters may be acceptable in a developmental test, they are not fully representative of an operational environment and do not provide high confidence that GMD will perform well operationally. Finally, because BMDS assets are being fielded based on developmental tests, which are not always representative of the operational environment, operational test officials have limited test data to determine whether all BMDS elements/components being fielded are effective and suitable for and survivable on the battlefield. MDA has added operational test objectives to its developmental test program, but many of the objectives are aimed at proving that military personnel can operate the equipment. In addition, limited flight test data is available for characterizing the BMDS' capability against intercontinental ballistic missiles. Up until 2007, the overall lack of data limited the Office of the Director of Operational Test and Evaluation, in annual assessments, to commenting on the operational realism of tests and recommending other tests needed to characterize system effectiveness and suitability. In 2007, tests provided sufficient information to partially quantify the effectiveness and suitability of the BMDS' midcourse capability (Aegis BMD and GMD) and to fully characterize a limited portion of the BMDS' terminal capability (PAC-3). However, according to the Office of the Director of Operational Test and Evaluation, further testing that incorporates realistic operational objectives and verification, validation, and accreditation of models and simulations will be needed before the performance, suitability, and survivability of the BMDS can be fully characterized. Efforts Underway to Improve BMDS Management and Oversight: Since its initiation in 2002, MDA has been given a significant amount of flexibility in executing the development of the BMDS. While the flexibility has enabled MDA to be agile in decision making and to field an initial capability relatively quickly, it has diluted transparency into MDA's acquisition processes, making it difficult to conduct oversight and hold the agency accountable for its planned outcomes and costs. As we reported in 2007, MDA operates with considerable autonomy to change goals and plans, which makes it difficult to reconcile outcomes with original expectations and to determine the actual cost of each block and of individual operational assets. In the past year, MDA has begun implementing two initiatives--a new block construct and a new executive board-to improve transparency, accountability, and oversight. These initiatives represent improvements over current practices, although they provide for less oversight than statutes provide for other major defense acquisition programs. In addition, Congress has directed that MDA's budget materials, after 2009, request funds using the appropriation categories of research, development, and evaluation, procurement, operations and maintenance, and military construction, which should promote accountability for and transparency of the BMDS.[Footnote 13] New Block Construct Offers Improvements, but Does Not Address All Issues: In 2007, MDA redefined its block construct to better communicate its plans and goals to Congress. The agency's new construct is based on fielding capabilities that address particular threats as opposed to the biennial time periods that were the agency's past approach to development and fielding. MDA's new block construct makes many positive changes. These include establishing unit cost for selected block assets, including in a block only those elements or components that will be fielded during the block, and abandoning the practice of deferring work from block to block. Features of New Block Construct: Table 5 illustrates MDA's new block construct for fielding the BMDS. Table 5: MDA New Block Construct: Block: Block 1.0: Initial Defense of U.S. from North Korea; Description: Includes quantities, schedule, performance metrics, engagement sequence groups, and cost needed to defend U.S. homeland from limited North Korean long-range threats. Block: Block 2.0: Initial Defense of Allied Forces; Description: Includes quantities, schedule, performance metrics, engagement sequence groups, and cost needed to defend allies and deployed forces from short-to medium-range threats in one region/theater. Block: Block 3.0: Initial Defense of U.S. from Iran; Description: Includes quantities, schedule, performance metrics, engagement sequence groups, and cost need to expand defense of U.S. to include limited Iranian long-range threats. Block: Block 4.0: Expanded Defense of U.S. and Allies from Iran; Description: Includes quantities, schedule, performance metrics, engagement sequence groups, and cost needed to defend allies and deployed forces in Europe from limited Iranian long-range threats and to expand protection of the U.S. homeland. Block: Block 5.0: Expanded Defense of Allied Forces; Description: Includes quantities, schedule, performance metrics, engagement sequence groups, and cost needed to expand defense of allies and deployed forces from short-to intermediate-range threats in two regions/theaters. Categories: Capability Development; Description: Includes BMDS elements that are not ready to be fielded, such as ABL and MKV. These programs have knowledge points tailored to critical risks. Categories: Sustainment; Description: Includes annual operations and sustainment costs. Categories: Mission Area Investment; Description: Investments that cut across several blocks and cannot be reasonably allocated to a specific block. Examples include modeling and simulation and intelligence and security. Categories: MDA Operations; Description: Contains operations support functions such as MDA headquarters management. Source: MDA data. Note: Capability development programs will transition to a block when a match exists between desired capability and resources. The decision to initiate a new block will be made by the Director, MDA. The Director will consider the severity of threats, the imminence of threats, achievement of knowledge points, proven technologies, cost estimates, and funding for each element prior to its transition. [End of table] MDA's new block construct provides a means for comparing the expected and actual unit cost of assets included in a block. As we noted in our fiscal year 2006 report, MDA's past block structure did not estimate unit costs for assets considered part of a given block or categorize block costs in a manner that allowed calculations of expected or actual unit costs.[Footnote 14] For example, the expected cost of Block 2006 GMD interceptors emplaced for operational use was not separated from other GMD costs. Even if MDA had categorized the interceptors' cost, it would have been difficult to determine the exact cost of these interceptors because MDA acquires and assembles components into interceptors over several blocks and it has been difficult to track the cost of components to a specific group of interceptors. Under the new block construct, MDA expects to develop unit costs for selected block assets--such as THAAD interceptors--and request an independent verification of that unit cost from DOD's Cost Analysis Improvement Group. MDA will also track the actual unit cost of the assets and report significant cost growth to Congress. However, MDA has not yet determined for which assets a unit cost will be developed and how much a unit cost must increase before that increase is reported to Congress. The new construct also makes it clearer as to which assets should be included in a block. Under the agency's prior block construct, assets included in a given block were sometimes not planned for delivery until a later block. For example, as we reported in March 2007, MDA included costs for ABL and STSS as part of its Block 2006 cost goal although those elements did not field or plan to field assets during Block 2006.[Footnote 15] Agency officials told us those elements were included in the block because they believed the elements could offer some emergency capability during the block timeframe. Finally, the new block construct should improve the transparency of each block's actual cost. Under its prior construct, MDA deferred work from one block to another; but it did not track the cost of the deferred work so that it could be attributed to the block that it benefited. For example, MDA deferred some work needed to characterize and verify the Block 2004 capability until Block 2006 and counted the cost of those activities as a cost of Block 2006. By doing so, it understated the cost of Block 2004 and overstated the cost of Block 2006. Because MDA did not track the cost of the deferred work, the agency was unable to adjust the cost of either block to accurately capture the cost of each. MDA officials told us that under its new block construct, MDA will no longer transfer work, along with its cost, to a future block. Rather, a block of work will not be considered complete until all work that benefits a block has been completed and its cost has been properly attributed to that block. Block Construct Does Not Resolve All Issues: Although improvements are inherent in MDA's new block construct, the new construct will not dispel all transparency and accountability concerns. MDA has not yet estimated the full cost of a block. Also, MDA has not addressed whether it will transfer assets produced during a block to a military service for production and operation at the block's completion, or whether MDA will continue its practice of concurrently developing and fielding BMDS elements and components. According to its fiscal year 2009 budget submission, MDA does not plan to initially develop a full cost estimate for any BMDS block. Instead, when a firm commitment can be made to Congress for a block of capability, MDA will develop a budget baseline for the block. This budget will include anticipated funding for each block activity that is planned for the 6 years included in DOD's Future Years Defense Plan. MDA officials told us that if the budget for a baselined block changes, MDA plans to report and explain those variations to Congress.[Footnote 16] At some future date, MDA does expect to develop a full cost estimate for each committed block and is in discussions with DOD's Cost Analysis Improvement Group on having the group verify each estimate; but documents do not yet include a timeline for estimating block cost or having that estimate verified. For accountability, other DOD programs are required to provide the full cost of developing and producing their weapon system before system development and demonstration can begin. Until the cost of a block of BMDS capability is fully known, it will be difficult for decision makers to compare the value of investing in a block of BMDS capability to the value of investing in other DOD programs or to determine whether the block of capability that is being initiated will be affordable over the long term. The new block construct does not address whether the assets included in a block will be transferred at the block's completion to a military service for production and operation. Officials representing multiple DOD organizations recognize that the transfer criteria established in 2002 are neither complete nor clear given the BMDS's complexity. Without clear transfer criteria, MDA has transferred the management of only one element--the Patriot Advanced Capability-3--to the military for production and operation. Joint Staff officials told us that for all other elements, MDA and the military services have been negotiating the transition of responsibilities for the sustainment of fielded elements--a task that has proven arduous and time consuming. Although MDA documents show that under its new block construct the agency should be ready at the end of each block to deliver BMDS components that are fully mission-capable, MDA officials could not tell us when MDA's Director will recommend that management of components, including production responsibilities, be transferred to the military. MDA officials maintain that even though a particular configuration of a weapon could be fully mission-capable, that configuration may never be produced because it could be replaced by a new configuration. Yet, by the block's end, a transfer plan for the fully mission-capable configuration will have been drafted, developmental ground and flight tests will be complete, elements and components will be certified for operations, and doctrine, organization, training, material, leadership, personnel, and facilities are expected to be in place. Another issue not addressed under MDA's new block construct is whether the concurrent development and fielding of BMDS elements and/or components will continue. Fully developing a component or element and demonstrating its capability prior to production increases the likelihood that the product will perform as designed and can be produced at the cost estimated. To field an initial capability quickly, MDA accepted the risk of concurrent development and fielding during Block 2004. For example, by the end of Block 2004, the agency realized that the performance of some Ground-based interceptors could be degraded because the interceptors included inappropriate or potentially unreliable parts. MDA has begun the process of retrofitting these interceptors, but work will not be completed until 2012. Meanwhile there is a risk that some interceptors might not perform as designed. MDA also continued to accept this risk during Block 2006 as it fielded assets before they were fully tested. MDA has not addressed whether it will accept similar performance risks under its new block construct or whether it will fully develop and demonstrate all elements/components prior to fielding. New Executive Board Offers Improved, but Not Full, Oversight: In March 2007, the Deputy Secretary of Defense established a Missile Defense Executive Board (MDEB) to recommend and oversee implementation of strategic policies and plans, program priorities, and investment options for protecting the United States and its allies from missile attacks. The MDEB was also to replace existing groups and structures, such as the Missile Defense Support Group (MDSG). However, while it has some oversight responsibilities, the MDEB was not established to provide full oversight of the BMDS program and it would likely be unable to carry out this mission even if tasked to do so. The MDEB will not receive some information that the Defense Acquisition Board relies upon to make program recommendations, and in other cases, MDA does not plan to seek the MDEB's approval before deciding on a course of action.[Footnote 17] In addition, there are parts of the BMDS program for which there will be no baseline against which progress can be measured, which makes oversight difficult. Functions of the New Board: According to its charter, the MDEB is vested with more responsibility than its predecessor, the MDSG. When the MDSG was chartered in 2002, it was to provide constructive advice to MDA's Director. However, the Director was not required to follow the advice of the group. According to a DOD official, although the MDSG met many times initially, it did not meet after June 2005. This led, in 2007, to the formation of the MDEB. This board's mission is to review and make recommendations on MDA's comprehensive acquisition strategy to the Deputy Secretary of Defense. It is also to provide the Under Secretary of Defense, Acquisition, Technology and Logistics, with a recommended strategic program plan and a feasible funding strategy based on "business case" analysis that considers the best approach to fielding integrated missile defense capabilities in support of joint MDA and warfighter objectives: The MDEB will be assisted by four standing committees. These committees, which are chaired by senior-level officials from the Office of the Secretary of Defense and the Joint Staff, could play an important oversight role as they are expected to make recommendations to the MDEB, which in turn will recommend courses of action to the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD AT&L) and the Director, MDA, as appropriate. The following table identifies the chair of each standing committee as well as key committee functions. Table 6: MDEB Standing Committee Functions: Standing committee: Policy Oversight; Committee chair: Principal Deputy Under Secretary of Defense for Policy; Function: * Advises the board on strategic missile defense policy direction to ensure full consistency with DOD policy; * Conducts and oversees international activities; * Represents the Department in inter-agency matters. Standing committee: Operational Forces; Committee chair: Vice Chairman, Joint Chiefs of Staff; Function: * Oversees fielding schedules and deployments to ensure consistency with planned schedules and DOD objectives; * Oversees agreements, documentation, and requirements between MDA, the DOD components, and the fielding organizations for ensuring appropriate funding policies for operational and support resources. Standing committee: Program, Acquisition and Budget Development Committee; Committee chair: Deputy Under Secretary of Defense, Acquisition, Technology, and Logistics; Function: * Ensures that missile defense program and budget development is integrated effectively into the board's oversight role and that missile defense programs are properly aligned with missions, taking appropriate account of relevant risk factors; * Oversees implementation of the missile defense acquisition guidance to include transition and transfer of responsibilities/authorities of the system from MDA to the services; * Provides oversight for missile defense system procurement, operation, and support. Standing committee: Test and Evaluation; Committee chair: Deputy Under Secretary of Defense, Acquisition, Technology, and Logistics; Function: * Oversees the test and evaluation planning and resource road map as it relates to MDA test requirements and test program; * Provides technical recommendations and oversight for the conduct of an integrated T&E program and investment strategy. Source: DOD. [A] The acronyms C4 and ISR stand for command, control, communications, and computers and intelligence, surveillance, and reconnaissance, respectively. [End of table] Limitations of the Board: The MDEB will not have access to all information normally available to oversight bodies. For other major defense acquisition programs, the Defense Acquisition Board must approve the program's progress through the acquisition cycle. Further, before a program can enter the System Development and Demonstration phase of the cycle, statute requires that certain information be developed. This information is then provided to the Defense Acquisition Board. However, in 2002, the Secretary of Defense allowed MDA to defer application of the defense acquisition system that among other things require programs to follow a defined acquisition cycle and obtain approval before advancing from one phase of the cycle to another. Because MDA does not follow this cycle, it does not enter System Development and Demonstration and it does not trigger the statutes requiring the development of information that the Defense Acquisition Board uses to inform its decisions. For example, most major defense acquisition programs are required by statute to obtain an independent verification of program cost prior to beginning system development and demonstration, and/or production and deployment.[Footnote 18] Independent life-cycle cost estimates provide confidence that a program is executable within estimated cost and along with other DOD-wide budget demands. Although MDA plans to develop unit cost for selected block assets and request that DOD's Cost Analysis Improvement Group verify the unit costs, the agency does not initially plan to develop a block cost estimate and, therefore, cannot seek an independent verification of that cost. In addition, even when MDA estimates block costs, the agency will not be required to obtain an independent verification of that cost, because, as noted earlier, the BMDS program operates outside of DOD's acquisition cycle. Although not required, MDA officials told us that they have initiated discussions with the Cost Analysis Improvement Group on independent verifications of block cost estimates. Statute also requires an independent verification of a system's suitability for and effectiveness on the battlefield before a program can proceed beyond low-rate initial production.[Footnote 19] After the test is completed, the Director for Operational Test and Evaluation assesses whether the test was adequate to support an evaluation of the system's suitability and effectiveness for the battlefield, whether the test showed the system to be acceptable, and whether any limitations in suitability and effectiveness were noted. However, a comparable assessment of the BMDS assets being produced for fielding will not be available to the MDEB. As noted earlier, the limited amount of testing completed, which has been primarily developmental in nature, and the lack of verified, validated, and accredited models and simulations prevent the Director of Operational Test and Evaluation from fully assessing the effectiveness, suitability, and survivability of the BMDS in annual assessments. MDA will also make some decisions without approval from the MDEB or any higher level DOD official. Although the charter of the MDEB includes the mission to make recommendations to MDA and the Under Secretary of Defense for AT&L on investment options, program priorities, and MDA's strategy for developing and fielding an operational missile defense capability, the MDEB will not have the opportunity to review and recommend changes to BMDS blocks. According to a briefing on the business rules and processes for MDA's new block structure, the decision to initiate a new block of BMDS capability will be made by MDA's Director. Also cost, schedule, and performance parameters will be established by MDA when technologies that the block depends upon are mature, a credible cost estimate can be developed, funding is available, and the threat is both imminent and severe. The Director will inform the MDEB as well as Congress when a new block is initiated, but he will not seek the approval of either. Finally, there will be parts of the BMDS program that will be difficult for the MDEB to oversee because of the nature of the work being performed. MDA plans to place any program that is developing technology in a category known as Capability Development. These programs, such as ABL, KEI, and MKV, will not have a firm cost, schedule, or performance baseline. This is generally true for technology development programs in DOD because they are in a period of discovery, which makes schedule and cost difficult to estimate. On the other hand, the scale of the technology development in BMDS is unusually large, ranging from $2 billion to about $5 billion dollars a year--eventually comprising nearly half of MDA's budget by fiscal year 2012. The MDEB will have access to the budgets planned for these programs over the next 5 or 6 years, each program's focus, and whether the technology is meeting short-term key events or knowledge points. But without some kind of baseline for matching progress with cost, the MDEB will not know how much more time or money will be needed to complete technology maturation. MDA's experience with the ABL program provides a good example of the difficulty in estimating the cost and schedule of technology development. In 1996, the ABL program believed that all ABL technology could be demonstrated by 2001 at a cost of about $1 billion. However, MDA now projects that this technology will not be demonstrated until 2009 and its cost has grown to over $5 billion. While the uncertainties of technology development must be recognized, some organizations suggest ways to establish a baseline appropriate for such efforts. For example, the Air Force Research Laboratory suggested a methodology to estimate a technology's cost once analytical and laboratory studies physically validate analytical predictions of separate elements of the technology. MDEB Could Play Key Role In Proposal to Return BMDS to Standard Requirements Process: In an effort to further improve oversight, the Joint Requirements Oversight Council proposed a plan to transition the BMDS into standard DOD processes. In August 2007, the Vice Chairman of the Joint Chiefs of Staff and Joint Requirements Oversight Council Chairman requested the Deputy Secretary of Defense approve a proposal to return MDA to the Joint Capabilities Integration and Development System process and direct the Joint Requirements Oversight Council to validate BMDS capabilities.[Footnote 20] The Vice Chairman believed that the council should exercise oversight of MDA in order to improve Department-wide capability integration. More specifically, he noted that: * In 2002, the Secretary of Defense exempted the BMDS program from the traditional requirements generation process to expedite fielding the system as soon as practicable. * Now that an initial capability for homeland defense has been deployed, there is no longer the same need for flexibility provided by the requirements exemption. * The current process, with MDA exempted, does not allow the Joint Requirements Oversight Council to provide appropriate military advice or to validate missile defense capabilities. Without this change, there is increasing potential that MDA-fielded systems will not be synchronized with other air and missile defense capabilities being developed. * The current process hinders the military departments' ability to plan and program resources for fielding and sustainment of MDA-developed systems. In responding to the proposal, the Acting Under Secretary of Defense for AT&L recommended that the Deputy Secretary of Defense delay his approval of the Joint Staff's proposal until the MDEB could review the proposal and provide a recommendation. However, he agreed that more Joint Requirements Oversight Council involvement was necessary for the BMDS, although he was not sure that returning BMDS to standard DOD processes was the appropriate solution to the agency's oversight issues. Instead, he noted that the Deputy Secretary of Defense recently established the MDEB to recommend and oversee the implementation of strategic policies and plans, program priorities, and investment options for the BMDS. He stated that since the MDEB is tasked with determining the best means of managing the BMDS throughout its life cycle, it should consider the Joint Staff's proposal. MDA Directed to Use Procurement Funding for the First Time: In an effort to improve the transparency of MDA's acquisition processes, Congress has directed that MDA's budget materials delineate between funds needed for research, development, and evaluation, procurement, operations and maintenance, and military construction.[Footnote 21] Using procurement funds will mean that MDA generally will be required to adhere to congressional policy that assets be fully funded in the year of their purchase, rather than incrementally funded over several years. The Congressional Research Service reported in 2006 that "incremental funding fell out of favor because opponents believed it could make the total procurement costs of weapons and equipment more difficult for Congress to understand and track, create a potential for DOD to start procurement of an item without necessarily stating its total cost to Congress, permit one Congress to 'tie the hands' of future Congresses, and increase weapon procurement costs by exposing weapons under construction to uneconomic start-up and stop costs."[Footnote 22] Our analysis of MDA developed costs, which are presented in table 7, also shows that incremental funding is usually more expensive than full funding, in part, because inflation decreases the buying power of the dollar each year. The National Defense Authorization Act for Fiscal Year 2008 directed MDA to submit a plan to transition from using research and development funds exclusively to using procurement, operations and maintenance, military construction, and research and development funds by March 1, 2008. However, it allowed MDA to continue to use research and development funds in fiscal year 2009 to incrementally fund previously approved missile defense assets. The act also directed that beginning in fiscal year 2009, the MDA budget request include, in addition to RDT&E funds, military construction funds and procurement funds for some long lead items such as those required for the third and fourth THAAD fire units and Aegis BMD SM-3 Block 1A missiles. MDA did not request long lead funding for either THAAD or SM-3 missiles in its fiscal year 2009 budget because MDA has slipped the schedule for procuring fire units 3 and 4 by one year, and the National Defense Authorization Act for Fiscal Year 2008 was not signed in time to allow MDA to adjust its budget request for SM-3 missiles. Congress also provided MDA with the authority to use procurement funds for fiscal years 2009 and 2010 to field its BMDS capabilities on an incremental funding basis, without any requirement for full funding. Congress has granted similar authority to other DOD programs. In the conference report accompanying the Fiscal Year 2008 National Defense Authorization Act, the conferees indicated that if MDA wishes to use incremental funding after fiscal year 2010, DOD must request additional authority for a specific program or capability. Conferees cautioned DOD that additional authority will be considered on a limited case-by-case basis and that future missile defense programs will be funded in a manner more consistent with other DOD acquisition programs. Since 2002, MDA has been granted the flexibility to incrementally fund the fielding of its operational assets with research and development funds. In some cases, the agency spreads the cost of assets across 5 to 7 budget years. After reviewing the agency's incremental funding plan for future procurements of THAAD fire units and Aegis BMD missiles, we analyzed the effect of fully funding these assets using present value techniques and found that the agency could save about $125 million by fully funding their purchase and purchasing them in an economical manner. Our analysis is provided in table 7. In addition, more detailed analysis is available in appendix III. Table 7: Incremental Funding Costs versus Full funding Costs for THAAD and Aegis BMD Fielded Assets: (Dollars in Then Year Thousands). BMDS element: THAAD; (Incrementally Funded from 2009-2013); Asset: Fire Units 3 and 4 (launchers, battle manager, radars, and 48 missiles); Cost if incrementally funded: $1,173,346; Cost if fully funded: $1,069,486; Savings provided by full funding: $103,860. BMDS element: Aegis BMD; (Missiles Incrementally Funded from 2008-2010; Shipsets Incrementally Funded from 2009-2015); Asset: 48 1B SM-3s for Blocks 2012 and 2014; Cost if incrementally funded: $519,000; Cost if fully funded: $501,600; Savings provided by full funding: $17,400. BMDS element: Asset: 19 4.0.1 shipset procurement and installs; Cost if incrementally funded: $512,120; Cost if fully funded: $508,000; Savings provided by full funding: $4,120. Total; Asset: [Empty]; Cost if incrementally funded: $2,204,466; Cost if fully funded: $2,079,086; Savings provided by full funding: $125,380. Source: MDA (data); GAO (analysis). [End of table] According to our analysis, fully funding the THAAD and Aegis BMD assets will, in all instances, save MDA money. For example, full funding would save the THAAD program approximately $104 million and the Aegis BMD program nearly $22 million. In addition, by providing funds upfront, the contractors should be able to arrange production in the most efficient manner. Conclusions: By the end of Block 2006, MDA posted a number of accomplishments for the BMDS, including fielding more assets, conducting several successful tests, and progressing with developmental efforts. As a result, fielded capability has increased. On the other hand, some problems continue that make it difficult to assess how well the BMDS is progressing relative to the funds it has received and the goals it has set for those funds. First, under the proposed block construct, MDA plans to develop a firm baseline for each block and have it independently reviewed. However, MDA has not yet developed estimates for full block costs, so the initial baseline incorporates the budget for each block only through DOD's Future Years Defense Plan. Second, while MDA expects to estimate unit costs and track increases, it is unclear as to what criteria will be used for reporting variances to Congress. Third, while MDA has gotten some contractors to lower the portion of work planned as level of effort, a substantial amount of work remains so planned. Fourth, while it may not be reasonable to expect the same level of accountability for technology development efforts as it is for development and production of systems, the high level of investment--up to half of its budget--MDA plans to make in technology development warrants some mechanism for reconciling the cost of these efforts with their progress. Finally, MDA fields assets before development testing is complete and without conducting operational testing. We have previously recommended that MDA return to its original non-concurrent, knowledge-based approach to developing, testing, and fielding assets. Short of that, the developmental testing that is done provides the primary basis for the Director of Operational Test and Evaluation to assess whether a block of BMDS capability is suitable and effective for the battlefield. So far, BMDS testing has not yielded sufficient data to make a full assessment. Recommendations for Executive Action: To build on efforts to improve the transparency, accountability, and oversight of the missile defense program, we recommend that the Secretary of Defense direct: * MDA to develop a full cost for each block and request an independent verification of that cost; * MDA to clarify the criteria that it will use for reporting unit cost variances to Congress; * MDA to examine a contractor's planning efforts when 20 percent or more of a contract's work is proposed as level of effort; * MDA to investigate ways of developing a baseline or some other standard against which the progress of technology programs may be assessed; and: * MDA and the Director of Operational Test and Evaluation to agree on criteria and incorporate corresponding scope into developmental tests that will allow a determination of whether a block of BMDS capability is suitable and effective for fielding. Agency Comments and Our Evaluation: DOD provided written comments on a draft of this report. These comments are reprinted in appendix I. DOD also provided technical comments, which we incorporated as appropriate. DOD concurred with three of our five recommendations--developing a full cost estimate for each block and requesting an independent verification of that cost, clarifying criteria for reporting unit cost variances to Congress, and examining contractors' planning efforts when 20 percent or more of a contract's work is proposed as level of effort. The Department indicated that MDA has already taken steps to develop new cost models aligned with its new block structure and met with DOD's Cost Analysis Improvement Group to initiate the planning process for the independent verifications of MDA's cost estimates. The cost estimates will extend until block completion and will not be limited by a 6-year Future Years Defense Plan window. MDA is also working to establish criteria for reporting unit cost variances and to incorporate them into an MDA directive. Finally, MDA has made a review of prime contractors' work planning efforts part of the Integrated Baseline Review process and the Defense Contract Management Agency has agreed to continuously validate the appropriateness of each contractor's planning methodology as part of its ongoing contract surveillance. DOD partially concurred with our recommendation that MDA investigate ways of developing a baseline or some other standard against which the progress of technology programs may be assessed. DOD observed that MDA uses knowledge points, technology readiness levels, and engineering and manufacturing readiness levels in assessing the progress of its technology programs and that it will continue to investigate other methods of making such assessments. While we recognize their value, these methods typically assess progress in the short term and do not provide an estimate of the remaining cost and time needed to complete a technology program. Because MDA must balance its efforts to improve the existing BMDS while developing new capability, DOD and MDA need to ensure that only the most beneficial technology programs in terms of performance, cost, and schedule are pursued. This will require an understanding of not only the benefit to be derived from the technology, but also an understanding of the cost and time needed to bring the technology to fruition. DOD also partially concurred with our last recommendation that MDA and the Director of Operational Test and Evaluation (DOT&E) agree on criteria and additional scope for developmental tests that will allow a full determination of the effectiveness and suitability of a BMDS block for fielding. DOD noted that it is MDA's mission to work with the warfighter, rather than DOT&E, to determine that the BMDS is ready for fielding, but that MDA will work closely with DOT&E to strengthen the testing of BMDS suitability and effectiveness. We agree that DOT&E is not responsible for fielding decisions, but its mission is to ensure that weapon systems are realistically and adequately tested and that accurate evaluations of operational effectiveness, suitability, and survivability are available for production decisions. MDA improved the operational realism of testing in 2007 and for the first time DOT&E considered tests at least partially adequate to make an assessment of the BMDS. However, a full assessment is not yet possible and we continue to recommend that MDA and DOT&E take steps to make as full a BMDS evaluation as possible. In doing so, MDA and DOT&E can work cooperatively to reduce the number of unknowns that will confront the warfighter when the system is required operationally and improve the likelihood that the BMDS will perform as needed in the field. We are sending copies of this report to the Secretary of Defense and to the Director, MDA. We will make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. If you, or your staff have any questions concerning this report, please contact me at (202) 512-4841. Contact Points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report. The major contributors are listed in appendix V. Signed by: Paul Francis: Director, Acquisition and Sourcing Management: List of Congressional Committees: The Honorable Carl Levin: Chairman: The Honorable John McCain: Ranking Member: Committee on Armed Services: United States Senate: The Honorable Daniel K. Inouye: Chairman: The Honorable Ted Stevens: Ranking Member: Subcommittee on Defense: Committee on Appropriations: United States Senate: The Honorable Ike Skelton: Chairman: The Honorable Duncan L. Hunter: Ranking Member: Committee on Armed Services: House of Representatives: The Honorable John P. Murtha: Chairman: The Honorable C.W. Bill Young: Ranking Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: [End of section] Appendix I: Comments from the Department of Defense: Acquisition, Technology, And Logistics: Office Of The Under Secretary Of Defense: 3000 Defense Pentagon: Washington, DC 20301-3000: March 12, 2008: Mr. Paul Francis Director, Acquisition and Sourcing Management: U. S. Government Accountability Office: 441 G. Street, N.W.: Washington, DC 20548: Dear Mr. Francis: This is the Department of Defense (DoD) response to the GAO draft report, "Defense Acquisitions: Progress Made in Fielding Missile Defense, but Program is Short of Meeting Goals," dated February 5, 2008 (GAO Code 120662/GAO-08-448). Detailed comments on the report recommendations are enclosed. The DoD concurs with three of the draft report's recommendations and partially concurs with two. The rationale for our position is included in the enclosure. I submitted separately a list of technical and factual errors for your consideration. As you note in the report, the Missile Defense Executive Board reviews and makes recommendations on MDA's acquisition strategy, plans, and funding. The Board will continue to promote the improvement of our nation's ballistic missile defense capabilities, ensure proper oversight of their acquisition, and align the Department's business processes to enhance life cycle management of the Ballistic Missile Defense System. We appreciate the opportunity to comment on the draft report. My point of contact for this effort is Mr. Greg Hulcher, (703) 695-2680, Greg.Hulcher@osd.mil. Sincerely, Signed by: David G. Ahern: Director: Portfolio Systems Acquisition: Enclosure: As stated: GAO Draft Report Dated February 5, 2008 GAO-08-448: (GAO CODE 120662): "Defense Acquisitions: Progress Made In Fielding Missile Defense, But Program Is Short Of Meeting Goals": Department Of Defense Comments To The Gao Recommendations: Recommendation 1: The GAO recommends that the Secretary of Defense direct the Missile Defense Agency (MDA) to develop a full cost for each initiated block and request an independent verification of that cost. (Page 41/GAO Draft Report) DoD Response: Concur. MDA is taking a step by step approach to building cost estimates for BMDS blocks. This is a complex undertaking that has to be done carefully. MDA is currently conducting a review of restructured cost models for MDA program elements. These new cost models are aligned with the new MDA block structure. At conclusion of these reviews, MDA will compile the relevant portions of each program element estimate to develop a full cost estimate for the entire block. The cost estimates will extend until block completion and will not be limited by a 6-year Future Years Defense Plan window. MDA has met with the Cost Analysis Improvement Group to initiate the planning process for the independent verification of MDA's cost estimates. The CAIG has been supportive and is working with MDA to develop a concept for completing the independent reviews. Because MDA is implementing these steps, no direction from the Secretary of Defense is warranted at this time. Recommendation 2: The GAO recommends that the Secretary of Defense direct the MDA to clarify the criteria that it will use for reporting unit cost variances to Congress. (Page 41/GAO Draft Report) DoD Response: Concur. MDA is now working to establish criteria for reporting unit cost variances and incorporate them in a MDA directive. No direction from the Secretary of Defense is required at this time. Recommendation 3: The GAO recommends that the Secretary of Defense direct the MDA to examine a contractor's planning efforts when 20 percent or more of a contract's work is proposed as level of effort. (Page 41/GAO Draft Report) DoD Response: Concur. The review of the planning is now a requirement of the Integrated Baseline Review process. This responsibility applies to all MDA elements regardless of their respective contract's percentage of Level of Effort (LOE). A memorandum of agreement with the Defense Contract Management Agency (DCMA) was signed in January 2008, in which DCMA agreed to continuously validate the appropriateness of MDA contractors' planning methodology as part of its ongoing contract surveillance. In December 2007, MDA began the process of reviewing each contract's LOE percentages during its formal program reviews with the goal to minimize the LOE percentage where appropriate. No direction from the Secretary of Defense is warranted at this time. Recommendation 4: The GAO recommends that the Secretary of Defense direct the MDA to investigate ways of developing a baseline or some other standard against which the progress of technology programs maybe assessed. (Page 41/GAO Draft Report) DoD Response: Partially concur. While GAO noted MDA's use of knowledge points in assessing the maturity of the Airborne Laser, GAO's draft report did not comment on the adequacy of MDA's current knowledge-based approach to assessing progress in complex technology programs. MDA uses standards”called knowledge points”to assess the progress of its technology programs. These knowledge points address both technical maturity and affordability issues. The Agency also pays close attention to technology readiness levels and engineering and manufacturing readiness levels in assessing the progress of technology programs. However, MDA continually seeks better engineering and business practices and will continue to investigate other means for assessing technology programs. No direction from the Secretary of Defense is warranted at this time. Recommendation 5: The GAO recommends that the Secretary of Defense direct the MDA and the Director of Operational Test and Evaluation to agree on criteria and incorporate corresponding scope into developmental tests that will allow a determination of whether a block of Ballistic Missile Defense System capability is suitable and effective for fielding. (Page 41/GAO Draft Report) DoD Response: Partially concur. MDA has the mission to enable the fielding of BMDS elements in coordination with the war fighter. The war fighter participates in ground and flight tests to build confidence and support capability activation through the Warfighter Operational Readiness and Acceptance program. DOT&E does not participate in fielding decisions. DOT&E provides independent oversight of combined developmental/operational testing and conducts capability and test adequacy assessments to support Congressional reporting requirements. MDA will continue working closely with DOT&E and the operational test agencies in the Services to strengthen our testing of Ballistic Missile Defense System suitability and effectiveness. [End of section] Appendix II: MDA Contracts: Overview of MDA Prime and Support Contracts: The Missile Defense Agency (MDA) employs prime contractors and support contractors to accomplish different tasks that are needed to develop and field the ballistic missile defense system. Prime contractors receive the bulk of funds MDA requests each year and work to provide the hardware and software for elements of the Ballistic Missile Defense System (BMDS). Support contractors provide a wide variety of useful services, such as special knowledge and skills not available in the government and the capability to provide temporary or intermittent services. Prime Contracts: MDA has prime contracts with four defense companies--Boeing, Raytheon, Lockheed Martin, and Northrop Grumman--to develop elements of the BMDS. All current contracts and agreements are cost reimbursement type that provide for payment of reasonable, allowable, and allocable incurred costs to the extent provided in the contract or agreement.[Footnote 23] The contracts also provide fee for the contractor performing the work, but the amount earned depends on many variables, including the type of cost contract, contractor performance, technical risk, and complexity of the requirement. All of the cost reimbursement contracts used for the BMDS elements include cost plus award fee aspects. Cost plus award fee contracts provide for a fee consisting of a base fee--fixed at the inception of the contract that may be zero--and an award amount based upon a subjective evaluation by the government, meant to encourage exceptional performance. It should be noted that some award fee arrangements include objective criteria such as Key Performance events.[Footnote 24] The Multiple Kill Vehicle (MKV) contract and Command, Control, Battle Management and Communications (C2BMC) Other Transaction Agreement differ somewhat from the other elements' contracts. The MKV prime contractor was awarded an indefinite delivery/indefinite quantity cost reimbursement contract. This type of contract allows MDA to order services as they are needed through a series of task orders. Without having to specify a firm quantity of services (other than a minimum or maximum quantity), the government has greater flexibility to align the tasks with available funding. The C2BMC element operates under an Other Transaction Agreement with cost reimbursement aspects. These types of agreements are not always subject to procurement laws and regulations meant to safeguard the government. MDA chose the Other Transaction Agreement to facilitate a collaborative relationship between industry, government, federally funded research and development centers, and university research centers. DOD requires that all contractors awarded cost reimbursement contracts or other agreements of $20 million or greater implement an Earned Value Management System (EVMS) to integrate the planning of work scope, schedule, and resources, and to provide insight into their cost and schedule performance. To implement this system, contractors examine the totality of the work directed by the contract and break it into executable work packages. Each work package is assigned a schedule and a budget that is expected to enable the work's completion. On a monthly basis, the contractor examines initiated work packages to determine whether the work scheduled for the month was performed on time and within budget. If more work was completed than scheduled and the cost of the work performed was less than budgeted, the contractor reports a positive schedule and cost variance. However, if the contractor was unable to complete all of the work scheduled and needed more funds to complete the work than budgeted, the contractor reports a negative schedule and cost variance. Of course, the results can be mixed. That is, the contractor may have completed more work than scheduled but at a cost that exceeded the budget. The contractor details its performance to MDA each month in Contract Performance Reports. These reports also identify the reasons that negative or positive variances are occurring. Used properly, the earned value concept allows program managers to identify problems early so that steps can be taken before the problems increase the contract's overall cost and/or schedule. In the course of subdividing the total work of the contract into smaller efforts, contractors plan work according to its type. Included in these classifications are discrete work--work that is expected to produce a product, such as a study, lines of software code, or a test- -and work considered to be level of effort (LOE). LOE is work that does not result in a product, but is of a general or supportive nature. Supervision and contract administration are examples of work that do not produce definable end products and are appropriately planned as LOE. Some BMDS Contracts Have a High Percentage of Level of Effort Work: Several contracts for BMDS systems have relatively high proportions of work planned as LOE. When work is incorrectly planned as LOE, the contractor's performance becomes less transparent because earned value does not recognize schedule variances for such work. Rather, it is assumed that the time budgeted for an LOE effort will produce the intended result. Although an LOE work package will report cost variances, those variances will only be measured against how much the program intended to spend at certain time intervals. If LOE were to be used on activities that could otherwise be measured discretely, the project performance data could be favorably distorted and contractors and program managers might not be able to discern the value gained for the time spent on the task. Specifically, the program's Contract Performance Reports would not indicate whether or not the work performed produced the product expected. By losing early insight into performance, the program could potentially need to spend more time and money to complete the task. Since earned value management is less suited for work that is not intended to produce a specific product, or work that is termed LOE, the Standard for Earned Value Management Systems Intent Guide instructs that although some amount of LOE activity may be necessary, it must be held to the lowest practical level.[Footnote 25] In addition, earned value experts such as Defense Contract Management Agency officials agree that if a contractor plans more than 20 percent of the total contract work as LOE, the work plan should be examined to determine if work is being properly planned. Although the amount of LOE should be minimized, some BMDS prime contracts have a relatively high percentage of LOE. As figure 2 illustrates, the MKV contractor planned much of the work for task orders open during fiscal year 2007 as LOE. Contractors for Aegis BMD SM-3 and C2BMC also planned a high percentage of their work as LOE. Both MDA's Earned Value Management Group and program office reviewers encouraged the SM-3 and C2BMC contractors to reduce their LOE percentages. By the end of the fiscal year, the SM-3 and C2BMC contractors had reduced the amount of work planned as LOE. Figure 2: Percentages of LOE as of December 2007: This figure is a vertical bar graph showing percentages of LOW as of December 2007. Aegis SM-3 (March 2007}: 73; Aegis SM-3 (September 2007): 18; Aegis AWS: 40; ABL: 57; C2BMC: 52; GMD: 28; FBX-T: 26; KEI: 40; MKV (TO 6): 3; STSS: 25; THAAD: 30. [See PDF for image] Source: MDA (data); GAO (presentation). Note: Percentages For Aegis Bmd Sm-3 (March 2007), Abl, C2bmc, And Mkv Were Updated In January 2008. [A] Aegis SM-3 (Mar 2007) represents development and initial production of Block 1A missiles delivered in March 2007. Aegis SM-3 (Sept 2007) represents the production of 20 additional Block 1A missiles, which began in February 2007. [B] Once the MKV program's Task Order 4 is baselined--planned for February 2008--the program office anticipates a decrease in the percentage of LOE. [End of figure] In December 2006, the Aegis BMD SM-3 contractor completed work to develop and produce initial Block 1A missiles with 73 percent of this work categorized as LOE--well above the 15 percent that the Aegis BMD SM-3 program reports as its industry standard. Although we have reported that the contractor completed this segment of work ahead of cost but slightly behind schedule, it is difficult to assess whether this represents the contractor's actual performance. The high percentage of LOE associated with this work may have limited our assessment and distorted whether the work completed was in all respects the work planned. Subsequently, the contractor initiated procurement of long lead materials to produce an additional 20 Block 1A missiles before work packages were developed. Once work packages were developed, only 18 percent of the work was planned as LOE. The C2BMC program was able to reduce the percentage of work planned as LOE, but the program continues to encourage further reductions. During fiscal year 2007, the C2BMC contractor replanned its work and reduced the amount of work planned as LOE from 73 to 52 percent. This change was implemented after two closely related reviews suggested the percentage of LOE work was too high. Both the program office and its contractor acknowledge the high level of LOE and have made plans to limit it in future work. As noted in figure 2, the MKV contractor considered all work being completed under two task orders--Task Orders 4 and 5--as LOE. The primary objective of Task Order 4 is to update the program plan and complete the systems engineering effort necessary to integrate the MKV warhead into the BMDS to the extent required for the systems requirements review. Both the system concept review, completed in July 2006, and the system requirements review, scheduled for December 2008, are major milestones. However, the contractor did not plan these milestone reviews as products. According to program officials, Task Order 4 will be reevaluated in February 2008 to reduce the amount of LOE and recognize more work as discrete. The MKV program also planned 100 percent of Task Order 5 work as LOE. Under this task order, the contractor was to design a prototype propulsion system, assemble and integrate the hardware for the prototype, and perform a static hot fire test of the integrated system. This effort culminates in hardware--a tangible end product--that is expected to exhibit certain performance characteristics during the static hot fire test. The contractor could have categorized this task order, at least in some part, as discrete work since the work was expected to deliver a defined product with a schedule that could slip or vary. Because the contractor categorized all of this task order as LOE, the program lost its ability to gauge performance and to make adjustments that might prevent contract cost growth. Prime Contractors Exceed Fiscal Year 2007 Budgeted Costs but Make Gains on Schedule: We analyzed Fiscal Year 2007 Contract Performance Reports for MDA's 10 prime contracts and determined that collectively the contractors overran budgeted costs by nearly $170 million but were ahead of schedule by nearly $200 million. However, the percentage of work planned as LOE should be scrutinized before accepting this as the contractors' actual performance because a high percentage of LOE, as noted above, can potentially distort the contractors' cost and schedule performance. The cumulative performance of one contractor is also distorted because it rebaselined part of its work. Rebaselining is an accepted EVM procedure that allows a contractor to reorganize all or part of its remaining contract work, add additional time or budget for the remaining effort, and, under some circumstances, set affected cost and/or schedule variances to zero. When variances are set to zero, the cumulative performance of the contractor appears more positive than it is. Four of the 10 contracts we reviewed also contained some kind of replanning activity during fiscal year 2007. Contractors may replan when they conclude that the current plan for completing the effort remaining on the contract is unrealistic. A replan can consist of any of the following: reallocating the budget for the remaining effort within the existing constraints of the contract, realigning the schedule within the contractually defined milestones, and setting cost and/or schedule variances to zero. During the course of replanning a contract, the contractor must provide traceability to previous baselines as well as ensure that available funding is not exceeded. Aegis BMD Contractors Perform within Budgeted Costs: The Aegis BMD program awarded two prime contracts for its major components, the Aegis BMD Weapon System and the Standard Missile-3. During the fiscal year, the contractors completed all work at less cost than budgeted. Both contractors ended the year with positive cumulative cost variances, but negative cumulative schedule variances. Based on our analysis, we project that if the contractors continue to perform at the same level, the weapon system contractor could underrun its budget by between $8.8 million and $17.7 million, while the SM-3 contractor could complete its work on 20 Block 1A missiles for $7.4 million to $11.1 million less than budgeted. The weapon system contractor's fiscal year 2007 cost performance resulted in a positive cost variance of $7.7 million. The positive variance was realized as two software packages required less effort than anticipated and were completed earlier than expected. Combined with its performance from earlier periods, the contractor finished the year with a cumulative positive cost variance of $7 million. This upward trend is depicted in figure 3. Figure 3: Aegis AWS Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing Aegis AWS fiscal year 2007 cost and schedule performance. Date: Oct. 2006; Cumulative Cost variance: -4.728; Cumulative schedule variance: 0.494. Date: Nov. 2006; Cumulative Cost variance: -8.533; Cumulative schedule variance: 0.118. Date: Dec. 2006; Cumulative Cost variance: -6.026; Cumulative schedule variance: -0.096. Date: Jan. 2007; Cumulative Cost variance: -4.324; Cumulative schedule variance: -0.089. Date: Feb. 2007; Cumulative Cost variance: -6.197; Cumulative schedule variance: -0.371. Date: Mar. 2007; Cumulative Cost variance: -6.945; Cumulative schedule variance: -0.489. Date: April 2007; Cumulative Cost variance: -8.024; Cumulative schedule variance: -0.515 Date: May 2007; Cumulative Cost variance: -7.666; Cumulative schedule variance: -0.515. Date: June 2007; Cumulative Cost variance: -7.551; Cumulative schedule variance: -0.049. Date: July 2007; Cumulative Cost variance: -4.332; Cumulative schedule variance: -2.098. Date: Aug. 2007; Cumulative Cost variance: 6.963; Cumulative schedule variance: -2.37. Date: Sept. 2007; Cumulative Cost variance: 6.979; Cumulative schedule variance: -3.258 (67% of contract complete) (40% level of effort) [See PDF for image] Source: Contractor (data); GAO presentation. [End of figure] The contractor produced a $3.8 million unfavorable schedule variance in fiscal year 2007. The contractor reported that the unfavorable cumulative variance was caused in part by a delay in receiving component materials for the radar's processor. During fiscal year 2007, the Aegis SM-3 contractor closed out work related to missile development and initial production of Block 1A missiles and began new work in February 2007 to manufacture an additional 20 Block 1A missiles. In performing the new work, the contractor underran its cost budget by $6.2 million, but failed to complete $4.0 million of planned work. The Aegis BMD SM-3 contractor's cumulative cost and schedule variances are highlighted in figure 4. Figure 4: Aegis BMD SM-3 Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing Aegis BMD SM-3 fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: 3.61; Cumulative schedule variance: -6.91. Nov. 2006; Cumulative cost variance: 4.83; Cumulative schedule variance: -5.26. Dec. 2006; Cumulative cost variance: 6.01; Cumulative schedule variance: -2.39. Jan. 2007; Cumulative cost variance: 6.56; Cumulative schedule variance: -1.56. Feb. 2007; Cumulative cost variance: 6.94; Cumulative schedule variance: -0.91. Mar. 2007; Cumulative cost variance: 7.25; Cumulative schedule variance: -0.46; Cumulative cost variance: 1.3; Cumulative schedule variance: -0.37. April 2007; Cumulative cost variance: 1.69; Cumulative schedule variance: -0.35. May 2007; Cumulative cost variance: 3.09; Cumulative schedule variance: 0.79. June 2007; Cumulative cost variance: 3.37; Cumulative schedule variance: -0.59. July 2007; Cumulative cost variance: 4.4; Cumulative schedule variance: -2.77. Aug. 2007; Cumulative cost variance: 5.07; Cumulative schedule variance: -2.99. Sept. 2007; Cumulative cost variance: 6.15; Cumulative schedule variance: -3.96. (66% of contract complete) (18% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). Note: In March 2007, the Aegis BMD SM-3 contractor delivered 12 Block 1A missiles and continued work that began in February 2007 to manufacture an additional 20 Block 1A missiles. [End of figure] The positive cost variance can be attributed to several factors including cost efficiencies realized from streamlining system engineering resources and lower than planned hardware costs. Our analysis predicts that if the SM-3 contractor continues to perform as it did through September 2007, it will underrun its budgeted costs for the 20 Block 1A missiles by between $7.4 million and $11.1 million. The contractor's negative cumulative schedule variance of $4 million for the 20 missiles was primarily caused by delayed qualification testing and integration of hardware components. Despite Rebaselining, ABL's Cumulative Cost and Schedule Performance Remain Negative: In May 2007, MDA allowed ABL's contractor to rebaseline one part of its contract after the work associated with a key knowledge point could not be completed on schedule. Because the contractor did not achieve this knowledge point as planned, the program was forced to postpone its lethality demonstration until August 2009. Technical issues including weapon system integration, beam control/fire control software modifications, and flight testing discoveries, all contributed to the delay in completing the knowledge point for the program. To provide funds and time to support the delay in the lethality demonstration, the program extended the contract's period of performance by approximately 1 year and increased the contract's ceiling cost by $253 million. Once the new baseline was incorporated, the contractor was able to complete fiscal year 2007 with positive cost and schedule variances of $3.7 million and $24.2 million, respectively. Figure 5 depicts the contractor's cumulative cost and schedule performance. Figure 5: ABL Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing ABL fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: 3.61; Cumulative schedule variance: -6.91 Nov. 2006; Cumulative cost variance: 4.83 Cumulative schedule variance: -5.26 Dec. 2006; Cumulative cost variance: 6.01; Cumulative schedule variance: -2.39. Jan. 2007; Cumulative cost variance: 6.56; Cumulative schedule variance: -1.56. Feb. 2007; Cumulative cost variance: 6.94; Cumulative schedule variance: -0.91. Mar. 2007; Cumulative cost variance: 7.25; Cumulative schedule variance: -0.46; Cumulative cost variance: 1.3; Cumulative schedule variance: -0.37. April 2007; Cumulative cost variance: 1.69; Cumulative schedule variance: -0.35. May 2007; Cumulative cost variance: 3.09; Cumulative schedule variance: 0.79. June 2007; Cumulative cost variance: 3.37; Cumulative schedule variance: -0.59. July 2007; Cumulative cost variance: 4.4; Cumulative schedule variance: -2.77. Aug. 2007; Cumulative cost variance: 5.07; Cumulative schedule variance: -2.99. Sept. 2007; Cumulative cost variance: 6.15; Cumulative schedule variance: -3.96. (83% of contract complete) (57% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). [End of figure] As shown in figure 5 above, the ABL contractor was not able to overcome the negative cost and schedule variances of prior years and ended the fiscal year with an unfavorable cumulative cost variance of $74.2 million and an unfavorable cumulative schedule variance of $25.8 million. We estimate that, at completion, the contract could overrun its budget by between $95.4 million and $202.5 million. C2BMC Program Incurs Negative Cumulative and Fiscal Year Variances: During fiscal year 2006, the C2BMC contractor did not report earned value because it was working on a replan of its Block 2006 increment of work (known as Part 4). Following the definitization of the Part 4 replan in November 2006, the C2BMC contractor resumed full EVM reporting with the first submittal covering February 2007 data. As part of the replan, the contractor adjusted a portion of its Part 4 work and set cost and schedule variances to zero in an effort to establish a baseline commensurate with the contractor's replanning efforts. However, even with the adjustment, the C2BMC program ended fiscal year 2007 with negative fiscal year cost and schedule variances of $11.1 million and $1.5 million, respectively. Figure 6 shows the contractor's cumulative performance in fiscal year 2007. Figure 6: C2BMC Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing C2BMC fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: -7.56; Cumulative schedule variance: -4.25. Nov. 2006; Cumulative cost variance: -9.01; Cumulative schedule variance: -8.79. Dec. 2006; Cumulative cost variance: -16.28; Cumulative schedule variance: -11.39; Jan. 2007; Cumulative cost variance: -3.49; Cumulative schedule variance: -0.78. Feb. 2007; Cumulative cost variance: -0.44; Cumulative schedule variance: 2.76. Mar. 2007; Cumulative cost variance: -0.19; Cumulative schedule variance: 2.78. April 2007; Cumulative cost variance: -1.63; Cumulative schedule variance: -1.56. May 2007; Cumulative cost variance: -3.09; Cumulative schedule variance: -2.56. June 2007; Cumulative cost variance: -4.72; Cumulative schedule variance: -2.93. July 2007; Cumulative cost variance: -11.33; Cumulative schedule variance: -7.38. Aug. 2007; Cumulative cost variance: -15.05; Cumulative schedule variance: -8.28. Sept. 2007; Cumulative cost variance: -14.48; Cumulative schedule variance: -3.54. (90% of contract complete) (52% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). [End of figure] The unfavorable fiscal year cost variance was largely due to adding staff to support a software release; while the unfavorable fiscal year schedule variance was attributable to delays in hardware delivery, initiation of a new training system, and completing training material for the new system. Added to prior year negative variances, the C2BMC contractor reported cumulative negative cost and schedule variances of $14.5 million and $3.5 million, respectively. The contractor completed Part 4 work in December 2007 and reported an overrun of $9.9 million. GMD Contractor Maintained Negative Cost and Schedule Variances Throughout the Fiscal Year: The GMD prime contractor's cost performance improved significantly in fiscal year 2007. The contractor experienced a budget overrun of $22.1 million for the fiscal year following budget overruns in both fiscal years 2005 and 2006 that exceeded $300 million. Program officials attribute this turnaround in performance to several factors, including rigorous management of the contract's estimate at completion, quality initiatives, and joint efforts by the contractor and program office to define scope, schedule, and price of change orders. The cumulative cost variance at the end of fiscal year 2007 was over $1 billion. We estimate that at completion the contract, with a target price of $15.54 billion, could exceed its budgeted cost by between $1.06 billion and $1.4 billion. The contractor was able to complete $84.9 million more work than scheduled for fiscal year 2007, but could not overcome poor performance in earlier years and ended the year with a negative cumulative schedule variance of $52.9 million. Figure 7 illustrates both cost and schedule trends in GMD fiscal year 2007 performance. Figure 7: GMD Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line chart showing GMD fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: -1069.93; Cumulative schedule variance: -149.41. Nov. 2006; Cumulative cost variance: -1137.45; Cumulative schedule variance: -154.87. Dec. 2006; Cumulative cost variance: -1116.15; Cumulative schedule variance: -142.85. Jan. 2007; Cumulative cost variance: -1120.22; Cumulative schedule variance: -109.72. Feb. 2007; Cumulative cost variance: -1137.75; Cumulative schedule variance: -122.2. Mar. 2007; Cumulative cost variance: -1132.03; Cumulative schedule variance: -124.99. April 2007; Cumulative cost variance: -1129.22; Cumulative schedule variance: -106.88. May 2007; Cumulative cost variance: -1125.29; Cumulative schedule variance: -99.2. June 2007; Cumulative cost variance: -1129.99; Cumulative schedule variance: -99.89. July 2007; Cumulative cost variance: -1136.41; Cumulative schedule variance: -106.76. Aug. 2007; Cumulative cost variance: -1121.08; Cumulative schedule variance: -113.88. Sept. 2007; Cumulative cost variance: -1081.76; Cumulative schedule variance: -52.89. (77% of contract complete) (28% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). [End of figure] The unfavorable fiscal year cost variance is primarily attributable to the EKV. During fiscal year 2007, the EKV contractor experienced negative cost variances as it incurred additional labor costs to recover delivery schedules, manufacturing schedule delays, hardware manufacturing problems, and embedded software development and system integration problems. With 18 percent of the EKV work remaining, the negative trends on this component could continue. As we reported last year, the contractor was in the process of developing a new contract baseline to incorporate the updated scope, schedule, and budget that the contractor was working toward. In September 2006, phase one of the new baseline, covering fiscal year 2006-2007 efforts, had been implemented and validated through the Integrated Baseline Review of the prime contractor and its major subcontractors. Phase two of the review was completed in December 2006. Subsequent to the reviews, fiscal year 2007 ground and flight tests were replanned to reflect a contract change that added additional risk mitigation effort to one planned flight test and added a radar characterization system test. KEI Replan Significantly Affects Cost and Schedule Variances: The KEI contractor replanned its work in April 2007 when MDA directed the program to focus, in the near term, on two main objectives: booster vehicle development and the 2008 booster flight developmental test. Prior to the replan, the KEI program was developing a land-mobile capability with fire control and communications and mobile launcher components. Although the contractor's primary objectives are now focused around the booster segment of work, it is still performing some activities related to the fire control and communications component. During fiscal year 2007, the contractor incurred a positive cost variance of $2.1 million and a negative schedule variance of $7.5 million. Combined with variances from earlier fiscal years, the cumulative cost variance is a positive $5.7 million and the cumulative schedule variance is a negative $12.8 million. Figure 8 illustrates KEI's cumulative performance over the course of the fiscal year. Figure 8: KEI Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line chart showing KEI fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: 2.72; Cumulative schedule variance: -3.16. Nov. 2006; Cumulative cost variance: 5.72; Cumulative schedule variance: -5.04. Dec. 2006; Cumulative cost variance: 5.3; Cumulative schedule variance: -11.65. Jan. 2007; Cumulative cost variance: 10.31; Cumulative schedule variance: -15.61. Feb. 2007; Cumulative cost variance: 11.84; Cumulative schedule variance: -22.6. Mar. 2007; Cumulative cost variance: 14.77; Cumulative schedule variance: -29.25. April 2007; Cumulative cost variance: 4.35; Cumulative schedule variance: -7.95. May 2007; Cumulative cost variance: 7.1; Cumulative schedule variance: -5.39. June 2007; Cumulative cost variance: 7.6; Cumulative schedule variance: -6.69. July 2007; Cumulative cost variance: 7.85; Cumulative schedule variance: -9.14. Aug. 2007; Cumulative cost variance: 8.05; Cumulative schedule variance: -9.72. Sept. 2007; Cumulative cost variance: 5.67; Cumulative schedule variance: -12.78. (10% of contract complete) (40% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). [End of figure] KEI's fiscal year favorable cost variance primarily results from completing work on the fire control and communications component, as well as systems engineering and integration with fewer staff than planned. We were unable to estimate whether the total contract is likely to be completed within budgeted cost since the contract is only 10 percent complete and trends cannot be developed until at least 15 percent of the contract is completed. Work related to the interceptor's booster and systems engineering and integration contributed to KEI's cumulative negative fiscal year schedule variance of $7.5 million. The contractor reports that the booster work was understaffed, which caused delays in finalizing designs that, in turn, delayed procurement of subcomponents and materials and delayed analysis and tests. While the reduction in staff for systems engineering and integration work reduced costs for the contractor, it also delayed completion of the weapon system's scheduled engineering, flight, and performance analysis products. Information Not Available to Assess All MKV Work: We could evaluate only two of five MKV task orders open during fiscal year 2007 because the contractor did not report sufficient earned value data to make an assessment of the other three meaningful. MDA awarded the MKV contract in January 2004 and has since initiated eight task orders through its indefinite delivery/indefinite quantity contract. During fiscal year 2007, the program worked on five of these task orders--Task Orders 4, 5, 6, 7, and 8. We evaluated the contractor's cost and schedule performance for Task Orders 5 and 6 only. Of the three task orders that we did not evaluate, the contractor began reporting full earned value on two so late in the fiscal year that little data was available for analysis. In the third case, the contractor's reports did not include all data needed to make a cost and schedule assessment. In June 2006, MDA issued Task Order 5 which directed the design, assembly, and integration of the hardware for a prototype propulsion system, and a static hot fire test of the integrated prototype. Because the contractor planned all activities for this task order as level of effort, the contractor reported zero schedule variance. Contract Performance Reports show that in preparation for the hot fire test in August 2007, the program discovered anomalies indicative of propellant contamination in the prototype's propulsion system. These anomalies led to multiple unplanned propellant tank anomaly investigations, which contributed to the unfavorable $2.3 million cost variance for the fiscal year. Additionally, during the hot fire test, one of the thrusters in the propulsion system's divert and attitude control component experienced anomalies due to foreign object contamination. This anomaly led to unplanned investigations which also contributed to increased costs. Figure 9 below depicts the unfavorable cumulative cost variance of $2.7 million and cumulative schedule variance of zero reported for Task Order 5. Based on our analysis, we predict the contractor will overrun its contract costs by between $2.6 million and $2.9 million. Figure 9: MKV Task Order 5 Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing MKV task order 5 fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: -0.138; Cumulative schedule variance: 0. Nov. 2006; Cumulative cost variance: -0.242; Cumulative schedule variance: 0. Dec. 2006; Cumulative cost variance: -0.235; Cumulative schedule variance: 0. Jan. 2007; Cumulative cost variance: -0.793; Cumulative schedule variance: 0. Feb. 2007; Cumulative cost variance: -1.452; Cumulative schedule variance: 0. Mar. 2007; Cumulative cost variance: -2.223; Cumulative schedule variance: 0. April 2007; Cumulative cost variance: -2.839; Cumulative schedule variance: 0. May 2007; Cumulative cost variance: -3.001; Cumulative schedule variance: 0. June 2007; Cumulative cost variance: -3.157; Cumulative schedule variance: 0. July 2007; Cumulative cost variance: -3.331; Cumulative schedule variance: 0. Aug. 2007; Cumulative cost variance: -3.534; Cumulative schedule variance: 0. Sept. 2007; Cumulative cost variance: -2.681; Cumulative schedule variance: 0. (93% of contract complete) (100% level of effort) [See PDF for image] Source: Contractor (data); GAO presentation. [End of figure] MKV's objective for Task Order 6 is to manufacture a prototype seeker capable of acquiring, tracking, and discriminating objects in space. The program plans to demonstrate the prototype seeker, which is a component of a carrier vehicle, through testing in 2009. In contrast to Task Order 5, the contractor correctly planned the bulk of Task Order 6 as discrete work and has been reporting the work's cost and schedule status since March 2007. During this time, the contractor has completed 37 percent of the work directed by the task order at $0.3 million less than budgeted. The contractor was also able to complete $0.9 million more work than planned. See Figure 10 for an illustration of cumulative cost and schedule variances for this task order. Figure 10: MKV Task Order 6 Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing MKV task order 6 fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: [Empty]; Cumulative schedule variance: [Empty]. Nov. 2006; Cumulative cost variance: [Empty]; Cumulative schedule variance: [Empty]. Dec. 2006; Cumulative cost variance: [Empty]; Cumulative schedule variance: [Empty]. Jan. 2007; Cumulative cost variance: [Empty]; Cumulative schedule variance: [Empty]. Feb. 2007; Cumulative cost variance: [Empty]; Cumulative schedule variance: [Empty]. Mar. 2007; Cumulative cost variance: 0.035; Cumulative schedule variance: -0.016. April 2007; Cumulative cost variance: 0.033; Cumulative schedule variance: 0.024. May 2007; Cumulative cost variance: -0.002; Cumulative schedule variance: 0.032. June 2007; Cumulative cost variance: 0.166; Cumulative schedule variance: 0.313. July 2007; Cumulative cost variance: 0.424; Cumulative schedule variance: 0.621. Aug. 2007; Cumulative cost variance: 0.355; Cumulative schedule variance: 0.528 Sept. 2007; Cumulative cost variance: 0.267; Cumulative schedule variance: 0.913 [See PDF for image] (37% of contract complete) (3% level of effort) Source: Contractor (date); GAO (presentation). [End of figure] The program attributes its favorable fiscal year cost and schedule variances for Task Order 6 to the early progress made on interface requirements, hardware procurements, component drawings, and the prototype seeker's architecture. Because detailed designs for the seeker are derived from models, the program is anticipating some rework will be needed as the designs are developed, processed, and released. Although program officials are expecting some degradation in cumulative cost and schedule variances to occur, the program does not expect an overrun of the contract's budgeted cost at completion. Based on the contractor's performance to date, we predict, at contract completion, the contractor will underrun costs by between $0.8 million and $2.5 million. Sensors' FBX-T Contractor Remains Ahead of Cost and on Schedule: The Sensors contractor's performance during fiscal year 2007 resulted in a positive cost variance of $3.9 million and an unfavorable schedule variance of $8.8 million. Added to variances from prior years, the contractor is reporting cumulative positive cost and schedule variances of $24.1 million and $17.8 million, respectively. The contractor's performance in 2007, suggests that at completion the contract will cost from $22.0 million to $46.8 million less than budgeted. The variances, depicted below in figure 11 represent the Sensors contractor's cumulative cost and schedule performance over fiscal year 2007. Figure 11: Sensors Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing sensors fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: 21; Cumulative schedule variance: 33.65. Nov. 2006; Cumulative cost variance: 19.65; Cumulative schedule variance: 34.47. Dec. 2006; Cumulative cost variance: 21.69; Cumulative schedule variance: 33.21. Jan. 2007; Cumulative cost variance: 22.9; Cumulative schedule variance: 33.4. Feb. 2007; Cumulative cost variance: 23.2; Cumulative schedule variance: 29.22. Mar. 2007; Cumulative cost variance: 22.7; Cumulative schedule variance: 21.93. April. 2007; Cumulative cost variance: 22.8; Cumulative schedule variance: 21.23. May 2007; Cumulative cost variance: 22.53; Cumulative schedule variance: 20.52. June 2007; Cumulative cost variance: 20.14; Cumulative schedule variance: 16.66. July 2007; Cumulative cost variance: 21.02; Cumulative schedule variance: 17.16. Aug. 2007; Cumulative cost variance: 21.37; Cumulative schedule variance: 14.5. Sept. 2007; Cumulative cost variance: 24.15; Cumulative schedule variance: 17.83. (65% of contract complete) (28% level of effort) [See PDF for image] Source: Contractor (data); GAO presentation. [End of figure] The contractor has reported favorable schedule and cost variances since the contract's inception because the program was able to leverage the hardware design of the THAAD radar to reduce development timelines and it implemented manufacturing efficiencies to reduce manufacturing costs. However, during fiscal year 2007, the contractor experienced a negative schedule variance as it struggled to upgrade software expected to provide an increased capability for the FBX-T radar. STSS Contractor's Cumulative Cost and Schedule Variances Remain Unfavorable: After replanning a portion of its work in October 2006, the STSS contractor in fiscal year 2007 experienced an unfavorable cost variance of $67.7 million and a favorable schedule variance of $84.7 million. Combined with performance from earlier periods, the contractor is reporting cumulative negative cost and schedule variances of $231.4 million and $19.7 million, respectively. Figure 12 shows both cost and schedule trends during fiscal year 2007. Figure 12: STSS Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line chart showing STSS fiscal year 2007 cost and schedule performance. Sept. 2006; Cumulative cost variance: -163.74; Cumulative schedule variance: -104.35. Oct. 2006; Cumulative cost variance: -166.15; Cumulative schedule variance: -7.59. Nov. 2006; Cumulative cost variance: -175.54; Cumulative schedule variance: -7.7. Dec. 2006; Cumulative cost variance: -180.83; Cumulative schedule variance: -9.47. Jan. 2007; Cumulative cost variance: -189.49; Cumulative schedule variance: -9.15. Feb. 2007; Cumulative cost variance: -193.37; Cumulative schedule variance: -9.43. Mar. 2007; Cumulative cost variance: -199.3; Cumulative schedule variance: -9.56. April. 2007; Cumulative cost variance: -206.17; Cumulative schedule variance: -11.17. May 2007; Cumulative cost variance: -212.36; Cumulative schedule variance: -14.2. June 2007; Cumulative cost variance: -217.96; Cumulative schedule variance: -15.65. July 2007; Cumulative cost variance: -221.76; Cumulative schedule variance: -18.86. Aug. 2007; Cumulative cost variance: -227.71; Cumulative schedule variance: -20.19 Sept. 2007; Cumulative cost variance: -231.41; Cumulative schedule variance: -19.67. (49% of contract complete) (25% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). [End of figure] During the fiscal year, the contractor was able to accomplish a significant amount of work ahead of schedule after a replan added additional time for planned work efforts. However, the contractor was unable to overcome the negative schedule variances incurred in prior years. Delays in hardware and software testing as well as integration issues contributed to fiscal year 2007's negative cost variance. We did not estimate the cost of the STSS contract at completion. The contract includes not only the effort to develop and launch two demonstration satellites (the Block 2006 capability) but also effort that will benefit future blocks. Block 2006 work is about 86 percent complete, while work on future blocks is about 16 percent complete. THAAD Contractor Improves Schedule, but Costs Continue to Be a Problem: The THAAD contractor overran its fiscal year 2007 budgeted costs by $91.1 million but accomplished $19.0 million more work than scheduled. Cumulatively, the contractor ended the year with an unfavorable cost variance of $195.2 million and a negative schedule variance of $9.1 million, as shown by figure 13. Figure 13: THAAD Fiscal Year 2007 Cost and Schedule Performance: This figure is a combination line graph showing THAAD fiscal year 2007 cost and schedule performance. Oct. 2006; Cumulative cost variance: -115.507; Cumulative schedule variance: -28.965. Nov. 2006; Cumulative cost variance: -127.284; Cumulative schedule variance: -28.959. Dec. 2006; Cumulative cost variance: -135.696; Cumulative schedule variance: -28.813. Jan 2007; Cumulative cost variance: -143.392; Cumulative schedule variance: -26.672. Feb. 2007; Cumulative cost variance: -154.16; Cumulative schedule variance: -28.131. Mar. 2007; Cumulative cost variance: -164.953; Cumulative schedule variance: -10.699. April. 2007; Cumulative cost variance: -176.408; Cumulative schedule variance: -10.084. May 2007; Cumulative cost variance: -183.955; Cumulative schedule variance: -10.969. June 2007; Cumulative cost variance: -187.942; Cumulative schedule variance: -7.667. July 2007; Cumulative cost variance: -190.388; Cumulative schedule variance: -7.19. Aug. 2007; Cumulative cost variance: -194.889; Cumulative schedule variance: -7.875. Sept. 2007; Cumulative cost variance: -195.232; Cumulative schedule variance: -9.058. (86% of contract complete) (30% level of effort) [See PDF for image] Source: Contractor (data); GAO (presentation). [End of figure] The THAAD prime contractor's cost overrun of $91.1 million was primarily caused by technical problems related to the element's missile, launcher, radar, and test components. Missile component cost overruns were caused by higher than anticipated costs in hardware fabrication, assembly, and support touch labor as well as subcontractor material costs for structures, propulsion, and other sub-assembly components. Additionally, design issues with the launcher's missile round pallet and the electronics assembly that controls the launcher caused the contractor to experience higher than anticipated labor and material costs. More staff than planned was required to resolve hardware design issues in the radar's prime power unit, causing the radar component to end the fiscal year with a negative cost variance. The contractor also experienced negative cost variances with the system test component because the Launch and Test Support Equipment required additional set-up time at the flight test range. THAAD's prime contractor fared better in performing scheduled work. It was able to reduce its negative cumulative schedule variance over the course of the fiscal year because subcontracted missile items were delivered early and three flight tests were removed from the test program to accommodate target availability and budget constraints, allowing staff more time to work on current efforts. The contractor projects an overrun of $174 million at contract completion, while we estimate that the overrun could range from $227.2 million to $325.8 million. To achieve its projection, the contractor needs to complete $1.04 worth of work for every dollar spent. In contrast, during fiscal year 2007, the contractor achieved an average of $0.82 worth of work for each dollar spent. Therefore, it seems unlikely that the contractor will be able to achieve its estimate at completion. MDA Relies on Award Fees to Motivate Contractor Performance: Like other DOD programs, MDA has not always effectively used award fees to encourage contractors toward exceptional performance but it is making efforts to revise its award fee policy to do so. Over the course of fiscal year 2007, the agency sometimes rolled over large percentages of award fee--in most cases for work that was moved to later periods but also for one contractor that exhibited poor performance. In addition, some award fee plans allow fee to be awarded to contractors for merely meeting the requirements of their contract. For two contractors, MDA awarded fee amounts that were linked to very good or outstanding work in the cost and/or program management performance elements. During their award fee periods, the contractors' earned value data showed declines in cost and/or schedule variances, although there are several other factors considered when rating contract performance. However, in June 2007, MDA issued a revised draft of its award fee guide in an effort to more closely link the amount of award fees earned with the level of contractor performance. In an effort to encourage its defense contractors to perform in an innovative, efficient, and effective way in areas considered important to the development of the BMDS, MDA offers its contractors the opportunity to collectively earn billions of dollars through monetary incentives known as award fees. Award fees are intended to motivate exceptional performance in subjective areas such as technical ingenuity, cost, and schedule. Award fees are appropriate when contracting and program officials cannot devise predetermined objective targets applicable to cost, technical performance, or schedule. Currently, all 10 of the contracts we assessed for BMDS elements utilize award fees in some manner to incentivize their contractor's performance. Each element's contract has an award fee plan that identifies the performance areas to be evaluated and the methodology by which those areas will be assessed. At the end of each period, the award fee evaluation board, made up of MDA personnel, program officials, and officials from key organizations knowledgeable about the award fee evaluation areas, begins its process. The board judges the contractor's performance and recommends to a fee determining official the amount of fee to be paid. For all BMDS prime contracts we assessed, the fee determining official is the MDA Director. Table 1 provides a summary of the award fee process. Table 8: MDA's General Process for Determining Award Fee Amounts: 1; DOD officials provide input on the contractor's performance for the evaluation period just ended. 2; Program officials compile data and prepare a briefing or summary for the award fee evaluation board. 3; Award fee evaluation board convenes meeting; contractor has the option to submit a self-assessment and brief the board. 4; Award fee evaluation board considers all inputs and recommends a fee rating for the contractor. 5; Fee determining official makes a final determination, including whether to rollover unearned fee; issues final determination to contractor; and notifies contracting officer. 6; Contracting officer processes contract modification authorizing payment. Sources: Air Force Award Fee Guide, Army Contracting Agency Award Fee Handbook, Navy/Marine Corp Award Fee Guide (data), MDA program officials; GAO (analysis). [End of table] DOD Has Not Linked Award Fees to Acquisition Outcomes: GAO has found in the past that DOD has not always structured and implemented award fees in a way that effectively motivates contractors to improve performance and achieve acquisition outcomes.[Footnote 26] Specifically, GAO cited four issues with DOD's award fee processes. GAO reported that in many evaluation periods when rollover--the process of moving unearned available award fee from one evaluation period to the next--was allowed, the contractor had the chance to earn almost the entire unearned fee, even in instances when the program was experiencing problems.[Footnote 27] Additionally, DOD guidance and federal acquisition regulations state that award fees should be used to motivate excellent contractor performance in key areas. However, GAO found that most DOD award fee contracts were paying a significant portion of the available fee from one evaluation period to the next for what award fee plans describe as "acceptable, average, expected, good, or satisfactory" performance.[Footnote 28] Furthermore, DOD paid billions of dollars in award fees to programs whose costs continued to grow and schedules increased by many months or years without delivering promised capabilities to the warfighter. GAO also found that some award fee criteria for DOD programs were focused on broad areas--such as how well the contractor was managing the program--instead of criteria directly linked with acquisition outcomes--such as meeting cost and schedule goals, and delivering desired capabilities. All of these DOD practices contribute to the difficulty in linking elements of contractor performance considered in award fee criteria to overall acquisition outcomes and may lessen the motivation for the contractor to strive for excellent performance. DOD Imposed Stricter Award Fee Rules: The Department of Defense has been working to close the gap between the amount of award fees earned and the level of contractor performance. In March 2006, the Office of the Under Secretary of Defense directed acquisition executives to: * address desired outcomes and the role that award fee should play in the overall acquisition strategy; * remind the acquisition workforce to follow existing policies that tie award fees closely to contractor performance; and: * provide guidance to the acquisition workforce on rollover that directs rollover be the exception rather than the rule and, when it is allowed, that only a portion of the rolled-over fee be awarded. The Under Secretary explained that while award fee arrangements should be structured to motivate excellent contractor performance, award fees must be commensurate with contractor performance over a range from satisfactory to excellent. The memo stated that it is appropriate to award a portion of the award fee pool for satisfactory performance-- although it should be considerably less than excellent performance--to ensure contractors receive an adequate fee on contracts. Performance less than satisfactory is not entitled to any award fee. In April 2007, the Under Secretary added an even more stringent policy regarding award fee ratings that is to be applied to contract solicitations commencing after August 1, 2007. This stricter provision does not allow more than 50 percent award fee for satisfactory ratings. The Under Secretary also placed limitations on the use of rollover. Stating that rollover should be the exception rather than the rule, the Under Secretary directed that the decision to add an award fee rollover provision to a contract should include a rationale as to why it is appropriate. He added that if rollover is used, contractors should earn only a portion of the fee that was rolled over--even for subsequent excellent performance. Finally the memo directed that if the fee determining official approves the use of rollover, the official contract file must be documented accordingly, and the contractor must be notified. Similar Problems Found in MDA Award Fee Practices: We assessed all award fee plans for the BMDS elements and fiscal year 2007 award fee letters for 9 of the 10 contractors. Our review revealed that during 2007 MDA experienced some of the same award fee problems that were prevalent in other DOD programs. MDA did not roll fee forward often, but when it did the contractor was, in one case, able to earn 100 percent of that fee. Also, MDA allowed another contractor to earn the unearned portion of fiscal year 2007 award fee in the same period through a separate pool composed of the unearned fee but tied to other performance areas. In two other instances, MDA awarded fee amounts that were linked to very good or outstanding work in the cost and/or program management performance element. However, during the award fee periods, earned value data indicates that these two contractors' cost and/or schedule performance continued to decline. Although DOD guidance discourages use of earned value performance metrics in award fee criteria, MDA includes this as a factor in several of its award fee plans. MDA considers many factors in rating contractors' performance and making award fee determinations, including considerations of earned value data that shows cost, schedule, and technical trends. Table 9 provides the award fee MDA made available to its contractors, as well as the fee earned during fiscal year 2007. Table 9: Fee Awarded during Fiscal Year 2007: Element: Aegis BMD AWS[A]; Total cost of prime contract: $1,039,858,153; Total fee available for the contract: $118,470,435; Total award fee available during fiscal year 2007: $27,231,954; Total award fee awarded during fiscal year 2007: $26,558,018; Percentage of award fee awarded during fiscal year 2007: 97.5%. Element: Aegis BMD SM-3[B]; Total cost of prime contract: 1,927,578,516; Total fee available for the contract: 280,787,286; Total award fee available during fiscal year 2007: 56,840,712; Total award fee awarded during fiscal year 2007: 55,105,617; Percentage of award fee awarded during fiscal year 2007: 96.9%. Element: ABL[C]; Total cost of prime contract: 3,627,836,954; Total fee available for the contract: 188,924,382; Total award fee available during fiscal year 2007: 32,646,465; Total award fee awarded during fiscal year 2007: 24,003,984; Percentage of award fee awarded during fiscal year 2007: 73.5%. Element: C2BMC; Total cost of prime contract: 790,997,756; Total fee available for the contract: 170,255,747; Total award fee available during fiscal year 2007: 19,584,246; Total award fee awarded during fiscal year 2007: 18,936,065; Percentage of award fee awarded during fiscal year 2007: 96.7%. Element: FBX-T; Total cost of prime contract: 1,486,984,365; Total fee available for the contract: 160,330,668; Total award fee available during fiscal year 2007: 31,052,207; Total award fee awarded during fiscal year 2007: 30,308,477; Percentage of award fee awarded during fiscal year 2007: 97.6%. Element: GMD; Total cost of prime contract: 16,953,743,541; Total fee available for the contract: 2,084,394,496; Total award fee available during fiscal year 2007: 330,560,006; Total award fee awarded during fiscal year 2007: 330,560,006; Percentage of award fee awarded during fiscal year 2007: 100.0%. Element: KEI; Total cost of prime contract: 6,697,430,000; Total fee available for the contract: 909,544,000; Total award fee available during fiscal year 2007: 31,457,055; Total award fee awarded during fiscal year 2007: 27,230,128; Percentage of award fee awarded during fiscal year 2007: 86.6%. Element: MKV (TO 5); Total cost of prime contract: 13,046,997; Total fee available for the contract: 1,872,051; Total award fee available during fiscal year 2007: 0; Total award fee awarded during fiscal year 2007: n/a; Percentage of award fee awarded during fiscal year 2007: n/ a. Element: STSS; Total cost of prime contract: 1,166,897,605; Total fee available for the contract: 79,101,120; Total award fee available during fiscal year 2007: 13,683,962; Total award fee awarded during fiscal year 2007: 9,102,341; Percentage of award fee awarded during fiscal year 2007: 66.5%. Element: THAAD; Total cost of prime contract: 5,075,908,305; Total fee available for the contract: 626,030,200; Total award fee available during fiscal year 2007: 97,219,054; Total award fee awarded during fiscal year 2007: 84,542,827; Percentage of award fee awarded during fiscal year 2007: 87.0%. Source: MDA (data); GAO (analysis). Notes: Data received from MDA program offices is as of August 2007 except for the ABL, C2BMC, MKV TO 5, and THAAD elements, which are current as of January 2008. [A] Aegis BMD AWS includes only award fee for products being developed for the U.S. government (domestic product). [B] Aegis BMD SM-3 includes only domestic award fee. [C] ABL's total fee available amount is the amount of fee remaining on the contract. The total cost of the prime contract shown is based on the cost for calculating award fee, fixed fee, and direct cost reimbursement contractor efforts. [End of table] MDA Awards Contractor Large Percentage of Rollover: MDA is awarding some BMDS contractors a large percentage of the fees rolled over from a prior period. The agency's award fee plans allow the fee determining official, at his discretion, to rollover all fee that is not awarded during one period to a future period. For example, in accordance with MDA's award fee policy, the fee determining official may consider award fee rollover when a slipped schedule moves an award fee event to another period, it is the desire of the fee determining official to add greater incentive to an upcoming period, and when the contractor improves performance to such a great extent that it makes up for previous shortfalls. During fiscal year 2007, MDA rolled fee forward for 3 of the 8 contractors for which award fee letters were available. Table 10 presents a synopsis of this data. Table 10: Amount of Fees Rolled over to Future Award Fee Periods During Fiscal Year 2007: Element: Aegis BMD AWS[A]; Total Fee Available during Fiscal Year 2007: 27,231,954; Total Unearned Fee During Fiscal Year 2007: 673,936; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 0; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 0%. Element: Aegis BMD SM-3[B]; Total Fee Available during Fiscal Year 2007: 56,840,712; Total Unearned Fee During Fiscal Year 2007: 1,735,095; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: n/a; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: n/a. Element: ABL; Total Fee Available during Fiscal Year 2007: 32,646,465; Total Unearned Fee During Fiscal Year 2007: 8,642,481; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 2,275,000; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 26%. Element: C2BMC; Total Fee Available during Fiscal Year 2007: 19,584,246; Total Unearned Fee During Fiscal Year 2007: 648,181; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 0; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 0%. Element: FBX-T; Total Fee Available during Fiscal Year 2007: 31,052,207; Total Unearned Fee During Fiscal Year 2007: 743,730; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 0; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 0%. Element: GMD; Total Fee Available during Fiscal Year 2007: 330,560,006; Total Unearned Fee During Fiscal Year 2007: 0; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 0; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: n/a. Element: KEI; Total Fee Available during Fiscal Year 2007: 31,457,055; Total Unearned Fee During Fiscal Year 2007: 4,227,470; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 0; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 0%. Element: MKV (TO 5)[C]; Total Fee Available during Fiscal Year 2007: 0; Total Unearned Fee During Fiscal Year 2007: n/a; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: n/a; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: n/a. Element: STSS; Total Fee Available during Fiscal Year 2007: 13,683,962; Total Unearned Fee During Fiscal Year 2007: 4,581,621; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 1,992,900; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 43%. Element: THAAD; Total Fee Available during Fiscal Year 2007: 97,219,054; Total Unearned Fee During Fiscal Year 2007: 12,676,227; Amount in Fiscal Year 2007 Rolled Over to Future Award Fee Periods: 12,676,227; Percentage of Unearned Fee in Fiscal Year 2007 Rolled Over to Future Award Fee Period: 100%. Source: MDA (data); GAO (analysis). [A] Aegis BMD AWS includes only award fee for performance related to work for the United States government, which is termed domestic award fee. [B] Aegis BMD SM-3 includes only domestic award fee. We did not include rollover for the SM-3 contractor because a shipboard anomaly - outside the control of the contractor - limited the contractor's ability to complete three fee-bearing events during one fiscal year 2007 award fee period. [C] Award fee was not available for the MKV contractor's work on Task Order 5 during 2007. [End of table] As noted in table 10, MDA rolled over a large percentage of the fee that was not earned by the THAAD contractor during fiscal year 2007. During its last award fee period in fiscal year 2007, the THAAD contractor did not earn any of the fee associated with cost management. The award fee letter cited unfavorable cost variances and a growing variance projected at completion of the contract as the reasons for not awarding any of the fee for cost management. However, the fee determining official decided to roll 100 percent of that portion of the unearned fee to a rollover pool tied to minimizing cost overruns. Fee will be awarded from this pool at the end of the contract. By rolling the fee forward, MDA provided the contractor an additional opportunity to earn fee from prior periods. Rolling over fee in this instance may have failed to motivate the contractor to meet or exceed expectations. The award fee plan for the GMD contract allowed the contractor to not only rollover fee, but to earn all unearned fee in the same period. During the fiscal year, the GMD contractor earned 97.7 percent of the $330 million dollars in award fees tied to performance areas outlined in the award fee plan. However, the award fee plan made provisions for the contractor to earn the unearned $7.5 million by creating a separate pool funded solely from this unearned portion and awarding the fee for performance in other areas. In this instance, the contractor did not have to wait to earn rolled over fees in later award fee periods--it was able to receive the unearned portion in the same period despite not meeting all of the criteria for its original objectives. GMD officials told us that this fee incentivized the contractor to achieve added objectives. In contrast, the fee determining official handled rollover of fee on the ABL contract in accordance with DOD's new policy. According to ABL's award fee plan, MDA was to base its 2007 award fee decision primarily on the outcome of three knowledge points. During this period, the contractor completed two of the knowledge points, but could not complete a third. To encourage the contractor to complete the remaining knowledge point in a timely manner, the fee determining official rolled over only 35 percent of the fee available for the event. MDA Award Fee Plans Allow Contractors to Earn Fee for Satisfactory or Lower Performance: All of the award fee plans we assessed allowed MDA to award fees for satisfactory ratings--that is, work considered to meet most of the requirements of the contract. Some award fee plans even allow fee for marginal performance or performance considered to meet some of the requirements of the contract. By paying for performance at the minimum standards or requirements of the contract, the intent of award fees to provide motivation for excellence above and beyond basic contract requirements is lost. While the definitions of satisfactory or marginal differed from element to element, the award fee plans allotted roughly more than 50 percent award fee to contractors performing at these levels.[Footnote 29] According to the award fee plans, MDA allows between 51 and 65 percent of available fee for work rated as marginal for the C2BMC and KEI contractor and no less than 66 percent of available fee for satisfactory performance by the ABL contractor. MDA's practice of allowing more than 50 percent of available fee for satisfactory or, even, marginal performance illustrates why DOD in April 2007 directed that no more than 50 percent of available fee be given for satisfactory performance on all contract solicitations commencing after August 1, 2007. Not All Award Fees Reflect Performance as Reported by the Earned Value Management System: Earned value is one of several factors that according to the award fee plans for the ABL and Aegis BMD Weapon System contractors will be considered in rating the contractors' cost and program management performance. During a good part of fiscal year 2007, earned value data for both contractors showed that they were overrunning their fiscal year cost budgets. In addition, the ABL contractor was not completing all scheduled work. Even considering these variances, MDA presented the contractors with a significant portion of the award fee specifically tied to cost and/or program management. In contrast, the THAAD contractor also experienced downward trends in its cost variance during its last award fee period in fiscal year 2007, but was not paid any of the award fee tied to cost management. The ABL and Aegis BMD Weapon System contractors received a large percentage of the 2007 award fee available to them for the cost and/or program management element. According to ABL's award fee plan, one of several factors that is considered in rating the contractor's performance as "very good" is whether earned value data indicates that there are few unfavorable cost, schedule, and/or technical variances or trends. During the award fee period that ran from February 2006 to January 2007, MDA rated the contractor's cost and program management performance as very good and awarded 88 percent of the fee available for these areas of performance. Yet, earned value data indicates that the contractor overran its budget by more than $57 million and did not complete $11 million of planned work. Similarly, the Aegis BMD weapon system contractor was to be rated in one element of its award fee pool as to how effectively it managed its contract's cost. Similar to ABL's award fee plan, the weapon system contractor's award fee plan directs that earned value data be one of the factors considered in evaluating cost management. During the fee period that ran from October 2006 through March 2007, MDA rated the contractor's performance in this area as outstanding and awarded the contractor 100 percent of the fee tied to cost management. Earned value data for this time period indicates that the contractor overran its budget by more than $6 million. MDA did not provide us with more detailed information as to other factors that may have influenced its decision as to the amount of fee awarded to the ABL and Aegis BMD contractors. In another instance, MDA more closely linked earned award fee to contractor performance. The THAAD contractor continued to overrun its 2007 cost budget, and was not awarded any fee tied to the cost management element during its last award fee period in fiscal year 2007. The award fee decision letter cites several examples of the contractor's poor cost performance including cost overruns and an increased projected cost variance at contract completion. These and other cost management issues led the fee determining official to withhold the $9.8 million to be awarded on the basis of cost management. MDA Is Making Changes to Its Award Fee Policy: MDA has made efforts to comply with DOD policy regarding some of GAO's recommendations and responded to the DOD issued guidance by releasing its own revised award fee policy in February 2007. According to the policy, every contract's award fee plan is directed to include: * a focus on developing specific award fee criteria for each element of contractor performance, * an emphasis on rewarding results rather than effort or activity, and: * an incentive to meet or exceed MDA requirements. Additionally, the directive calls for using the Award Fee Advisory Board, established to make award fee recommendations to the fee determining official, to biannually review and report to the Director on the consistency between MDA's award fees and DOD's Contractor Performance Assessment Report--which provides a record, both positive and negative, on a given contract for a specific period of time. MDA's directive also requires program managers to implement MDA's new award fee policy at the earliest logical point, which is normally the beginning of the next award fee period. MDA is currently constructing a revised draft of its award fee guide that addresses the rollover and rating scale issues from DOD's March 2006 and April 2007 memorandums. In the latest draft, MDA limits rollover to exceptional cases and adopts the Under Secretary's limitation of making only a portion of award fee available for rollover. MDA's latest draft of the guide also makes use of the latest ratings scale, referencing the Under Secretary's April 2007 direction, and applies the usage of the new scales to contract solicitations beginning after July 31, 2007. MDA Tightens Controls on Unpriced Changes and Task Orders: MDA sometimes finds that events such as funding changes, technology advances, and concurrent development and deployment of the BMDS arise that make changes to the contract's provisions or terms necessary. MDA describes contract changes that are within the scope of the contract but whose final price, or cost and fee, the agency and its contractor have not agreed upon as unpriced changes. MDA has followed the FAR in determining how quickly the agency should reach agreement on such unpriced changes' price, or cost and fee. According to the FAR, an agreement should be reached before work begins if it can be done without adversely affecting the interest of the government. If a significant cost increase could result from the unpriced change, and time doesn't permit negotiation of a price, the FAR requires the negotiation of a maximum price unless it is impractical to do so.[Footnote 30] In 2007, MDA began applying tighter limits on definitization of price. MDA also issues unpriced task orders. MDA uses this term to describe task orders issued under established contract ordering provisions, such as an indefinite delivery/indefinite quantity contract, for which a definitive order price has not yet been agreed upon. MDA has followed the FAR requirements that task orders placed under an indefinite delivery/indefinite quantity contract must contain, at least, an estimated cost or fee. During Block 2006--January 1, 2006 through December 31, 2007--MDA authorized 137 unpriced changes and task orders with a value of more than $6 billion. Consistent with the FAR requirements noted above, of the total 137 unpriced changes and unpriced task orders, 61 percent of these--totaling $5.9 billion--were not priced for more than 180 days. Agreement on the price of several was not reached for more than a year, and agreement on the price of one was not reached for more than two and a half years. Table 11 below shows the value of unpriced changes and task orders issued on behalf of each BMDS element and the number of days after the contractor was authorized to proceed with the work before MDA and its contractor agreed to a price, or cost and fee, for the work. Table 11: MDA's Block 2006 Unpriced Changes and Unpriced Task Orders from January 1, 2007 to September 26, 2007: Element: Aegis BMD; Total number of unpriced changes and task orders: 0; Number unpriced more than 180 days before reaching price agreement: 0; Percentage unpriced more than 180 days before reaching price agreement: 0%; Total dollar value of unpriced change and task orders (Dollars in millions): $0. Element: ABL; Total number of unpriced changes and task orders: 0; Number unpriced more than 180 days before reaching price agreement: 0; Percentage unpriced more than 180 days before reaching price agreement: 0%; Total dollar value of unpriced change and task orders (Dollars in millions): $0. Element: C2BMC; Total number of unpriced changes and task orders: 23; Number unpriced more than 180 days before reaching price agreement: 4; Percentage unpriced more than 180 days before reaching price agreement: 17%; Total dollar value of unpriced change and task orders (Dollars in millions): $389.3. Element: GMD; Total number of unpriced changes and task orders: 70; Number unpriced more than 180 days before reaching price agreement: 52; Percentage unpriced more than 180 days before reaching price agreement: 74%; Total dollar value of unpriced change and task orders (Dollars in millions): $5,100.9. Element: KEI; Total number of unpriced changes and task orders: 1; Number unpriced more than 180 days before reaching price agreement: 0; Percentage unpriced more than 180 days before reaching price agreement: 0%; Total dollar value of unpriced change and task orders (Dollars in millions): $12.3. Element: MKV [A]; Total number of unpriced changes and task orders: 4; Number unpriced more than 180 days before reaching price agreement: 3; Percentage unpriced more than 180 days before reaching price agreement: 75%; Total dollar value of unpriced change and task orders (Dollars in millions): $89.6. Element: Sensors; Total number of unpriced changes and task orders: 16; Number unpriced more than 180 days before reaching price agreement: 6; Percentage unpriced more than 180 days before reaching price agreement: 38%; Total dollar value of unpriced change and task orders (Dollars in millions): $171.0. Element: STSS; Total number of unpriced changes and task orders: 4; Number unpriced more than 180 days before reaching price agreement: 2; Percentage unpriced more than 180 days before reaching price agreement: 50%; Total dollar value of unpriced change and task orders (Dollars in millions): $161.6. Element: THAAD[ B]; Total number of unpriced changes and task orders: 19; Number unpriced more than 180 days before reaching price agreement: 16; Percentage unpriced more than 180 days before reaching price agreement: 84%; Total dollar value of unpriced change and task orders (Dollars in millions): $172.8. Element: Total [C]; Total number of unpriced changes and task orders: 137; Number unpriced more than 180 days before reaching price agreement: 83; Percentage unpriced more than 180 days before reaching price agreement: 61%; Total dollar value of unpriced change and task orders (Dollars in millions): $6,097.3. Source: MDA (data); GAO (analysis). [A] Total for the MKV element does not include one unpriced change or task order whose amount was not available as of September 26, 2007. [B] The THAAD element's data is current as of January 28, 2008. [C] Total may not be exact due to rounding. [End of table] Realizing that unpriced changes and unpriced task orders may greatly reduce the government's negotiation leverage and typically result in higher cost and fee for the overall effort, MDA, in February 2007, issued new contract guidance that required tighter limits on the timeframes for reaching agreement on price, or cost and fee. The agency now applies some of the Defense Federal Acquisition Regulation Supplement guidelines established for undefinitized contract actions to unpriced changes and unpriced task orders.[Footnote 31] Undefinitized contract actions are different from MDA's unpriced changes or unpriced task orders in that they are contract actions on which performance is begun before agreement on all contract terms, including price, or cost and fee, is reached. A contract modification or change will not be considered an undefinitized contract action if it is within the scope and under the terms of the contract. MDA has elected to follow some of the stricter undefinitized contract action guidelines because the agency believes the guidelines will lead to better cost results. Similar to the undefinitized contract action guidelines, the agency's new guidelines require that MDA's unpriced changes and unpriced task orders be definitized within 180 days, that the contractor be given a dollar value that it cannot exceed until price agreement is reached, and that approval for the unpriced change or task order be obtained in advance. MDA's new policy also, to the maximum extent practicable, limits the amount of funds that a contractor may be given approval to spend on the work before agreement is reached on price to less than 50 percent of the work's expected price. Support Contracts: MDA officials maintain that support contracts provide necessary personnel and are instrumental in developing the BMDS quickly. The agency contracts with 45 different companies that provide the majority of the personnel who perform a variety of tasks. Table 12 illustrates the broad categories of job functions that MDA support contractors carry out. Table 12: MDA Support Contractor Job Functions: Acquisition Management; International Affairs. Administrative Services; Legal Services. Advanced Technology Development; Legislative Affairs. BMDS Information Assurance; Logistics. Business & Financial Management & Cost; Operations Support. Command & Staff; Personnel Services. Contracts; Public Affairs. Director; Safety, Quality & Mission. Facility Services & Supplies; Security. Information Assurance Certification; Test. Information Technology Support; Total Engineering. Intel and Counter Intel; Worldwide Deployment Support. Internal Review; [Empty]. Source: MDA (data); GAO (presentation). [End of table] Last year we reported that MDA had 8,186 approved personnel positions. This number has not changed appreciably in the last year. According to MDA's manpower database, about 8,748 personnel positions--not counting prime contractors--currently support the missile defense program. These positions are filled by government civilian and military employees, contract support employees, employees of federally funded research and development centers (FFRDC), researchers in university and affiliated research centers, as well as a small number of executives on loan from other organizations. MDA funds around 95 percent of the total 8,748 positions through its research and development appropriation. Of this 95 percent, 2,450, or about 29 percent, of the positions are set aside for government civilian personnel. Another 60 percent, or 5,005 positions, are allotted for support contractors. The remaining 11 percent are positions either being filled, or expected to be filled, by employees of FFRDCs and university and affiliated research centers that are on contract or under other types of agreements to perform missile defense tasks. MDA officials noted that nearly 500 of the 8,748 personnel positions available were currently vacant. Table 13 shows the staffing levels within the BMDS elements. Table 13: Program Office Full-time Equivalent Staffing Levels: [See PDF for image] Source: MDA (data). Note:The table includes positions that may be presently vacant. [A] A detailee is a non-Missile Defense Agency position that is temporarily assigned, for a specified period of time, to the Missile Defense Agency for duties and responsibilities other than its permanent component position. During the detail, employee costs such as salary, leave, and other benefits are charged to the component where his or her permanent position is located. [B] Intergovernmental Personnel Act (IPA) positions provide for the temporary assignment of positions between the federal government and state and local governments, colleges and universities, federally funded research and development centers, and other eligible organizations " to facilitate cooperation between the federal government and the non-state entity." Typically only senior executive level positions will be filled by an IPA agreement for 2 years. The recipient of the appointment will continue to be paid by his or her parent organization at full salary and benefits; the parent organization will receive a negotiated reimbursement from the federal government. [C] Liaison positions are those that require its representative to serve as an intermediary and coordinate activities between two different organizations. [D] This one ABL full-time equivalent civilian government position is an MDA employee who will be relocated to another category. [E] The GMD manpower total will be adjusted to 869 full-time equivalents with confirmation for transfer of 95 full-time equivalents from GMD to Sensors for X-band radars and upgrades to early-warning radars. [F] The KEI manpower total will be adjusted to 37 full-time equivalents for changes in FFRDC and contractor support services. [End of table] Support contractors in MDA program and functional offices may perform tasks that closely support those tasks described in the FAR as inherently governmental.[Footnote 32] According to the FAR, tasks such as determining agency policy and approving requirements for prime contracts should only be performed by government personnel.[Footnote 33] Contract personnel that, for example, develop statements of work, support acquisition planning, or assist in budget preparation are carrying out tasks that may closely support tasks meeting this definition.[Footnote 34] Having support contractors perform these tasks may create a potential risk that the contractors may influence the government's control over and accountability for decisions. MDA officials told us that when support contractors perform tasks that closely support those reserved for government employees the agency mitigates its risk by having knowledgeable government personnel provide regular oversight or final approval of the work to ensure that the data being generated is reasonable. [End of section] Appendix III: Incremental Funding: Incremental Versus Full-funding of BMDS Assets: In the tables below we provide more information comparing the cost of purchasing THAAD and Aegis BMD assets incrementally versus fully- funding the assets. Table 14 presents MDA's incremental funding plans for THAAD fire units 3 and 4, 48 Aegis BMD (SM-3) missiles to be produced during Blocks 2012 and 2014, and 19 shipsets intended to improve the performance of Aegis BMD ships. Tables 15 through 17 present our analysis of the cost of purchasing these same assets with procurement funds and following Congress' full-funding policy. Table 14: MDA's Incremental Funding Plan for THAAD and Aegis BMD Assets: (Dollars in thousands). BMDS element: THAAD; (Asset: Fire units 3 and 4 (launchers, battle manager, radars, and 48 missiles); Fiscal year 2009: 64,815; Fiscal year 2010: 409,973; Fiscal year 2011: 354,870; Fiscal year 2012: 259,254; Fiscal year 2013: 84,434; Fiscal year 2014: [Empty]; Fiscal year 2015: [Empty]; Total: $1,173,346. BMDS element: Aegis BMD; Asset: 48 SM-3 IB missiles for Blocks 2012 and 2014; Fiscal year 2009: 86,400; Fiscal year 2010: 144,000; Fiscal year 2011: 144,000; Fiscal year 2012: 119,000; Fiscal year 2013: 25,600; Fiscal year 2014: [Empty]; Fiscal year 2015: [Empty]; Total: $519,000. BMDS element: [Empty]; Asset: 19 4.0.1 shipset procurement and installs[A]; Fiscal year 2009: : 61,400; Fiscal year 2010: : 83,600; Fiscal year 2011: : 101,410; Fiscal year 2012: : 108,880; Fiscal year 2013: : 111,060; Fiscal year 2014: : 22,660; Fiscal year 2015: : 23,110; Total: : $512,120. Total; Asset: [Empty]; Fiscal year 2009: 212,615; Fiscal year 2010: 637,573; Fiscal year 2011: 600,280; Fiscal year 2012: 487,134; Fiscal year 2013: 221,094; Fiscal year 2014: 22,660; Fiscal year 2015: 23,110; Total: $2,204,466. Source: MDA (data); GAO (analysis). Note: Inflation factors were applied to estimate future costs. Factors for 2014 and 2015 were estimated based on factors provided for prior years as these factors are not yet available from DOD. DOD currently provides factors through the current Future Years Defense Plan which ends in 2013. [A] Under the Aegis BMD program incremental funding plan, 4.0.1 shipsets are purchased in one year and installed 2 years later. According to program officials, there is no need for long lead procurement for the 4.0.1 shipsets because all components can be purchased, assembled, and installed within the 3-year life of a procurement appropriation. [End of table] Table 15: Full Funding Alternative for THAAD Fire Units versus MDA Incremental Funding Plan: (Dollars in thousands). THAAD activity: Purchase Long-lead items; Fiscal year 2008: -; Fiscal year 2009: 218,800; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $218,800. THAAD activity: Purchase Fire Unit (launchers, Battle Manager, and 48 missiles); Fiscal year 2008: -; Fiscal year 2009: -; Fiscal year 2010: 392,486; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $392,486. THAAD activity: Purchase Radars; Fiscal year 2008: -; Fiscal year 2009: -; Fiscal year 2010: 458,200; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $458,200. Total cost for fire units 3 and 4; Fiscal year 2008: [Empty]; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Total: $1,069,486. Total MDA incremental funding cost for fire units 3 and 4; Fiscal year 2008: [Empty]; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Total: $1,173,346. Total full funding savings; Fiscal year 2008: [Empty]; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Total: $ 103,860. Source: GAO analysis: Note: According to the THAAD Program Office, to avoid obsolescence, production gaps, and a less economical production rate, THAAD fire units 3 and 4 should be purchased together. [End of table] Table 16: Full Funding Alternative for Aegis BMD Standard Missile-3s versus MDA Incremental Funding Plan: (Dollars in thousands). Aegis BMD SM-3 Activity: Purchase long-lead items for 24 Block 2012 SM- 3 IBs; Fiscal year 2008: 24,100; Fiscal year 2009: -; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $24,100. Aegis BMD SM-3 Activity: Fully fund remaining cost of 24 Block 2012 SM- 3 IBs; Fiscal year 2008: -; Fiscal year 2009: 223,900; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $223,900. Aegis BMD SM-3 Activity: Purchase long-lead items for 24 Block 2014 SM- 3 IBs; Fiscal year 2008: -; Fiscal year 2009: 24,700; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $24,700. Aegis BMD SM-3 Activity: Fully fund remaining cost of 24 Block 2014 SM- 3 IBs; Fiscal year 2008: -; Fiscal year 2009: -; Fiscal year 2010: 228,900; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Total: $228,900. Total full funding cost of 48 SM-3 IBs; Fiscal year 2008: [Empty]; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Total: $501,600. Total MDA incremental funding cost of 48 SM-3 IBs; Fiscal year 2008: [Empty]; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Total: $519,000. Cost savings reaped from full funding; Fiscal year 2008: [Empty]; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Total: $ 17,400. Source: GAO analysis. [End of table] Table 17: Full Funding Alternative for Aegis BMD Shipsets versus MDA Incremental Funding Plan: Aegis BMD Ship Activity: Procure and install 3 BMD 4.0.1 Shipsets; Fiscal year 2009: 76,700; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Fiscal year 2014: -; Fiscal year 2015: -; Total: $76,700. Aegis BMD Ship Activity: Procure and install 4 BMD 4.0.1 Shipsets; Fiscal year 2009: -; Fiscal year 2010: 104,600; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: -; Fiscal year 2014: -; Fiscal year 2015: -; Total: $104,600. Aegis BMD Ship Activity: Procure and install 4 BMD 4.0.1 Shipsets; Fiscal year 2009: -; Fiscal year 2010: -; Fiscal year 2011: 106,800; Fiscal year 2012: - ; Fiscal year 2013: -; Fiscal year 2014: -; Fiscal year 2015: -; Total: $106,800. Aegis BMD Ship Activity: Procure and install 4 BMD 4.0.1 Shipsets; Fiscal year 2009: -; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: 108,900; Fiscal year 2013: -; Fiscal year 2014: -; Fiscal year 2015: -; Total: $108,900. Aegis BMD Ship Activity: Procure and install 4 BMD 4.0.1 Shipsets; Fiscal year 2009: -; Fiscal year 2010: -; Fiscal year 2011: -; Fiscal year 2012: -; Fiscal year 2013: 111,000; Fiscal year 2014: -; Fiscal year 2015: -; Total: $111,000. Aegis BMD Ship Activity: Total full funding cost of 19 4.0.1 shipsets; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Fiscal year 2014: [Empty]; Fiscal year 2015: [Empty]; Total: $508,000. Aegis BMD Ship Activity: Total MDA incremental funding cost of 19 4.0.1 shipsets; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Fiscal year 2014: [Empty]; Fiscal year 2015: [Empty]; Total: $512,120. Aegis BMD Ship Activity: Total full funding savings; Fiscal year 2009: [Empty]; Fiscal year 2010: [Empty]; Fiscal year 2011: [Empty]; Fiscal year 2012: [Empty]; Fiscal year 2013: [Empty]; Fiscal year 2014: [Empty]; Fiscal year 2015: [Empty]; Total: $4,120. Source: GAO analysis. [End of table] [End of section] Appendix IV: Scope and Methodology: To examine the progress MDA made in fiscal year 2007 toward its Block 2006 goals, we examined the accomplishments of nine BMDS elements. The elements included in our review collectively accounted for 77 percent of MDA's fiscal year 2007 research and development budget request. We evaluated each element's progress in fiscal year 2007 toward Block 2006 schedule, testing, performance, and cost goals. In assessing each element we examined Program Execution Reviews, test plans and reports, production plans, Contract Performance Reports, and MDA briefing charts. We developed data collection instruments that were completed by MDA and each element program office. The instruments gathered detailed information on completed program activities including tests, prime contracts, and estimates of element performance. To understand performance issues, we talked with officials from MDA's Deputy for Engineering and Program Director for Targets and Countermeasures, each element program office, as well as the office of DOD's Director, Operational Test and Evaluation. To assess each element's progress toward its cost goals, we reviewed Contract Performance Reports and, when available, the Defense Contract Management Agency's analyses of these reports. We applied established earned value management techniques to data captured in Contract Performance Reports to determine trends and used established earned value management formulas to project the likely costs of prime contracts at completion. We also interviewed MDA officials within the Deputy for Acquisition Management office to gather detailed information regarding BMDS prime contracts. We reviewed 10 prime contracts for the 9 BMDS elements and also examined fiscal year 2007 award fee plans, award fee letters, and gathered data on the number of and policy for unpriced changes and unpriced task orders. We became familiar with sections of the Federal Acquisition Regulation and Defense Federal Acquisition Regulation Supplement dealing with contract type, contract award fees, and undefinitized contract actions. To develop data on support contractors, we held discussions with officials in MDA's Office of Business Operations. We also collected data from MDA's Pride database on the numbers and types of employees supporting MDA operations. In assessing MDA's accountability, transparency, and oversight, we interviewed officials from the Office of the Under Secretary of Defense's Office for Acquisition, Technology, and Logistics and Joint Staff officials. We also examined a Congressional Research Service report, U.S. Code, DOD acquisition system policy, the MDEB Charter, and various MDA documents related to the agency's new block structure. In determining whether MDA would save money if it fully funded THAAD and Aegis BMD assets rather than funding them incrementally, we used present value techniques to restate dollars that MDA planned to expend over a number of years to the equivalent number of dollars that would be needed if MDA fully funded the assets in the fiscal year that incremental funding was to begin. We also considered whether MDA would need to acquire long lead items for the assets and stated those dollars in the base year that their purchase would be required. We then compared the total cost of incrementally funding the assets, as shown in MDA's funding plans, to the fully funded cost that our methodology produced. To ensure that MDA-generated data used in our assessment are reliable, we evaluated the agency's management control processes. We discussed these processes with MDA senior management. In addition, we confirmed the accuracy of MDA-generated data with multiple sources within MDA and, when possible, with independent experts. To assess the validity and reliability of prime contractors' earned value management systems and reports, we interviewed officials and analyzed audit reports prepared by the Defense Contract Audit Agency. Finally, we assessed MDA's internal accounting and administrative management controls by reviewing MDA's Federal Manager's Financial Integrity Report for Fiscal Years 2003, 2004, 2005, 2006, and 2007. Our work was performed primarily at MDA headquarters in Arlington, Virginia. At this location, we met with officials from the Aegis Ballistic Missile Defense Program Office; Airborne Laser Program Office; Command, Control, Battle Management, and Communications Program Office; BMDS Targets Office, and MDA's Agency Operations Office. We also met with DOD's Office of the Director, Operational Test and Evaluation and the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics in Washington, DC. In addition, in Huntsville, Alabama, we met with officials from the Ground-based Midcourse Defense Program Office, the Terminal High Altitude Area Defense Project Office, the Kinetic Energy Interceptors Program Office, the Multiple Kill Vehicle Program Office, and BMDS Tests Office. We also met with Space Tracking and Surveillance System officials in Los Angeles, California. We conducted this performance audit from May 2007 to March 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Paul Francis (202) 512-4841 or FrancisP@gao.gov: Acknowledgments: In addition to the individual named above, Barbara Haynes, Assistant Director; LaTonya Miller; Sigrid McGinty; Michele R. Williamson; Michael Hesse; Steven Stern; Meredith Allen Kimmett; Kenneth E. Patton; and Alyssa Weir made key contributions to this report. [End of section] Footnotes: [1] National Defense Authorization Act for Fiscal Year 2002, Pub. L. No. 107-107, 232(g) (2001); National Defense Authorization Act for Fiscal Year 2006, Pub. L. No. 109-163, 232 (2006); the John Warner National Defense Authorization Act for 2007, Pub. L. No. 109-364, 224 (2006) recently extended the requirement for GAO assessment. [2] We did not assess MDA's progress in fiscal year 2002 as the agency did not establish goals for that fiscal year. We delivered the following reports for fiscal years 2003 through 2006: GAO, Missile Defense: Actions Are Needed to Enhance Testing and Accountability, GAO- 04-409 (Washington, D.C.: Apr. 23, 2004); Defense Acquisitions: Status of Ballistic Missile Defense Program in 2004, GAO-05-243 (Washington, D.C.: Mar. 31, 2005); Defense Acquisitions: Missile Defense Agency Fields Initial Capability but Falls Short of Original Goals, GAO-06-327 (Washington, D.C.: Mar.15, 2006;), Defense Acquisitions: Missile Defense Acquisition Strategy Generates Results but Delivers Less at a Higher Cost, GAO-07-387 (Washington, D.C. : Mar.15, 2007). [3] The BMDS also includes a 10th element, Patriot Advanced Capability- 3 (PAC-3), which has been transferred to the Army for production, operation, and sustainment. This report does not evaluate PAC-3 because its initial development is complete and is now being managed by the Army. [4] MDA's new block construct is discussed in detail later in this report. [5] GAO, Defense Acquisitions: Missile Defense Acquisition Strategy Generates Results but Delivers Less at a Higher Cost, GAO-07-387 (Washington, D.C.: Mar.15, 2007). [6] Earned Value Management (EVM) is a program management tool that integrates the technical, cost, and schedule parameters of a contract. During the planning phase, an integrated baseline is developed by time phasing budget resources for defined work. As work is performed and measured against the baseline, the corresponding budget value is "earned". Using this earned value metric, cost and schedule variances can be determined and analyzed. From these basic variance measurements, the program manager can identify significant cost and schedule drivers, forecast future cost and schedule performance, and construct corrective action plans to get the program back on track. EVM therefore encompasses both performance measurement (the status of the program) and performance management (what can be done to bring the program back on track). EVM is program management that provides significant benefits to both the Government and the contractor. [7] MDA employs 10 prime contractors to develop the 9 BMDS elements. There is one prime contractor for each element except Aegis BMD, which has 2 prime contractors. One Aegis BMD contractor is responsible for the weapon system component and another for the SM-3 missile. [8] Work under the Aegis BMD contract is divided into contract line items. The contractor initially was directed, under two separate contract line items, to develop, flight test, and deliver 12 Block 1A missiles. When this work was completed, the contractor was directed, under a third contract line item, to initiate the production of 20 additional missiles. The MKV program uses task orders issued under an indefinite delivery/indefinite quantity contract to direct its contractor to accomplish certain work. More than one task order may be under way at any one time during the contract's life. [9] Atmospheric compensation is performed by the ABL's beam control system. It measures atmospheric disturbance between ABL and the target so that the laser beam can be shaped, preventing the atmosphere from scattering and weakening the beam's energy. [10] thermal vacuum test verifies that the temperature control design will maintain the spacecraft and all its elements within allowable flight temperature ranges while operating over the environmental extremes expected for the mission. [11] BMDS performance goals included a numerical goal for the probability of a successful BMDS engagement, a defined area from which the BMDS would prevent an enemy from launching a ballistic missile, and a defined area that the BMDS would protect from ballistic missile attacks. [12] GAO-06-327. [13] he National Defense Authorization Act for Fiscal Year 2008; Pub. L. No. 110-181, 223. [14] GAO-07-387. [15] GAO-07-387. [16] MDA expects to initially develop budget baselines and report variances to this baseline for Blocks 1.0, 2.0, and a portion of 3.0. [17] The Defense Acquisition Board advises the Under Secretary of Defense for Acquisition, Technology, and Logistics on critical acquisition decisions. [18] MDA is subject to a requirement enacted in section 234(e) of the Fiscal Year 2005 National Defense Authorization Act, Pub. L. No. 108- 375 that requires the Director, MDA, to establish and report annually to Congress a cost, schedule, and performance baseline for each block configuration being fielded. The National Defense Authorization Act for 2008 extended this to include all MDA elements that have entered the equivalent of System Development and Demonstration or are produced and acquired for operational fielding. Modification to the baseline and variations against the baseline must also be reported. MDA is also subject to a statutory requirement that life-cycle costs be considered. MDA asserts that DOD's independent Cost Analysis Improvement Group has completed independent cost analyses of three BMDS Component Program Offices. In a February 2002 memorandum, the Under Secretary of Defense delegated to the Director, MDA, the full responsibility and authority for baselining each BMDS capability and configuration. In addition, Section 223 of the Fiscal Year 2008 National Defense Authorization Act contains requirements for unit cost reporting and independent cost estimates for certain MDA elements. [19] 10 USC 2399 requires completion of initial operational test and evaluation of a weapon system before a program can proceed beyond low- rate initial production. According to DOD policy, low-rate initial production is intended to result in completion of manufacturing development in order to ensure adequate and efficient manufacturing capability and to produce the minimum quantity necessary to provide production or production-representative articles for operational test and evaluation, establish an initial production base for the system; and permit an orderly increase in the production rate for the system, sufficient to lead to full-rate production upon successful completion of operational (and live-fire, where applicable) testing. [20] Joint Capabilities Integration and Development System (JCIDS) is the formal DOD procedure that defines acquisition requirements and evaluation criteria for future defense programs. JCIDS was created to replace the previous service-specific requirements generation system, which allegedly created redundancies in capabilities and failed to meet the combined needs of all US military services. In order to correct these problems, JCIDS is intended to guide the development of requirements for future acquisition systems to reflect the needs of all four services (Army, Navy, Marines, and Air Force) by focusing the requirements generation process on needed capabilities as requested or defined by one of the US combatant commanders. In the JCIDS process, regional and functional combatant commanders give feedback early in the development process to ensure that their requirements are met. [21] The National Defense Authorization Act for Fiscal Year 2008, Pub. L. No. 110-181, 223. [22] Congressional Research Service, Defense Procurement: Full Funding Policy--Background, Issues, and Options for Congress (Oct. 20, 2006). [23] As with most cost reimbursement research and development contracts, BMDS contractors are responsible to put forth their best efforts on the development of the BMDS capability. If, given that effort, the BMDS capability falls short of needs, the government has the option of stopping the effort or allowing the contractor to continue with no additional fee. [24] Key Performance Events measure the contractor's timely and effective completion of those events essential to successful development of the planned capabilities. [25] The Standard for Earned Value Management Systems Intent Guide was created by the National Defense Industrial Association Program Management Systems Committee to provide additional insight into some of the EVMS guidelines included in the American National Standards Institute/Electronic Industries Alliance Standard-748-A Standard for Earned Value Management Systems. [26] GAO, Defense Acquisitions: DOD Has Paid Billions in Award and Incentive Fees Regardless of Acquisition Outcomes, GAO-06-66 (Washington, D.C.: Dec. 19, 2005). [27] GAO-06-66. [28] According to the Federal Acquisition Regulation (FAR) 16.405- 2(a)(2), a cost plus award fee contract should include an award amount that is sufficient to provide motivation for excellence in such areas as quality, timeliness, technical ingenuity, and cost-effective management. [29] We reviewed the award fee plans for all 10 BMDS contracts. All except for one identified the percentage of award fee that would be provided for satisfactory performance. GMD's plan was silent as to the percentage of fee that would be awarded for this level of performance. [30] FAR 43.102(b). The Federal Acquisition Regulation also requires change orders-a type of contract modification that is included in MDA's "unpriced changes"--to have an agreed upon price in "the shortest practicable time." FAR 43.204. [31] The Defense Federal Acquisition Regulation Supplement is DOD's supplement to the FAR. [32] The Federal Acquisition Regulation provides guidance to federal executive agencies on acquiring goods and services with appropriated funds. [33] FAR 7.503(c)(5), (12)(iii). [34] FAR 7.503(d)(1),(6),(9). GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548:

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.