Defense Acquisitions
DOD Can Improve Its Management of Configuration Steering Boards Gao ID: GAO-11-640 July 7, 2011GAO has previously reported that requirements changes are factors in poor cost and schedule outcomes on Department of Defense (DOD) weapon programs. In 2007, DOD introduced Configuration Steering Boards (CSBs) to review requirement and configuration changes that could adversely affect programs. In 2008, Congress made annual CSB meetings a requirement for all of the military departments' major defense acquisition programs. In response to the Senate report accompanying the bill for the Ike Skelton National Defense Authorization Act for Fiscal Year 2011, GAO assessed (1) the extent to which DOD has complied with the statutory requirements for CSBs, and (2) the extent to which CSBs have been effective in controlling requirements and mitigating cost and schedule risks. To conduct this work, GAO surveyed DOD's major defense acquisition programs, reviewed CSB documentation, and interviewed relevant military service and program officials.
The military departments varied in their compliance with the CSB requirements in statute. The Air Force and Navy did not fully comply with the requirement to hold annual CSB meetings for all major defense acquisition programs in 2010, while the Army did. In total, the military departments held an annual CSB meeting for 74 of 96 major defense acquisition programs they managed in 2010. According to GAO's survey results, when the military departments held CSB meetings, 19 programs endorsed requirements or configuration changes. In most of these cases, strategies were developed to mitigate the effects of these changes--a key provision in the statute and DOD policy. However, key acquisition and requirements personnel were often absent from Air Force and Navy CSB meetings when these issues were discussed. Two major defense acquisition programs--the Ballistic Missile Defense System and the Chemical Demilitarization-Assembled Chemical Weapons Alternatives programs--are not subject to the CSB provisions in statute because the statute only applies to programs overseen by military departments; the programs are managed by other DOD components. These programs are subject to DOD's CSB policy, which differs from the statute in that it only requires major defense acquisition programs that are in development to hold annual CSB reviews. Individual programs varied in the extent to which they utilized CSBs to control requirements and mitigate cost and schedule risks. According to GAO's survey results, the majority of CSB meetings neither reviewed requirement changes nor discussed options to moderate requirements or reduce the scope of programs. There were a number of specific instances where CSB meetings were effective in mitigating the effect of necessary changes, rejecting other changes, facilitating discussion of requirements, and endorsing "descoping" options with the potential to improve or preserve cost or schedule. However, in response to a survey, program officials cast some doubts about the effectiveness of CSBs, and in interviews, acquisition officials indicated that program managers may be reluctant to recommend descoping options due to cultural biases that encourage meeting warfighters' stated needs rather than achieving cost savings, a preference not to elevate decisions to higher levels of review, and concerns that future funding may be cut if potential savings are identified. In response, the Army and Air Force have issued additional descoping guidance and set savings or budget targets. The types of discussions for which CSBs were useful changed based on whether programs were in development or production. Development programs found them more useful to consider requirements changes and descoping options, and production programs found CSBs more useful to prevent changes. In an effort to further increase effectiveness and efficiency of CSBs, some of the military departments have taken steps to coordinate CSB meetings among programs that provide similar capabilities and align CSB meetings with other significant reviews. Among GAO's recommendations for DOD components are that they amend their CSB policies to be consistent with statute and align CSBs with other reviews when possible. In comments on a draft of this report, DOD concurred or partially concurred with all seven of GAO's recommendations and agreed to take action to address six of them.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Michael J. Sullivan Team: Government Accountability Office: Acquisition and Sourcing Management Phone: (937) 258-7915GAO-11-640, Defense Acquisitions: DOD Can Improve Its Management of Configuration Steering Boards
This is the accessible text file for GAO report number GAO-11-640
entitled 'Defense Acquisitions: DOD Can Improve Its Management of
Configuration Steering Boards' which was released on July 7, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Committee on Armed Services, U.S. Senate:
July 2011:
Defense Acquisitions:
DOD Can Improve Its Management of Configuration Steering Boards:
GAO-11-640:
GAO Highlights:
Highlights of GAO-11-640, a report to the Committee on Armed Services,
U.S. Senate.
Why GAO Did This Study:
GAO has previously reported that requirements changes are factors in
poor cost and schedule outcomes on Department of Defense (DOD) weapon
programs. In 2007, DOD introduced Configuration Steering Boards (CSBs)
to review requirement and configuration changes that could adversely
affect programs. In 2008, Congress made annual CSB meetings a
requirement for all of the military departments‘ major defense
acquisition programs. In response to the Senate report accompanying
the bill for the Ike Skelton National Defense Authorization Act for
Fiscal Year 2011, GAO assessed (1) the extent to which DOD has
complied with the statutory requirements for CSBs, and (2) the extent
to which CSBs have been effective in controlling requirements and
mitigating cost and schedule risks. To conduct this work, GAO surveyed
DOD‘s major defense acquisition programs, reviewed CSB documentation,
and interviewed relevant military service and program officials.
What GAO Found:
The military departments varied in their compliance with the CSB
requirements in statute. The Air Force and Navy did not fully comply
with the requirement to hold annual CSB meetings for all major defense
acquisition programs in 2010, while the Army did. In total, the
military departments held an annual CSB meeting for 74 of 96 major
defense acquisition programs they managed in 2010. According to GAO‘s
survey results, when the military departments held CSB meetings, 19
programs endorsed requirements or configuration changes. In most of
these cases, strategies were developed to mitigate the effects of
these changes-”a key provision in the statute and DOD policy. However,
key acquisition and requirements personnel were often absent from Air
Force and Navy CSB meetings when these issues were discussed. Two
major defense acquisition programs-”the Ballistic Missile Defense
System and the Chemical Demilitarization-Assembled Chemical Weapons
Alternatives programs”-are not subject to the CSB provisions in
statute because the statute only applies to programs overseen by
military departments; the programs are managed by other DOD
components. These programs are subject to DOD‘s CSB policy, which
differs from the statute in that it only requires major defense
acquisition programs that are in development to hold annual CSB
reviews.
Individual programs varied in the extent to which they utilized CSBs
to control requirements and mitigate cost and schedule risks.
According to GAO‘s survey results, the majority of CSB meetings
neither reviewed requirement changes nor discussed options to moderate
requirements or reduce the scope of programs. There were a number of
specific instances where CSB meetings were effective in mitigating the
effect of necessary changes, rejecting other changes, facilitating
discussion of requirements, and endorsing ’descoping“ options with the
potential to improve or preserve cost or schedule. However, in
response to a survey, program officials cast some doubts about the
effectiveness of CSBs, and in interviews, acquisition officials
indicated that program managers may be reluctant to recommend
descoping options due to cultural biases that encourage meeting
warfighters‘ stated needs rather than achieving cost savings, a
preference not to elevate decisions to higher levels of review, and
concerns that future funding may be cut if potential savings are
identified. In response, the Army and Air Force have issued additional
descoping guidance and set savings or budget targets. The types of
discussions for which CSBs were useful changed based on whether
programs were in development or production. Development programs found
them more useful to consider requirements changes and descoping
options, and production programs found CSBs more useful to prevent
changes. In an effort to further increase effectiveness and efficiency
of CSBs, some of the military departments have taken steps to
coordinate CSB meetings among programs that provide similar
capabilities and align CSB meetings with other significant reviews.
What GAO Recommends:
Among GAO‘s recommendations for DOD components are that they amend
their CSB policies to be consistent with statute and align CSBs with
other reviews when possible. In comments on a draft of this report,
DOD concurred or partially concurred with all seven of GAO‘s
recommendations and agreed to take action to address six of them.
View [hyperlink, http://www.gao.gov/products/GAO-11-640] or key
components. For more information, contact Michael J. Sullivan at (202)
512-4841 or sullivanm@gao.gov.
[End of section]
Contents:
Letter:
Background:
Compliance with the CSB Provisions in Statute Varied by Military
Department:
CSB Meetings Had Some Positive Effects on Programs' Efforts to Control
Requirements and Costs:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contact and Acknowledgments:
Tables:
Table 1: Configuration Steering Board Meetings Held for Major Defense
Acquisition Programs by Military Department in 2010:
Table 2: Explanations Provided by Military Departments for Not Holding
2010 Configuration Steering Boards:
Table 3: Descoping Options Endorsed at CSB Meetings:
Table 4: Program Officials' Opinions on the Utility of CSB Meetings by
Acquisition Phase:
Abbreviations:
ASD (NCB): Assistant Secretary of Defense for Nuclear, Chemical, and
Biological Programs:
BMDS: Ballistic Missile Defense System:
CSB: Configuration Steering Board:
DOD: Department of Defense:
MDA: Missile Defense Agency:
OSD: Office of the Secretary of Defense:
SAR: Selected Acquisition Report:
USD (AT&L): Undersecretary of Defense for Acquisition, Technology and
Logistics:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
July 7, 2011:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Department of Defense's (DOD) major defense acquisition programs
have historically cost more and taken longer to field capabilities to
the warfighter than initially planned.[Footnote 1] The total
acquisition cost of DOD's portfolio of major programs has increased by
$135 billion since 2008, and the average delay in delivering initial
capability is now 22 months.[Footnote 2] We have previously reported
that requirements changes and the inability of program managers to
defer requirements that could not be completed under existing cost and
schedule targets are factors in poor acquisition program outcomes.
[Footnote 3] To address this issue, the Under Secretary of Defense
(USD) for Acquisition, Technology and Logistics (AT&L) introduced
Configuration Steering Boards (CSBs) in 2007 to review requirement and
configuration changes that could adversely affect cost and schedule
for major programs in development. Congress has also identified the
CSB as a way to enable this process. In the Duncan Hunter National
Defense Authorization Act for Fiscal Year 2009, Congress made annual
CSB meetings a requirement for all major defense acquisition programs.
[Footnote 4]
In our 2010 assessment of selected weapons programs, we found that few
programs reported holding CSB meetings in 2009[Footnote 5]. In
response, the Senate report that accompanied the bill for the Ike
Skelton National Defense Authorization Act for Fiscal Year 2011 asked
us to review DOD's use of CSBs in fiscal year 201[Footnote 6]0. This
report assesses: (1) the extent to which DOD has complied with the
statutory requirements for CSBs and (2) the extent to which CSBs have
been effective in controlling requirements and mitigating cost and
schedule risks.
To determine the extent to which DOD complied with the statutory
requirement to hold annual CSB meetings, we identified 98 active major
defense acquisition programs using the DOD's Defense Acquisition
Management Information Retrieval System.[Footnote 7] We defined an
active program as one that issued a selected acquisition report in
December 2009.[Footnote 8] For each program, we asked the acquisition
organization overseeing it--the Army, Navy, Air Force, Missile Defense
Agency, and the Assistant Secretary of Defense for Nuclear, Chemical,
and Biological Defense Programs--to provide the minutes and lists of
attendees from the CSB meetings held in calendar year 2010.
To determine the extent to which CSBs have been effective in
controlling requirements and mitigating cost and schedule risks, we
surveyed all 98 program offices to gather information on their
programs, on the CSB meetings their programs held in fiscal year 2010,
and on the utility of these meetings. We collected fiscal-year data in
our survey because the Senate report language that contained our
mandate focused on fiscal-year 2010. All 98 programs completed the
survey. We also conducted interviews with 17 programs to collect more
information about how requirements changes were reviewed, how they
mitigated cost and schedule changes, and what made CSB meetings
effective or ineffective. We selected these programs based on the
types of activities that took place during their CSB meetings as
reported in their survey responses and minutes. Appendix I contains
more information regarding our scope and methodology.
We conducted this performance audit from September 2010 to July 2011
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
In July 2007, the USD (AT&L) established CSBs for every current and
future major defense acquisition program in development as a measure
to limit requirements change and avoid cost increases. The CSBs were
to have a broad membership, including senior representatives from the
offices of USD (AT&L) and Joint Staff. CSBs were intended to review
all requirements and significant technical configuration changes with
the potential to adversely affect the program.[Footnote 9] The USD
(AT&L) directed that these changes should generally be rejected or
deferred unless funds and schedule adjustments could be identified to
mitigate their effects. In addition, program managers were asked to
identify options to reduce program cost or moderate requirements,
referred to as "descoping" options, on a roughly annual basis. USD
(AT&L) also instructed that, while policy would be to keep within
planned costs as much as possible even at the expense of scope and
content, all expected increases in program costs must be budgeted at
the absolute earliest opportunity. USD (AT&L) incorporated CSBs into
DOD's primary acquisition policy--DOD Instruction 5000.02--in December
2008.
In October 2008, Congress enacted the Duncan Hunter National Defense
Authorization Act for Fiscal Year 2009, which required the
establishment of CSBs for the major defense acquisition programs of
the military departments.[Footnote 10] According to the statute, a CSB
must meet at least once each year for each of these programs.[Footnote
11] The statute also provided direction on CSB membership and
responsibilities. It requires CSBs to:
* include the appropriate service acquisition executive as chair and
include representatives from USD (AT&L), the Chief of Staff for the
armed forces, representatives from other armed forces as appropriate,
the Joint Staff, the comptroller of the military department, the
military deputy to the service acquisition executive, the program
executive officer for the program concerned, and others as appropriate;
* prevent unnecessary changes to programs that could have an adverse
impact on program cost or schedule, mitigate adverse cost and schedule
effects of changes that may be required, and ensure that each program
delivers as much planned capability as possible at or below the
planned cost and schedule;
* review and approve or disapprove any proposed changes to program
requirements or system configuration with the potential to adversely
affect cost and schedule; and:
* review and recommend proposals that could reduce requirements and
improve cost and schedule.
In addition, the statute provided program managers the authority to:
* object to adding new requirements that would be inconsistent with
previously established parameters unless approved by the CSB and:
* propose opportunities to reduce program requirements to improve cost
and schedule consistent with program objectives.
In our March 2010 assessment of selected weapon programs, we reported
that only 7 of the 42 programs we assessed held CSB meetings in 2009.
[Footnote 12] As a result, in the Senate report accompanying the bill
for the Ike Skelton National Defense Authorization Act for Fiscal Year
2011, the Senate Armed Services Committee directed USD (AT&L) to take
appropriate steps to ensure that CSBs meet at least once a year to
consider the full range of proposed changes to program requirements or
system configuration for each major defense acquisition program.
[Footnote 13]
Compliance with the CSB Provisions in Statute Varied by Military
Department:
The military departments' compliance with statutory CSB requirements
varied. The Air Force and Navy did not fully comply with the
requirement to hold annual CSB meetings for all major defense
acquisition programs in 2010; the Army did comply. In total, the
military departments held an annual CSB meeting for 74 of 96 major
defense acquisition programs they managed in 2010. According to our
survey results, when the military departments held CSB meetings, 19
programs endorsed requirements or configuration changes. In most of
these cases, strategies were developed to mitigate any effect on a
program's cost and schedule--a key provision in the statute and DOD
policy. However, key acquisition and requirements personnel were often
absent from Air Force and Navy CSB meetings when these issues were
discussed. Two major defense acquisition programs--the Ballistic
Missile Defense System (BMDS) and the Chemical Demilitarization-
Assembled Chemical Weapons Alternatives programs, which are managed by
DOD components rather than military departments--are not subject to
the CSB provisions in statute, but rather to DOD policy, because the
statute only applies to programs overseen by military departments.
This policy differs from the statute in that it only requires major
defense acquisition programs in development to hold annual CSB reviews
and does not require the same members, including the comptroller of
the military department.
The Air Force and Navy Did Not Hold CSB Meetings for All Programs,
While the Army Did:
The Air Force and Navy did not hold CSB meetings for all of their
major defense acquisition programs in 2010. The Air Force did not hold
CSB meetings for 13 of 31 programs, and the Navy did not hold CSB
meetings for 9 of 37 programs. The Army held a CSB meeting for each of
its 28 major defense acquisition programs. Of the 96 major defense
acquisition programs managed by the military departments, 74 held CSB
meetings in 2010 and 22 failed to do so. Table 1 shows how many
programs had CSB meetings by military department.
Table 1: Configuration Steering Board Meetings Held for Major Defense
Acquisition Programs by Military Department in 2010:
Military department: Air Force;
Programs with CSB meetings: 18;
Programs without CSB meetings: 13;
Total programs: 31.
Military department: Army;
Programs with CSB meetings: 28;
Programs without CSB meetings: 0;
Total programs: 28.
Military department: Navy;
Programs with CSB meetings: 28;
Programs without CSB meetings: 9;
Total programs: 37.
Military department: Total;
Programs with CSB meetings: 74;
Programs without CSB meetings: 22;
Total programs: 96.
Source: GAO analysis of information from DOD components.
[End of table]
Of the 22 programs that did not have CSB meetings in 2010, 9 programs
had meetings in early 2011. In addition, according to the Air Force
and Navy, 8 other programs were in the process of being completed or
canceled. Table 2 includes explanations from the Air Force and Navy
about why CSB meetings were not held for individual programs.
Table 2: Explanations Provided by Military Departments for Not Holding
2010 Configuration Steering Boards:
Air Force:
Advanced Medium Range Air-to-Air Missile (AIM-120);
CSB held in December 2009 and January 2011.
C-130J Hercules;
CSB held in January 2011.
C-17A Globemaster III;
Exceeded 90 percent of quantities delivered[A].
C-5 Avionics Modernization Program;
Exceeded 90 percent of quantities delivered[A].
C-5 Reliability Enhancement and Reengining Program;
CSB held in January 2011.
Joint Air-to-Surface Standoff Missile/Joint Air-to-Surface Standoff
Missile Extended Range;
CSB held in December 2009 and January 2011.
Joint Cargo Aircraft;
Program in process of transferring from the Army.
Joint Direct Attack Munition;
Exceeded 90 percent of quantities delivered[A].
Joint Primary Aircraft Training System;
CSB held in January 2011.
Large Aircraft Infrared Countermeasures;
CSB held in January 2011.
Minuteman III Propulsion Replacement Program;
Exceeded 90 percent of quantities delivered[A].
National Airspace System;
CSB held in February 2011.
National Polar-orbiting Operational Environmental Satellite System;
Program in process of cancellation.
Navy:
Advanced Anti-Radiation Guided Missile (AGM-88E);
CSB held in January 2011.
EA-6B Improved Capability III;
Exceeded 90 percent of quantities delivered[A].
Expeditionary Fighting Vehicle;
Program in process of cancellation.
Joint High Speed Vessel;
CSBs held in October 2009 and March 2011.
Littoral Combat Ship;
No need for CSB as the configuration is locked.
Mobile User Objective System;
No need for CSB as requirements are stable.
Nimitz Class Carrier (CVN 68);
Exceeded 90 percent of quantities delivered[A].
Remote Minehunting System;
Other review held.
Zumwalt Class Destroyer (DDG 1000);
Other reviews held.
Source: GAO presentation of information from DOD components.
[A] According to DOD, programs with 90 percent of items delivered are
no longer covered by the statute as changes to requirements or
configuration could no longer occur after a program reaches its
inventory objective.
[End of table]
Most Programs That Made Changes Developed Ways to Mitigate the Cost
and Schedule Effects:
For each of the military departments, when a CSB meeting reviewed
requirements or configuration changes, most were endorsed and
strategies to mitigate the effects on a program's cost and schedule
were developed and discussed. However, most of the programs we
surveyed did not present requirements or configuration changes to be
approved or rejected at their fiscal-year-2010 CSB meetings.
Specifically, our survey showed the following results:
* Air Force: 6 CSB meetings reviewed requirements or configuration
changes, 5 of these meetings endorsed changes, and 4 discussed the
cost and schedule effects and ways to mitigate them.
* Army: 6 CSB meetings reviewed and endorsed requirements or
configuration changes, and 4 of these discussed the cost and schedule
effects and ways to mitigate them.
* Navy: 10 CSB meetings reviewed requirements or configuration
changes; 8 meetings endorsed changes, and 7 of these discussed the
cost and schedule effects and ways to mitigate them.
The Navy did not hold CSB reviews for all programs that experienced
requirements changes in fiscal-year 2010. According to our survey
results, three Navy programs changed system requirements or
specifications yet did not hold a CSB meeting. Two of these programs,
the Advanced Anti-Radiation Guided Missile and the Remote Minehunting
System, held other high-level reviews during this period--two program
management reviews and a critical Nunn-McCurdy breach review,[Footnote
14] respectively--and officials reported that a third program, the
Expeditionary Fighting Vehicle, did not conduct its CSB meeting
because DOD proposed canceling the program.
Key Acquisition and Requirements Personnel Were Absent from Many CSB
Meetings:
Key acquisition and requirements personnel were absent from many of
the CSB meetings held by the Air Force and Navy in 2010. The CSB
provision in the Duncan Hunter National Defense Authorization Act for
Fiscal Year 2009 lists seven officials or offices that should be part
of a CSB, including the service acquisition executive who should serve
as the chairperson of the CSB; representatives from the acquisition,
requirements, and funding communities; and others as appropriate. Army
CSB meetings held in 2010 included the full array of board members in
all but one case. Although USD (AT&L) was invited to the meeting in
this case, Army officials reported that the office did not send a
representative. The medium of CSB board members' participation also
varied among the military departments. The Army conducted all its CSB
meetings in person, whereas both the Air Force and the Navy conducted
virtual, otherwise known as paper,[Footnote 15] CSB meetings for
certain programs in 2010 and early 2011.
The Air Force held all of its 2010 CSB meetings without key
acquisition participants listed in the CSB statute. According to Air
Force officials, their CSB meetings may be chaired by either the
service acquisition executive or the principal military deputy to
provide for flexibility in scheduling meetings.[Footnote 16]
Generally, the principal military deputy acts as chair in the place of
the service acquisition executive and does not attend those meetings
that the service acquisition executive chairs. According to the
attendee lists provided by the Air Force, only 2 of the 18 CSB
meetings held were attended and chaired by the service acquisition
executive. At one of those meetings neither the principal military
deputy nor a representative of the comptroller was in attendance
although officials report that both had been invited. The CSB meetings
the service acquisition executive did not attend included numerous
discussions of changes that could affect programs' costs and
schedules, including requirements and configuration changes or
descoping opportunities. For example, one meeting discussed changes to
the Space Based Infrared System's architecture that could accelerate
the program's delivery of initial capability by 2 years but would cost
an additional $45 million.
The Air Force also allows paper CSBs to fulfill the requirement for an
annual CSB for programs it believes are stable. A program is eligible
to conduct paper CSB meetings if (1) it has a Probability of Program
Success score of greater than 80;[Footnote 17] (2) it has made no
requirements and/or significant technical configuration changes since
the last CSB that have the potential to affect the cost and schedule
of the program; (3) when in production, it is in steady state
production but has not reached 90 percent of planned expenditures
completed or 90 percent of quantities delivered; and (4) descoping
options will not yield any real cost savings. The Air Force did not
conduct any paper CSBs in 2010; however, 6 of the 13 Air Force
programs that did not hold a CSB meeting in 2010 conducted paper
reviews in January 2011. According to Air Force officials, the process
for these paper reviews began in December 2010.
The Navy held most of its 2010 CSB meetings without key acquisition
and requirements personnel. The Navy has incorporated CSB meetings
into the Navy's gate review process and uses the gate 6 review, with
the service acquisition executive or his designee acting as chair, to
fulfill the requirement for an annual CSB.[Footnote 18] However, the
Navy's policy on gate reviews does not include the Joint Staff--a key
player in the requirements process and a participant required by
statute and DOD policy--as a participant, and at least 22 of the 28
CSB meetings held in 2010 lacked a representative of the Joint Staff.
As a result of our review, Navy officials reported that they are
revising their policy and procedures for CSBs to ensure the Joint
Staff is invited to future CSB meetings.
Navy policy allows the service acquisition executive to delegate the
chair to another official within the Navy's acquisition office, which
officials stated provides flexibility in scheduling CSBs. In practice,
this resulted in meetings where required members of the CSB did not
participate in discussions of requirements, configuration, or
descoping. In 2010, the Navy service acquisition executive chaired and
attended 12 of the 28 CSB meetings and participated in at least 2
others, both CSBs conducted via paper. According to our review of CSB
documentation, six CSB meetings clearly discussed descoping options,
and the service acquisition executive did not attend any of the five
held in person. The sixth meeting was a paper CSB and it is unclear
whether the service acquisition executive participated. When the Navy
service acquisition executive or others chair the CSB meeting, the
principal military deputy typically does not attend. In addition, at
least three CSB meetings in 2010 did not include a representative from
USD (AT&L).
The Navy also allows paper CSBs to fulfill the requirement for an
annual CSB. In four cases, the Navy used paper CSBs to review
requirement and configuration changes sometimes requiring millions of
dollars or tens of millions of dollars in additional funding.
According to Navy officials, Navy policy allows CSB members to reach
decisions on issues of requirements and configuration by circulating
briefing slides and memoranda rather than holding an actual meeting;
however, there are not clear criteria specifying the circumstances
under which a program may hold a paper CSB. Multiple Navy program
managers stated that they do not understand which programs are
eligible or when and how to request a paper CSB. In one case, a
program manager stated that although the program was planning for and
preferred a CSB meeting in person, Navy officials changed the format
to a paper CSB a few days before the scheduled meeting time.
Two Major Defense Acquisition Programs Are Not Covered By The CSB
Statute:
Two major defense acquisition programs--the Ballistic Missile Defense
System (BMDS) and the Chemical Demilitarization-Assembled Chemical
Weapons Alternatives programs, which are managed by DOD components
rather than military departments--are not subject to the CSB
provisions in statute because the statute only applies to major
defense acquisition programs overseen by the military departments.
However, DOD acquisition policy, which requires CSBs for all major
defense acquisition programs in development, applies to these programs.
The Missile Defense Agency (MDA), which is responsible for the
management of BMDS, did not hold a CSB for the system in 2010;
however, it did conduct reviews that discussed many of the same issues
and included some of the same participants as those required for CSBs.
The Program Change Board manages the development, fielding, and
integration of BMDS through separate program elements and ensures the
integrity of the system as a whole. This board, which is the primary
forum for discussing and mitigating changes to program elements'
requirements and configuration, met 42 times in 2010. The Program
Change Board is chaired by the equivalent of a service acquisition
executive--the director of MDA--and, according to an MDA official,
includes the equivalent of the comptroller, the program executive
officer, and the program manager. MDA policy also requires USD (AT&L)
to be invited to Program Change Boards, and allows for the military
services' participation when deemed appropriate, but does not include
the Joint Staff. The Missile Defense Executive Board oversees
implementation of strategic plans and reviews the priorities and
budget for BMDS as a whole. The Missile Defense Executive Board
includes the Joint Staff as well as the MDA director and an array of
Office of the Secretary of Defense (OSD) and military service
representatives, but according to DOD it does not generally discuss
requirements and configuration at the element level. The executive
board met seven times in 2010.
The Assistant Secretary of Defense for Nuclear, Chemical, and
Biological Defense Programs, who is responsible for the management of
the Chemical Demilitarization-Assembled Chemical Weapons Alternatives
program, also did not hold a CSB in 2010. However, a similar board--
the Chemical Demilitarization Program Strategic Governance Board--met
three times in 2010 to discuss program progress, including how it is
performing against its requirements and funding issues, including
those related to significant cost and schedule growth. In 2010, the
Assistant Secretary acted as the chair for this board which also
includes representatives from the OSD comptroller, the Joint Staff,
and the Army.
DOD Policy Is Not Consistent with the CSB Provisions in Statute:
The CSB requirements in DOD's primary acquisition instruction are not
fully consistent with the provisions in statute. Most significantly,
the instruction only requires CSB meetings for major defense
acquisition programs in development, rather than major defense
acquisition programs in development and production.[Footnote 19]
Additionally, the instruction does not include the comptroller as a
CSB member. According to USD (AT&L) officials, the CSB provisions in
statute may not have been fully incorporated into USD (AT&L)'s
December 2008 revision of DOD's acquisition instruction because the
statute was enacted in October 2008 and there was not enough time to
reconcile them. USD (AT&L) is in the process of updating the
instruction and is considering changes to the CSB requirements. USD
(AT&L), according to officials, has also not consistently tracked
whether programs are fulfilling the current requirements in DOD policy
because the statute makes CSBs a military department responsibility.
CSB Meetings Had Some Positive Effects on Programs' Efforts to Control
Requirements and Costs:
Individual programs varied in the extent to which they utilized CSBs
to control requirements and mitigate cost and schedule risks.
According to our survey results, the majority of CSB meetings neither
reviewed requirement changes nor discussed options to reduce
requirements or the scope of programs. We found a number of instances
in which CSB meetings were effective in mitigating the effect of
necessary changes, rejecting other changes, facilitating discussion of
requirements, and endorsing descoping options with the potential to
improve or preserve cost or schedule. Program managers, however, may
be reluctant to recommend descoping options because of cultural biases
about the role of a program manager, a preference not to elevate
decisions to higher levels of review, and concerns that future funding
will be cut. In an effort to increase descoping proposals, the Army
and Air Force have issued additional descoping guidance and set
savings or budget targets. The perceived effectiveness of the CSB
meetings also varied based on the acquisition phase of a program and
which CSB members participated. To further increase effectiveness and
efficiency of CSBs, some of the military departments have taken steps
to coordinate CSB meetings among programs that provide similar
capabilities and align CSB meetings with other significant reviews.
Programs Have Had Some Success in Using CSBs to Control and Reduce
Requirements:
We identified individual examples from each military department in
which CSB meetings were used to prevent or reject requirements or
configuration changes, mitigate the cost and schedule effects of
endorsed changes, facilitate the prioritization of requirements, and
provide program managers with opportunities to reduce requirements or
suggest other programmatic changes to lower costs and field systems
faster. However, most of the program officials who held CSB meetings
and responded to our survey reported that CSB meetings were not useful
for preventing changes to requirements or configuration, mitigating
the potential effects on cost and schedule when changes were endorsed,
or recommending ways to improve a program's cost and schedule by
moderating requirements. In interviews with program officials, some
explained that they did not utilize the CSB meetings to control
requirements because they addressed requirement issues as they arose
within the program rather than waiting for their program's scheduled
CSB meeting to occur. Others stated that their program was stable and
that there were no requirement changes or descoping options to
discuss. According to our survey results, reviews of CSB
documentation, and interviews:
* 26 percent of the programs in our survey with CSB meetings reported
that these meetings were useful forums to prevent changes to
requirements. Moreover, 35 percent reported that the meetings were
useful to make necessary changes to requirements. In an interview,
several program officials stated that the mere suggestion of convening
a CSB meeting to discuss a new requirement was enough to deter changes.
* 25 percent of the programs in our survey with CSB meetings reported
that these meetings were useful forums to prevent changes to technical
configuration. Conversely, 23 percent reported that the meetings were
useful to make necessary changes to technical configurations. Our
review of minutes and presentations also show at least one CSB meeting
that rejected a change that had the potential to adversely affect
program cost; the August 2010 CSB review for the LPD 17 amphibious
ship program rejected a proposed configuration change that would have
added new equipment to the ship at an estimated cost of $26 million.
* Some CSB meetings also included discussions of how to prioritize
requirements. For example, according to officials, the Air Force used
a June 2010 CSB meeting for the Global Hawk--an unmanned surveillance
aircraft--to prioritize joint urgent operational needs. According to
program officials, the Global Hawk program has received numerous
requests to add new capabilities to the platform due to its use in
current operations. The program manager stated that the CSB meeting
provided the opportunity to present the costs and benefits of those
requests to decision makers and receive guidance from them on which
ones to pursue or defer.
* 28 percent of the programs in our survey with CSB meetings reported
that these meetings were useful forums to mitigate the potential cost
and schedule effects of changes brought to the CSB for consideration.
Moreover, 18 percent of programs reported CSB meetings were useful
forums to mitigate the potential cost and schedule effects of changes
made as a result of the CSB. The Vertical Take Off and Landing
Tactical Unmanned Aerial Vehicle program used a CSB meeting to discuss
ways to restructure the program in response to cost growth. At the
meeting, the members of the CSB encouraged the program manager to go
beyond his proposals and investigate changes to program quantities,
contract strategy, and operational plans when restructuring the
program, in order to reduce cost.
* CSB meetings seem to have been effective in mitigating the cost and
schedule effects of changes or only endorsing changes that would not
affect costs and schedules. Of the 19 programs in our survey in which
a CSB meeting endorsed changes to requirements or technical
configuration, 1 reported an increase in program cost and 2 reported a
delay in the delivery of an initial operational capability.
* 30 percent of programs in our survey with CSB meetings reported that
these meetings were useful forums to offer options to lower costs and
field systems faster. Survey results show that descoping options were
presented for 19 programs and those options were endorsed for 8 of
them. For example, at the December 2009 CSB meeting for the Air
Force's Joint Air-to-Surface Standoff Missile, the program office
recommended adopting the extended range version's lower reliability
requirement for the baseline missile. The program office stated the
existing baseline requirement, which was 5 percent higher, had the
potential to become a cost driver in testing for the program. The CSB
endorsed the program office's recommendation.
* Program officials also reported that the exercise of formulating
descoping options, regardless of whether or not they were endorsed,
helped their office identify and develop mitigation strategies in the
event costs increased.
Table 3 provides examples of programs across the military departments
that used CSB meetings to endorse requirement, configuration, or other
programmatic changes to improve or preserve cost or schedule.
Table 3: Descoping Options Endorsed at CSB Meetings:
Air Force:
Program: Joint Air-to-Surface Standoff Missile;
Action endorsed: Relaxed reliability requirement;
Result: May avoid test costs.
Program: Joint Strike Fighter;
Action endorsed: Deleted requirement to jettison stores at supersonic
speeds;
Result: Avoided test and development cost.
Program: Predator;
Action endorsed: Transferred two test units to the Army;
Result: May reduce costs.
Army:
Program: Excalibur;
Action endorsed: Reduced quantity of projectiles procured;
Result: Reduced total program cost by $893.5 million.
Program: Family of Medium Tactical Vehicles;
Action endorsed: Eliminated the self-recovery winch on most variants;
Result: Reduced unit cost by $9,535 per vehicle.
Program: Increment 1 Early Infantry Brigade Combat Team;
Action endorsed: Ceased development of three portions of the program;
Result: Reduced total program cost by $112.8 million.
Program: Joint Land Attack Cruise Missile Defense Elevated Netted
Sensor System;
Action endorsed: Reduced quantities and relaxed requirement for
emplacement time;
Result: May result in preservation or improvement of program cost or
schedule.
Program: Joint Tactical Radio System Airborne & Maritime/Fixed Station;
Action endorsed: Relaxed requirement for startup time;
Result: May result in preservation or improvement of program cost or
schedule.
Program: Joint Tactical Radio System Handheld, Manpack, and Small Form
Fit Radio;
Action endorsed: Eliminated requirements for two radios as well as a
requirement for radios to operate one waveform;
Result: May result in preservation or improvement of program cost or
schedule.
Program: Joint Tactical Radio System Network Enterprise Domain;
Action endorsed: Eliminated an information assurance requirement for
one waveform;
Result: Avoided costs of $75 million.
Program: Stryker Family of Vehicles;
Action endorsed: Canceled production of flat bottom variants and
rearranged decision points for others;
Result: Avoided $1.7 billion and $24.1 million, respectively, in costs.
Navy:
Program: AIM-9X Air-to-Air Missile;
Action endorsed: Limited use of missiles in training;
Result: Possible maintenance cost avoidance of 60%.
Program: CH-53K;
Action endorsed: Deferred a communications requirement to future
increments;
Result: Avoided adverse impact on program cost and schedule.
Program: Joint High Speed Vessel;
Action endorsed: Reduction of the ship's transit speed;
Result: Avoided significant redesign and the possibility of increased
cost and schedule.
Source: GAO analysis of CSB minutes.
[End of table]
Program Managers May Be Reluctant to Offer Options to Moderate
Requirements:
Program managers may be reluctant to recommend descoping options to
moderate requirements during a CSB meeting because of cultural biases
about the role of a program manager, a preference not to elevate
decisions to higher levels of review, and concerns that future funding
will be cut. According to several acquisition officials, there is a
cultural bias throughout DOD that the role of the program manager is
to meet the requirements handed to them, not to seek to reduce them to
achieve cost savings. In this context, if a program manager recommends
reducing requirements, it may suggest the person is not managing the
program or serving the warfighter well. Still others preferred to
reduce requirements that were within their span of control through
their program's internal change-management process rather than waiting
for a CSB meeting to ask permission. For example, the DDG-51 program
office proposed changes to the ships' configuration to reduce cost by
removing or relocating equipment and the CH-53K program avoided cost
by relaxing a requirement for self-sealing fuel tanks. Our interviews
with program officials also suggest that there may be a reluctance to
present descoping options at a CSB meeting because it could be
interpreted as an opportunity to reduce the program's budget.
Army and Air Force Are Encouraging More Descoping Options to Reduce
Costs:
The Army and Air Force have both taken steps to encourage or require
program managers to seek options to lower costs by reducing scope.
Acquisition officials noted that the presentation of descoping options
and the focus on reducing costs has increased in importance since CSBs
were first established, as the budget environment has become more
constrained. In a November 2010 memorandum, the Army emphasized the
need for program officials to aggressively seek descoping
opportunities with the goal of reducing per-unit or total program
costs by 5 percent. Army officials stated that the memorandum was
signed by senior leaders from the requirements, acquisition, and
budgeting communities specifically to address the bias that reducing
requirements is unacceptable.
According to officials, the Air Force amended its guidance for CSB
meetings to require programs to present three to four descoping
options along with the effect of those options on performance and
program execution, the dollar amount already invested, and the
estimated savings likely to result. Program managers are instructed to
treat the descoping options as a budgeting exercise and to present the
decisions that would need to be made if the program's current budget
were reduced by 10, 20, and 30 percent. Several program offices told
us that forcing programs to present options to reduce requirements or
scope led them to spend time preparing options that were not viable or
that they would have to recommend against implementing.
CSBs' Effects Differ for Programs in Development and Production:
The types of discussions for which CSBs were useful changed based on
whether programs were in development or production. According to our
survey results, programs in development found CSB meetings to be more
useful than programs in production for making necessary changes to
requirements or technical configuration, mitigating the potential cost
and schedule effects of changes, and recommending proposals to improve
program costs and schedule. Table 4 presents our survey results of
program officials' opinions on the usefulness of CSB meetings.
Table 4: Program Officials' Opinions on the Utility of CSB Meetings by
Acquisition Phase:
CSB was useful for: Preventing changes to requirements;
Percentage of programs in development that responded "Yes": 18;
Percentage of programs in production that responded "Yes": 30;
Percentage of all programs that responded "Yes": 26.
CSB was useful for: Preventing changes to technical configuration;
Percentage of programs in development that responded "Yes": 12;
Percentage of programs in production that responded "Yes": 30;
Percentage of all programs that responded "Yes": 25.
CSB was useful for: Making necessary changes to requirements;
Percentage of programs in development that responded "Yes": 53;
Percentage of programs in production that responded "Yes": 28;
Percentage of all programs that responded "Yes": 35.
CSB was useful for: Making necessary changes to technical
configuration;
Percentage of programs in development that responded "Yes": 24;
Percentage of programs in production that responded "Yes": 23;
Percentage of all programs that responded "Yes": 23.
CSB was useful for: Mitigating the potential cost and schedule impacts
of changes for which CSB convened;
Percentage of programs in development that responded "Yes": 35;
Percentage of programs in production that responded "Yes": 25;
Percentage of all programs that responded "Yes": 28.
CSB was useful for: Mitigating the potential cost and schedule impacts
of changes made as a result of the CSB;
Percentage of programs in development that responded "Yes": 31;
Percentage of programs in production that responded "Yes": 13;
Percentage of all programs that responded "Yes": 18.
CSB was useful for: Recommending proposals to improve the program
costs and schedule;
Percentage of programs in development that responded "Yes": 35;
Percentage of programs in production that responded "Yes": 28;
Percentage of all programs that responded "Yes": 30.
Source: GAO analysis of survey data.
[End of table]
Programs in development also proposed changes to requirements or
configuration, presented options for reducing scope, and had those
options endorsed at a higher rate than those in production. Even so,
an official for one program in development stated that its CSB meeting
was not effective because the program was meeting cost and schedule
targets and its requirements were narrowly defined, which decreased
opportunities for reducing scope.
According to our survey results, a higher percentage of programs in
production reported that CSBs were useful in preventing changes
compared to programs in development. We have previously reported that
stabilizing a program's requirements and design well before production
is important because changes have increasingly negative effects on
cost and schedule the further a program progresses.[Footnote 20]
Program officials were wary about using CSB meetings to try to reduce
costs for programs in production either through requirements changes
or reductions in scope because the configuration should be locked, the
available trade space is probably limited, and potential changes could
be disruptive. For instance, the E-2D program reported in its April
2010 CSB meeting that its configuration was extremely stable and, with
development and demonstration almost complete, reducing the scope of
the programs could prove detrimental because it could lead to
redesigns or decreases in capability. Changes at this stage of a
program can still have a positive effect on cost if they do not
require extensive design changes. For example, the program manager for
the Family of Medium Tactical Vehicles--which is well into production
with over 40,000 vehicles fielded--recommended removing the self-
recovery winch from some vehicles, resulting in savings of $9,535 per
vehicle.
Broad Senior Level Participation in CSB Meetings Facilitates Decision
Making:
CSBs provide a unique opportunity for program managers to address
programmatic issues in front of a broad group of high-level decision
makers that includes the acquisition, requirements, and funding
communities. In some cases, the makeup of the CSB helped to accelerate
the resolution of issues and facilitate decision making. For example,
the Grey Eagle program utilized its CSB meeting to endorse an increase
in the number of active units from 13 to 17. The program office
reported that this decision, which otherwise may have taken years to
approve and fund, was made and implemented quickly by the CSB because
of the senior leadership present. Other program offices stated that
the broad membership on CSBs, which includes key stakeholders and
other interested parties, helps to create institutional buy-in for
programmatic changes. CSB meetings also raised stakeholders' awareness
of cost increases. Specifically, CSB meetings provided the Joint Staff
with its first knowledge of cost growth on at least four programs and
triggered separate reviews by the Joint Requirements Oversight Council.
When critical stakeholders are absent, the decision-making ability of
the CSB may be limited. In particular, some programs with users from
across the military services and organizations external to DOD
reported that the utility of CSBs was limited when those users were
not represented. For example, the primary users of the Air Force's
Global Positioning System IIIA program include the Army, Navy, and
other organizations external to DOD. The September 2010 CSB meeting
for the system did not include these stakeholders, and program
officials stated that as a result, the CSB was not empowered to make
significant changes to the program.
Aligning CSB Meetings with Other Reviews May Increase Effectiveness
and Efficiency:
The decisions made at CSB meetings can affect complementary programs,
as well as the funding required for programs. As a result, acquisition
and program officials told us there is value in aligning CSB meetings
so they are held together with reviews of similar programs or
sequencing them to occur before key funding decisions are made. For
example, in 2010, the Army grouped programs into capability
portfolios, such as aviation or precision fires capabilities, and held
one CSB meeting to discuss requirement changes and descoping options
for all the programs. These CSB meetings generally occurred after the
Army's capability portfolio reviews--which revalidate, modify, or
terminate requirements and ensure the proper allocation of funds
across programs--and reviewed, endorsed, and implemented the
recommendations coming from them. Holding CSB meetings for capability
portfolios can facilitate discussions about interoperability and
interdependency and promote an examination of requirements and
capabilities across programs, including potential redundancies.
Officials also stated that if two well-executed, high-performing
programs within the same portfolio were reviewed independently, those
discussions might not take place. For example, the Army's Excalibur--a
precision-guided munition--and Guided Multiple Launch Rocket System
were both relatively stable programs in production. However, according
to officials, during a capability portfolio review, the Army
identified an overlap in the two programs' capabilities and missions
and recommended reducing the number of Excalibur munitions to be
procured. At the subsequent April 2010 CSB meeting, the Army reviewed
and implemented the proposal, which reduced the cost of the Excalibur
program by $893.5 million. According to acquisition officials,
grouping programs in this manner can also ease the difficulty of
scheduling a large number of meetings that require senior leadership
participation.
According to program officials, when CSB meetings were aligned with
budget deliberations, it enabled an informed discussion of funding
issues and rapid changes to program budgets. USD (AT&L)'s 2007
memorandum establishing CSBs stressed the importance of making
necessary budget adjustments, especially those involving expected
increases in program costs, at the earliest opportunity. In one
example, the Army's November 2009 CSB for the Patriot and Medium
Extended Air Defense System programs corresponded with the service's
fiscal-year-2011 budget-formulation process. Program officials stated
that this helped facilitate the transfer of funds and efforts among
the two programs, which had been endorsed by senior leaders from the
acquisition and funding communities at the CSB. However, it may be
functionally challenging to align CSB meetings with the budget
formulation process in all cases, as CSB meetings in some cases must
be event driven while the budget process is calendar driven.
Conclusions:
With the prospect of slowly growing or flat defense budgets for years
to come, DOD must get better returns on its weapon system investments
than it has in the past. CSBs, which are intended to ensure that a
program delivers as much planned capability as possible at or below
the expected cost, can be a key tool in furthering this goal. They
represent a unique forum that brings together a broad range of high-
level decision makers from the acquisition, requirements, and funding
communities, who can make and implement decisions quickly.
DOD's experience with CSBs to date has already demonstrated their
potential value--costly new requirements have been rejected, and
options to moderate requirements and reduce program costs by millions
of dollars have been endorsed. However, the efficiency and
effectiveness of CSBs can still be improved. Ensuring key CSB members
from the acquisition and requirements community are present at
meetings could help build consensus more quickly and make decisions
more efficiently. Similarly, while the law is silent on whether paper
CSB meetings may be used to meet the annual requirement, holding in-
person meetings may be more effective because a paper meeting may not
provide the opportunity for in-depth discussion or proper oversight.
Holding CSBs in conjunction with capability portfolio reviews and
other similar meetings has the potential to expand opportunities to
review and rationalize requirements across programs. Improving the
connection between CSBs and the budget process and other reviews can
help further efforts to match weapon system requirements with funding
resources. Reviewing programs at CSBs on a case-by-case basis well
into production would help decision makers identify cost savings and
shift funding as warfighter needs and funding priorities change. Taken
together, these steps have the potential to improve not only the
efficiency and effectiveness of CSBs but also the affordability and
execution of DOD's major defense acquisition programs.
Recommendations for Executive Action:
We recommend that the Secretary of Defense take the following seven
actions directing:
* the Navy to amend its policy on CSBs to ensure that all statutorily
required participants, particularly the Joint Staff, are included;
* the MDA to amend its policy to ensure that all statutorily required
participants for military department CSBs are included in MDA's
Program Change Board, particularly the Joint Staff, if it is to serve
as an equivalent review;
* USD (AT&L) to amend its acquisition instruction to:
- ensure that all statutorily required participants, in particular the
comptroller, are included on CSBs;
- require CSB meetings for major defense acquisition programs in
production as well as development but also coordinate with the
military departments and the Congress to evaluate the effectiveness of
CSB meetings for programs well into production; and:
- develop the means to better track CSBs and ensure compliance with
the requirement that CSBs hold a meeting at least once each year;
* USD (AT&L) to work with DOD components to determine whether paper
CSBs are as effective as in-person meetings and, if not, amend the
acquisition instruction accordingly; and:
* DOD components to amend their policies to encourage alignment
between CSB meetings and other complementary reviews whenever possible.
Agency Comments and Our Evaluation:
DOD provided us with written comments on a draft of this report. In
its comments, DOD concurred or partially concurred with all seven of
our recommendations and agreed to take action to address six of them.
The comments are reprinted in appendix II. DOD also provided technical
comments, which we addressed in the report, as appropriate.
In concurring with our recommendation that the Navy amend its policy
on CSBs to include all statutorily required participants, DOD stated
that the Navy has already issued two policy memorandums that do so.
DOD also stated that the Navy will continue to issue policy guidance
consistent with our recommendation. This will be particularly
important as the Navy is currently in the process of revising its
primary acquisition instruction.
DOD also concurred with our recommendations to amend its acquisition
instruction to ensure that all statutorily required participants are
included in CSBs and that meetings occur for programs in development
as well as those in production. DOD did not address the portion of our
recommendation to coordinate with the military departments and the
Congress to evaluate the effectiveness of CSB meetings for programs
well into production. Given our mixed findings on the utility of CSB
meetings late in production, we continue to believe it would be in the
interest of the department to study this issue.
DOD partially concurred with our recommendation that MDA amend its
policy to ensure that all statutorily required participants for
military department CSBs, in particular the Joint Staff, are included
in MDA's Program Change Board, if it is to serve as an equivalent
review. In its comments, DOD stated that Joint Staff participation
would provide little value because of the role of the Joint Staff in
the acquisition of BMDS. In addition, DOD pointed out that the Joint
Staff participates in the Missile Defense Executive Board, a forum in
which strategic direction and funding priorities are established.
However, we continue to believe that if the Program Change Board is to
act as the forum for discussing configuration and requirements
changes, it is important that the user communities, as represented by
the Joint Staff, participate in these discussions.
DOD partially concurred with our recommendations on improving the
tracking of CSB meetings, determining the effectiveness of paper CSBs,
and aligning complimentary reviews with CSB meetings, when possible.
In its comments, DOD stated that it would address these issues in
"best practices" guidance to the military departments. With regard to
developing the means to better track CSB meetings and compliance with
the requirement to hold a meeting at least once each year, DOD stated
the best practices guidance will direct the military departments to
ensure adequate tracking vehicles are in place. We continue to believe
that USD (AT&L) should play a role in tracking compliance and holding
the military departments accountable, given our findings that the
military departments did not hold CSBs for all the required programs.
We are sending copies of this report to the Secretary of Defense; the
Secretaries of the Army, Navy, and Air Force; USD (AT&L); and the
Director of the Office of Management and Budget. In addition, the
report will be made available at no charge on the GAO Web site at
[hyperlink, http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact me at (202) 512-4841. Contact points for our offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. Staff members making key contributions to this
report are listed in appendix III.
Signed by:
Michael J. Sullivan:
Director, Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
Selection and Classification of Major Defense Acquisition Programs:
This report presents information on the Department of Defense's (DOD)
use of Configuration Steering Boards (CSB) for the major defense
acquisition program portfolio in 2010. We used the Defense Acquisition
Management Information Retrieval system to identify 98 active major
defense acquisition programs. We defined an active program as one that
issued a selected acquisition report in December 2009. This report
presents information on all of these programs. One program, the
Ballistic Missile Defense System, is managed by the Missile Defense
Agency (MDA), which reports acquisition information on the system by
functional elements. We reviewed nine elements and analyzed them
separately from the rest of the major programs.
We categorized programs by the five acquisition organizations
designated as having oversight--Army, Navy, Air Force, MDA, and the
Assistant Secretary of Defense for Nuclear, Chemical, and Biological
Defense programs--to assess trends in the use of CSBs. The selected
acquisition report for each program designates the program's
acquisition organization. As the lead authority for joint programs
rotates among the acquisition organizations as determined by the
Office of the Secretary of Defense, we categorized all joint programs
according to the service that was designated as the lead authority in
the December 2009 selected acquisition report.
All of the programs in our audit fall into one of two phases:
engineering manufacturing and development (referred to as development)
or production and sustainment (referred to as production). Development
generally begins with the initiation of an acquisition program as well
as the start of engineering and manufacturing development and
generally ends with entry into production. Production generally begins
with the decision to enter low-rate initial production. For most
programs in our assessment, the placement of programs in one of these
two phases was determined by the dates of their Milestone B/II and
Milestone C/III decisions. For instance, we categorized programs that
have held a Milestone B/II decision but not a Milestone C/III as in
the development phase and those that have held a Milestone C/III
decision as in the production phase. The dates of milestone decisions
for the programs used in the audit were determined through use of the
Defense Acquisition Management Information Retrieval system.
Due to the nature of individual programs, select programs were not
classified by milestone decision because they either have multiple
increments that may begin production in advance of the notional
Milestone C/III date,[Footnote 21] or the programs do not report
milestone dates.[Footnote 22] In these cases, we used the program's
selected acquisition reports to determine the appropriate phase. The
Navy often authorizes shipbuilding programs to begin production of the
lead ship at Milestone B/II. We classified these programs as in the
production phase.[Footnote 23] As the MDA programs develop systems'
capabilities incrementally instead of following the standard DOD
acquisition model, we did not identify acquisition phases for
Ballistic Missile Defense System elements.
Compliance with Statutory Requirements:
To assess the extent that DOD has complied with the statutory
requirements for CSB meetings in 2010, we compared CSB execution to
provisions in the statute that call for annual CSB meetings and
discussion of specific content. To determine the extent to which DOD
complied with the requirement to hold an annual CSB for each program,
we analyzed CSB records provided by the acquisition organization we
reviewed and, using these records, calculated the number of CSBs held
for each program in calendar-year 2010. To determine whether the
components established boards that included the statutorily required
participants, we analyzed policy and procedure documentation from each
of the components as well as attendance lists of CSBs held in calendar-
year 2010, provided by the acquisition organizations we reviewed. To
identify issues discussed at CSBs and actions resulting from these
CSBs, we reviewed CSB documents and questionnaire data and interviewed
acquisition officials. We also reviewed and analyzed current and draft
documentation related to department and service-level CSB policies,
directives, guidance, and instructions to determine if they establish
a structure that would facilitate compliance with the statute;
examples of these documents include Department of Defense Instruction
5000.02, Department of the Army Pamphlet 70-3 regarding Army
Acquisition Procedures, SECNAV Instruction 5000.2D, Air Force
Instruction 63-101, and Missile Defense Agency Directive 5010.18
regarding Acquisition Management.
We also interviewed officials representing organizations that
participate in CSBs or their equivalents including the Office of the
Under Secretary of Defense for Acquisition, Technology and Logistics,
Joint Staff, military service and MDA offices, program offices, and
capabilities and requirements offices to address department, military
service, and MDA policies and execution.
Effectiveness of Configuration Steering Boards:
To assess how effective CSBs have been controlling requirements and
mitigating cost and schedule risks on programs, we analyzed CSB
documentation to identify actions proposed and actions taken as a
result of the CSB and their effect on cost, schedule, performance, and
system configuration. We also asked program officials in our
questionnaire to identify requirement changes or descoping options
discussed at the CSB, the impact of decisions made, perceived
effectiveness of the CSB, and explanations for not conducting a CSB,
if applicable. To further analyze the effectiveness, challenges, and
benefits of holding CSBs, we selected 17 programs for interviews. We
based our selection on answers to our questionnaire, discussions with
officials, and programmatic factors such as acquisition organization
and phase. Specifically, we met with program officials at Wright
Patterson Air Force Base, Ohio; Redstone Arsenal, Alabama; Washington
Navy Yard in Washington DC; the Naval Air Station Patuxent River in
Patuxent River, Maryland; and conducted video teleconferences with
program officials at Picatinny Arsenal in New Jersey and at Los
Angeles Air Force Base in El Segundo, California. We also interviewed
acquisition officials, reviewed selected acquisition reports, and
examined documentation related to service-level CSB policies,
directives, guidance, and instructions to determine whether other
reviews or acquisition processes influenced the effectiveness of CSBs.
DOD Major Defense Acquisition Programs Questionnaire:
To collect information about DOD's use of CSBs in fiscal year 2010, we
developed and administered a Web-based questionnaire to the program
offices of all 98 programs. Fiscal-year data was collected in our
survey to be consistent with the Senate report language that contained
our mandate. We administered separate questionnaires to nine Ballistic
Missile Defense System elements and analyzed the results separately
from the rest of the programs in our review. We fielded the survey
from October 2010 to December 2010, and after extensive follow-up, we
received responses from all 98 programs.
Our questionnaire of the 98 program offices, was not a sample
questionnaire, so it has no sampling errors. However, the practical
difficulties of conducting any questionnaire may introduce errors,
commonly referred to as nonsampling errors. For example, difficulties
in interpreting a particular question or limitations in the sources of
information available to respondents can introduce unwanted
variability into the questionnaire results. We took steps in
developing the questionnaire, collecting the data, and analyzing the
responses to minimize such nonsampling errors. For example, social
science survey specialists designed the questionnaire in collaboration
with GAO's subject-matter experts. We conducted pretests with program
managers to check that (1) the questions were clear and unambiguous,
(2) terminology was used correctly, (3) the questionnaire did not
place an undue burden on agency officials, (4) the information could
feasibly be obtained, and (5) the questionnaire was comprehensive and
unbiased. For the pretests, we selected programs from each military
department and from various phases of the acquisition life cycle. We
conducted four pretests. We made changes to the content and format of
the questionnaire after each pretest, based on the feedback received.
When we analyzed the data, an independent analyst checked all computer
programs to reduce risk of error. Since this was a Web-based
questionnaire, respondents entered their answers directly into the
electronic questionnaire, eliminating the need to key data into a
database, minimizing error.
We did not validate the data provided by the program offices, but
reviewed the data and performed various checks to determine that the
data were reliable enough for our purposes. Where we discovered
discrepancies from reviewing responses and interviewing program
offices, we clarified the data with the program office and made
changes to the questionnaire data accordingly.
[End of section]
Appendix II: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
Acquisition, Technology And Logistics:
3000 Defense Pentagon:
Washington, DC 20301-3000:
June 24, 2011:
Mr. Michael J. Sullivan:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Sullivan:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GA0-11-640, "Defense Acquisitions: DoD Can Improve Management
of Programs Using Configuration Steering Boards," dated June 1, 2011
(GAO Code 120935).
The DoD concurs with three of the draft report's recommendations and
partially concurs with four. The rationale for our position is
enclosed. I submitted separately a list of technical and factual
errors for your consideration.
We appreciate the opportunity to comment on the draft report. My point
of contact for this effort is Mr. Ronald Woods, Ronald.Woods@osd.mil,
703-697-8183.
Sincerely,
Signed by:
David G. Ahern:
Deputy Assistant Secretary of Defense:
Portfolio Systems Acquisition:
Enclosure: As stated:
[End of letter]
GAO Draft Report Dated June 1, 2011:
GAO-11-640 (GAO Code 120935):
"Defense Acquisitions: DOD Can Improve Management Of Programs
Using Configuration Steering Boards"
Department Of Defense Comments To The GAO Recommendations:
Recommendation 1: The GAO recommends that the Secretary of Defense
direct the Navy to amend its policy on Configuration Steering Boards
(CSB) to ensure that all statutorily required participants,
particularly the Joint Staff and USD (AT&L), are included. (See
page 24/GAO Draft Report.)
DOD Response: Concur. The Navy has already issued two policy memos
that provide guidance that invitations to Configuration Steering
Boards must include the Joint Staff and OSD. The Department of the
Navy will continue to issue policy guidance ensuring all statutorily
required participants are included in the conduct of Configuration
Steering Boards.
Recommendation 2: The GAO recommends that the Secretary of Defense
direct the Missile Defense Agency (MDA) to amend its policy to ensure
that all statutorily required participants for military department
CSBs are included in MDA's program change board, particularly the
Joint Staff, if it is to serve as an equivalent review. (See page
24/GAO Draft
Report.)
DOD Response: Partially Concur. Joint Staff participation in Missile
Defense Agency (MDA) Program Change Boards would provide little value
because the Joint Staff is not a stakeholder in the detailed execution
of the MDA Ballistic Missile Defense System (BMDS) acquisition
process. Requirements for the MDA BMDS are not derived from the Joint
Capabilities Integration Development System, MDA BMDS acquisition
decision reviews do not include Joint Staff participation, and the
Joint Staff does not oversee the BMDS Accountability Report. Joint
Staff participation in the Missile Defense Executive Board, a forum
where strategic direction and funding priorities are established,
provides the desired guidance and oversight.
Recommendation 3: The GAO recommends that the Secretary of Defense
direct the USD (AT&L) to amend its acquisition instruction to ensure
that all statutorily required participants, in particular the
comptroller, are included on CSBs. (See page 24/GAO Draft Report.)
DOD Response: Concur. The Under Secretary of Defense for Acquisition,
Technology and Logistics will amend acquisition policy to require that
all statutorily required participants, including the comptroller of
the relevant military department, are included on Configuration
Steering Boards.
Recommendation 4: The GAO recommends that the Secretary of Defense
direct the USD (AT&L) to amend its acquisition instruction to require
CSB meetings for major defense acquisition programs in production as
well as development but work with the military departments and the
Congress to evaluate the effectiveness of CSB meetings for programs
well into production. (See page 24/GAO Draft Report.)
DOD Response: Concur. The Under Secretary of Defense for Acquisition,
Technology and Logistics will amend acquisition policy to ensure that
the Configuration Steering Board meet at least annually for
Acquisition Categories I and IA programs in either development or
production.
Recommendation 5: The GAO recommends that the Secretary of Defense
direct the USD (AT&L) to amend its acquisition instruction to develop
the means to better track CSBs and ensure compliance with the
requirement that CSBs hold a meeting at least once each year. (See
page 24/GAO Draft Report.)
DOD Response: Partially concur. The Department will issue
Configuration Steering Board (CSB) "best practices" to the Military
Departments. We will address the need for the Military Departments to
ensure adequate tracking vehicles are in place to ensure compliance
with CSB statute and policy.
Recommendation 6: The GAO recommends that the Secretary of Defense
direct the USD (AT&L) to work with DOD components to determine whether
paper CSBs are as effective as in person meetings and, if not, amend
the acquisition instruction accordingly. (See page 24/GAO Draft
Report.)
DOD Response: Partially concur. The Department will issue
Configuration Steering Board (CSB) "best practices" to the Military
Departments. We will address the appropriateness of paper CSBs in
those "best practices."
Recommendation 7: The GAO recommends that the Secretary of Defense
direct DoD components to amend their policies to encourage alignment
between CSB meetings and other complementary reviews whenever possible.
(See page 24/GAO Draft Report.)
DOD Response: Partially concur. The Department will issue
Configuration Steering Board (CSB) "best practices" to the Military
Departments. We will address the advantages of aligning CSBs with
complementary reviews.
[End of section]
Appendix III: GAO Contact and Acknowledgments:
GAO Contact:
Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov:
Acknowledgments:
In addition to the contact named above, Ronald E. Schwenn, Assistant
Director; Noah B. Bleicher; MacKenzie Cooper; Morgan Delaney Ramaker;
J. Kristopher Keener; Jean McSween; Kenneth E. Patton; and Brian
Schwartz made key contributions to this report.
[End of section]
Footnotes:
[1] Major defense acquisition programs are those identified by the
Undersecretary of Defense for Acquisition, Technology, and Logistics
that will eventually require a total expenditure for research
development, test, and evaluation of more than $365 million or
procurement funding, including all increments, of more than $2.19
billion (in fiscal-year-2000 constant dollars) or those designated by
the milestone decision authority as a major defense acquisition
program.
[2] About $65 billion of this growth can be attributed to quantity
changes.
[3] GAO, Best Practices: Better Support of Weapons Systems Program
Managers Needed to Improve Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-06-110] (Washington, D.C.: Nov. 30,
2005) and Defense Acquisitions: Assessments of Selected Weapons
Programs, [hyperlink, http://www.gao.gov/products/GAO-10-388SP]
(Washington, D.C.: Mar. 30, 2010).
[4] Pub L. No. 110-417, § 814 (2008).
[5] [hyperlink, http://www.gao.gov/products/GAO-10-388SP].
[6] S. Rep. No. 111-201, at 170 (2010).
[7] We excluded two of these programs, the Ballistic Missile Defense
System and Chemical Demilitarization-Assembled Chemical Weapons
Alternatives, from some of our analysis because the CSB provisions in
statute only apply to military department major defense acquisition
programs. These programs are managed by the Missile Defense Agency
(MDA) and the Assistant Secretary of Defense (ASD) for Nuclear,
Chemical, and Biological Defense programs (NCB) respectively.
[8] DOD is required to submit selected acquisition reports (SAR) to
Congress at the end of each fiscal-year quarter on current major
defense acquisition programs, though certain exemptions apply. SARs
for the first quarter of a fiscal year are known as comprehensive
annual SARs. Each comprehensive annual SAR is required to be submitted
within 60 days after the date on which the President transmits the
Budget to Congress for the following fiscal year. 10 U.S.C. §
2432(b)(1), (c)(4), (f).
[9] A requirement is an established need justifying the timely
allocation of resources to achieve a capability to accomplish approved
military objectives, missions, or tasks. These are often communicated
in requirements and other documentation as key performance parameters,
key systems attributes, or contract specifications. Configuration
refers to the functional and physical characteristics of a product.
[10] Pub L. No. 110-417, § 814 (2008).
[11] The statute does not require CSBs to be established for programs
outside the military departments, such as the Ballistic Missile
Defense System and Chemical Demilitarization-Assembled Chemical
Weapons Alternatives, which are managed by the Missile Defense Agency
(MDA) and the Assistant Secretary of Defense (ASD) for Nuclear,
Chemical, and Biological Defense programs (NCB) respectively. These
programs are covered by the CSB provision in DODI 5000.02.
[12] [hyperlink, http://www.gao.gov/products/GAO-10-388SP].
[13] S. Rep. No. 111-201, at 170 (2010).
[14] A breach of the critical cost growth threshold occurs when the
program's acquisition unit cost or the procurement unit cost increases
by at least 25 percent over the current baseline estimate or at least
50 percent over the original baseline estimate. 10 U.S.C. § 2433.
[15] Although the law requires a CSB to "meet ... at least once each
year," it is silent with regard to in-person or virtual "paper"
meetings. Pub. L. No. 110-417, § 814(c)(4) (2008).
[16] While the law states that the service acquisition executive
should chair the CSB itself, it does not address whether the chair for
a particular CSB meeting can be delegated.
[17] The Probability of Program Success model, as developed and
implemented within DOD, reviews the factors and metrics that
contribute to the success of a program with the goal of projecting a
program's future performance.
[18] The Navy has six gate reviews, which recur over time. While
program officials can present and discuss issues related to
requirements or configuration at any of the gate reviews, only the
gate 6 review is generally used to fulfill the requirement for an
annual CSB.
[19] The statute does not specify a point at which meetings are no
longer required, only that the CSBs for military departments must be
held for each major defense acquisition program at least once a year.
According to DOD, programs with 90 percent of items delivered are no
longer covered by the statute as changes to requirements or
configuration could no longer occur after a program reaches its
inventory objective; in addition, at this point, official reporting
through the SAR is no longer required.
[20] GAO, Best Practices: Capturing Design and Manufacturing Knowledge
Early Improves Acquisition Outcomes, [hyperlink,
http://www.gao.gov/products/GAO-02-701] (Washington, D.C.: July 15,
2002).
[21] Global Hawk (RQ-4A/B); Guided Multiple Launch Rocket
System/Guided Multiple Launch Rocket System Alternative Warhead;
Patriot/Medium Extended Air Defense System Combined Aggregate Program.
[22] Airborne Signals Intelligence Payload; Ballistic Missile Defense
System; Chemical Demilitarization–Chemical Materials Agency; Chemical
Demilitarization–Assembled Chemical Weapons Alternatives; CVN-68;
Family of Advanced Beyond Line-of-Sight Terminals; Family of Medium
Tactical Vehicles; Global Broadcast Service; Joint Tactical Radio
System, Network Enterprise Domain; Space-Based Space Surveillance
Block 10; T-AKE Lewis and Clark Class Dry Cargo/Ammunition Ship;
Wideband Global SATCOM.
[23] Cobra Judy Replacement; DDG 1000 Zumwalt Class Destroyer; CVN 78;
LHA 6 America Class Amphibious Assault Ship; Littoral Combat Ship; LPD
17; SSN 774 Virginia Class Submarine.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
