Defense Acquisitions
Analysis of Processes Used to Evaluate Active Protection Systems Gao ID: GAO-07-759 June 8, 2007Active Protection Systems (APS) protect vehicles from attack by detecting and intercepting missiles or munitions. In 2005, the lead systems integrator for the Army's Future Combat Systems (FCS) program sought proposals for an APS developer and design and to deliver APS prototypes on vehicles by fiscal year 2009. Raytheon was chosen the APS developer. At the same time, the Department of Defense's Office of Force Transformation (OFT) evaluated near-term APS for potential use in Iraq. GAO was asked to review the Army's actions on APS/FCS: (1) the process for selecting the subcontractor to develop an APS for FCS and if potential conflicts of interest were avoided; (2) the timing of the trade study and if it followed a consistent methodology to evaluate alternatives, and the results; (3) the role the Army and Boeing played in selecting the developer; and (4) the process followed to provide a near-term APS solution for current forces.
In selecting the APS developer, the Army and Boeing--the FCS lead systems integrator--followed the provisions of the FCS lead systems integrator contract, as well as the Federal Acquisition Regulation, in addressing organizational conflicts of interest. No officials from the offering companies participated in the evaluation and all offerors were evaluated based on the same criteria. Four proposals were evaluated and three were determined to be comparable in terms of cost and schedule. The winner--Raytheon--was chosen on technical merit, as being more likely to meet APS requirements although its design had less mature technology. The APS development contract required the source selection winner to perform a trade study to assess alternatives and select the best design for development, and the Raytheon design was chosen. The trade study applied a consistent methodology to all alternatives before selecting Raytheon's vertical launch design. While the role played by Raytheon in the trade study was in accordance with its contract, the rationale for having the trade study follow the source selection is not entirely clear. The purpose of the trade study was to select the best concept; yet the source selection process that preceded it had, in fact, chosen Raytheon primarily on the technical merits of its vertical launch design concept. Although the vertical launch technology is not mature, the Army estimated that it could be available for prototype delivery to current force vehicles in fiscal year 2009 and tested on a FCS vehicle in 2011. This may be an optimistic estimate, as the FCS vehicle is yet to be fully developed. The Army and Boeing were extensively involved in APS source selection and the trade study. FCS officials actively participated and concurred in the final selection of the APS developer. FCS officials and technical experts from Army research centers took part in the trade study and helped choose the vertical launch design. Boeing officials took part in various ways and, with the Army's concurrence, selected Raytheon as the APS developer, participated in the trade study, and recommended the vertical launch approach. In its pursuit of a different APS concept, OFT was responding to an urgent need statement issued by the Central Command with potential for near-term fielding. This evaluation centered on the results of physical testing of the most technically mature candidate system, the Trophy. Decisions on how to proceed with Trophy involved disagreement between OFT and the Army. While the Trophy tests were successful, the Joint Rapid Acquisition Cell decided to defer fielding the APS system, based in part on the recommendation of Army officials, who believed that testing had not been realistic and integrating it on the platform would delay fielding other useful capabilities. OFT officials proposed additional testing of Trophy to answer these questions, but funding for further OFT testing of this system was discontinued after the Joint Rapid Acquisition Cell's decision.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-759, Defense Acquisitions: Analysis of Processes Used to Evaluate Active Protection Systems
This is the accessible text file for GAO report number GAO-07-759
entitled 'Defense Acquisitions: Analysis of Processes Used to Evaluate
Active Protection Systems' which was released on June 11, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Air and Land Forces, Committee
on Armed Services, House of Representatives:
United States Government Accountability Office:
GAO:
June 2007:
Defense Acquisitions:
Analysis of Processes Used to Evaluate Active Protection Systems:
GAO-07-759:
GAO Highlights:
Highlights of GAO-07-759, a report to Subcommittee on Air and Land
Forces, Committee on Armed Services, House of Representatives
Why GAO Did This Study:
Active Protection Systems (APS) protect vehicles from attack by
detecting and intercepting missiles or munitions. In 2005, the lead
systems integrator for the Army‘s Future Combat Systems (FCS) program
sought proposals for an APS developer and design and to deliver APS
prototypes on vehicles by fiscal year 2009. Raytheon was chosen the APS
developer. At the same time, the Department of Defense‘s Office of
Force Transformation (OFT) evaluated near-term APS for potential use in
Iraq.
GAO was asked to review the Army‘s actions on APS/FCS: (1) the process
for selecting the subcontractor to develop an APS for FCS and if
potential conflicts of interest were avoided; (2) the timing of the
trade study and if it followed a consistent methodology to evaluate
alternatives, and the results; (3) the role the Army and Boeing played
in selecting the developer; and (4) the process followed to provide a
near-term APS solution for current forces.
What GAO Found:
In selecting the APS developer, the Army and Boeing--the FCS lead
systems integrator--followed the provisions of the FCS lead systems
integrator contract, as well as the Federal Acquisition Regulation, in
addressing organizational conflicts of interest. No officials from the
offering companies participated in the evaluation and all offerors were
evaluated based on the same criteria. Four proposals were evaluated and
three were determined to be comparable in terms of cost and schedule.
The winner”Raytheon”was chosen on technical merit, as being more likely
to meet APS requirements although its design had less mature
technology.
The APS development contract required the source selection winner to
perform a trade study to assess alternatives and select the best design
for development, and the Raytheon design was chosen. The trade study
applied a consistent methodology to all alternatives before selecting
Raytheon‘s vertical launch design. While the role played by Raytheon in
the trade study was in accordance with its contract, the rationale for
having the trade study follow the source selection is not entirely
clear. The purpose of the trade study was to select the best concept;
yet the source selection process that preceded it had, in fact, chosen
Raytheon primarily on the technical merits of its vertical launch
design concept. Although the vertical launch technology is not mature,
the Army estimated that it could be available for prototype delivery to
current force vehicles in fiscal year 2009 and tested on a FCS vehicle
in 2011. This may be an optimistic estimate, as the FCS vehicle is yet
to be fully developed.
The Army and Boeing were extensively involved in APS source selection
and the trade study. FCS officials actively participated and concurred
in the final selection of the APS developer. FCS officials and
technical experts from Army research centers took part in the trade
study and helped choose the vertical launch design. Boeing officials
took part in various ways and, with the Army‘s concurrence, selected
Raytheon as the APS developer, participated in the trade study, and
recommended the vertical launch approach.
In its pursuit of a different APS concept, OFT was responding to an
urgent need statement issued by the Central Command with potential for
near-term fielding. This evaluation centered on the results of physical
testing of the most technically mature candidate system, the Trophy.
Decisions on how to proceed with Trophy involved disagreement between
OFT and the Army. While the Trophy tests were successful, the Joint
Rapid Acquisition Cell decided to defer fielding the APS system, based
in part on the recommendation of Army officials, who believed that
testing had not been realistic and integrating it on the platform would
delay fielding other useful capabilities. OFT officials proposed
additional testing of Trophy to answer these questions, but funding for
further OFT testing of this system was discontinued after the Joint
Rapid Acquisition Cell‘s decision.
What GAO Recommends:
GAO recommends that the Secretary of Defense support additional testing
and demonstration of APS systems to help develop tactics, techniques,
procedures, and concepts of operations for active protection systems
and provide useful data on the use of APS. DOD did not agree to support
such testing.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-759].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Paul Francis at (202) 512-
4841 or francisp@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
APS Source Selection Avoided Organizational Conflicts of Interest:
Trade Study Used Consistent Method in Reaffirming Vertical Launch
Concept:
Army and Lead Systems Integrator Had Extensive Roles in APS
Subcontractor Selection and Trade Study:
OFT Process for Evaluating APS Was More Test-Based and Near-Term
Oriented:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Composition of APS Trade Study Technical Team:
Table 2: Composition of APS Trade Study Stakeholders:
Table 3: APS Concepts Considered in Trade Study:
Table 4: Top-Level Criteria and Associated Weights:
Table 5: Technology Maturity Levels of APS Alternatives as of May 2006:
Table 6: APS Candidate Systems and Developers:
Figures:
Figure 1: FCS APS Major Players:
Figure 2: Timelines for OFT and Army Processes for Evaluating Active
Protection Systems:
Abbreviations:
APS: Active Protection System:
DOD: Department of Defense:
FAR: Federal Acquisition Regulation:
FCS: Future Combat System:
JUON: Joint Urgent Operational Need:
RFP: Request for Proposal:
OFT: Office of Force Transformation:
OSD: Office of the Secretary of Defense:
SAIC: Science Applications International Corporation:
TRL: Technology Readiness Level:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
June 8, 2007:
The Honorable Neil Abercrombie:
Chairman:
The Honorable Jim Saxton:
Ranking Member:
Subcommittee on Air and Land Forces:
Committee on Armed Services:
House of Representatives:
An Active Protection System (APS) is intended to protect a vehicle from
attack by detecting and intercepting missiles or munitions before they
hit the vehicle. It has significant possibilities for offering light
vehicles some of the protection that armor gives, without the extra
weight. In April 2005, BAE, a subcontractor for the Army's Future
Combat System (FCS) program, issued a draft request for proposals to
choose the developer for an APS for FCS and current force manned ground
vehicles. To avoid an organizational conflict of interest, the lead
systems integrator, the Boeing Company, ultimately reissued the draft
request for proposals, after BAE decided to submit its own bid on this
contract. Raytheon was eventually chosen as the APS developer and a
subsequent trade study reached the conclusion that Raytheon's vertical
launch concept was the optimal solution to meet the APS requirements.
Also, in April 2005, the U.S. Central Command issued an urgent
operational need statement to field a combination of near-term
technologies, including a different APS, on a modified Stryker vehicle,
the Full-Spectrum Effects Platform, for demonstration and potential use
in Iraq. The effort made in response to this need was managed by the
Office of Force Transformation (OFT), which reported to the Secretary
of Defense. After evaluating six candidate APS, the OFT chose to test
the Trophy APS because they considered it to be the most technically
mature. In June 2006, the Joint Rapid Acquisition Cell decided to defer
fielding the APS system.
Both the Army's FCS program and the OFT effort were looking for APS
applications for current Army forces, but with somewhat different
objectives in mind. Although the primary focus of the APS subcontractor
selection and trade study was to pick a developer and a design for the
FCS APS, another objective was to field an APS on current force combat
vehicle platforms in the 2009 time period.[Footnote 1] On the other
hand, the OFT worked on a separate objective aimed at providing these
capabilities sooner to forces currently in theater, to provide a nearer-
term demonstration of APS.
This letter is to respond to your request for a review of the process
that led to the Army's decision to pursue a new APS system under the
FCS program. You asked us to review this process because of uncertainty
surrounding the facts that led to the Army's decision to pursue a
vertical launch APS concept. Specifically, this report addresses:
1. The process for selecting the subcontractor to develop an APS for
FCS and whether potential conflicts of interest were avoided.
2. The timing of the trade study, whether it followed a consistent
methodology to evaluate all the alternatives, and the results.
3. The role the Army and the lead systems integrator played in
selecting the APS developer.
4. The process followed for evaluating a near-term APS solution for
current forces.
In conducting our work, we held discussions with FCS, Army, and other
DOD officials. We reviewed documents relevant to the decision the Army
made about which APS system to pursue and held discussions with key
officials about various decisions that have been made in pursuing this
technology. We conducted our work between October 2006 and June 2007 in
accordance with generally accepted government auditing standards.
Appendix I further discusses our scope and methodology.
Results in Brief:
In choosing the developer for the APS system, the FCS lead systems
integrator ran a competition, with Army participation in the process
and concurrence in the selection. The contractor followed the
organizational conflict of interest clause developed in accordance with
the FCS contract and applicable to all subcontract tiers under the FCS
contract, as well as the applicable Federal Acquisition Regulation
(FAR) provisions. According to these terms, subcontractors cannot
conduct or participate in a subcontract source selection if any part of
its organization submits a proposal. Since BAE was the integrator of
the hit avoidance system--of which the APS is a subset--and was
submitting a proposal to be the APS developer, the lead systems
integrator, Boeing Company, issued the final request for proposals and
completed the source selection without BAE's involvement. The lead
systems integrator chose the APS developer--the Raytheon Company--after
all offerors were evaluated based on the same criteria contained in the
request for proposals. Proposals from three companies were evaluated in
the subcontractor selection process and all three were found to be
comparable in the areas of cost, management/schedule, and past
performance. The primary discriminating factor became technical merit.
Based on input from the source selection evaluation team and source
selection advisory council, the source selection executive decided that
the Raytheon vertical launch concept would be more likely to meet all
the APS requirements, even though it was based on less mature
technology than other proposals. In March 2006, the APS contract was
awarded to Raytheon.
The APS development contract required the winner of the subcontractor
selection to perform a trade study that would identify and assess APS
alternatives and select the best design for further development from
among the competing alternatives. Even though the role played by
Raytheon in the trade study was in accordance with its contract, the
rationale for having the trade study follow the source selection is not
entirely clear. While the trade study applied a consistent methodology
to all alternatives before selecting Raytheon's preferred design as the
best solution, this was a likely outcome given that the selection of
Raytheon as the APS developer was based largely on this design concept.
According to the study documentation, only one of the seven
alternatives evaluated was less technologically mature than the
vertical launch concept at the time the study was conducted. However,
in assessing it against the weighted criteria, vertical launch scored
the highest in all categories except risk. The study concluded that the
vertical launch concept would be a high-payoff approach, albeit at a
high risk. The Army estimated that, despite the immaturity of this APS,
it could be available for prototype delivery to current force vehicles
in fiscal year 2009 and tested on a FCS manned ground vehicle in 2011.
This may be an optimistic estimate as the FCS manned ground vehicles
are in the early stages of development.
Both the Army and the lead systems integrator were extensively involved
in the APS subcontractor selection and the trade study. The Army FCS
program officials were active participants in the source selection
evaluation team and source selection advisory council and concurred in
the final selection of the APS developer. Army FCS officials, as well
as technical experts from Army research centers, were members of the
trade study technical team and also concurred in the choice of the
vertical launch design. Lead systems integrator officials were members
of the source selection evaluation team and, with the Army's
concurrence, made the source selection of Raytheon as the APS
developer. In addition, the lead systems integrator participated in the
trade study and was one of the trade study members who recommended the
vertical launch approach.
The process followed by OFT to meet the urgent needs of the Central
Command included a simpler evaluation of active protection systems with
potential for near-term fielding and then centered on the results of
physical testing of the most mature of the candidate APS systems, the
Trophy. Decisions on how to proceed with the Trophy system involved
considerable disagreement between OFT and the Army. While the testing
of Trophy had a high success rate, the Joint Rapid Acquisition Cell
recommended that the Central Command defer fielding the Trophy, based
in part on the recommendations of the Army, which believed that the
testing was not realistic and the Trophy's integration on the platform
would delay fielding of other useful capabilities by at least 6 to 14
months. Further, the Army estimated that it would take 5 years to
integrate and field Trophy on other current force manned ground
vehicles. OFT officials recommended that additional testing of Trophy
be conducted to answer the questions raised by the Army, but the Joint
Rapid Acquisition Cell funding for further testing of the Trophy by OFT
ceased after the Joint Rapid Acquisition Cell decision.
Because of the likelihood that the Army will introduce APS into its
forces, we recommend that the Secretary of Defense support additional
testing and demonstration of near-term APS systems on the Full Spectrum
Effects Platform or similar vehicles to, at a minimum, help develop
tactics, techniques, procedures, and concepts of operations for both
near-term and long-term active protection systems.
In commenting on a draft of this report, DOD did not agree with our
recommendation that it support additional testing and demonstration of
near-term active protection systems on the Full Spectrum Effects
Platform that could respond to the Central Command's need. It stated
that there are no active protection systems mature enough at this time
to integrate on this vehicle, regardless of any additional testing and
demonstration efforts. DOD also stated that it continues to pursue
active protection, citing its efforts with the vertical launch system
for use on the FCS, among other things. However, we believe that
testing near term active protection systems on the Full Spectrum
Effects Platform or similar vehicles is valuable for answering
remaining questions about such systems and future systems. This is
particularly important given the likelihood that the Army will field
some form of APS to its forces. We have broadened our recommendation to
capture the value of continued testing of near-term APS for tactics,
techniques and procedures and concepts of operations.
Background:
The FCS concept is designed to be part of the Army's Future Force,
which is intended to transform the Army into a more rapidly deployable
and responsive force that differs substantially from the large division-
centric structure of the past. The FCS family of weapons is now
expected to include 14 manned and unmanned ground vehicles, air
vehicles, sensors, and munitions that will be linked by an advanced
information network. Fundamentally, the FCS concept is to replace mass
with superior information--allowing soldiers to see and hit the enemy
first rather than to rely on heavy armor to withstand a hit. The Army
envisions a new way of fighting that depends on networking the force,
which involves linking people, platforms, weapons, and sensors
seamlessly together in a system of systems.
Within the FCS program, eight types of manned ground vehicles are being
developed, each having a common engine, chassis, and other components.
One of the other common components is a hit avoidance system that
features a set of capabilities to detect, avoid, and/or defeat threats
against the manned ground vehicles. One of its subsystems is the APS,
which is intended to protect a vehicle from attack by detecting a
threat in the form of an incoming round or rocket propelled grenade
(threat) and launching an interceptor round from the vehicle to destroy
the incoming weapon. An APS system consists of a radar to detect the
incoming weapon, a launcher, an interceptor or missile, and a computing
system.
The Army has employed a management approach for FCS that centers on a
lead systems integrator to provide significant management services to
help the Army define and develop FCS and reach across traditional Army
mission areas. Boeing, along with its subcontractor, the Science
Applications International Corporation (SAIC), serves as the lead
systems integrator for the FCS system development and demonstration
phase of acquisition, which is expected to extend until 2014. The lead
systems integrator has a close partner-like relationship with the Army
and its responsibilities include requirements development, design, and
source selection of major system and subsystem subcontractors. In the
case of APS, the first-tier subcontractors are the manned ground
vehicle integrators, BAE and General Dynamics Land Systems, who are
responsible for developing individual systems. BAE was designated the
hit avoidance integrator, a role that covers more than active
protection, and was responsible for awarding the subcontract to the APS
developer. This subcontract has three elements: a base contract, option
A to support the current force (the short-range solution) and option B
to support the FCS manned ground vehicles (short-and long-range
solution). Figure 1illustrates these relationships.
Figure 1: FCS APS Major Players:
[See PDF for image]
Source: Army (data); GAO (presentation).
[End of figure]
A separate initiative involving active protection resulted from a Joint
Urgent Operational Needs Statement,[Footnote 2] issued by Central
Command and the Multi-National Corps in Iraq in April of 2005, which
requested 14 special-equipped vehicles with a host of distinctive
capabilities, one of which was an APS. The need statement called for a
capability to field a combination of near-term technologies that would
be useful in conducting force protection missions, reconnaissance and
crowd control in Iraq and an evaluation of an active protection
capability against rocket-propelled grenades as part of this suite of
capabilities. To respond to this need statement, the Joint Rapid
Acquisition Cell, a group within the Office of the Secretary of Defense
(OSD) that seeks solutions to urgent needs and focuses on near-term or
off-the-shelf equipment to meet these needs, provided funding to the
Army, which worked with the OFT to evaluate various technologies,
including an APS, for inclusion on the vehicles. The OFT was also an
office within the OSD, and its role was to examine unanticipated needs
and experiment with innovative technologies that could be used to meet
warfighter needs.
Both the process for evaluating APS sources and concepts to meet FCS
needs and the urgent needs of the Central Command occurred nearly
simultaneously, as shown in figure 3.
Figure 2: Timelines for OFT and Army Processes for Evaluating Active
Protection Systems:
[See PDF for image]
Source: Army and OFT (data); GAO (presentation and analysis).
[End of figure]
As can be seen in figure 3, many events took place at the same time.
The lead systems integrator for FCS completed its subcontractor
selection for APS shortly before decisions were made on the near term
system being considered to meet the Central Command need. The Trophy
system was evaluated as a candidate system in both processes.
APS Source Selection Avoided Organizational Conflicts of Interest:
In choosing the developer for the APS system, the FCS lead systems
integrator, with Army support and concurrence, conducted a source
selection and followed the FCS lead systems integrator subcontract
provisions for avoiding organizational conflicts of interest. The
purpose was to select the subcontractor for the APS that would be best
able to develop the overall APS architecture to address the FCS
requirements to defeat the short-and long-range antiarmor threats as
well as meet the current force needs for defeating short-range rocket-
propelled grenade attacks. The subcontractor selected would support the
hit avoidance integrator in integrating APS technology into the FCS
manned ground vehicles and also apply this architecture to the Army's
current force. The contract included two options that were to supply
the specific design for the APS system: Option A for the short-range
APS for the current force; and Option B for the short-and long-range
solution for the FCS. These options would be awarded later, based on
the results of trade studies subsequently performed.
To protect against organizational conflicts of interest, contracts
between the FCS lead systems integrator and its subcontractors preclude
a subcontractor from conducting or participating in a source selection
for other FCS subcontracts if any part of its organization submits a
proposal.[Footnote 3] Under normal circumstances, since the APS would
be part of the hit avoidance system of the FCS manned ground vehicles,
the hit avoidance integrator, BAE, would have had the primary
responsibility to issue the requests for proposals, conduct the source
selection evaluation, and award the contract. In this capacity, BAE
issued a draft request for proposals for the APS in April 2005. When
the firm subsequently decided to submit a proposal on the APS
subcontract, it was required, under the FCS lead systems integrator
subcontract organizational conflict of interest provisions, to notify
the lead systems integrator, Boeing, of its intention. BAE did so and
the lead systems integrator reissued the request for proposals for APS
in September 2005 and assumed the source selection responsibilities.
BAE submitted its proposal but then had no further role in the
evaluation of proposals or the actual source selection. After the
source selection was complete, the lead systems integrator transferred
contract responsibility to BAE, and BAE assumed the responsibility for
awarding and administering the APS contract.
From our review, the documentation from the APS source selection
process shows that (1) no officials from the offering companies
participated in the source selection process, and (2) all offerors were
evaluated based on the same criteria contained in the request for
proposals.[Footnote 4] In response to this request for proposals, four
proposals were received. Three proposals were considered competitive,
while the fourth was eliminated from consideration as it was considered
"unsatisfactory" in technical merit and its architectural approach did
not meet the requirements. Proposals from the remaining three
companies--BAE, Raytheon, and General Dynamics Land Systems--were
evaluated in the source selection process and no officials from these
companies were on the evaluating or selecting teams. The source
selection evaluation team consisted of 53 members, with 27 lead systems
integrator representatives and 26 government representatives, including
personnel from the FCS program manager's office, Army research centers,
and the Defense Contract Management Agency. After evaluating each of
the proposals against the criteria spelled out in the request for
proposals, the source selection evaluation team made its recommendation
to the lead systems integrator source selection executive, who accepted
its recommendation.
Our review of the documentation shows that the criteria were ranked in
order of importance, with technical merit considered most important,
then cost, management/schedule and finally past performance. The
technical merit criteria were divided into six sub-factors: systems
engineering and architecture; expertise in APS technologies;
simulation, modeling and test; fratricide and collateral damage;
specialty engineering; and integration capability. Cost criteria were
based on the realism, reasonableness, completeness, and affordability
of the proposal. Management/ schedule criteria included such areas as
expertise and experience in key positions. The past performance risk
rating category was based on whether the respondents' past performance
raised doubts about their being able to perform the contract.
Since all three proposals were deemed comparable in the areas of cost,
management/schedule, and past performance, the primary discriminating
factor became technical merit. According to the evaluation
documentation, the technical merit scores were assessed based on
whether the proposal demonstrated that the contractor understood the
requirements and on its approach to meeting these requirements in each
of the six technical merit sub-factors. Also, part of the technical
score was a proposal risk evaluation, defined as the degree any
proposal weaknesses could cause disruption of schedule, increase in
cost, or degradation in performance.
While the source selection's stated purpose was to choose the company
best able to develop the APS and not a specific design, each proposal
used a specific APS system as an "artifact" to illustrate how they
intended to meet the requirements. Even though, in theory, one company
could have been chosen as the APS developer while another company's
preferred design could have been selected for development, much of the
source selection assessment of technical merit was based on the
"artifact" used for illustration. For example, in the technical merit
category of APS expertise, the source selection evaluation of Raytheon
states that "the vertical launch concept solves several design and
integration problems." Similarly, the BAE evaluation in the criteria of
APS expertise states that "the proposed long-range
countermeasure—design has effectiveness against the full spectrum of
threats." The General Dynamics Land System's evaluation discusses the
relatively high technology readiness level (TRL)[Footnote 5] of the
"proposed Trophy system." Therefore, while each company's proposed
solution was not the only aspect of the proposals to be evaluated, the
evaluation documentation shows that the technical merit category was a
key factor in the evaluation.
The source selection evaluation team decided that the BAE and Raytheon
proposals had the highest technical merit. BAE had a lower-risk
approach and its solution had been tested in a relevant environment:
however, the source selection evaluation team stated that this low-risk
approach could prevent BAE from considering higher-risk options that
would enable them to meet the full range of the performance
requirements, such as protection from top-attack weapons. In addition,
the source selection evaluation team determined that, while both
Raytheon and BAE could develop the design presented in the BAE
proposal, Raytheon would have the advantage if the vertical launch
design was chosen. The evaluation team concluded that the Raytheon
approach would have the best chance of meeting all the requirements.
Based on the team's recommendation, the lead systems integrator
selected Raytheon. The integrator accepted the higher risk because it
concluded that the Raytheon proposal had excellent technical merit and
the firm would be better able to develop the vertical launch
technology, if that were the design decided upon in the trade study.
Trade Study Used Consistent Method in Reaffirming Vertical Launch
Concept:
The APS development contract required the winner of the source
selection to perform a trade study identifying and assessing competing
APS alternatives. The trade study used a methodology consistent with
Army guidance to evaluate all alternatives, ultimately selecting
Raytheon's vertical launch as the best design. According to the Army
and the lead systems integrator, conducting the trade study after
choosing the APS subcontractor could have resulted in selecting a
different concept than Raytheon's vertical launch design. However, in
our view, this possibility appears remote given the selection of
Raytheon as APS developer was based largely on the technical merits of
its vertical launch design and the fact that it would be best able to
develop that design.
The development contract's terms required the source selection winner
to perform a trade study that would identify and assess APS
alternatives and select an APS design from among competing
alternatives. Therefore, once Raytheon won the development contract in
March 2006, it was required to conduct the trade study rather than
simply develop its own design. Since the trade study was not a source
selection, FAR contract provisions regarding organizational conflicts
of interest did not apply and Raytheon was free to participate in the
study as the responsible contractor. The trade study's specific
objective was to choose a single short-range APS architecture (launcher
and interceptor) that best met active protection requirements for FCS
manned ground vehicles, with consideration for application to the
current force. The study was conducted in May 2006 and Raytheon's
vertical launch concept was selected as the design.
Based on the trade study documentation, the study was conducted using a
methodology prescribed by Army guidance and this methodology was
applied consistently to all APS alternatives. Seven alternatives
survived a screening process and were then evaluated against a set of
weighted criteria. The study concluded that Raytheon's vertical launch
was the best design approach.
According to general Army guidance for trade studies, steps in the
trade study process should include such elements as incorporating
stakeholders, identifying assumptions, determining criteria,
identifying alternatives, and conducting comparative analyses. The APS
trade study process consistently applied such methodology to all APS
alternatives by using separate, independent roles for a technical team
and stakeholders; operating under a set of assumptions; using
validated, protected technical data on each alternative; having a
screening process to filter out non-viable alternatives; and using a
set of weighted criteria to assess alternatives that survived the
screening process.
The trade study was performed by a technical team and stakeholders--
each having separate roles and operating independently from one
another. The technical team provided technical input and expertise to
the stakeholders, who were the voting members of the study and made the
final selection. The technical team, 21 members from industry and
government as shown in table 1, included individuals who were subject
matter experts as well as those from organizations participating in
development of the short-range APS. Raytheon had 11 members on the
technical team--the most from any single organization. The Army stated
that this representation included administrators and observers and
occurred because Raytheon had been designated APS developer, was thus
required to conduct the trade study, and could gain knowledge from
attending subject matter experts.
Table 1: Composition of APS Trade Study Technical Team:
Industry: Boeing/SAIC (1);
Government: U.S. Army Program Manager, FCS (1).
Industry: BAE (1);
Government: U.S. Army Armament Research, Development, and Engineering
Center (1).
Industry: General Dynamics Land Systems (3);
Government: U.S. Army Material Systems Analysis Activity (1).
Industry: Raytheon (11);
Government: Department of Energy - Idaho National Lab (2).
Source: Army (data), GAO (presentation and analysis).
[End of table]
The stakeholders made the final selection. The composition and number
of stakeholders are shown in table 2. The stakeholders were program
leads from the Army, lead systems integrator, and subcontractors
responsible for integrating the FCS manned ground vehicles. According
to the Army, Raytheon's APS program manager was included as a
stakeholder because Raytheon as developer had responsibility for
developing the design chosen by the trade study process.
Table 2: Composition of APS Trade Study Stakeholders:
Industry: Raytheon (1);
Government: U.S. Army Program Manager, FCS (1).
Industry: BAE (2);
Government: U.S. Army Training and Doctrine Command (1).
Industry: General Dynamics Land Systems (2);
Government: U.S. Army Program Executive Office for Ground Combat
Systems (1).
Industry: Boeing/SAIC (1);
Government: [Empty].
Source: Army (data), GAO (presentation and analysis).
[End of table]
The technical team and stakeholders operated the trade study under
assumptions that set parameters for screening and evaluating each
alternative. These assumptions were tied to such areas as performance
and threat. Additionally, they conducted the study using data that was
previously validated and remained protected throughout the study's
course. The primary source of the data was the Army Research,
Development, and Engineering Command's APS database, which contained
data gathered and validated by the Command's subordinate labs. This
data was protected by third parties, including the Department of
Energy's Idaho National Lab, to ensure it was not changed during the
study.
The technical team used initial screening processes to eliminate four
alternatives and identify seven viable alternatives for further
assessment. The screening process filtered out the four alternatives
that could not meet one or both of two criteria: (1) ability to grow to
meet 360-degree hemispherical requirements, and (2) ability to be
procured within a program schedule that would meet the need for
prototype delivery of a short-range solution to the current force in
fiscal year 2009. The seven alternatives that survived the screening
process are shown in table 3, along with the respective government
organizations and industry associated with each.
Table 3: APS Concepts Considered in Trade Study:
Government organization/industry: U.S. Army Aviation and Missile
Research, Development, and Engineering Center/Boeing;
Alternative: Close-In Active Protection System.
Government organization/industry: U.S. Army Aviation and Missile
Research, Development, and Engineering Center/Boeing;
Alternative: Close-In Active Protection System II.
Government organization/industry: U.S. Army Tank and Automotive
Research, Development, and Engineering/BAE;
Alternative: Close-In Countermeasure.
Government organization/industry: U.S. Army Tank and Automotive
Research, Development, and Engineering/Chang Industries;
Alternative: Full Spectrum Close-In Layered Shield.
Government organization/industry: U.S. Army Tank and Automotive
Research, Development, and Engineering/BAE and Northrup Grumman;
Alternative: Integrated Army Active Protection System.
Government organization/industry: Israel/Rafael and General Dynamics
Land Systems;
Alternative: Trophy.
Government organization/industry: U.S. Army Program Manager, FCS/
Raytheon;
Alternative: Vertical Launch.
Source: Army (data), GAO (presentation and analysis).
[End of table]
The technical team assessed the seven alternatives against a set of
five weighted criteria. According to the Army, these were the same top-
level criteria mandated in all FCS trade studies, and their weights
were assigned by FCS chief engineers. Table 4 defines each of the
criteria and provides information on respective weights.
Table 4: Top-Level Criteria and Associated Weights:
Criteria: Performance;
Description: Survivability, logistics, and reliability;
Weight: 35%.
Criteria: Cost;
Description: Average unit production cost for fitting 500 vehicles;
Weight: 25%.
Criteria: Burdens;
Description: Includes signature, weight, volume, power, and integration
complexity;
Weight: 20%.
Criteria: Risk;
Description: Technical, schedule, and cost risk;
Weight: 15%.
Criteria: Growth potential;
Description: System growth potential to counter full spectrum of
threats;
Weight: 5%.
Source: Army (data), GAO (presentation and analysis).
[End of table]
The vertical launch concept scored highest in every category of
criteria except risk. The Army indicated that the concept had about one-
third better overall weighted performance than the other alternatives.
Army officials described the vertical launch design as having technical
advantages over the other alternatives--including the need for less
space, weight, and power--as well as cost benefits.
The Army and lead systems integrator officials told us that the trade
study could have resulted in the selection of a design other than
Raytheon's. They also stated that, had this occurred, Raytheon as APS
developer would have been required to develop this design rather than
the vertical launch. While in theory the APS source selection chose a
developer and the trade study chose the design to develop, in reality
it is difficult to separate the trade study results and the source
selection decision. In our view, in both the source selection and trade
study, criteria related to technical aspects of the designs were
deciding factors. Considering that the source selection evaluation
relied on artifacts representing specific systems--and Raytheon won the
source selection based in large part on the technical merit of its
artifact--it seems unlikely that the APS trade study would have
resulted in the selection of any system other than Raytheon's vertical
launch.
Although the trade study concluded that vertical launch was a high-
payoff approach, it also noted that it was a high risk due to its low
technology maturity. At the time of the trade study, as shown in table
5, the vertical launch was less technologically mature than the other
alternatives except for one.
Table 5: Technology Maturity Levels of APS Alternatives as of May 2006:
Alternative: Close-In Active Protection System;
Technology maturity: 6.
Alternative: Close-In Active Protection System II;
Technology maturity: 3.
Alternative: Close-In Countermeasure;
Technology maturity: 6.
Alternative: Full Spectrum Close-In Layered Shield;
Technology maturity: 6.
Alternative: Integrated Army APS;
Technology maturity: 6.
Alternative: Trophy;
Technology maturity: 6.
Alternative: Vertical Launch;
Technology maturity: 5.
Source: Army (data), GAO (presentation and analysis).
[End of table]
The Army expects the design to reach TRL 6 (system model or prototype
demonstration in a relevant environment) by August or September 2007.
The Army expects the vertical launch concept to be available for
prototype delivery to current force combat vehicles in fiscal year 2009
and for testing on a FCS vehicle in 2011. These estimates appear
optimistic. At a TRL 5, the vertical launch will require additional
technology development and demonstration before it is ready for either
application. Also, the FCS vehicles have not been fully developed yet.
Assuming all goes as planned, most FCS vehicle prototypes are expected
to be available in 2011 for developmental testing. As we noted in our
March 2007 report,[Footnote 6] the Army has in general been accepting
significant risks with immature technologies for the FCS program,
coupled with compressed schedules for testing and evaluating
prototypes.
Army and Lead Systems Integrator Had Extensive Roles in APS
Subcontractor Selection and Trade Study:
The Army and the lead systems integrator were both extensively involved
in preparing for and conducting the APS subcontractor selection and the
trade study. Prior to the selection, FCS program officials assisted in
APS requirements development and reviewed and approved the scope of
work, schedule, and evaluation criteria for the request for proposals.
After the proposals were received, FCS program officials, technical
experts from various Army research centers, representatives of the Tank-
Automotive and Armaments Command and the Training and Doctrine Command
were active participants in the selection evaluation team and reviewed
the proposals along with the lead systems integrator members. The
Source Selection Advisory Council, who advise the Source Selection
Executive, provided oversight to the evaluation team and also had
representatives from the FCS program manager's office and the Army
research community. Similarly, Army FCS officials, as well as technical
experts from Army research centers, were members of the trade study
technical team and also concurred in the choice of the vertical launch
concept. The co-lead of the trade study was an FCS official. The lead
systems integrator's office assumed responsibility for the selection
process, was the selection executive, and made the final choice of an
APS developer. In addition to its lead role in the APS subcontractor
selection, the lead systems integrator was represented on the trade
study technical team and was one of the stakeholders.
As our previous body of work on the FCS program has shown, the Army's
participation in the APS subcontractor selection and trade study is
consistent with the Army's general approach to FCS. Army leadership set
up the FCS program in such a way that it would create more competition
and have more influence over the selection of suppliers below the lead
systems integrator. In setting up FCS, Army leadership noted that
traditionally, once the Army hired a prime contractor, that contractor
would bring its own supplier chains. The Army was not very involved in
the choice of the suppliers. In FCS, the Army called for the lead
systems integrator to hold a competition for the next tier of
contractors. The Army had veto power over these selections. In
addition, the Army directed that the lead systems integrator employ
integrators at lower levels in the program, for high-cost items such as
sensors and active protection systems and the Army has been involved
with these selections. These integrators were also to hold competitions
to select suppliers for those systems. This strategy was designed to
keep the first tier of contractors from bringing their own supplier
chains and pushed competition and Army visibility down lower in the
supplier chain. The fact that the decisions on the APS subcontractor
selection and trade study lend themselves to after-the-fact examination
is due in part to the Army's focus on competition at lower supplier
levels on FCS.
OFT Process for Evaluating APS Was More Test-Based and Near-Term
Oriented:
The process followed by OFT to meet the urgent needs of the Central
Command was characterized by a simpler evaluation of active protection
systems with potential for near term fielding, followed by actual
physical testing of the APS candidate system that the OFT considered
most technically mature, the Trophy. The Army's Program Manager's
Office for Close Combat Systems was also involved in this evaluation.
While the testing of Trophy had a high success rate, the Joint Rapid
Acquisition Cell decided to defer fielding the Trophy based, at least
in part, on the recommendations of the Army that the testing was not
realistic and the Trophy's integration on the platform would delay
fielding of other useful capabilities. OFT officials did not agree with
the Army's position and thought the system's success in testing
indicated it should be further evaluated.
To meet the Central Command's need, OFT began an effort, the Full-
Spectrum Effects Platform, to incorporate and test various improvements
for potential application to existing military vehicles such as the
Stryker. The platform itself is a modified Stryker vehicle.[Footnote 7]
The program was divided into spirals: spiral 0 was to evaluate the
synergy of the different systems, including the APS, on the vehicle and
to compile lessons learned to aid in future concepts of operations,
development and integration. Spiral 1 was intended to field a limited
number of such systems to current forces in-theater in 2007, for
purposes of an operational assessment of the various capabilities. The
Full Spectrum Effects Platform is not part of or associated with FCS.
OFT, in association with the Naval Surface Warfare Center, evaluated
six candidate APS systems. Army representatives from the Program
Manager, Close Combat Systems were also involved in this evaluation.
The six candidate systems evaluated are shown in table 6.
Table 6: APS Candidate Systems and Developers:
Candidate: Close-in Active Protection System;
Developer: Aviation and Missile Research, Development and Engineering
Center.
Candidate: Close-in Countermeasure;
Developer: British Aerospace (formerly United Defense).
Candidate: Canister-Launched Area Denial System;
Developer: Northrop Grumman.
Candidate: Tactical Rocket-Propelled Grenade Airbag Protection System;
Developer: Innovative Survivability Technologies.
Candidate: Full-Spectrum Close-in Layered Shield;
Developer: Chang Industries.
Candidate: Trophy;
Developer: Rafael/General Dynamics Land Systems.
Source: OFT.
[End of table]
These systems were evaluated because the OFT and Navy and Army
officials considered them to be the most promising APS solutions
available within the required schedule. They evaluated each system
based on such criteria as the feasibility of the operational concept,
its cost and schedule factors, as well as its weight, size, and power
requirements. Trophy was selected as the most promising system because
it was the most technically mature system and was being developed by
Israeli defense forces that had done initial work to integrate it on a
light armored vehicle.
OFT subsequently sponsored tests of the Trophy APS as part of the Full-
Spectrum Effects Platform at Naval Surface Warfare Center in Dahlgren,
Virginia. A representative from the Army's Program Manager, Close
Combat Systems, was part of the oversight team for these tests. In
these test firings, the Trophy APS did well, destroying 35 of 38
incoming rocket-propelled grenades. However, the process for deciding
how to proceed based on the test results was not agreed to in advance.
A disagreement subsequently arose between OFT and the Army Close Combat
System officials on how best to proceed from the testing.
Although the tests were not designed to represent the Trophy's
capabilities in a realistic operational environment, OFT officials
concluded that Trophy showed enough promise that they recommended
continued testing to demonstrate its capabilities under various
conditions. These officials estimated that an additional $13 million
would cover the cost for this testing. They believed that Trophy could
be integrated in the near term on existing light-armored vehicles and
meet the urgent need for an immediate APS capability.
The Army officials disagreed with OFT's assessment that further testing
of Trophy for inclusion on the Full Spectrum Effects Platform was
justified. According to the Army officials, Trophy was not tested in a
realistic environment for collateral damage or effectiveness. They
believed that it would not be sufficiently tested for operational and
safety issues within the time period required for the first spiral of
the Full Spectrum Effects Platform. A delay in its integration on the
Platform would delay, by at least 6 to 14 months, demonstration of
other potentially useful capabilities,that could be immediately
incorporated. Further, the Army estimated that it would take 5 years to
integrate and field Trophy on other current force manned ground
vehicles. The Army recommended to the Joint Rapid Acquisition Cell that
the Trophy APS be excluded from Spiral 1 of the Full-Spectrum Effects
Platform. In lieu of putting this technology in the field, the Army
recommended that slat armor[Footnote 8] be incorporated on Spiral 1,
since it has been effective in defeating the current rocket-propelled
grenade threat. OFT officials disagreed, reasoning that although the
use of slat armor on the current force has seemed to mitigate the
effects of the rocket-propelled grenades currently in use, improved
munitions will soon be available, and the slat armor will no longer be
effective against these threats. They believed that the Trophy should
be tested further in order to answer the questions raised by the Army
and to provide insight into its capabilities. OFT officials based their
position on the Trophy's success in these tests, its high level of
technical maturity when compared to other active protection systems,
and the criticality of the need.
The Joint Rapid Acquisition Cell presented this information to Central
Command and recommended slipping the active protection capability to a
later platform spiral, once it was more mature. Currently, there are no
plans for further evaluation of active protection for future platform
spirals. Upon the removal of the Trophy APS system from the Full-
Spectrum Effects Platform vehicle, the Joint Rapid Acquisition Cell
discontinued funding for further testing and evaluation of the Trophy.
The disagreement between Army and OFT officials notwithstanding, we did
not find information that would challenge the decision to defer the
introduction of the Trophy on light-armored vehicles. On the other
hand, the 5 years the Army estimated would be needed to integrate the
comparatively mature Trophy system on the existing Stryker vehicle does
not appear consistent with its estimates that the less mature vertical
launch system could be ready for prototype delivery on Strykers in 2
years and on the yet-to-be developed FCS prototypes in 3 years.
Conclusions:
The FCS lead systems integrator, with support from the Army, followed a
consistent and disciplined process in both selecting Raytheon to
develop the APS for FCS and in conducting the trade study and followed
the lead systems integrator subcontract and FAR provisions for avoiding
organizational conflicts of interest. While the role played by Raytheon
in the trade study was in accordance with its contract and thus not
improper, the rationale for having the trade study follow the source
selection is not entirely clear. The purpose of the trade study was to
select the best concept; yet, the source selection process that
preceded it had, in fact, chosen Raytheon primarily on the technical
merits of its vertical launch design concept. It was thus improbable
that the trade study would reach a different conclusion. Both the Army
and the lead systems integrator were closely involved throughout the
source selection and trade study processes and concurred in the
selection of Raytheon's APS concept.
The process for evaluating the Trophy system to meet the urgent needs
of the Central Command was different. It centered more directly on the
results of physical testing, followed a less-disciplined decision-
making process, and was characterized by considerable disagreement
between OFT and the Army. While the decision to defer the use of the
Trophy on fielded vehicles appears prudent in light of the limited
realism of the testing, the promising results of the testing likewise
appeared to warrant additional testing of the Trophy system to either
confirm or dispel potential risks in the use of APS capabilities.
Discontinuing all testing of the Trophy systems may thus have been
premature, particularly in light of the need to better understand
tactics, techniques and procedures and concepts of operations for both
near-term and long-term applications.
Recommendations for Executive Action:
Because of the likelihood that the Army will introduce APS into its
forces, we recommend that the Secretary of Defense support additional
testing and demonstration of near-term APS systems on the Full Spectrum
Effects Platform or similar vehicles to, at a minimum, help develop
tactics, techniques, procedures, and concepts of operations for both
near-term and long-term active protection systems.
Agency Comments and Our Evaluation:
DOD provided us with written comments on a draft of this report. The
comments are reprinted in appendix II. DOD did not concur with our
recommendation. DOD also provided technical comments, which we
incorporated where appropriate.
DOD did not concur with our recommendation that the Secretary of
Defense support additional testing and demonstration of near-term
active protection systems on the Full Spectrum Effects Platform that
could respond to the Central Command's need. It stated that the
original decision in May 2006 that delayed delivering Full Spectrum
Effects Platform capabilities due to technical development and
performance risks remains true today. DOD added that there are no
active protection systems mature enough at this time to integrate on a
Full Spectrum Effects Platform regardless of any additional testing and
demonstration efforts. This represents a much more decided opinion than
was rendered at the time of the OFT tests. At that time, Army officials
believed that the Trophy would not be sufficiently tested for
operational and safety issues in time for the first spiral of the Full
Spectrum Effects Platform. OFT officials believed that the Trophy
should be tested further to answer the questions raised by the Army and
to provide insight into its capabilities. Ultimately, the Joint Rapid
Acquisition Cell recommended slipping the active protection capability
to a later spiral of the Full Spectrum Effects Platform. This was the
basis for our recommendation for additional testing of near-term active
protection systems on the Full Spectrum Effects Platform.
DOD stated that it continues to pursue active protection, citing the
Army's vertical launch system for FCS. As stated in our report, this
system is technically immature and the Army's estimates for testing it
appear optimistic. According to the Institute of Defense Analysis, the
vertical launch system is ambitious, with much enabling technology not
yet demonstrated. Given the criticality of active protection for the
FCS manned ground vehicles, additional testing of near-term active
protection systems could provide valuable insights into operations and
tactics that would benefit future applications, such as FCS. DOD noted
that the Trophy system is being tested on the Wolf Pack Platoon
Project, an OSD Rapid Reaction Technology Office (formerly OFT) effort.
However, this project is not directed toward development of APS
tactics, techniques, procedures, or concepts of operations. In
addition, it will not include testing against live targets.
Testing near-term active protection systems on the Full Spectrum
Effects Platform or similar vehicles is valuable for answering
remaining questions about such systems and to provide insights for the
employment of future systems. This is particularly important given the
likelihood that the Army will field some form of APS to its forces. We
have broadened our recommendation to capture the value of continued
testing of near-term APS for tactics, techniques and procedures and
concepts of operations.
Please contact me on (202) 512-4841 if you or your staff has any
questions concerning this report. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report.
Signed by;
Paul L. Francis:
Director:
Acquisition and Sourcing Management:
[End of section]
Appendix I: Scope and Methodology:
To develop the information on the U.S. Army's decision to pursue a new
APS system under the FCS program, we interviewed officials of the
Office of the Assistant Secretary of the Army (Acquisition, Logistics
and Technology); the Tank-Automotive and Armaments Command; the Joint
Rapid Acquisition Cell; the Office of Force Transformation; the Naval
Surface Warfare Center (Dahlgren Division); the Program Manager for the
Future Combat System (Brigade Combat Team); and the Future Combat
System Lead Systems Integrator.
We reviewed the APS subcontractor selection documentation, including
the APS request for proposal, current force and FCS operational
requirements documents, subcontract proposals, criteria used to rate
those proposals, and the APS development contract to determine if
procedures for avoiding organizational conflicts of interest were
followed and how the APS subcontractor was selected. In addition, we
held discussions with key Army officials and lead systems integrator
representatives regarding this process and their roles in it. To
determine why the trade study was conducted after source selection, we
reviewed the trade study process and results and Army guidelines for
conducting trade studies. To identify the roles played by both the Army
and lead systems integrator in the selection of an APS, we reviewed
documentation concerning their roles in these processes. We also
reviewed these materials to determine whether consideration was given
to a separate APS solution for current forces and, in conjunction with
this issue, we reviewed test reports and other documentation and
discussed the testing of an alternative APS system, the Trophy, with
the parties involved.
In evaluating the APS subcontractor selection and trade study
processes, we did not attempt to determine if the best technical
solution was chosen, but only if these processes followed lead systems
integrator provisions for organizational conflicts of interest and used
a consistent methodology for the trade study. We conducted our work
between October 2006 and June 2007 in accordance with generally
accepted government auditing standards.
[End of section]
Appendix II: Comments from the Department of Defense:
Office Of The Under Secretary Of Defense:
3000 Defense Pentagon:
Washington, DC 20301-3000:
Acquisition, Technology And Logistics:
Jun 4 2007:
Paul L. Francis:
Director, Acquisition and Sourcing Management:
U.S. Government Accountability Office:
Washington, D.C. 20548.
Dear Mr. Francis:
This is the Department of Defense (DoD) response to the GAO Draft
Report, GAO-07-759, "Defense Acquisitions: Analysis of Processes Used
to Evaluate Active Protection Systems," dated April 30, 2007 (GAO Code
120624).
The report recommends that the Secretary of Defense reconsider
supporting additional testing and demonstration of near term Active
Protection Systems' systems on the Full Spectrum Effects Platform
(FSEP) that could respond to the Central Command's need.
The Department non-concurs with the GAO recommendation. The original
decision in May of 2006 that delayed delivering FSEP capabilities due
to technical development and performance risks remains true today. Our
comments on the recommendation are enclosed.
The Department appreciates the opportunity to comment on the draft
report. Technical comments were provided separately. For further
questions concerning this report, contact Mr. Robert Maline at (703)
693-9414.
Sincerely,
Signed by:
David G. Ahem:
Director:
Portfolio Systems Acquisition:
Enclosure:
As stated:
GAO Draft Report Dated April 30, 2007 GAO-07-759 (GAO Code 120624):
"Defense Acquisitions: Analysis Of Processes Used To Evaluate Active
Protection Systems"
Department Of Defense Comments To The GAO Recommendation:
Recommendation: We recommends that the Secretary of Defense reconsider
supporting additional testing and demonstration of near term APS
systems on the Full Spectrum Effects Platform that could respond to the
Central Command's need. (p. 21/GAO Draft Report):
DoD Response: Nonconcur. There are no Active Protection Systems (APS)
mature enough at this time to integrate onto a Full Spectrum Effects
Platform (FSEP) regardless of any additional testing and demonstration
efforts. The recommendations of this report are not supported by the
report findings. The Director, Joint Rapid Acquisition Cell, consulted
with technical experts and U.S. Central Command before deciding that
deferring Trophy (the Israeli Active Protection System) on the FSEP was
a reasonable approach.
The Department continues to pursue Active Protection. In addition to
the work on Active Protection System development spearheaded by Future
Combat Systems, the Rapid Reaction Technology Office (formerly the
Office of Force Transformation) is also testing the Trophy APS as part
of the Wolf Pack Platoon Project. Additionally, the Defense Advanced
Research and Programs Agency is developing potential systems and the
Foreign Comparative Test Office is monitoring active protection
systems.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
Paul Francis, (202) 512-4841:
Acknowledgments:
Other contributors to this report were Assistant Director William R.
Graveline, Marie P. Ahearn, Beverly Breen, Tana Davis, Letisha Jenkins,
Kenneth E. Patton, and Robert Swierczek.
FOOTNOTES
[1] The Army defines its "Current Force" as including the Stryker,
Bradley, and Abrams combat vehicles. Of the three, the Stryker is the
only system for which the Army has definite plans to incorporate an
APS.
[2] A joint urgent operational need is a need that can be considered
life or combat-mission-threatening based on unforeseen military
requirements that must be resolved in days, weeks, or months. Chairman
of the Joint Chiefs of Staff Instructions 3470.01, Rapid Validation and
Resourcing of Joint Urgent Operational Needs (JUONS) in the Year of
Execution, July 15, 2005.
[3] The FCS lead systems integrator contract required that an
organizational conflict of interest provision be included in the
subcontracts under the FCS lead systems integrator contract and flowed
down to all lower level subcontract tiers. The lead systems integrator
developed a specific organizational conflicts of interest clause
applicable to all subcontract tiers under this contract. Additionally,
the lead systems integrator contract provided that pursuant to the FAR
9.5, subcontractors could neither prepare RFP documents nor conduct or
participate in a source selection if any part of its organization
submits a proposal.
[4] We reviewed the source selection documentation for reporting
purposes and did not conduct a legal review of the evaluation or
selection decision.
[5] Technology readiness levels characterize the readiness of
technologies for hand-off to project implementers. Nine levels are
defined, representing concepts from fundamental research level (TRL 1)
through technologies fully qualified and demonstrated (TRL 9).
[6] GAO, Defense Acquisitions: Key Decisions to Be Made on Future
Combat System, GAO-07-376 (Washington D.C.: Mar. 15, 2007).
[7] The OFT had initiated a program called Project Sheriff in 2004,
prior to the Central Command's need statement, to address urgent
warfighter needs for lethal and non-lethal weapons, and force
protection. After the needs statement, this evolved into the Full-
Spectrum Effects Platform effort.
[8] Slat armor forms a metal frame barrier on the front of the vehicle
that detonates the rocket-propelled grenade away from the vehicle,
preventing it from boring through the vehicle and causing damage and
injury.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
