Defense Acquisitions
Status of the Safety, Performance, and Reliability of the Expeditionary Fire Support System Gao ID: GAO-09-189R November 18, 2008The Expeditionary Fire Support System (EFSS)--which consists of two kinds of motorized vehicles, a 120-mm mortar, an ammunition trailer, and fire direction equipment--is being developed to meet the United States Marine Corps' need for a weapon system that can be carried inside the MV-22 Osprey to support assault operations. The Marine Corps Operational Test and Evaluation Activity (MCOTEA), the independent test agency for the Marines, conducted initial operational testing and evaluation of the EFSS from May to July 2007, and reported in September 2007, among other things, that it experienced several safety, performance, reliability, and mechanical problems. We briefed Congress on these and other issues related to the EFSS in September 2007. Subsequently, at congressional request, the Marine Corps delayed full-rate production of the EFSS until after GAO reported on the system. In December 2007, we issued our report, which described the system's safety, performance, reliability, and mechanical problems. MCOTEA retested the system in February and March 2008, focusing on determining whether the problems identified in 2007 were resolved. It reported its analysis of the test results in May 2008. In October 2008, Congress asked us to provide Congress with a brief assessment of the Marine Corps' conclusions regarding whether the concerns we reported have been addressed. To do so, we reviewed MCOTEA's May 2008 Independent Evaluation Report from the EFSS' Follow-on Operational Test and Evaluation as well as documentation of the system's insensitive munition certification, and compared the results with the concerns we reported in 2007. We also reviewed documentation of EFSS' full-rate production decision. We interviewed EFSS program officials, a Marine Corps Combat Development Command official, and the MCOTEA official who oversaw EFSS testing to obtain their perspectives regarding whether and how the previously reported concerns were addressed. We conducted this performance audit from October 2008 through November 2008 in accordance with generally accepted government auditing standards.
Based on the May 2008 independent test report, most of the EFSS' safety, performance, reliability, and mechanical issues we reported in 2007 have been addressed through a combination of design changes and increased training. While some issues remain, MCOTEA did not judge them to be serious. The test report indicates that the EFSS' major safety issue we previously reported on--the system's inability to tow the ammunition trailer safely--has been addressed. The 2008 report also indicates that most of the performance issues we reported from the 2007 initial operational testing have been addressed. For example, follow-on testing showed that the system met all requirements associated with timed events, except the maximum rate of fire requirement, and resolved problems with the mortar's sight. Because the maximum rate of fire was frequently, but not consistently achieved, MCOTEA did not report it as a serious shortcoming and a Marine Corps requirements official stated that it was not a problem. EFSS vehicles are still not capable of securely carrying all required equipment, but Marine Corps officials attributed this problem to the space constraints imposed by the need to fit the system inside the V-22 Osprey, rather than to a design problem. The 2008 follow-on testing indicated that all of the reliability issues we reported in 2007 have been addressed, including issues related to the mortar's need for maintenance, transport barrel clamp weakness, and inconsistent vehicle configuration. The mechanical issues we reported on in 2007 appear to have been addressed, although not all of them could be directly tested during the 2008 follow-on test events. Although most of EFSS's earlier problems have been addressed, MCOTEA reiterated in its 2008 test report that the EFSS is a survivable platform provided it is used within its concept of employment and that employing the EFSS outside of the concept of employment would present a significant survivability liability to the operators given its limited protection.
GAO-09-189R, Defense Acquisitions: Status of the Safety, Performance, and Reliability of the Expeditionary Fire Support System
This is the accessible text file for GAO report number GAO-09-189R
entitled 'Defense Acquisitions: Status of the Safety, Performance, and
Reliability of the Expeditionary Fire Support System' which was
released on November 18, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
GAO-09-189R:
United States Government Accountability Office:
Washington, DC 20548:
November 18, 2008:
The Honorable Carl Levin:
Chairman:
Committee on Armed Services:
United States Senate:
Subject: Defense Acquisitions: Status of the Safety, Performance, and
Reliability of the Expeditionary Fire Support System:
The Expeditionary Fire Support System (EFSS)--which consists of two
kinds of motorized vehicles, a 120-mm mortar, an ammunition trailer,
and fire direction equipment--is being developed to meet the United
States Marine Corps' need for a weapon system that can be carried
inside the MV-22 Osprey to support assault operations. The Marine Corps
Operational Test and Evaluation Activity (MCOTEA), the independent test
agency for the Marines, conducted initial operational testing and
evaluation of the EFSS from May to July 2007, and reported in September
2007, among other things, that it experienced several safety,
performance, reliability, and mechanical problems. We briefed you on
these and other issues related to the EFSS in September 2007.
Subsequently, at your request, the Marine Corps delayed full-rate
production of the EFSS until after GAO reported on the system. In
December 2007, we issued our report,[Footnote 1] which described the
system's safety, performance, reliability, and mechanical problems.
MCOTEA retested the system in February and March 2008, focusing on
determining whether the problems identified in 2007 were resolved. It
reported its analysis of the test results in May 2008.
In October 2008, you asked us to provide you with a brief assessment of
the Marine Corps' conclusions regarding whether the concerns we
reported have been addressed. To do so, we reviewed MCOTEA's May 2008
Independent Evaluation Report from the EFSS' Follow-on Operational Test
and Evaluation as well as documentation of the system's insensitive
munition certification[Footnote 2], and compared the results with the
concerns we reported in 2007. We also reviewed documentation of EFSS'
full-rate production decision. We interviewed EFSS program officials, a
Marine Corps Combat Development Command official, and the MCOTEA
official who oversaw EFSS testing to obtain their perspectives
regarding whether and how the previously reported concerns were
addressed. We conducted this performance audit from October 2008
through November 2008 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
Results in Brief:
Based on the May 2008 independent test report, most of the EFSS'
safety, performance, reliability, and mechanical issues we reported in
2007 have been addressed through a combination of design changes and
increased training. While some issues remain, MCOTEA did not judge them
to be serious. The test report indicates that the EFSS' major safety
issue we previously reported on--the system's inability to tow the
ammunition trailer safely--has been addressed. The 2008 report also
indicates that most of the performance issues we reported from the 2007
initial operational testing have been addressed. For example, follow-on
testing showed that the system met all requirements associated with
timed events, except the maximum rate of fire requirement, and resolved
problems with the mortar's sight. Because the maximum rate of fire was
frequently, but not consistently achieved, MCOTEA did not report it as
a serious shortcoming and a Marine Corps requirements official stated
that it was not a problem. EFSS vehicles are still not capable of
securely carrying all required equipment, but Marine Corps officials
attributed this problem to the space constraints imposed by the need to
fit the system inside the V-22 Osprey, rather than to a design problem.
The 2008 follow-on testing indicated that all of the reliability issues
we reported in 2007 have been addressed, including issues related to
the mortar's need for maintenance, transport barrel clamp weakness, and
inconsistent vehicle configuration. The mechanical issues we reported
on in 2007 appear to have been addressed, although not all of them
could be directly tested during the 2008 follow-on test events.
Although most of EFSS's earlier problems have been addressed, MCOTEA
reiterated in its 2008 test report that the EFSS is a survivable
platform provided it is used within its concept of employment and that
employing the EFSS outside of the concept of employment would present a
significant survivability liability to the operators given its limited
protection.
Background:
The EFSS is designed to use its M327 120-mm rifled mortar to provide
mobile fire support for expeditionary forces. The EFSS includes two
kinds of prime mover vehicles and a trailer, in addition to the mortar.
One vehicle tows the mortar, while the other tows the trailer that
carries ammunition for the mortar. MCOTEA reported that the EFSS has a
capability that is not currently fielded in the operating forces. It
provides greater range and lethality than the mortars currently in use
by the Marine Corps and it can be internally transported via assault
support aircraft, in particular, the MV-22 Osprey.
MCOTEA conducted initial and follow-on operational testing and
evaluation for the EFSS to ensure that the system meets its mission
essential tasks as well as the criteria of Operational Effectiveness
and Operational Suitability by providing effective fire support and
proving its reliability, availability, and maintainability. The Initial
Operational Test and Evaluation, which we reported on in 2007, was
conducted in three phases: amphibious, desert, and high-altitude
operations. Follow-on testing was conducted in two phases: cold weather
at the Cold Regions Test Center, Fort Greely, Alaska, from February 17,
2008, through March 2, 2008, and desert terrain at the Marine Corps Air
Ground Combat Center, Twentynine Palms, California, from March 6-9,
2008. Data from both the initial testing and the follow-on testing were
used by the Marine Corps Operational Test and Evaluation Activity to
prepare the May 2008 Independent Evaluation Report for the Follow-on
Operational Test and Evaluation.
On June 2, 2008, the Marine Corps Systems Command approved the EFSS
program to move forward with full-rate production of the system. In
addition, the Command approved full fielding of the system once certain
conditions have been met.[Footnote 3] The program office reported that
the request for proposal for full-rate production for fiscal year 2009
has been issued, but they have not yet entered into a contract.
Most EFSS Safety, Performance, Reliability, and Mechanical Issues
Identified in 2007 Have Been Addressed:
In the 2008 follow-on testing, MCOTEA determined that the EFSS is
operationally effective and operationally suitable. The test report
indicates that most of the safety, performance, and reliability issues
identified in 2007 initial operational testing have been addressed.
Substantial design changes were made to the system in response to
issues identified in initial operational testing and the test unit
participants received substantially more training in the use of the
system than in previous tests. The Operational Test Project Officer
stated that the prime mover was "like a completely different vehicle"
and the director of the follow-on testing reported that the system
"proved it can operate in extreme conditions with close to normal
response times." Although substantial improvements have been made to
the EFSS, it remains critical that the system be employed within its
concept of employment in order to be survivable. Table 1 summarizes the
status of these issues based on the 2008 testing.
Table 1: Comparison of Selected Results from 2007 and 2008 EFSS
Testing:
Issue: Safety;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Ammunition
trailer risked injury to crewmember;
Status Based on 2008 Test Results: Addressed.
Issue: Performance;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Measures
associated with timed events were not met, including: rate of fire,
first round response, shift out of traverse, emplacement, and
displacement;
Status Based on 2008 Test Results: Addressed except maximum rate of
fire was not consistently met.
Issue: Performance;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Mortar
sight difficult to use;
Status Based on 2008 Test Results: Addressed[A].
Issue: Performance;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Shock
mount problems;
Status Based on 2008 Test Results: Minor problems persist.
Issue: Performance;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Vehicles
carry required equipment, but not securely;
Status Based on 2008 Test Results: Not addressed, but officials report
this is due to the vehicles' size requirements.
Issue: Reliability;
Factors Limiting EFSS Capability as Reported by GAO in 2007: The
mortar's mean rounds between operational mission failure did not meet
the threshold;
Status Based on 2008 Test Results: Addressed.
Issue: Reliability;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Transport
barrel clamps contributed to brake damage;
Status Based on 2008 Test Results: Addressed.
Issue: Reliability;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Vehicle
physical configuration not considered to be production representative;
Status Based on 2008 Test Results: Addressed.
Issue: Mechanical failures;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Problems
with Central Tire Inflation System;
Status Based on 2008 Test Results: Addressed.
Issue: Mechanical failures;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Cooling
system insufficient during operations;
Status Based on 2008 Test Results: Considered by MCOTEA to be
addressed[B].
Issue: Mechanical failures;
Factors Limiting EFSS Capability as Reported by GAO in 2007: Vehicles
had difficulty starting at high altitude;
Status Based on 2008 Test Results: Considered by MCOTEA to be
addressed[B].
Source: GAO analysis based on Marine Corps Operational Test and
Evaluation Activity's Expeditionary Fire Support System Follow-on Test
and Evaluation Independent Evaluation Report (May 2008) and GAO,
Defense Acquisitions: Status of the Expeditionary Fire Support System,
GAO-08-331R (Washington, D.C., Dec. 21, 2007).
[A] MCOTEA reported that the operators could use the mortar sight
without significant problem, but the mortar does require additional
training for the operators.
[B] Although the EFSS was not retested under these conditions in follow-
on testing, the Operational Test Project Officer stated that he was
satisfied with the changes because testing for a system with the same
cooling and starting components resulted in satisfactory operation in
high heat and at an elevation of over 8,000 feet.
[End of table]
Safety:
The 2008 follow-on test report indicates that the EFSS's major safety
issue identified in initial operational testing--the system's inability
to tow the ammunition trailer safely, which posed a risk of injury to
the crewmember riding in rear seat of vehicle--has been addressed.
Three incidents involving the trailer occurred during the 2007 test
events. Prior to the 2008 follow-on test design changes were made,
including lengthening the tongue on the ammunition trailer to increase
the trailer's stability, redesigning the ammunition trailer's brakes,
and lowering tire pressure. In addition, a cage and roll bar were
installed on the rear of the prime mover to protect the Marine riding
in the third seat and safe towing speeds were emphasized during
testing. The test report indicated that throughout the follow-on
testing, the Marines operated the prime mover and trailer in accordance
with training standards and no incidents occurred to indicate the
trailer would pose a hazard to the operators. In addition, in 2007 we
reported that the EFSS mortar had not yet completed insensitive
munition certification--a certification that a munition will not
detonate under any condition other than its intended mission. The
program office reported and provided documentation in October 2008,
that this certification is now complete.
Performance:
The report from follow-on testing indicates that most of the
performance issues identified in initial operational testing have been
addressed. In follow-on testing, the EFSS successfully and consistently
met requirements for almost all timed events that MCOTEA retested due
to performance deficiencies in initial operational testing. Marine
Corps officials attributed this improvement primarily to more thorough
training and the test unit's higher level of performance. During
initial operational testing, the EFSS did not meet the critical
requirement related to maximum rate of fire. It also did not meet some
non-critical requirements for timed events, which involved sustained
rate of fire, first-round response, shift out of traverse, and
emplacement and displacement. Although required times were achieved on
occasion for these events, the preponderance of observed times took
longer than required. These events were all retested in 2008, during
which the test unit successfully and consistently performed those timed
events except for the events related to rate of fire. MCOTEA assessed
the sustained rate of fire requirement as met.[Footnote 4] However, the
maximum rate of fire critical requirement was not met. Although the
test unit frequently achieved the maximum rate of fire of 4 rounds per
minute, the average for all trials was 3.5 rounds per minute. MCOTEA
did not report the rate of fire results as a serious shortcoming and a
Marine Corps requirements official stated that he did not see the
system's maximum rate of fire as a problem because the maximum rate of
fire was shown in testing to be achievable.
The 2008 test report also indicates that performance issues associated
with the mortar were largely addressed. Initial testing in 2007 found
that the mortar sight was difficult for operators to use and its
declinations were difficult to read in low light. The program office
reported that, in order to help address these difficulties, the sight's
eyepiece was lengthened and a better training package was developed to
help users learn how to use the sight more effectively. MCOTEA reported
in 2008 that operators could use the mortar sight without significant
problems. In addition, in initial operational testing, the shock mount
used to protect the sight from firing vibration caused issues with
boresight verification. The shock mount was redesigned and officials
stated that most of the problems associated with it were resolved.
However, the test report indicated that the shock mount still posed
some challenges with verification of the boresight. The EFSS program
manager stated that a modification has been made to the shock mount
latching mechanism to address the problems caused by the shock mount.
The remaining performance issue is the limited storage space offered by
the system, which may not be resolvable given the system's need to be
small enough to be transported inside a V-22 tilt-rotor aircraft. Both
the 2007 and 2008 test reports found that the EFSS vehicles were
capable of carrying all required equipment, but not securely. The
reports stated that section and personal equipment were loaded and
carried haphazardly due to insufficient storage space. Because space
inside the vehicle is limited, the Marines need to tie down their gear
on a "gypsy rack" or on other parts of the prime movers. A Marine Corps
requirements official stated that this limitation will be a challenge
in terms of packing equipment, but several officials, including the
Test Director stated that it is a function of the need to fit the
system inside the V-22.
Reliability:
Follow-on testing indicated that all of the reliability issues
identified in initial operational testing had been addressed. In
initial operational testing, the mortar's mean rounds between
operational mission failure did not meet the threshold. However, in
follow-on testing the achieved availability for the launcher was met,
meaning it did not experience a large amount of down time for
maintenance relative to the amount of operating time. In addition,
initial operational testing indicated that the EFSS' transport barrel
clamps were not stout enough to prevent its mortar tube from rotating,
which contributed to brake damage on one mortar during movement. This
issue has been addressed--the transport barrel clamp has been augmented
with a stop block and follow-on testing indicates that the augmentation
effectively prevents the mortar tube from rotating during travel.
Lastly, initial operational testing showed that each prime mover
vehicle's physical configuration was different--the vehicles were not
considered to be production representative. Spare and repair parts were
also nonstandard, and the test team could not interchange parts between
vehicles on several occasions. MCOTEA's EFSS Operational Test Project
Officer stated that in follow-on testing, the vehicles' physical
configuration was consistent and the system appeared to be production
representative. Although substandard spare and repair parts continued
to be an issue in follow-on testing, the program office stated that a
complete spare parts package will be produced and will undergo quality
inspection for delivery with the full-rate production systems.
Mechanical Issues:
The mechanical issues we reported on in 2007 appear to have been
addressed, although not all of them could be directly tested during the
2008 follow-on test events. Initial testing indicated that the
compressor for the air ride system and Central Tire Inflation System
was not robust enough to support all the air-powered systems on the
vehicle. However, for follow-on testing, the system's operators were
properly trained to use the systems, which operated satisfactorily.
Substantial engineering changes were made in response to other
mechanical issues raised in initial operational testing and the
Operational Test Project Officer indicated he was satisfied with those
changes. The 2007 testing indicated that EFSS' cooling system was not
able to cool the engine and transmission sufficiently during
operations. In addition, the same tests indicated that the vehicles did
not start reliably in the mornings during high-altitude operations. To
address these and other issues, the radiator and cooling system have
been redesigned and expanded. The program office described these
changes as significant. No cooling issues occurred during follow-on
testing although desert temperatures were not considered high enough to
challenge the system. The Operational Test Project Officer indicated
that he was satisfied with the changes because testing for another
Marine Corps system with the same cooling system resulted in
satisfactory operation in high heat and at an elevation of over 8,000
feet.
Other Considerations:
The Marine Corps' 2008 test report pointed out that the EFSS would
enhance the capabilities of the Marine Corps' artillery community
because it has a capability not currently fielded in the operating
forces, but stated that the platform is only survivable[Footnote 5]
when used within its concept of employment. MCOTEA reported that the
system provides greater range and lethality than the 60 and 81
millimeter mortars currently in use and the entire EFSS can be
internally transported via assault support aircraft, particularly the
MV-22 Osprey. The EFSS's 2008 follow-on test report, like the initial
test report, indicated that the EFSS is a survivable platform provided
it is used within its concept of employment. The EFSS concept of
employment indicates that the EFSS will be flown in via MV-22 Osprey or
helicopter. Once on the ground, most Marines would travel by foot, with
the EFSS accompanying them at comparable speeds and at an equivalent
distance. The system's concept of employment does not suggest
employment in areas with asymmetric threats such as the urban areas of
Iraq or Afghanistan. Both test reports stated that employing the EFSS
outside of the concept of employment would present a significant
survivability liability to the operators given its limited protection.
Agency Comments and Our Evaluation:
We provided a draft of this report to Department of Defense officials
for review and comment and they chose to provide only technical
comments, which were incorporated as appropriate.
We are sending copies of this report to the Secretary of Defense, the
Commandant of the Marine Corps, and interested congressional
committees. We will also make copies available to others upon request.
In addition, the report will be available at no charge on GAO's Web
site at [hyperlink, http://www.gao.gov]. If you have any questions
concerning this report, please contact me at (202) 512-4841. Key
contributors to this assignment are David Best, Laura Holliday, and
Greg Campbell.
Signed by:
Paul Francis, Director:
Acquisition and Sourcing Management:
[End of section]
Footnotes:
[1] GAO, Defense Acquisitions: Status of the Expeditionary Fire Support
System, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-331R]
(Washington, D.C., Dec. 21, 2007).
[2] Certification that a munition will not detonate under any condition
other than its intended mission.
[3] These conditions include: verification that safety testing has been
successfully completed and that manuals and training protocols address
all necessary operating cautions and safety procedures, validation of
the production process for ammunition manufacturing, and certification
that support equipment and special tools have been provided and
required technical manuals have been made available.
[4] Sustained rate of fire was not directly evaluated because the
entire basic load of ammunition was not fired--the assessment of the
sustained rate of fire was derived from the maximum rate of fire
evaluation.
[5] According to DOD, survivability is a concept that includes all
aspects of protecting personnel, weapons, and supplies while
simultaneously deceiving the enemy.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
