Military Readiness
DOD Needs to Better Manage Automatic Test Equipment Modernization Gao ID: GAO-03-451 March 31, 2003The services have billions of dollars worth of outdated and obsolete automatic test equipment (ATE) used to test components on military aircraft or weapon systems. Department of Defense (DOD) policy advocates the development and acquisition of test equipment that can be used on multiple types of weapon systems and aircraft and used interchangeably between the services. At the request of the Subcommittee's Chairman, GAO examined the problems that the Air Force, Navy, and Marine Corps are facing with this aging equipment and their efforts to comply with DOD policy.
DOD and the services face growing concerns regarding obsolete automatic test equipment, given the high costs of modernizing or replacing it and its potential effect on aircraft readiness. The Navy and Air Force, for example, estimate that they will spend billions of dollars to modernize or replace this equipment, much of which was acquired in the 1970s and 1980s. In the meantime, the aging testers are becoming increasingly out of date and more difficult to support. When the testers do not work properly, maintenance can suffer and readiness can be adversely affected. Since 1994, DOD policy has advocated the acquisition of test equipment that can be used on multiple weapon systems and aircraft and can be used interchangeably between the services; progress in this regard has been slow. For example, although the Navy set out in 1991 to replace 25 major tester types with one standard tester by 2000, budget cuts and delays in developing software have resulted in delays in completing the replacement of these obsolete testers until 2008. The Air Force has only recently initiated a test equipment modernization plan. However, little evidence suggests that consideration is being given to the acquisition of equipment that would have common utility for more than one weapon system as DOD policy advocates. For procurement of new weapon systems, the Air Force is giving little consideration to the use of a common tester, while a common tester is planned for use as the primary tester for the Joint Strike Fighter. Although DOD tasked the Navy as its Executive Agent for automatic test equipment in 1994, the agent has made only limited progress in achieving compliance across all the services with DOD policy advocating the development of common systems. While the Executive Agent can point to some successes in individual systems, its officials acknowledged that the organization does not have sufficient authority or resources to fully implement the policy and achieve the maximum commonality possible.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-451, Military Readiness: DOD Needs to Better Manage Automatic Test Equipment Modernization
This is the accessible text file for GAO report number GAO-03-451
entitled 'Military Readiness: DOD Needs to Better Manage Automatic Test
Equipment Modernization' which was released on April 11, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products‘ accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
Report to the Chairman, Subcommittee on National Security, Emerging
Threats and International Relations, Committee on Government Reform,
House of Representatives:
United States General Accounting Office:
GAO:
March 2003:
Military Readiness:
DOD Needs to Better Manage Automatic Test Equipment Modernization:
DOD Automated Test Equipment Modernization:
GAO-03-451:
GAO Highlights:
Highlights of GAO-03-451, a report to the Chairman, Subcommittee on
National Security, Emerging Threats and International Relations, House
Committee on Government Reform
Why GAO Did This Study:
The services have billions of dollars worth of outdated and obsolete
automatic test equipment (ATE) used to test components on military
aircraft or weapon systems. Department of Defense (DOD) policy
advocates the development and acquisition of test equipment that can be
used on multiple types of weapon systems and aircraft and used
interchangeably between the services.
At the request of the Subcommittee‘s Chairman, GAO examined the
problems that the Air Force, Navy, and Marine Corps are facing with
this aging equipment and their efforts to comply with DOD policy.
What GAO Found:
DOD and the services face growing concerns regarding obsolete automatic
test equipment, given the high costs of modernizing or replacing it and
its potential effect on aircraft readiness. The Navy and Air Force, for
example, estimate that they will spend billions of dollars to modernize
or replace this equipment, much of which was acquired in the 1970s and
1980s. In the meantime, the aging testers are becoming increasingly out
of date and more difficult to support. When the testers do not work
properly, maintenance can suffer and readiness can be adversely
affected.
Since 1994, DOD policy has advocated the acquisition of test equipment
that can be used on multiple weapon systems and aircraft and can be
used interchangeably between the services; progress in this regard has
been slow. For example, although the Navy set out in 1991 to replace 25
major tester types with one standard tester by 2000, budget cuts and
delays in developing software have resulted in delays in completing the
replacement of these obsolete testers until 2008. The Air Force has
only recently initiated a test equipment modernization plan. However,
little evidence suggests that consideration is being given to the
acquisition of equipment that would have common utility for more than
one weapon system as DOD policy advocates. For procurement of new
weapon systems, the Air Force is giving little consideration to the use
of a common tester, while a common tester is planned for use as the
primary tester for the Joint Strike Fighter.
Although DOD tasked the Navy as its Executive Agent for automatic test
equipment in 1994, the agent has made only limited progress in
achieving compliance across all the services with DOD policy advocating
the development of common systems. While the Executive Agent can point
to some successes in individual systems, its officials acknowledged
that the organization does not have sufficient authority or resources
to fully implement the policy and achieve the maximum commonality
possible.
What GAO Recommends:
GAO recommends that the Secretary of Defense reemphasize the policy and
reconsider the organizational placement and authority of the Executive
Agent for ATE.
DOD concurred with GAO‘s recommendations and agreed that its Executive
Agent for ATE should be given the authority and resources to more
effectively fulfill the Department‘s oversight responsibilities.
www.gao.gov/cgi-bin/getrpt?GAO-03-451.
To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Neal Curtin at (757) 552-8100 or
curtinn@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Aging ATE Presents Major Challenges to DOD:
DOD Has Had Limited Success in Fostering Commonality:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Defense:
Related GAO Products:
Figures:
Figure 1: Schematic of an Automatic Test System, Including ATE and Test
Program Set Components:
Figure 2: CASS Station:
This is a work of the U.S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO‘s product.
Abbreviations:
ATE: automatic test equipment:
CASS: Consolidated Automated Support System:
DOD: Department of Defense:
GAO: General Accounting Office:
JSF: Joint Strike Fighter:
United States General Accounting Office:
Washington, DC 20548:
March 31, 2003:
The Honorable Christopher Shays
Chairman, Subcommittee on National Security,
Emerging Threats and International Relations
Committee on Government Reform
House of Representatives:
Dear Mr. Chairman:
One of the major challenges facing the military services is that of
modernizing billions of dollars‘ worth of aging and increasingly
obsolete automatic test equipment[Footnote 1] (ATE) used to
troubleshoot and diagnose components of aircraft or weapon systems.
Since 1994, Department of Defense (DOD) policy has advocated a
reduction in the proliferation of testers that are unique to one type
of weapon system or aircraft, favoring the development or acquisition
of testers that (1) are interoperable[Footnote 2] within a service and
between the services and (2) can be used on many different components
of multiple types of aircraft and weapon systems. As the services
modernize ATE, they are challenged to adhere to DOD policy and reduce
the number of unique testers.
ATE with its test program sets, including test software, an interface
device that connects the ATE to the item being tested, and
documentation make up an automatic test system. (See fig. 1.):
Figure 1: Schematic of an Automatic Test System, Including ATE and Test
Program Set Components:
[See PDF for image]
[End of figure]
ATE is used by the services at all maintenance levels (from flight
lines for routine maintenance to depots for major overhauls and
factories for production and acceptance testing) to test electronic
systems and components that are difficult or impossible to test
manually, to isolate system malfunctions, and to verify that systems
are operating properly. These testers can be made to examine a single
aircraft system, various components of an aircraft, or multiple
components of different aircraft.
Because of your concerns regarding DOD‘s continued reporting of spare
parts shortages and the potential impact that ATE obsolescence could
have on the readiness of military aircraft, you asked us to determine
whether DOD and the services are giving adequate attention to ATE
modernization efforts. Specifically, our objectives were to identify
(1) what problems the Air Force and Navy[Footnote 3] are facing with
their ATE and (2) how successful DOD, the Air Force, and the Navy have
been in addressing the proliferation of unique testers.
Our review included ATE for aircraft managed by the Air Force and the
Navy and included information on ATE acquisition for two fighter
aircraft currently under development: the multiservice Joint Strike
Fighter (JSF) and the Air Force‘s F/A-22. Our scope and methodology are
described in more detail in appendix I. We performed our review from
January 2002 through March 2003 in accordance with generally accepted
government auditing standards.
Results in Brief:
DOD and the services face growing concerns regarding obsolete ATE,
given the high costs of modernizing or replacing this type of equipment
and its potential adverse impact on aircraft readiness. ATE acquired in
the 1970s and 1980s is becoming increasingly out-of-date and more
difficult to support. These obsolescence issues are further aggravated
by new technologies that, in some cases, make ATE obsolete even before
the new testers can be fully fielded. Also, older testers are kept much
longer than initially planned because the weapon systems they support
are being kept longer. Repair parts for older ATE are becoming
increasingly scarce, as more contractors discontinue their support.
Although exact cost figures are not available, the services estimate
that they will need several billion dollars in the coming years either
to acquire new testers or modernize existing ones. Although the
services do not maintain data that allow them to measure the extent to
which obsolete ATE affects readiness, according to DOD readiness
reports, only 28 percent of Air Force, Navy, and Marine Corps key
aircraft models met their readiness goals in 2002. Although a
combination of factors affects readiness goals, the availability of
spare parts is a key contributor to readiness, and the performance of
ATE significantly affects the supply of spare parts.
For years, DOD‘s policy has aimed to minimize the acquisition of ATE
that is unique to a particular weapon system; however, the
implementation of this policy has been slow. In 1994, DOD appointed the
Navy as its Executive Agent to oversee policy implementation; however,
according to Executive Agent officials, the Navy has not had the
authority or resources to effectively carry out this oversight. The
services lack ATE modernization plans, and there is no DOD-wide
approach to ensure that all ATE acquisitions and modernizations are
identified in an early enough stage to ensure that commonality[Footnote
4] and interoperability are adequately considered. Without sufficient
information concerning the magnitude of the services‘ modernization
efforts or a departmentwide approach to accomplish ATE modernization,
the department faces a very expensive and time-consuming ATE
modernization effort, with the continued proliferation of unique
testers and no assurance that resources are allocated in the most
effective manner. As a result, some ATE modernization and acquisition
planning is being done with little consideration to commonality. For
example, Executive Agent officials said that they have not had contact
with the Air Force‘s F/A-22 project office concerning ATE since 1994,
and it does not appear that commonality is being considered or that
unique ATE development will be minimized.
We are making several recommendations aimed at reinforcing DOD‘s stated
goal of achieving more commonality of test equipment and strengthening
the department‘s oversight. DOD concurred with our recommendations and
agreed to reemphasize its policy that common automatic test equipment
be developed to the maximum extent possible. In addition, DOD agreed
that its Executive Agent for ATE should be given the authority and
resources to more effectively fulfill its oversight responsibilities.
DOD‘s comments on our report are reprinted in their entirety in
appendix II.
Background:
It is estimated that DOD employs more than 400 different tester types.
This equipment is used to diagnose problems in aircraft avionics and
weapon system components so that the component can be repaired and
replaced on the aircraft or put into the supply system for future use.
For example, testers may be used to diagnose problems with aircraft
radars, guidance and control systems, or weapon systems. According to
DOD, the department spent over $50 billion in its acquisition and
support of ATE from 1980 through 1992, and the procurement was
characterized by the proliferation of testers designed to support a
specific weapon system or component. These testers are quickly becoming
obsolete and more difficult and costly to maintain because they may no
longer be in production and parts may not be readily available. Over
the years, various studies have criticized the continued proliferation
of unique ATE and highlighted the need for the development and
acquisition of testers that can be used to test more than one system or
component.
In September 1993, the House Appropriations Committee recommended that
the Secretary of Defense develop a DOD-wide policy requiring ATE
commonality among the services, along with a formal implementation
mechanism with sufficient authority, staffing, and funding to ensure
compliance.[Footnote 5] In 1994, DOD established a policy stating that
managers of DOD programs should select families of testers or
commercial off-the-shelf components to meet all ATE acquisition needs
and that the introduction of unique testers should be minimized. DOD
designated the Navy at that time as its Executive Agent to oversee
policy implementation in all services, and identified a goal of
reducing life-cycle costs and providing greater ATE commonality and
interoperability. Additional DOD guidance published in 1996 and 1997
required that all ATE acquisitions be part of the approved families of
testers or commercial off-the-shelf.
Aging ATE Presents Major Challenges to DOD:
DOD faces major challenges with aging and increasingly obsolete ATE.
These problems include the high costs of maintaining and replacing
ATE and the declining availability of spare parts for the aging
testers. In addition, several DOD organizations, including the Navy
Inspector General, have suggested that aging and obsolete ATE may
adversely affect aviation readiness.
Modernization Costs Are Substantial:
Departmentwide estimates of funds needed for ATE modernization and
acquisition are not readily available. However, according to Air Force
and Navy ATE managers, most of the services‘ ATE is obsolete and will
need to be upgraded or replaced over the next several years. Our study
confirmed that replacement and modernization costs would be
substantial. The Navy, for example, spent about $1.5 billion from
fiscal years 1990 through 2002 for the acquisition of its primary
family of testers and plans to spend an additional $430 million through
fiscal year 2007. Additionally, the Navy estimates that it plans to
spend $584 million through fiscal year 2007 to adapt existing test
program sets necessary to perform specific tests of the various
aircraft components supported by this family of testers. The Navy also
anticipates spending an additional $584 million to develop program test
sets for new weapon system requirements.
Information on the Air Force‘s spending for ATE modernization is
somewhat sketchy, as limited data are available centrally for
individual weapon systems. According to a recent study done for the Air
Force, the service has not developed a plan that allows modernization
funding requirements to be determined. However, estimates are available
for selected systems. The F-15 fighter program office, for example, is
spending approximately $325 million on just one tester that will be
fielded in 2004. It also plans to upgrade its electronic warfare
tester, which is one of seven primary testers for the aircraft, at a
cost of over $40 million. A 2002 study of B-52 bomber ATE identified
obsolescence issues associated with six of the aircraft‘s seven major
testers that will require more than $140 million in the near future.
Similarly, the upgrade of a unique B-1 bomber tester is expected to
exceed $15 million, even though the Air Force is considering replacing
this tester and has already begun planning the acquisition. The latest
estimate for the new tester is $190 million. Current ATE estimates for
the F/A-22, which is still under development, are not available.
However, estimates made early in the development phase exceeded
$1.5 billion.
Readiness Could Be Adversely Affected:
ATE is becoming increasingly out-of-date and more difficult to support.
And, according to service officials, using this outdated equipment to
perform required tests in a timely manner is becoming increasingly
challenging. Although the services could not quantify the extent that
tester problems affect readiness, service officials noted that without
adequate test equipment to diagnose problems, components cannot be
repaired in a timely manner and the mission capability of military
aircraft can be adversely affected. In August 2000, the Navy Inspector
General identified shortfalls in ATE as having a negative impact on
naval aviation and, in particular, on the availability of repaired
components. During the same time frame, a Navy operational advisory
group, recognizing the importance of ATE in maintaining readiness,
ranked support equipment, including ATE, as one of its top 20 readiness
issues.
We have issued several reports in the recent past addressing the
shortage of spare parts--a potential result of ATE problems. In
addition, according to DOD readiness reports, only 28 percent of Air
Force, Navy, and Marine Corps key aircraft models met their readiness
goals in fiscal year 2002. Although difficulties in meeting these goals
are caused by a complex combination of interrelated logistical and
operational factors, the shortage of spare parts was a major cause. ATE
plays a significant role in the supply of available spares, since this
equipment affects both how many parts are taken out of service for
repair and how quickly they are repaired and returned. We reported that
maintenance and repair facilities routinely work around spare parts
shortages by removing a working part from one aircraft to replace a
nonworking part in another aircraft, a practice called
’cannibalization.“[Footnote 6] And, although the services do not record
increases in cannibalizations that are caused by ATE problems, the
services use cannibalization as a routine maintenance practice when
testers are not available or not working properly.
In July 2001, we reported that as a result of ATE not working properly,
unfilled requisitions were adversely affecting the mission capability
of F-14 aircraft.[Footnote 7] In another case, more than 1,200 Air
Force B-1 bomber components were backlogged and could not be repaired
because of the same reason. Although we were unable to measure specific
reductions in the readiness of F-14 and B-1 aircraft as a result of
this problem, mission capable rates for the B-1 in fiscal years 1998-
2002 averaged approximately 55 percent, compared with the goal of
67 percent, while mission capable rates for the F-14D, during the same
period, averaged 67 percent, compared with a goal of 71 percent.
Additionally, the Air Force‘s 2002 B-52 study concluded that six of the
seven major testers used to test B-52 components need to be modified or
replaced or the availability of the aircraft will be adversely affected
as early as 2006. Air Force officials believe that similar problems
will continue unless the service undertakes a major ATE modernization
or replacement program.
DOD Has Had Limited Success in Fostering Commonality:
Since the early 1990s, DOD policies have addressed the need for
commonality in ATE acquisition and modernization. Although the services
have been making some progress, efforts to comply with these policies
have been slow. For example, although the Navy has developed a single
family of testers to work on many of its aircraft components, after
11 years, the replacement of its obsolete testers aboard aircraft
carriers and shore maintenance facilities has not been completed. In
addition, strategic planning for the modernization of automatic test
equipment at Navy depots has only recently been initiated.
Historically, the Air Force has not had a service-level ATE
standardization policy and has essentially pursued unique ATE solutions
for each weapon system. Since individual aircraft program offices have
been doing their own planning for modernization, the Air Force has
given little consideration to having common ATE or testers that are
interoperable with those of other services. Planning for the Air
Force‘s latest aircraft acquisition, the F/A-22, calls for the
development of automatic test equipment that will be unique to that
aircraft. In August 2002, the Air Force initiated a planning effort to
determine its long-term servicewide ATE modernization needs.
The Navy Has Been Slow in Fielding Its Common Tester:
According to Navy reports, obsolete ATE results in higher backlogs and
increased flying hour costs, and adversely affects aircraft readiness.
The Navy recognized years ago, and prior to the establishment of DOD‘s
1994 ATE standardization policy, that its ATE was becoming obsolete.
In the 1980s the Navy embarked upon an ATE standardization program to
replace 25 of its testers with one standard ATE family, the
Consolidated Automated Support System (CASS), to minimize unique types
of testers. The Navy designed CASS to be used at maintenance activities
both ashore and afloat. In 1991, the Navy began to produce CASS for the
general purpose testing of equipment such as radios, radars, and
electro-optics. (See fig. 2.):
Figure 2: CASS Station:
[See PDF for image]
[End of figure]
CASS‘s replacement of 25 types of obsolete testers, in support of
2,458 weapon system components, was scheduled for completion by
fiscal year 2000. However, according to Navy officials, because of
budget cuts that caused delays in developing the test program sets,
only 4 of the 25 have been completely replaced by CASS, and 8 test sets
have been partially replaced. Navy officials told us that the
completion schedule has slipped to fiscal year 2008 for aircraft
carriers and shore maintenance facilities and could be much longer for
aviation depots.
The Navy reports that the replacement of these testers with CASS
stations, when complete, will reduce the number of test-related
enlisted occupational specialties from 32 to 4, thus reducing training
requirements. In addition, CASS will reduce the requirement for test
equipment operators aboard each aircraft carrier from 105 to 54, and at
the same time reduce space requirements for testers from 2,700 to 1,900
square feet. Spare parts needed to repair testers will be reduced from
30,000 to 3,800. According to Navy officials, however, the revised
completion schedule will not allow for the timely replacement of aging
ATE, and these delays will adversely affect aircraft readiness.
In addition to schedule slippage, the original CASS equipment was
fielded about 10 years ago, uses 15-year-old technology and, according
to Navy ATE program managers, is in need of an upgrade. Accordingly,
by 2006, the first production units will have reached the point where
wear and obsolete components will drive supporting costs to
unacceptable levels and create a need for replacement and
modernization. The Navy has begun modernization planning for CASS,
including upgrades through fiscal year 2014.
Integrating CASS into Navy depots may further delay ATE commonality
within the service. For example, a 2001 Navy report, addressing total
ATE ownership costs, noted that the depots have not maximized the use
of CASS because of the limited availability of capital investment
funds. In addition, at one depot we found some reluctance to use CASS.
This depot had four CASS stations that had never been used--two were
delivered in 1999 and installed in December 2000 and February 2001,
while two others delivered in 2000 were still in crates. Depot
officials said that they had elected not to put the equipment on-line,
as they wanted to avoid paying for overhead and maintenance, especially
without the workload to justify their use. They also noted that the
development of the test program sets needed to use the CASS has been
slow, thereby slowing the fielding of the equipment. The Navy has only
recently begun a servicewide planning effort to modernize its depot-
level testers and determine how best to integrate CASS into its depot
maintenance strategy.
Air Force‘s Approach Has Resulted in Limited Commonality:
Unlike the Navy, the Air Force has not made commonality a priority but
has pursued unique ATE solutions for each weapon system. In addition,
it has only recently initiated efforts to collect information on ATE in
its inventory, including the equipment‘s condition and its need for
modernization or replacement. Because the Air Force has not made
concerted efforts to use one system to service multiple aircraft
platforms, it has not taken advantage of efficiencies and potential
savings such as those expected by the Navy as a result of CASS.
Although the Air Force is developing plans to modernize its ATE, and
although its policy is to consider developing common testers, it does
not yet have an overall plan to guide its modernization efforts and has
made limited progress in this area. Furthermore, it does not have a
process in place to ensure that commonality is given adequate
consideration in its ATE acquisition and modernization.
The Air Force has been primarily upgrading--rather than replacing--
aging ATE; leaving ATE management up to individual program managers. In
most cases, it relies on contractors to provide support for ATE,
leaving it vulnerable to contractors who may decide to stop supporting
testers when maintaining them is no longer profitable.
In early 2001, the Air Force organized the Warner Robins Air Logistics
Center Automatic Test System Division to work with program offices
on ATE issues. The Division has recently initiated efforts to establish
a database of all contractors that are capable of supporting existing
ATE to help identify emerging supportability issues. Although the
office is responsible for fostering the adoption and use of common
families of testers, it has no final decision-making authority
regarding ATE modernizations and no control over funding decisions on
these matters. Division officials told us that they work with
individual project offices to encourage them to use common ATE, but
individual project offices make the final decisions.
In our opinion, leaving these ATE decisions to the individual Air Force
project offices has led to some questionable and unnecessary
expenditures. For example:
* The Air Force will spend approximately $325 million to replace a
tester for the F-15 with one that has been under development for almost
10 years and is already obsolete. The new tester, called the Electronic
System Test Set, is not expected to be fielded until 2004. However,
this electronic tester already needs an upgrade that will cost more
than $24 million. Because the new tester will not be able to perform
all the required tests, the Air Force will have to keep the old tester
too.
* The Air Force is spending over $15 million for an interim
modernization of its intermediate automatic test equipment for its B-1
aircraft while, at the same time, a new tester is being developed. If
the Air Force had taken the necessary steps to replace this obsolete
tester in a timely manner, these duplicative costs could likely have
been avoided, and overall ATE modernization costs reduced. According to
an Air Force official, the program office should have begun the
acquisition of a replacement tester several years ago, but funding was
not available. The service is now considering acquiring a replacement
tester estimated to cost $190 million.
The Air Force‘s Warner Robins Air Logistics Center Automatic Test
System Division is developing a strategic plan that is expected to
serve as a management plan for meeting long-term ATE needs. The
Division plans to develop a baseline of its current tester
capabilities, address supportability and sustainability issues, and
determine whether tester failures adversely affect the availability of
aircraft weapon systems. In addition, it will evaluate replacement and
modernization alternatives, taking into account life-cycle costs and
the potential for developing common testers. The plan‘s implementation
is expected to take years to complete.
Services‘ Approaches in Developing Testers for Two New Aircraft Differ:
While most of our work focused on ATE for the current aircraft
inventory, we also wanted to see how the services were approaching
development of testers for two new aircraft, the Joint Strike Fighter
and the F/A-22. We found that very different approaches are being taken
in the development of ATE for these two aircraft. The JSF, for example,
will have a single tester, made up almost entirely of commercial
components, which will test all components on the aircraft. The F/A-22
project office has no assurance that commonality is being considered in
its tester development or that DOD‘s policy to minimize unique ATE
development is being followed.
The JSF originated in the early 1990s through the restructuring and
integration of several tactical aircraft and technology initiatives
already under way. The goal was to use the latest technology in a
common family of aircraft to meet the future strike requirements of the
services and U.S. allies. The JSF support strategy is built upon a
single tester to be used by the Air Force, Navy, and Marine Corps, as
well as by foreign partners, to test all avionics and weapon systems on
the aircraft.
The JSF tester, referred to as the LM-STAR, is made up almost entirely
of commercially available components, contributing to readily
available spares and less complicated upgrades. It will be used during
development and after the aircraft is fielded. Vendors participating in
the development of avionics and weapon system components for the
aircraft are required to produce these components so that their testing
can be done by the LM-STAR. A total of $99 million has been allocated
for the purchase and support of 88 of these testers during the
development phase. While a final decision has not been made on whether
maintenance support for the aircraft will be provided by the contractor
or at a military facility, the system project office is taking steps to
ensure that this tester can be used regardless of where maintenance is
accomplished.
By contrast, Air Force F/A-22 program officials told us that they have
not made a decision as to what testers will be used to support this new
aircraft, which began development in 1991. The project office has not
ensured that all components for the F/A-22 can be tested with a single
tester. Project officials told us that the F/A-22 is a very complex
aircraft and that opportunities to take advantage of common equipment
will be limited. Yet, the same contractor that is developing the F/A-22
is also involved in the JSF, which is also very advanced and complex
and which uses a common family of testers. While current projections of
ATE costs are not available, estimates made early in the F/A-22
development phase exceeded $1.5 billion.[Footnote 8]
DOD Oversight Needs Strengthening:
In 1993, the House Appropriations Committee recommended that a DOD-wide
policy be adopted requiring that the introduction of unique ATE be
minimized and that DOD establish an oversight system with sufficient
authority, staffing, and funding to ensure compliance. DOD established
a policy requiring the services to minimize unique types of testers to
reduce redundant investments and lessen long-term costs, leveraging its
investments in testers across the entire DOD establishment. In 1994,
DOD appointed the Navy as its Executive Agent for ATE to oversee the
implementation of this policy. As part of the tasking, the Executive
Agent for ATE was directed to establish a process so that programs
proposing not to use the DOD-designated standard of ATE families would
have to request a waiver. In accordance with the direction provided by
DOD, the Executive Agent established a waiver process. According to
data provided by the Executive Agent, since its inception, 30 requests
for waivers were submitted for their review. Our analysis indicated
that 15 of these requests resulted in waivers or concurrence. The
remaining requests were never finalized, were returned to the
originating office for further action, or were determined not to
require waivers. According to Executive Agent officials, the Executive
Agent makes recommendations concerning the waiver requests, but it does
not have the authority to disapprove them.
Executive Agent officials told us however, that they have no assurance
that all tester acquisitions and modifications are identified or that
all required waivers are requested. As a result, they may not be aware
of all ATE modifications or acquisitions or they may not be made aware
of such until the process is already under way and it is too late to
affect any change. For example, the Air Force did not request a waiver
for a $77 million modification to ATE supporting the low altitude
navigation and targeting infrared for night (LANTIRN). LANTIRN is a pod
system that supports the F-15, F-16, and F-14 aircraft in low-level
navigation and lazing targets. In its technical comments on our draft
report, however, Air Force officials indicated that owing to the nature
of the LANTIRN modification, a DOD waiver was not required. We continue
to believe, however, that the Executive Agent should be notified of
tester modifications of this magnitude.
In addition to having no assurance that all tester acquisitions and
modifications are identified, Executive Agent officials told us they do
not have the necessary enforcement authority or resources to
effectively implement the waiver process even when they know of the
planned acquisition or modification. For example, Executive Agent
officials held several discussions with F/A-22 program officials, early
in the development phase, concerning the use of common testers;
however, there was no evidence of the Executive Agent‘s involvement in
F/A-22 ATE development since November 1994. Executive Agent officials
do not know whether common testers are being considered.
As DOD‘s Executive Agent for ATE, the Navy has achieved some success in
encouraging the development of common testers and in dealing with
technical issues affecting all services. In September 1998, the
Executive Agent for ATE reported that DOD had avoided $284 million in
costs by implementing DOD‘s policy and cited one example in which the
Army and the Navy achieved savings of $80 million by jointly developing
an electro-optics test capability. Navy officials also told us that
they believe ATE planning for the Joint Strike Fighter, which calls for
vendors to use standardized test equipment or equipment having
commercially available components, can also be considered an
accomplishment. In addition, the Executive Agent established integrated
process teams to research technical issues dealing with tester
commonality, such as efforts to develop open systems architecture. In
this regard, DOD provided funds to the Executive Agent during fiscal
years 1995 to 1998 for its research and development efforts. Currently,
the Navy is leading a joint service technology project aimed at
demonstrating that the most advance technologies can be combined into a
single tester. The Executive Agent also implemented a process whereby
ATE modernization and acquisitions would be reviewed for compliance
with DOD policy, and developed the ATE Selection Process Guide and the
ATE Master Plan to aid the services in complying with DOD‘s ATE
policies.
ATE officials, responsible for oversight of ATE, noted that their role
is essential; however, its current placement in one service (the Navy)
makes it difficult to ensure other services comply with DOD guidance.
A report recently prepared by a joint service working group[Footnote 9]
noted continuing problems in the implementation of DOD policy,
including ATE obsolescence, delays in modernization efforts, a lack of
ATE interoperability among the services, upgrading difficulties, rising
support costs, proliferation of equipment that is difficult to support,
and systems that are not easily deployed.
Conclusions:
The services have made limited progress in achieving DOD‘s commonality
goals for ATE, as established in the early 1990s. The department does
not have a joint service forum or body that can oversee the total scope
of ATE acquisition and modernization and better promote ATE commonality
and the sharing of information and technology across platforms and
services. DOD does not have sufficient information concerning the
magnitude of the services‘ modernization efforts or a departmentwide
approach to accomplish ATE modernization in the most cost-effective
manner. Without such an approach, the department faces a very expensive
and time-consuming ATE modernization effort, with the continued
proliferation of unique testers. It will also have no assurance that
resources are allocated in the most effective manner to exploit
commonality and commercially available technology and products. A
single entity within DOD--rather than in one service--may be in the
best position to provide overarching oversight and coordination between
the services in planning for the modernization of ATE. We believe that
high-level management commitment within DOD and all the services will
be needed to achieve a cultural change that fosters the development of
common ATE.
Recommendations for Executive Action:
We recommend that the Secretary of Defense reemphasize the policy that
common ATE be developed to the maximum extent possible. We also
recommend that the Secretary reconsider whether placing its Executive
Agent for ATE in the Navy--or any single service--is the most effective
way to implement the policy. Wherever the Executive Agent is placed
organizationally, we recommend that the Secretary give it authority and
resources to:
* include representatives from all of the services, with a scope to
include the oversight of ATE acquisition and modifications for all
weapon systems;
* establish a mechanism to ensure that all ATE acquisitions and
modernizations are identified in an early enough stage to be able to
provide a comprehensive look at commonality and interoperability and to
ensure a coordinated effort between service entities;
* direct the services to draw up modernization plans for its review so
it can identify opportunities to maximize commonality and technology
sharing between and within the services; and:
* continue efforts to research technical issues dealing with tester
commonality such as the development of open system architecture and
other joint service applications.
Agency Comments and Our Evaluation:
The Department of Defense provided written comments on a draft of this
report, which are reprinted in their entirety in appendix II. The
department also provided technical comments which we have incorporated,
as appropriate, into the report. DOD concurred with our recommendations
and agreed that it should reemphasize the policy that common automatic
test equipment be developed to the maximum extent possible. DOD
indicated that it would propose that an ATE acquisition policy
statement be included in the next issuance of DOD Instruction 5000.2,
’Operation of the Defense Acquisition System,“ April 5, 2002. DOD also
agreed to reconsider whether the placement of its Executive Agent in
the Navy--or any single service--is the most effective way to implement
its ATE policy. The department further concurred that an Executive
Agent for ATE should have the authority and resources to direct the
services to draw up modernization plans for its review to maximize
commonality, interoperability, and technology sharing between the
services. In this regard, DOD agreed that there should be a mechanism
to ensure all automatic test equipment acquisitions and modernizations
are identified in an early enough stage in order to have a coordinated
effort among service entities. Finally, DOD agreed that the Executive
Agent for ATE should include representatives from all services. DOD
intends to use its authority recently published in DOD Directive
5100.88, ’DOD Executive Agent,“ September 3, 2002, to reconsider the
placement of the Executive Agent and to provide it with sufficient
authority, resources, and mechanisms to carry out its responsibilities.
In addition, DOD intends to include the funding for the Executive Agent
as part of the Planning, Programming, Budgeting and Execution process
and to identify such funding separately so that it is visible within
the DOD budget.
As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
from the date of this report. At that time, we will send copies of this
report to interested congressional committees; the Secretaries of
Defense, the Navy, the Air Force, and the Army; the Commandant, U.S.
Marine Corps; and the Director, Office of Management and Budget. We
will also make copies available to other interested parties on request.
In addition, the report will be available at no charge on the GAO Web
site at http://www.gao.gov/. If you or your staff have any questions
about the report, please contact me at (757) 552-8100.
Key contributors to this assignment were Ken Knouse, William Meredith,
Harry Taylor, Hugh Brady, and Stefano Petrucci.
Sincerely yours,
Neal P. Curtin
Director, Defense Capabilities and Management:
Signed by Neal P. Curtin
[End of section]
Appendix I: Scope and Methodology:
We reviewed and analyzed available reports, briefings, documents, and
records and interviewed officials at the Office of the Secretary of
Defense and at Air Force and Navy headquarters organizations,
Washington, D.C.; the Naval Air Systems Command located at Patuxent
River, Maryland; Air Force Material Command and system program offices
located at Wright-Patterson Air Force Base, Ohio; Warner Robins Air
Logistics Center, Georgia; the North Island Naval Aviation Depot,
California; the Navy‘s Aviation Intermediate Maintenance Department,
Oceana Master Jet Base, Virginia; and the intermediate maintenance
department aboard an aircraft carrier based in San Diego, California.
The Army was not included in the scope of this study because our focus
was primarily on fixed-wing aircraft and because of the Army‘s efforts
to standardize its automatic test equipment (ATE) around a single
family of testers, a situation similar to that of the Navy‘s.
To identify the problems that Air Force and Navy aviation (including
Marine Corps) is facing with regard to ATE, we interviewed personnel
responsible for policies and oversight, obtained applicable regulations
and other guidance, and analyzed data provided by the services on
various testers. We provided a proforma for the Air Force‘s and Navy‘s
use in documenting their inventory of ATE, identifying obsolete
testers, and providing estimates of modernization and replacement time
frames and cost. The Navy‘s data on ATE were provided by the central
office that manages common test equipment--PMA-260, within the Naval
Air Systems Command, and the Air Force‘s Automatic Test System Division
at Warner Robins Air Logistics Center. We also discussed obsolescence
issues and ATE problems with the managers of shore-based, aircraft
carrier, and depot maintenance activities. We reviewed and analyzed our
prior reports and ongoing efforts, and reports of other organizations
to provide a historical and contextual framework for evaluating ATE
policies and issues, for documenting readiness rates of selected
aircraft, and documenting the processes put in place by the Department
of Defense (DOD) to oversee the services‘ efforts to acquire and
modernize ATE.
To determine how successful DOD and the services have been in
addressing the proliferation of unique testers, we held discussions
with the responsible offices within each service and DOD, analyzed
regulations and guidance, and reviewed studies and other documentation.
We focused our work concerning this objective at the Navy office
designated as DOD‘s Executive Agent for Automatic Test Equipment--PMA-
260 within the Naval Air Systems Command--and the Air Force‘s Automatic
Test System Division at Warner Robins Air Logistics Center. At these
offices, which have responsibility for ATE acquisition or sustainment,
modernization, and oversight, we held discussions with responsible
officials, obtained documentation regarding responsibilities and
decisions, and reviewed files for specific ATE acquisition and
modernization programs. We also obtained information from individual
system program offices, for selected aircraft, located at Wright-
Patterson Air Force Base and selected Navy and Air Force depots and
intermediate maintenance facilities. Because we found that Air Force
testers are generally unique to specific aircraft, we selected the F-
15, B-1B, and B-2 for more detailed analysis, as these are considered
to be front-line aircraft depended upon heavily by the Air Force to
accomplish its mission. We also obtained information on ATE acquisition
for two fighter aircraft currently under development: the Joint Strike
Fighter and the F/A-22.
We performed our review from January 2002 through March 2003 in
accordance with generally accepted government auditing standards.
[End of section]
Appendix II: Comments from the Department of Defense:
DEPUTY UNDER SECRETARY OF DEFENSE FOR LOGISTICS AND MATERIEL READINESS
3500 DEFENSE PENTAGON WASHINGTON, DC 20301-3500:
MAR 24 2003:
Mr. Neal P. Curtin:
Director, Defense Capabilities and Management U.S. General Accounting
Office:
Washington, DC 20548:
Dear Mr. Curtin:
This is the Department of Defense (DoD) response to the GAO draft
report, GAO-03-451, ’Military Readiness: DoD Needs to Better Manage
Automatic Test Equipment Modernization,“ dated February 24, 2003 (GAO
Code 350137).
The Department concurs with all six recommendations. Additionally,
since portions of the draft report supporting the recommendations could
be misleading, technical comments are provided to improve the accuracy
and clarity of the draft report. The Department appreciates the
opportunity to comment on the draft report.
Sincerely,
Allen W. Beckett, Principal Assistant:
Signed by Allen W. Beckett:
Enclosure: As stated:
GAO CODE 350137/GAO-03-451:
’MILITARY READINESS: DOD NEEDS TO BETTER MANAGE AUTOMATIC TEST
EQUIPMENT MODERNIZATION“:
DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:
RECOMMENDATION 1: The GAO recommended that the Secretary of Defense
reemphasize the policy that common automatic test equipment be
developed to the maximum extent possible. (Page 15/Draft Report):
DoD RESPONSE: Concur. We will propose an ATE acquisition policy
statement for the next issuance of DoD Instruction 5000.2.
RECOMMENDATION 2: The GAO recommended that the Secretary of Defense
reconsider whether placing its executive agent for automatic test
equipment in the Navy - or any single Service - is the most effective
way to implement the policy. (Page 15/Draft Report):
DoD RESPONSE: Concur. DoD Directive 5100.88 ’DoD Executive Agent“,
September 3, 2002 was recently published in order to provide a DoD-wide
definition of DoD Executive Agent, and to provide DoD approval
authority for assigning DoD Executive Agent responsibilities,
functions, and authorities within the DoD. The new Directive goes on to
state that DoD Executive Agent assignments shall be identified in a DoD
Directive and will cite the Secretary of Defense‘s or the Deputy
Secretary of Defense‘s authority assigning DoD Executive Agency.
The DoD Directive assigning the Executive Agent must identify funding
and other resource arrangements for the DoD Executive Agent to carry
out assigned responsibilities, functions, and authorities; specify
other DoD Components, if any, that provide operational missions or
administrative or other designated activities in support of the DoD
Executive Agent.
The DoD Executive Agent funding methods and resource requirements,
including force structure to the extent permitted by law, shall be
included as a part of the Planning, Programming, Budgeting and
Execution process; and the funding and costs in support of each DoD
Executive Agent assignment and associated arrangements shall be
identified separately and shall be visible within the DoD budget.
DoD will reconsider assigning the Navy as Executive Agent for Automatic
Test Equipment consistent with the new DoD Directive 5100.88 outlined
above.
RECOMMENDATION 3: The GAO recommended that the Secretary of Defense
give the executive agent the authority and resources to include
representatives from all of the Services, with a scope to include the
oversight of automatic test equipment acquisition and modifications for
all weapon systems. (Page 15-16/Draft Report):
DoD RESPONSE: Concur. See response to Recommendation 2.
RECOMMENDATION 4: The GAO recommended that the Secretary of Defense
give the executive agent the authority and resources to establish a
mechanism to insure that all automatic test equipment acquisitions and
modernizations are identified in an early enough stage so as to be able
to provide a comprehensive look at commonality and interoperability and
to ensure a coordinated effort among Service entities. (Page 16/Draft
Report):
DoD RESPONSE: Concur. See response to Recommendation 2.
RECOMMENDATION 5: The GAO recommended that the Secretary of Defense
give the executive agent the authority and resources to direct the
Services to draw up modernization plans for its review so it can
identify opportunities to maximize commonality and technology sharing
between and within the Services. (Page 16/Draft Report):
DoD RESPONSE: Concur. See response to Recommendation 2.
RECOMMENDATION 6: The GAO recommended that the Secretary of Defense
give the executive agent the authority and resources to continue
efforts to research technical issues dealing with tester commonality
such as the development of open system architecture and other joint
Service applications. (Page 16/Draft Report):
DoD RESPONSE: Concur. See response to Recommendation 2.
[End of section]
Related GAO Products:
Defense Inventory: Better Reporting on Spare Parts Spending Will
Enhance Congressional Oversight. GAO-03-18. Washington, D.C.:
October 24, 2002.
Defense Inventory: Improved Industrial Base Assessments for Army
War Reserve Spares Could Save Money. GAO-02-650. Washington, D.C.: July
12, 2002.
Defense Inventory: Trends in Services‘ Spare Parts Purchased from the
Defense Logistics Agency. GAO-02-452. Washington, D.C.: April 30, 2002.
Defense Logistics: Opportunities to Improve the Army‘s and Navy‘s
Decision-Making Process for Weapons Systems Support. GAO-02-306.
Washington, D.C.: February 28, 2002.
Military Aircraft: Services Need Strategies to Reduce Cannibalizations.
GAO-02-86. Washington, D.C.: November 21, 2001.
Defense Logistics: Actions Needed to Overcome Capability Gaps in the
Public Depot System. GAO-02-105. Washington, D.C.: October 12, 2001.
Defense Logistics: Air Force Lacks Data to Assess Contractor Logistics
Support Approaches. GAO-01-618. Washington, D.C.: September 7, 2001.
Defense Inventory: Navy Spare Parts Quality Deficiency
Reporting Program Needs Improvement. GAO-01-923. Washington, D.C.:
August 16, 2001.
Army Inventory: Parts Shortages Are Impacting Operations and
Maintenance Effectiveness. GAO-01-772. Washington, D.C.: July 31, 2001.
Navy Inventory: Parts Shortages Are Impacting Operations and
Maintenance Effectiveness. GAO-01-771. Washington, D.C.: July 31, 2001.
Air Force Inventory: Parts Shortages Are Impacting Operations and
Maintenance Effectiveness. GAO-01-587. Washington, D.C.: June 27, 2001.
Defense Inventory: Information on the Use of Spare Parts Funding Is
Lacking. GAO-01-472. Washington, D.C.: June 11, 2001.
Defense Inventory: Approach for Deciding Whether to Retain or
Dispose of Items Needs Improvement. GAO-01-475. Washington, D.C.:
May 25, 2001.
Military Aircraft: Cannibalizations Adversely Affect Personnel and
Maintenance. GAO-01-93T. Washington, D.C.: May 22, 2001.
Defense Inventory: Army War Reserve Spare Parts Requirements Are
Uncertain. GAO-01-425. Washington, D.C.: May 10, 2001.
FOOTNOTES
[1] The test hardware and software of an integrated assembly of
stimulus, measurement, and switching components under computer control
that is capable of processing software routines designed specifically
to test a particular item or group of items.
[2] ’Interoperability“ is the ability of systems to provide data or
material to and accept the same from other systems and to operate
effectively together.
[3] The term ’Navy,“ as presented in this report, represents Naval
Aviation, which includes the Marine Corps.
[4] Test equipment that can be used on multiple airframes and weapon
systems.
[5] House Report No 103-254, Sept. 22, 1993.
[6] See U.S. General Accounting Office, Military Aircraft: Services
Need Strategies to Reduce Cannibalizations, GAO-02-86 (Washington,
D.C.: Nov. 21, 2001).
[7] See U.S. General Accounting Office, Navy Inventory: Parts Shortages
Are Impacting Operations and Maintenance Effectiveness, GAO-01-771
(Washington, D.C.: July 31, 2001).
[8] In providing technical comments on our draft report, the F/A-22
project office reiterated that because of the complexity of this
aircraft, opportunities to use common test equipment were limited. The
project office indicated that designing one set of ATE to test all
components could make that tester overly complex and expensive. In
addition, the project office indicated that it had taken advantage of
commercial testers and incorporated diagnostics into the avionics
themselves. Finally, the project office indicated that the estimate for
ATE of more than $1.5 billion made early in the development phase was
correct but misleading since the support philosophy had changed. We
continue to believe that the F/A-22 project office has not ensured that
tester commonality is being considered. The project office was not able
to provide information concerning the ATE used or planned for the F/A-
22 or estimates of ATE costs. Furthermore, there was no evidence of
Executive Agent involvement in the F/A-22 program since November 1994,
and Executive Agent officials do not know whether common testers are
being considered.
[9] The DOD Executive Agent for ATE established this working group to
develop a jointly funded demonstration project whereby the services
would develop and share ATE innovative technologies for inclusion in
future ATE acquisitions and modernizations.
GAO‘s Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO‘s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO‘s Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as ’Today‘s Reports,“ on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select ’Subscribe to daily E-mail alert for newly
released products“ under the GAO Reports heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
