Military Personnel
The Navy Has Not Provided Adequate Justification For Its Decision to Invest in MCTFS Gao ID: GAO-07-1139R July 25, 2007In June 2006, the Navy completed a feasibility assessment of both the Marine Corps Total Force System (MCTFS)--the Marine Corps' integrated personnel and payroll system--and the Defense Integrated Military Human Resources System DIMHRS. Based on the results of this assessment, the Navy expressed a preference to deploy MCTFS instead of DIMHRS. In August 2006, the Defense Business Systems Management Committee (DBSMC) made a business decision to accept the Navy's plan to proceed with the development and implementation of MCTFS. However, the Department of Defense (DOD) had already committed to deploy DIMHRS to provide a joint, integrated, standardized military personnel and pay system across all military components. According to the department, as of September 2006, it had spent over $668 million on the program. The John Warner National Defense Authorization Act for Fiscal Year 2007, Pub. L. No. 109-364, 324 (2006) directed the Secretary of the Navy to prepare a report about MCTFS, including (1) an analysis of alternatives to MCTFS, including a comparison between the costs of deploying and operating MCTFS within the Navy and the cost of including the Navy in DIMHRS; (2) a business case analysis of the costs and benefits to both the Navy and DOD of the alternatives to MCTFS considered in the first objective; and (3) an analysis of the compatibility of MCTFS with the department's business enterprise architecture. The Navy concluded in its report--which we received on April 25, 2007--that (1) either MCTFS or DIMHRS could provide basic personnel and pay capability for the Navy uniformed force at approximately equivalent cost; (2) the DIMHRS alternative has substantially higher risks on cost, schedule, and function because MCTFS is already operational; and (3) MCTFS is fully compatible and compliant with the department's business enterprise architecture. The Act directs the DBSMC to determine--not sooner than 120 days after April 25, 2007--whether the deployment of MCTFS by the Navy is in the best interest of DOD. The Act directed us to review the Navy's report and provide a written assessment to the congressional defense committees and the Chairman of the DBSMC within 90 days of our receipt of the Navy's report. Specifically, we determined whether the Navy in its congressionally mandated report has justified its decision to invest in MCTFS. As part of our review, we also determined whether the department's business enterprise architecture was sufficient to guide and constrain the acquisition of MCTFS.
In its report, the Navy has not adequately justified its decision to invest in MCTFS. Specifically, the analysis of alternatives and the business case analysis of both MCTFS and DIMHRS were unreliable, because the cost and benefit estimates used in these analyses were not sufficiently comprehensive, accurate, documented, or credible. The Navy's analysis shows that MCTFS is sufficiently compatible with DOD's evolving business enterprise architecture to allow the Navy to begin acquisition. However, we found that the architecture lacks sufficient content to be used effectively to guide and constrain the acquisition of MCTFS. DOD officials from the Navy, the Marine Corps, and the Business Transformation Agency provided oral comments on the facts presented in this report and we have incorporated these comments as appropriate.
GAO-07-1139R, Military Personnel: The Navy Has Not Provided Adequate Justification For Its Decision to Invest in MCTFS
This is the accessible text file for GAO report number GAO-07-1139R
entitled 'Military Personnel: The Navy Has Not Provided Adequate
Justification For Its Decision to Invest in MCTFS' which was released
on July 26, 2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
July 25, 2007:
Congressional Committees:
Subject: Military Personnel: The Navy Has Not Provided Adequate
Justification For Its Decision to Invest in MCTFS:
In June 2006, the Navy completed a feasibility assessment of both the
Marine Corps Total Force System (MCTFS)--the Marine Corps' integrated
personnel and payroll system--and the Defense Integrated Military Human
Resources System (DIMHRS)--the department's solution to provide a
joint, integrated, standardized military personnel and pay system
across all military components. Based on the results of this
assessment, the Navy expressed a preference to deploy MCTFS instead of
DIMHRS. In August 2006, the Defense Business Systems Management
Committee (DBSMC)[Footnote 1] made a business decision to accept the
Navy's plan to proceed with the development and implementation of MCTFS
instead of DIMHRS. However, the Department of Defense (DOD) had already
committed to deploy DIMHRS . According to the department, as of
September 2006, it had spent over $668 million[Footnote 2] on the
DIMHRS program.
The John Warner National Defense Authorization Act for Fiscal Year
2007, Pub. L. No. 109-364, §324 (2006) directed the Secretary of the
Navy to prepare a report about MCTFS, including (1) an analysis of
alternatives to MCTFS, including a comparison between the costs of
deploying and operating MCTFS within the Navy and the cost of including
the Navy in DIMHRS; (2) a business case analysis of the costs and
benefits to both the Navy and DOD of the alternatives to MCTFS
considered in the first objective; and (3) an analysis of the
compatibility of MCTFS with the department's business enterprise
architecture.[Footnote 3] The Navy concluded in its report--which we
received on April 25, 2007--that (1) either MCTFS or DIMHRS could
provide basic personnel and pay capability for the Navy uniformed force
at approximately equivalent cost; (2) the DIMHRS alternative has
substantially higher risks on cost, schedule, and function because
MCTFS is already operational; and (3) MCTFS is fully compatible and
compliant with the department's business enterprise architecture. The
Act directs the DBSMC to determine--not sooner than 120 days after
April 25, 2007--whether the deployment of MCTFS by the Navy is in the
best interest of DOD.
The Act directed us to review the Navy's report and provide a written
assessment to the congressional defense committees and the Chairman of
the DBSMC within 90 days of our receipt of the Navy's report. The DBSMC
is to consider our report in making its decision. Specifically, we
determined whether the Navy in its congressionally mandated report has
justified its decision to invest in MCTFS. As part of our review, we
also determined whether the department's business enterprise
architecture was sufficient to guide and constrain the acquisition of
MCTFS.
Scope and Methodology:
To perform our assessment, we reviewed the Navy report,[Footnote 4]
which summarized the results of the analysis of alternatives, the
business case analysis of the costs and benefits of the alternatives,
and the analysis of the compatibility of MCTFS with the department's
enterprise architecture. Using our Cost Assessment Guide and relevant
Office of Management and Budget (OMB) criteria,[Footnote 5] we
evaluated the analysis of alternatives and the business case analysis.
According to these criteria, the decision to invest in any system
should be based on reliable analyses of estimated costs and expected
benefits over the life of the system. Best practices state that
agencies should prepare a business case analysis--a comparative
analysis that presents facts and supporting details about competing
alternatives (i.e., analysis of alternatives). A business case analysis
should consider all life-cycle costs and nonquantifiable as well as
quantifiable benefits. It should consider all possible alternatives and
should not be developed solely to support a predetermined solution. A
business case analysis should be rigorous enough that independent
auditors can review it and clearly understand why a particular
alternative was chosen. We evaluated the cost estimates that the Navy
used in the analysis of alternatives and the business case analysis to
determine whether these estimates were comprehensive, accurate, well-
documented, and credible. Our Cost Assessment Guide considers an
estimate to be: (1) 'comprehensive' if it includes all relevant cost
elements; (2) 'accurate' if it is unbiased and neither overly
conservative nor overly optimistic and is based on an assessment of the
costs most likely to be incurred; (3) 'well-documented' if it can be
traced back to the documentation, which should include the source of
the data, detailed calculations and results, and explanations of why a
particular method or reference was chosen; and (4) 'credible' if
limitations of the data and assumptions are documented.
Using applicable guidance,[Footnote 6] we evaluated the efforts taken
by the Navy to show the extent to which MCTFS is compatible with the
architecture. This guidance states that agencies should develop and use
enterprise architectures to guide and constrain their investment
decisions. We obtained and analyzed the documents provided--such as
MCTFS architecture products, the results of the Navy's analysis of
MCTFS using the Architecture Compliance and Requirements Traceability
Tool, and a Gartner, Inc. report[Footnote 7]--the technology standards,
and version 4.0 of DOD's business enterprise architecture.
We interviewed officials to discuss the results of our work and to
obtain explanations for the discrepancies we identified. Specifically,
we interviewed officials from the following:
* Several organizations within the Department of the Navy, including
the Office of the Chief of Naval Operations; the Assistant Secretary of
the Navy Financial Management and Comptroller; the Manpower and Reserve
Affairs, U. S. Marine Corps; and the Naval Center for Cost Analysis.
These officials were responsible for developing the analysis of
alternatives, the business case analysis, and the analysis of the
compatibility of MCTFS with the department's business enterprise
architecture.
* The Enterprise Program Management Office-DIMHRS, which is responsible
for acquiring and implementing DIMHRS, including the DIMHRS Enterprise
Deputy Program Manager.
* The Personnel and Readiness Information Management Office, which is
responsible for developing the DIMHRS architecture, including staff
supporting the investment management process.
* The Business Transformation Agency, which is responsible for
maintaining and updating the department's business enterprise
architecture, including the chief architect for the Business Mission
Area.
Our work was conducted from April through July 2007 in accordance with
generally accepted government auditing standards.
Summary:
In its report, the Navy has not adequately justified its decision to
invest in MCTFS. Specifically, the analysis of alternatives and the
business case analysis of both MCTFS and DIMHRS were unreliable,
because the cost and benefit estimates used in these analyses were not
sufficiently comprehensive, accurate, documented, or credible. The
Navy's analysis shows that MCTFS is sufficiently compatible with DOD's
evolving business enterprise architecture to allow the Navy to begin
acquisition.[Footnote 8] However, we found that the architecture lacks
sufficient content to be used effectively to guide and constrain the
acquisition of MCTFS. DOD officials from the Navy, the Marine Corps,
and the Business Transformation Agency provided oral comments on the
facts presented in this report and we have incorporated these comments
as appropriate.
Analysis of Alternatives and Business Case Analysis Were Unreliable:
The Navy's report included an analysis of alternatives and a business
case analysis of MCTFS and DIMHRS; however, these analyses were
unreliable because the cost and benefit estimates used for these
analyses were not sufficiently comprehensive, accurate, documented, or
credible. First, the cost estimates were not comprehensive, because
they did not include all possible alternatives. For example, the Navy's
report included a third alternative that involves deploying MCTFS
initially and then deploying DIMHRS after the Army and Air Force have
successfully implemented it. However, the analysis of alternatives and
the business case analysis did not include cost estimates for this
alternative. In March 2007, DBSMC tasked the Navy to prepare the cost
and benefit estimates for this alternative, after which Navy officials
said that they discussed it with the Secretary of the Navy. These
estimates were based on those that had been used for both the MCTFS and
DIMHRS alternatives and we had concluded that these estimates were
unreliable. Second, the cost estimates were not accurate, because the
Navy excluded the third alternative, which was more costly than the
MCTFS and DIMHRS alternatives. This exclusion skewed the results of the
analyses by showing that the MCTFS alternative was the least costly
solution. Third, the cost estimates were not well-documented, because
source data typically was not provided and the calculations performed
for several cost elements were not explained. Finally, the cost
estimates were not credible, because there were neither cross-checks of
these estimates nor a current independent cost estimate; both of these
are necessary in order to determine whether a different estimating
method would have produced similar results.
Moreover, the analyses based on these cost estimates did not take into
account the following risks: (1) the potential impact of moving MCTFS
development operations from Kansas City to Indianapolis as part of the
Base Realignment and Closure 2005 actions and (2) the potential impact
of the contractor discontinuing its support of DIMHRS after 2013, as
planned. A Navy official responsible for MCTFS development operations
stated his intent to delay the closing of the Kansas City site as long
as possible in order to mitigate its impact on MCTFS. However, this
official did not provide documentation of his authority to carry out
this intent or of the Navy's plans for doing so within the time
limitations established for the Base Realignment and Closure 2005
actions. After the Navy issued its report, the contractor changed its
policy and stated its intent to support DIMHRS indefinitely; however,
this support will be limited. For example, 'sustaining support' does
not include key services, such as new updates, product fixes, security
alerts, critical patch and regulatory updates, and certification of
DIMHRS with new products developed by the contractor. It is unclear
what the costs, if any, will be for this sustaining support. However,
the contractor's change in policy is not a binding legal agreement; the
contractor could later decide to discontinue its support of DIMHRS or
increase the cost prohibitively for sustaining support. Since the
accuracy of the cost estimates is in question, the benefits or cost
savings to be realized are also in question.
According to best practices,[Footnote 9] the decision to invest in any
system should be based on reliable analyses of estimated costs and
expected benefits over the life of the system. Best practices state
that agencies should prepare a business case analysis--a comparative
analysis that presents facts and supporting details about competing
alternatives. A business case analysis considers all life-cycle costs
and quantifiable and nonquantifiable benefits. It should consider all
possible alternatives and should not be developed solely to support a
predetermined solution. A business case analysis should be rigorous
enough that independent auditors can review it and clearly understand
why a particular alternative was chosen.
Navy officials said that the report did not include the strategic
alternative showing the Navy's intent to ultimately transition to
DIMHRS because the mandate did not require them to do so. However, best
practices require that an agency consider all possible alternatives
when preparing a business case analysis, so this alternative should
have been included in the report. In November 2006, the Navy had tasked
the Naval Center for Cost Analysis to develop cost estimates for the
business case analysis to be included in the report by February 2007.
Naval Center for Cost Analysis officials said that this deadline give
them adequate time to meet best practices for developing comprehensive,
accurate, well-documented, and credible cost estimates. These officials
stated, and we agree, that it typically takes 6 months to develop
reliable cost estimates. DIMHRS program officials stated that the cost
and benefit estimates the Navy used in its analyses were outdated and
did not reflect the current DIMHRS program. They also stated that they
had not been contacted by the Naval Center for Cost Analysis to provide
updated information for the Navy report. However, officials from the
Naval Center for Cost Analysis provided documentation showing that they
had in fact contacted DIMHRS program officials and that these officials
had refused to answer programmatic and technical questions about the
DIMHRS program. Because the analyses were unreliable, Congress and the
DBSMC cannot be assured that the most cost-effective solution will be
selected.
MCTFS Is Compatible with the Business Enterprise Architecture, but the
Architecture Is Insufficient to Guide Investment Decisions:
The Navy's analysis of the compatibility of MCTFS with the department's
business enterprise architecture was sufficient for the Navy to begin
acquisition of the system. However, we found that the architecture
lacks sufficient content to be used effectively to guide and constrain
investments for systems such as MCTFS. As a result of this lack of
architectural constraint on systems, the Navy determined the
architecture requirements MCTFS will need to meet to fit within the
department's vision as outlined in the evolving architecture. According
to best practices and DOD guidance,[Footnote 10] agencies should
develop and use enterprise architectures to guide and constrain
investment decisions. Senior DOD and Navy officials acknowledged that
the architecture cannot be used at this time to effectively guide and
constrain component-level system investments such as MCTFS. As a
result, there is a risk that the department will not be able to achieve
its vision to deploy standard systems such as DIMHRS. By deploying
standard systems, DOD may be able to address the weaknesses in its
current system environment, which is comprised of numerous systems that
are duplicative in functionality, are not interoperable, are
unnecessarily costly to maintain and interface, and do not optimize
mission performance and accountability.
Conclusion:
Our assessment of the Navy's report shows that the analyses are based
on flawed estimates and an insufficient architecture. If the DBSMC
relies solely on the Navy's report in deciding whether or not to deploy
MCTFS, it risks recommending an investment decision that would not be
in the best interest of DOD. As it deliberates, the DBSMC should
consider the issues we have identified.
We are sending copies of this report to selected congressional
committees and the Chairman of the Defense Business Systems Management
Committee. We will also make copies available to others upon request.
In addition, the report will be available at no charge on GAO's Web
site at http://www.gao.gov.
If your offices or staffs have any questions concerning this report,
please contact me at (202) 512-3604 or by e-mail at farrellb@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to the report are listed in enclosure I.
Signed by:
Brenda S. Farrell:
Director, Defense Capabilities and Management:
Enclosure - 1:
List of Committees:
The Honorable Daniel Inouye:
Chairman:
The Honorable Ted Stevens:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
United States Senate:
The Honorable Carl Levin:
Chairman:
The Honorable John McCain:
Ranking Member:
Committee on Armed Services:
United States Senate:
The Honorable John P. Murtha:
Chairman:
The Honorable C.W. Bill Young:
Ranking Member:
Subcommittee on Defense:
Committee on Appropriations:
House of Representatives:
The Honorable Ike Skelton:
Chairman:
The Honorable Duncan Hunter:
Ranking Member:
Committee on Armed Services:
House of Representatives:
[End of section]
Enclosure I: GAO Contact and Staff Acknowledgments:
GAO Contact:
Brenda S. Farrell (202) 512-3604 or farrellb@gao.gov:
Acknowledgments:
In addition to the contact above, Cynthia Jackson, Assistant Director;
Brian Bothwell; Harold J. Brumm, Jr; Jennifer K. Echard; Joanne
Landesman; Sonya Phillips; Karen A. Richey; Matthew S. Spiers; Randolph
Tekeley; Joseph J. Watkins; and Angela D. Watson made key contributions
to this report.
(351049):
FOOTNOTES
[1] The Deputy Secretary of Defense established the DBSMC, which
oversees DOD's business transformation, and tasked it with
responsibility for overall departmental transformation pursuant to 10
U.S.C. § 186.
[2] The cost figure includes costs incurred by the Enterprise Program
Management Office-DIMHRS, the Army, the Air Force, the U. S. Marine
Corps, and the Defense Finance and Accounting Office; it does not
include costs incurred by the Navy since program inception.
[3] An enterprise architecture provides a clear and comprehensive
picture of an entity, whether it is an organization (e.g., a federal
department) or a functional or mission area that cuts across more than
one organization (e.g., human resources). It is an essential tool for
effectively and efficiently engineering business processes and for
implementing and evolving systems that support the business processes.
[4] Department of the Navy, Report from the Secretary of the Navy to
the Congressional Defense Committees and the Comptroller General on the
Marine Corps Total Force System (MCTFS), April 2007.
[5] GAO, Cost Assessment Guide: Best Practices for Developing and
Managing Program Costs, July 2007; Office of Management and Budget
(OMB) Circular A-11, Preparation, Submission, and Execution of the
Budget, Part 7: Planning, Budgeting, Acquisition, and Management of
Capital Assets (June 2006); and OMB Circular A-94, Guidelines and
Discount Rates for Benefit-Cost Analysis of Federal Programs (Revised
Oct. 29, 1992).
[6] Office of Management and Budget (OMB) Capital Programming Guide,
Version 2.0 (June 2006); GAO, Information Technology: A Framework for
Assessing and Improving Enterprise Architecture Management (Version
1.1), GAO-03-584G (Washington, D.C.: April 2003); Department of
Defense, DOD IT Business Systems Investment Review Process, Business
Enterprise Architecture (BEA) Compliance Guidance, Version 1.0 (Apr.
10, 2006); and Department of Defense, Department of Defense
Architecture Framework, Version 1.5, Volume 1 (Apr. 23, 2007).
[7] Gartner, Inc., A Report for U. S. Marine Corps and Defense Finance
and Accounting Service: Marine Corps Total Force System Program
Evaluation (June 19, 2006).
[8] Our analysis determined the extent to which MCTFS was compatible
with version 4.0 of the department's business enterprise architecture.
[9] See GAO, Cost Assessment Guide: Best Practices for Developing and
Managing Program Costs, OMB Circular A-11, and OMB Circular A-94.
[10] Office of Management and Budget (OMB) Capital Programming Guide,
Version 2.0 ; GAO-03-584G; Department of Defense, DOD IT Business
Systems Investment Review Process, Business Enterprise Architecture
(BEA) Compliance Guidance; and Department of Defense, Department of
Defense Architecture Framework, Version 1.5, Volume 1.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
