Office of Personnel Management
Retirement Modernization Planning and Management Shortcomings Need to Be Addressed Gao ID: GAO-09-529 April 21, 2009For the past two decades, the Office of Personnel Management (OPM) has been working to modernize the paper-intensive processes and antiquated systems used to support the retirement of federal employees. By moving to an automated system, OPM intends to improve the program's efficiency and effectiveness. In January 2008, GAO recommended that the agency address risks to successful system deployment. Nevertheless, OPM deployed a limited initial version of the modernized system in February 2008. After unsuccessful efforts to address system quality issues, OPM suspended system operation, terminated a major contract, and began restructuring the modernization effort, also referred to as RetireEZ. For this study, GAO was asked to (1) assess the status of OPM's efforts to plan and implement the RetireEZ program and (2) evaluate the effectiveness of the agency's management of the modernization initiative. To do this, GAO reviewed OPM program documentation and interviewed agency and contractor officials.
OPM remains far from achieving the modernized capabilities it had planned. Specifically, the agency has partially implemented two of eight planned capabilities: (1) an integrated database of retirement information accessible to OPM and agency retirement processing personnel and (2) enhanced customer service capabilities that support customer needs and provide self-service tools. However, the remaining six capabilities have yet to be implemented because they depended on deliverables that were to be provided by a contract that is now terminated. Examples of these missing capabilities include: (1) automated submission of retirement information through interfaces with federal agencies and (2) Web-accessible self-service retirement information for active and retired federal employees. Further, OPM has not yet developed a complete plan that describes how the program is to proceed without the system that was to be provided under the terminated contract. Although agency documents describe program implementation activities, they do not include a definition of the program, its scope, lines of responsibility and authority, management processes, and a schedule. Also, modernization program documentation does not describe results-oriented performance goals and measures. Until the agency completes and uses a plan that includes all of the above elements to guide its efforts, it will not be properly positioned to move forward with its restructured retirement modernization initiative. Further, OPM has significant weaknesses in five key management areas that are vital for effective development and implementation of its modernization program: cost estimating, earned value management (a recognized means for measuring program progress), requirements management, testing, and oversight. For example, the agency has not developed a cost estimating plan or established a performance measurement baseline--prerequisites for effective cost estimating and earned value management. Further, although OPM is revising its previously developed system requirements, it has not established processes and plans to guide this work or addressed test activities even though developing processes and plans, as well as planning test activities early in the life cycle, are recognized best practices for effective requirements development and testing. Finally, although OPM's Executive Steering Committee and Investment Review Board have recently become more active regarding RetireEZ, these bodies did not exercise effective oversight in the past, which has allowed the aforementioned management weaknesses to persist and OPM has not established guidance regarding how these entities are to intervene when corrective actions are needed. Until OPM addresses these weaknesses, many of which GAO and others made recommendations to correct, the agency's retirement modernization initiative remains at risk of failure. Institutionalizing effective management is critical not only for the success of this initiative, but also for that of other modernization efforts within the agency.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-09-529, Office of Personnel Management: Retirement Modernization Planning and Management Shortcomings Need to Be Addressed
This is the accessible text file for GAO report number GAO-09-529
entitled 'Office Of Personnel Management: Retirement Modernization
Planning and Management Shortcomings Need to Be Addressed' which was
released on April 21, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
April 2009:
Office Of Personnel Management:
Retirement Modernization Planning and Management Shortcomings Need to
Be Addressed:
GAO-09-529:
GAO Highlights:
Highlights of GAO-09-529, a report to congressional committees.
Why GAO Did This Study:
For the past two decades, the Office of Personnel Management (OPM) has
been working to modernize the paper-intensive processes and antiquated
systems used to support the retirement of federal employees. By moving
to an automated system, OPM intends to improve the program‘s efficiency
and effectiveness. In January 2008, GAO recommended that the agency
address risks to successful system deployment. Nevertheless, OPM
deployed a limited initial version of the modernized system in February
2008. After unsuccessful efforts to address system quality issues, OPM
suspended system operation, terminated a major contract, and began
restructuring the modernization effort, also referred to as RetireEZ.
For this study, GAO was asked to (1) assess the status of OPM‘s efforts
to plan and implement the RetireEZ program and (2) evaluate the
effectiveness of the agency‘s management of the modernization
initiative. To do this, GAO reviewed OPM program documentation and
interviewed agency and contractor officials.
What GAO Found:
OPM remains far from achieving the modernized capabilities it had
planned. Specifically, the agency has partially implemented two of
eight planned capabilities:
* an integrated database of retirement information accessible to OPM
and agency retirement processing personnel and;
* enhanced customer service capabilities that support customer needs
and provide self-service tools.
However, the remaining six capabilities have yet to be implemented
because they depended on deliverables that were to be provided by a
contract that is now terminated. Examples of these missing capabilities
include:
* automated submission of retirement information through interfaces
with federal agencies and;
* Web-accessible self-service retirement information for active and
retired federal employees.
Further, OPM has not yet developed a complete plan that describes how
the program is to proceed without the system that was to be provided
under the terminated contract. Although agency documents describe
program implementation activities, they do not include a definition of
the program, its scope, lines of responsibility and authority,
management processes, and a schedule. Also, modernization program
documentation does not describe results-oriented performance goals and
measures. Until the agency completes and uses a plan that includes all
of the above elements to guide its efforts, it will not be properly
positioned to move forward with its restructured retirement
modernization initiative.
Further, OPM has significant weaknesses in five key management areas
that are vital for effective development and implementation of its
modernization program: cost estimating, earned value management (a
recognized means for measuring program progress), requirements
management, testing, and oversight. For example, the agency has not
developed a cost estimating plan or established a performance
measurement baseline”prerequisites for effective cost estimating and
earned value management. Further, although OPM is revising its
previously developed system requirements, it has not established
processes and plans to guide this work or addressed test activities
even though developing processes and plans, as well as planning test
activities early in the life cycle, are recognized best practices for
effective requirements development and testing. Finally, although OPM‘s
Executive Steering Committee and Investment Review Board have recently
become more active regarding RetireEZ, these bodies did not exercise
effective oversight in the past, which has allowed the aforementioned
management weaknesses to persist and OPM has not established guidance
regarding how these entities are to intervene when corrective actions
are needed. Until OPM addresses these weaknesses, many of which GAO and
others made recommendations to correct, the agency‘s retirement
modernization initiative remains at risk of failure. Institutionalizing
effective management is critical not only for the success of this
initiative, but also for that of other modernization efforts within the
agency.
What GAO Recommends:
To improve OPM‘s effort to plan and implement its retirement
modernization program, GAO is recommending that OPM correct management
weaknesses. In written comments on a draft of this report, the Director
of OPM agreed with GAO‘s recommendations and described steps the agency
is taking to address them.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/products/GAO-09-529]. For more
information, contact Valerie C. Melvin at (202) 512-6304 or
melvinv@gao.gov.
[End of section]
Contents:
Letter:
Background:
Few Planned Capabilities Have Been Implemented, and Future Efforts Are
Not Guided by a Complete Plan:
OPM Is Not Positioned to Effectively Manage Its Retirement
Modernization Initiative:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Comments from the Office of Personnel Management:
Appendix III: GAO Staff Contact and Acknowledgments:
Tables:
Table 1: Key Retirement Modernization Contracts:
Table 2: Retirement Modernization Management Entities:
Table 3: Status of Implementing Retirement Modernization Capabilities:
Table 4: Fiscal Year 2009 Initiatives:
Figures:
Figure 1: Estimated Federal Employees Eligible and Likely to Retire by
2016:
Figure 2: Simplified View of Retirement Application Process:
Figure 3: Timeline of Retirement Modernization Initiatives:
Abbreviations:
CIS: Center for Information Services:
CRIS: Center for Retirement and Insurance Services:
CSRS: Civil Service Retirement System:
DBTS: defined benefits technology solution:
EVM: earned value management:
FAPS: FERS Automated Processing System:
FERS: Federal Employees Retirement System:
GSA: General Services Administration:
OMB: Office of Management and Budget:
OPM: Office of Personnel Management:
RSM: Retirement Systems Modernization:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
April 21, 2009:
The Honorable Richard J. Durbin:
Chairman:
The Honorable Susan Collins:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
United States Senate:
The Honorable José E. Serrano:
Chairman:
The Honorable Jo Ann Emerson:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
House of Representatives:
For the past two decades, the Office of Personnel Management (OPM) has
been attempting to modernize the paper-intensive processes and
antiquated systems it uses to support the retirement of federal
employees, in part because the current process does not provide prompt
and complete benefit payments upon retirement. The agency's retirement
modernization initiative, also referred to as RetireEZ, was intended to
address this situation and improve the efficiency and effectiveness of
the retirement program, which serves federal employees who are eligible
to receive benefits in the future, employees who are already retired,
and their survivors and beneficiaries.
In January 2008, we reported on challenges that OPM faced in deploying
RetireEZ, noted a number of concerns regarding the agency's management
of the initiative, and recommended corrective actions.[Footnote 1] In
late February 2008, OPM deployed a version of the modernized retirement
system; however, the agency suspended the system's operation in May
2008. After 5 months of attempting to address system quality issues,
the agency terminated its contract with the primary modernization
vendor in October 2008 and began restructuring the retirement
modernization initiative in November 2008.
At your request, we conducted a follow-up study of OPM's management of
RetireEZ. Specifically, our objectives were to:
* assess the status of OPM's efforts toward planning and implementing
the RetireEZ program and:
* evaluate the effectiveness of the agency's management of the
modernization initiative.
To accomplish our objectives, we reviewed relevant program
documentation and interviewed appropriate OPM and contractor officials.
Specifically, to assess the status of the program, we reviewed
documentation such as the retirement modernization program management
plan and supporting documents. We also visited OPM's retirement
operations facilities in Washington, D.C., and Boyers, Pennsylvania. To
evaluate the effectiveness of the agency's management of the
modernization initiative, we reviewed OPM's cost estimation, earned
value management (EVM), requirements management, test management
activities, and oversight. We compared OPM's plans and activities to
best practices and assessed the agency's progress toward implementing
our prior recommendations.
We conducted this performance audit from May 2008 through March 2009 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives. See appendix II for a more
complete description of our objectives, scope, and methodology.
Background:
OPM's mission is to ensure that the federal government has an effective
civilian workforce. In this regard, one of the agency's major human
resources tasks is to manage and administer the retirement program for
federal employees. According to the agency, the program serves federal
employees by providing (1) retirement compensation and (2) tools and
options for retirement planning. OPM's Center for Retirement and
Insurance Services administers the two defined benefit retirement plans
that provide retirement, disability, and survivor benefits to federal
employees. The first plan, the Civil Service Retirement System (CSRS),
provides retirement benefits for most federal employees hired before
1984. The second plan, the Federal Employees Retirement System (FERS),
covers most employees hired in or after 1984 and provides benefits that
include Social Security and a defined contribution system.[Footnote 2]
According to OPM, there are approximately 2.9 million active federal
employees and nearly 2.5 million retired federal employees. The
agency's March 2008 analysis of federal employment retirement data
estimates that nearly 1 million active federal employees will be
eligible to retire and almost 600,000 will most likely retire by 2016.
[Footnote 3] Figure 1 summarizes the estimated number of employees
eligible and likely to retire.
Figure 1: Estimated Federal Employees Eligible and Likely to Retire by
2016:
[Refer to PDF for image: multiple line graph]
Fiscal year: 2009;
Likely to retire: 184,461;
Eligible to retire: 494,619.
Fiscal year: 2010;
Likely to retire: 246,209;
Eligible to retire: 566,801.
Fiscal year: 2011;
Likely to retire: 307,159;
Eligible to retire: 637,645.
Fiscal year: 2012;
Likely to retire: 366,707;
Eligible to retire: 707,750.
Fiscal year: 2013;
Likely to retire: 424,546;
Eligible to retire: 775,035.
Fiscal year: 2014;
Likely to retire: 480,464;
Eligible to retire: 836,516.
Fiscal year: 2015;
Likely to retire: 534,411;
Eligible to retire: 896,335.
Fiscal year: 2016;
Likely to retire: 586,339;
Eligible to retire: 956,613.
Source: OPM.
[End of figure]
Federal Employee Retirement Application Processing:
OPM and employing agencies' human resources and payroll offices are
responsible for processing federal employees' retirement applications.
The process begins when an employee submits a paper retirement
application to his or her employer's human resources office and is
completed when the individual begins receiving regular monthly benefit
payments (as illustrated in figure 2).
Figure 2: Simplified View of Retirement Application Process:
[Refer to PDF for image: illustration]
Agency:
Employee submits retirement application;
Agency: Human Resources (HR):
* Required meeting in which agency benefits officer checks paperwork,
provides retirement counseling, and provides benefits estimate;
* Develop retirement package and separation form;
* Review retirement package and provide final documents to agency
payroll;
Agency: Payroll:
* Receive package from HR and perform analysis;
* Prepare individual retirement record;
* Electronically submit summary file and mail hardcopy of complete
package to OPM;
OPM:
* Agency retirement package received;
* Interim payment calculated based on agency summary file and
authorized;
* Development actions taken as needed to adjudicate application;
- Ensure package has all required information;
- Regular benefit payment is calculated and package is submitted for
authorization;
- Package and benefit payment reviewed and verified by second
adjudicator;
* Regular benefit payment authorized.
Source: GAO analysis of OPM data.
[End of figure]
Once an employee submits an application, the employing agency's human
resources office provides retirement counseling services to the
employee and augments the retirement application with additional
paperwork, such as a separation form that finalizes the date the
employee will retire. Then the agency provides the retirement package
to the employee's payroll office. After the employee separates for
retirement, the payroll office is responsible for reviewing the
documents for correct signatures and information, making sure that all
required forms have been submitted, and adding any additional paperwork
that will be necessary for processing the retirement package. Once the
payroll office has finalized the paperwork, the retirement package is
mailed to OPM to continue the retirement process. Payroll offices are
expected to submit the package to OPM within 30 days of the retiree's
separation date.
Upon receipt of the retirement package, OPM calculates an interim
payment based on information provided by the employing agency. The
interim payments are partial payments that typically provide retirees
with 80 percent of the total monthly benefit they will eventually
receive.[Footnote 4] OPM then starts the process of analyzing the
retirement application and associated paperwork to determine the total
monthly benefit amount to which the retiree is entitled. This process
includes collecting additional information from the employing agency's
human resources and payroll offices or from the retiree to ensure that
all necessary data are available before calculating benefits. After OPM
completes its review and authorizes payment, the retiree begins
receiving 100 percent of the monthly retirement benefit payments. OPM
then stores the paper retirement folder at the Retirement Operations
Center in Boyers, Pennsylvania.
According to the agency's 2008 performance report, the average
processing time from the date OPM receives the initial application to
the time the retiree receives a full payment is 42 days. According to
the Deputy Associate Director for the Center of Retirement and
Insurance Services, about 200 employees are directly involved in
processing the approximately 100,000 retirement applications OPM
receives annually. This processing includes functions such as
determining retirement eligibility, inputting data into benefit
calculators, and providing customer service. The agency uses over 500
different procedures, laws, and regulations, which are documented on
the agency's internal Web site, to process retirement applications. For
example, the site contains memorandums that outline new procedures for
handling special retirement applications, such as those for disability
or court orders. Further, OPM's retirement processing involves the use
of over 80 information systems that have approximately 400 interfaces
with other internal and external systems. For instance, 26 internal
systems interface with the Department of the Treasury to provide, among
other things, information regarding the total amount of benefit
payments to which an employee is entitled.
OPM has stated that the federal employee retirement process currently
does not provide prompt and complete benefit payments upon retirement,
and that customer service expectations for more timely payments are
increasing. The agency also reports that a greater workload is expected
due to an anticipated increase in the number of retirement applications
over the next decade, yet current retirement processing operations are
at full capacity. Further, the agency has identified several factors
that limit its ability to process retirement benefits in an efficient
and timely manner. Specifically, it noted that:
* current processes are paper-based and manually intensive, resulting
in a higher number of errors and delays in providing benefit payments;
* the high costs, limited capabilities, and other problems with the
existing information systems and processes pose increasing risks to the
accuracy of benefit payments;
* current manual capabilities restrict customer service;
* federal employees have limited access to their retirement records,
making planning for retirement difficult; and:
* attracting qualified personnel to operate and maintain the antiquated
retirement systems, which have about 3 million lines of custom
programming, is challenging.
OPM Has a Long History of Retirement Modernization Initiatives:
In the late 1980s, OPM recognized the need to automate and modernize
its retirement processing and began retirement modernization
initiatives that have continuously called for automating its antiquated
paper-based processes. The agency's previously established program
management plans included the objectives of having timely and accurate
retirement benefit payments and more efficient and flexible processes.
For example, the agency's plans call for processing retirement
applications and providing retirees 100 percent of their monthly
benefit payments the day it is due versus providing interim monthly
payments. Its initial modernization vision called for providing prompt
and complete benefit payments by developing an integrated system and
automated processes. However, the agency has faced significant and long-
standing challenges in doing so.
In early 1987, OPM began a program called the FERS Automated Processing
System (FAPS). However, after 8 years of planning, the agency decided
it needed to reevaluate the program, and the Office of Management and
Budget (OMB) requested that an independent board conduct a review to
identify critical issues impeding progress and recommend ways to
address the issues. The review identified various management
weaknesses, including the lack of an established strategic plan, cost
estimation methodologies, and baseline; improperly defined and
ineffectively managed requirements; and no clear accountability for
decision making and oversight.[Footnote 5] Accordingly, the board
suggested areas for improvement and recommended terminating the program
if immediate action was not taken. In mid-1996, OPM terminated the
program.
In 1997, OPM began planning a second modernization initiative, called
the Retirement Systems Modernization (RSM) program. The agency
originally intended to structure the program as an acquisition of
commercially available hardware and software that would be modified in-
house to meet its needs. From 1997 to 2001, OPM developed plans and
analyses and began developing business and security requirements for
the program. However, in June 2001, it decided to change the direction
of the retirement modernization initiative.
In late 2001, retaining the name RSM, the agency embarked upon its
third initiative to modernize the retirement process and examined the
possibility of privately sourced technologies and tools. To this end,
OPM issued a request for information to obtain private sourcing options
and determined that contracting was a viable alternative that would be
cost efficient, less risky, and more likely to be completed on time and
on budget. In 2006, the agency awarded three contracts for: (1) a
commercially available, defined benefits technology solution (DBTS) to
automate retirement processing; (2) services to convert paper records
to electronic files; and (3) consulting services to support the
redesign of its retirement operations. The contract for DBTS was
awarded to Hewitt Associates, and the additional contracts to support
the technology were awarded to Accenture Ltd. and Northrop Grumman
Corporation, as reflected in table 1.
Table 1: Key Retirement Modernization Contracts:
Contractor: Hewitt Associates;
Award date: May 2006;
Contract value (in millions): $290.0;
Deliverables: A defined benefits technology solution (DBTS) that
enables automating retirement processes.
Contractor: Accenture Ltd.;
Award date: May 2006;
Contract value (in millions): $40.0;
Deliverables: Redesigned retirement processes to support DBTS.
Contractor: Northrop Grumman Corporation;
Award date: September 2006;
Contract value (in millions): $30.7;
Deliverables: Converting paper retirement records to electronic files.
Source: OPM.
[End of table]
OPM produced a December 2007 program management plan that, among other
things, described capabilities the agency expected to implement as
outcomes of retirement modernization. Among these capabilities, the
agency expected to implement retirement benefit modeling and planning
tools for active federal employees, a standardized retirement benefit
calculation system, and a consolidated system to support all aspects of
retirement processing.
In February 2008, OPM renamed the program RetireEZ and deployed a
limited initial version of DBTS. As the foundation of the modernization
initiative, DBTS was to be a comprehensive technology solution that
would provide capabilities to substantially automate retirement
processing. This technology was to be provided by the contractor for a
period of 10 years and was intended to provide, among other things, an
integrated database with calculation functionality for retirement
processing. In addition to calculating retirement benefit amounts, DBTS
was intended to provide active and retired federal employees with self-
service, Internet-based tools for accessing accounts, updating
retirement records, submitting transactions, monitoring the status of
claims, and forecasting retirement income. The technology was also
expected to enhance customer service by providing OPM and agency
personnel with the capability to access retirement information online.
Further, the technology was expected to be integrated with OPM and
federal agency electronic retirement records and processes. When fully
implemented, the modernized program was expected to serve OPM
retirement processing personnel, federal agency human resources and
payroll offices, active federal employees, retirees, and the
beneficiaries of retirees.
According to the agency, in late February 2008, the DBTS was deployed
with limited functionality to 26,000 federal employees serviced by the
General Services Administration's (GSA) payroll offices. In April 2008,
OPM reported that 13 of the 37 retirement applications received from
GSA's payroll office had been processed through DBTS with manual
intervention and provided the retirees 100 percent of their monthly
benefits within 30 days from their retirement date. However, a month
later, the agency determined that DBTS had not worked as expected and
suspended system operation. In October 2008, after 5 months of
attempting to address system quality issues, the agency terminated the
contract.
In November 2008, OPM began restructuring the program and reported that
its efforts to modernize retirement processing would continue. Figure 3
illustrates the timeline of retirement modernization initiatives from
1987 to the present.
Figure 3: Timeline of Retirement Modernization Initiatives:
[Refer to PDF for image: timeline illustration]
FAPS: First effort: about 8 years: 1987-1996.
RSM (in-house): Second effort: about 4 year: 1997-2001.
RSM (privately sourced: Third effort: about 7 years: 2001-2008;
DBTS deployed (RSM renamed RetireEZ): 2008.
RetireEZ (restructures): Fourth effort: 2009 and beyond.
Source: GAO analysis of OPM data.
[End of figure]
Current Retirement Modernization Management:
Various entities within OPM are responsible for managing RetireEZ.
Specifically, the management is composed of committees, a program
office, and operational support, as reflected in table 2.
Table 2: Retirement Modernization Management Entities:
Title: Executive Steering Committee;
Description: The Executive Steering Committee is responsible for
overall program direction and decision making. The committee is chaired
by the Director of the Office of Modernization and Human Resources Line
of Business.
Title: Investment Review Board;
Description: The Investment Review Board is responsible for monitoring
the progress of continued investment in the program and determining
appropriate corrective action when necessary. The board is chaired by
the agency's Chief Information Officer.
Title: Office of Modernization and Human Resources Line of Business;
Description: The Office of Modernization and Human Resources Line of
Business is responsible for overseeing the program office for the
retirement modernization. The Director of Modernization and Human
Resources Line of Business maintains overall administration of program
activities and resources.
Title: Center for Information Services;
Description: The Center for Information Services (CIS), within the
Management Services Division, is responsible for providing support and
oversight for acquisition, systems, contract, and security management.
The Deputy Associate Director of CIS (also the Chief Information
Officer) oversees the maintenance and operation of legacy retirement
systems.
Title: Center for Retirement and Insurance Services;
Description: The Center for Retirement and Insurance Services (CRIS),
within the Human Resources Products and Services Division, is
responsible for retirement processes and is expected to assume
ownership of the retirement modernization. The Deputy Associate
Director of CRIS oversees current operations and allocates resources as
necessary to support the program office.
Source: GAO analysis of OPM data.
[End of table]
Prior GAO Reviews of OPM's Retirement Modernization Initiatives:
Since 2005, we have conducted several studies of OPM's retirement
modernization noting weaknesses in its management of the initiative. In
February of that year, we reported that the agency lacked processes for
retirement modernization acquisition activities, such as determining
requirements, developing acquisition strategies, and implementing a
risk program. Further, the agency had not established effective
security management, change management, and program executive
oversight.[Footnote 6] We recommended that the Director of OPM ensure
that the retirement modernization program office expeditiously
establish processes for effective oversight of the retirement
modernization in the areas of system acquisition management,
information security, organizational change management, and information
technology (IT) investment management. In response, between 2005 and
2007, the agency initiated steps toward establishing management
processes for retirement modernization and demonstrated the completion
of activities with respect to each of our nine recommendations.
However, in January 2008, we reported that the agency still needed to
improve its management of the program to ensure a successful outcome
for its modernization efforts.[Footnote 7] Specifically, we reported
that initial test results had not provided assurance that DBTS would
perform as intended, the testing schedule increased the risk that the
agency would not have sufficient resources or time to ensure that all
system components were tested before deployment, and trends in
identifying and resolving system defects had indicated a growing
backlog of problems to be resolved prior to deployment. Further, we
reported that although the agency had established a risk management
process, it had not reliably estimated the program costs, and its
progress reporting was questionable because it did not reflect the
actual state of the program. We recommended that the Director of OPM
address these deficiencies by conducting effective system tests and
resolving urgent and high priority system defects prior to system
deployment, in addition to improving program cost estimation and
progress reporting.
In response to our report, OPM stated that it concurred with our
recommendations and was taking steps to address them. However, in March
2008, we determined that the agency was moving forward with system
deployment and had not yet implemented its planned actions.[Footnote 8]
OPM subsequently affirmed its agreement with our recommendations in
April 2008 and reported that it had implemented or was in the process
of implementing each recommendation. As of March 2009, however, these
recommendations still had not been fully addressed.
Few Planned Capabilities Have Been Implemented, and Future Efforts Are
Not Guided by a Complete Plan:
OPM remains far from fully implementing the retirement modernization
capabilities described when it documented its plans for RetireEZ in
2007. The agency only partially implemented two of eight capabilities
that it identified to modernize retirement processing. The remaining
six capabilities, which were to be delivered through the DBTS contract,
have not been implemented, and OPM's plans to continue implementing
them are uncertain. While the agency has taken steps to restructure the
RetireEZ program without the DBTS contract, it has not developed a plan
to guide its future modernization efforts.
Few Retirement Modernization Capabilities Have Been Implemented:
OPM's retirement modernization plans from 2007 described eight
capabilities that were to be implemented to achieve modernized
processes and systems. As of late March 2009, the agency had partially
implemented two of these capabilities while the remaining six had not
been implemented (see table 3).
Table 3: Status of Implementing Retirement Modernization Capabilities:
Capability: An integrated database of retirement information accessible
to OPM and agency retirement processing personnel;
Implemented: Partially.
Capability: Enhanced customer service that support customer needs and
self-service tools;
Implemented: Partially.
Capability: Automated submission of retirement information through
interfaces with federal agencies;
Implemented: No.
Capability: Web-accessible, self-service retirement information for
active and retired federal employees;
Implemented: No.
Capability: A consolidated system to support all aspects of retirement
processing;
Implemented: No.
Capability: Electronic case management system to manage retirement
applications and processes;
Implemented: No.
Capability: Standardized systems for determining and calculating
retirement benefit that is integrated with the retirement information
database and is accessible to OPM and federal agencies;
Implemented: No.
Capability: Retirement benefit modeling and planning tools for active
federal employees;
Implemented: No.
Source: GAO analysis of OPM data.
[End of table]
Specifically, it had achieved partial implementation of an integrated
database of retirement information that was intended to be accessible
to OPM and agency retirement processing personnel. In this regard, the
agency implemented a new database, populated with images of retirement
information, which is accessible to OPM retirement processing personnel
online. This database contains over 8 million files which, according to
agency officials, represent approximately 80 to 90 percent of the
available retirement information for all active federal employees.
However, the capability for the information in the database to be
integrated with OPM's legacy retirement processing systems and to be
accessible to other agency retirement processing personnel has not yet
been implemented.
OPM has also partially implemented enhanced customer service
capabilities. Specifically, the agency acquired a new telephone
infrastructure (i.e., additional lines) and hired additional customer
service representatives to reduce wait times and abandonment rates.
However, the agency has not yet developed the capabilities for OPM
retirement processing personnel to provide enhanced customer support to
active and retired federal employees through online account access and
management.
Moreover, six other capabilities have not been implemented--and plans
to implement them are uncertain--because they were to be delivered
through the now-terminated DBTS contract, which had been expected to
provide a single system that would automate the processing of
retirement applications, calculations, and benefit payments.
Among the capabilities not implemented was one for other agencies'
automated submissions of retirement information to OPM that could be
used to process retirement applications. While OPM began developing
this capability by establishing interfaces with other agencies as part
of its effort to implement DBTS, it discontinued the use of the
interfaces for processing retirement applications when the DBTS
contract was terminated. Thus, federal agencies that submit retirement
information to OPM continue to provide paper packages and information
when employees are ready to retire.
Further, OPM has not implemented a planned capability for active and
retired federal employees to access online retirement information
through self-service tools. While the agency provided demonstrations of
DBTS in April 2008 that showed the ability for employees to access
information online, including applying for retirement and modeling
future retirement benefits, this capability was to be provided by DBTS,
and thus, no longer exists. The contractor had also been expected to
deliver a consolidated system to support all aspects of retirement
processing and an electronic case management system to support
retirement processing. In the absence of these capabilities, the agency
continues to manage cases through paper tracking and stand-alone
systems. Additionally, OPM and federal agencies continue to rely on
nonstandardized systems to determine and calculate retirement benefits,
and federal retirees currently have only limited online, self-service
tools.
OPM Does Not Have a Complete Plan for the Future of the RetireEZ
Program:
Program management principles and best practices emphasize the
importance of using a program management plan that, among other things,
establishes a complete description that ties together all program
activities.[Footnote 9] An effective plan includes a description of the
program's scope, implementation strategy, lines of responsibility and
authority, management processes, and a schedule. Such a plan
incorporates all the critical areas of system development and is to be
used as a means of determining what needs to be done, by whom, and
when. Furthermore, establishing results-oriented (i.e., objective,
quantifiable, and measurable) goals and measures, that can be included
in a plan, provides stakeholders with the information they need to
effectively oversee and manage programs.
A plan for the future of the RetireEZ program has not been completed.
In November 2008, OPM began restructuring the program and reported it
was continuing toward retirement modernization without the DBTS
contract. The restructuring efforts have resulted in a wide variety of
documentation, including multiple descriptions of the program in formal
agency reports, budget documentation, agency briefing slides, and
related documents.
For example, OPM's November Fiscal Year 2008 Agency Financial Report
described what the RetireEZ program is expected to achieve (e.g.,
provide retirement modeling tools for federal employees) once
implemented. The agency's Annual Performance Report, dated January
2009, outlined that the new vision for the restructured program is "to
support benefit planning and management throughout a participant's
lifecycle through an enhanced federal retirement program." The agency
also presented information to OMB that identified eight fiscal year
2009 program initiatives, as listed in table 4.
Table 4: Fiscal Year 2009 Initiatives:
Initiative:
* Determine next steps for technology tools and develop an execution
plan.
* Increase customer contact center capacity.
* Complete conversion of paper records.
* Work with agencies to improve the quality, timeliness, and
completeness of retirement data.
* Begin activities to provide agencies access to electronic data.
* Improve performance metrics for interim payments.
* Partner with the Enterprise Human Resources Integration initiative[A]
for data storage, cleansing, and transmission of recurring data.
* Begin imaging all incoming paper records.
Source: OPM data.
[A] OPM's Enterprise Human Resources Integration initiative is intended
to provide federal agencies access to personnel folders online.
[End of table]
The agency has developed a variety of informal program documents and
briefing slides that describe retirement modernization activities. For
instance, one document prepared by the program office describes a five-
phased approach that is intended to replace its previous DBTS-reliant
strategy. The approach includes the following activities: (1)
collecting electronic retirement information, (2) automating the
retirement application process, (3) integrating retirement information,
(4) developing retirement calculation technologies and tools, and (5)
improving post-retirement processes through a technology solution. In
addition, briefing slides also prepared by the program office outline a
schedule for efforts to identify new technologies to support retirement
modernization by drafting a request for information, which OPM expects
to issue in late April 2009.
Regardless, OPM's various reports and documents describing its planned
retirement modernization activities do not provide a complete plan for
its restructured program. Specifically, although agency documents
describe program implementation activities, they do not include a
definition of the program, its scope, lines of responsibility and
authority, management processes, and schedule. Also, the modernization
program documentation does not describe results-oriented (i.e.,
objective, quantifiable, and measurable) performance goals and
measures. According to the RetireEZ program manager, the agency is
developing plans, but they will not be ready for release until the new
OPM director has approved them, which is expected to occur in April
2009. Until the agency completes and uses a plan that includes all of
the above elements to guide its efforts, it will not be properly
positioned to obtain agreement with relevant stakeholders (e.g.,
Congress, OMB, federal agencies, and OPM senior executives) for its
restructured retirement modernization initiative. Further, the agency
will also not have a key mechanism that it needs to help ensure
successful implementation of future modernization efforts.
OPM Is Not Positioned to Effectively Manage Its Retirement
Modernization Initiative:
OPM has significant management weaknesses in five areas that are
important to the success of its retirement modernization program: cost
estimating, EVM, requirements management, testing, and program
oversight. For example, the agency has not performed key steps,
including the development of a cost estimating plan or completion of a
work breakdown structure, both of which are necessary to develop a
reliable program cost estimate. Also, OPM has not established and
validated a performance measurement baseline, which is essential for
reliable EVM. Further, although OPM is revising its previously
developed system requirements, it has not established processes and
plans to guide this work. Nor has the agency addressed test activities,
even though developing processes and planning test activities early in
the life cycle are recognized best practices for effective testing.
Furthermore, although OPM's Executive Steering Committee and Investment
Review Board have recently become more active regarding RetireEZ, these
bodies did not exercise effective oversight in the past, which has
allowed the aforementioned management weaknesses to persist. Notably,
OPM has not established guidance regarding how these entities are to
engage with the program when corrective actions are needed. Until OPM
addresses these weaknesses, many of which we and others made
recommendations to correct, the agency's retirement modernization
initiative remains at risk of failure.
OPM Has Yet to Complete Key Steps in Developing a Reliable Cost
Estimate for Retirement Modernization:
The establishment of a reliable cost estimate is a necessary element
for informed investment decision making, realistic budget formulation,
and meaningful progress measurement. A cost estimate is the summation
of individual program cost elements that have been developed by using
established methods and validated data to estimate future costs.
According to federal policy, programs must maintain current and well-
documented estimates of program costs, and these estimates must span
the full expected life of the program.[Footnote 10] Our Cost Estimating
and Assessment Guide includes best practices that agencies can use for
developing and managing program cost estimates that are comprehensive,
well-documented, accurate, and credible, and provide management with a
sound basis for establishing a baseline to measure program performance
and formulate budgets.[Footnote 11] This guide identifies a cost
estimating process that includes initial steps such as defining the
estimate's purpose (i.e., its intended use, scope, and level of
detail); developing the estimating plan (i.e., the estimating approach,
team, and timeline); defining the program (e.g., technical baseline
description); and determining the estimating structure (e.g., work
breakdown structure). According to best practices, these initial steps
in the cost estimating process are of the utmost importance, and should
be fully completed in order for the estimate to be considered valid and
reliable.
OPM officials stated that they intend to complete a modernization
program cost estimate by July 2009. However, the agency has not yet
fully completed initial steps for developing the new estimate.
Specifically, the agency has not yet fully defined the estimate's
purpose, developed the estimating plan, defined program characteristics
in a technical baseline description, or determined the estimating
structure.
* With respect to the estimate's purpose, agency officials stated that
the estimate will inform the budget justification of RetireEZ for
fiscal year 2011 and beyond. However, the agency has not clearly
defined the scope or level of detail of the estimate.
* Regarding the estimating plan, agency officials stated that they have
created a timeline for the estimate's completion by July 2009. However,
the agency has not documented an estimating plan that includes the
approach and resources required to complete the estimate in the time
period identified.
* With respect to the technical baseline description, agency officials
stated that they are in the advanced stages of developing a request for
information and a concept of operations that will serve as the basis
for a technical baseline description. These documents are expected to
be reviewed for approval in April 2009.
* Regarding the estimating structure, the agency has developed a work
breakdown structure that identifies elements of the program to be
estimated. However, the agency has not yet developed a work breakdown
structure dictionary that clearly defines each element.
Weaknesses in the reliability of OPM's retirement modernization cost
estimate have been long-standing. We first reported on the agency's
lack of a reliable cost estimate in January 2008 when we noted that
critical activities, including documentation of a technical baseline
description, had not been performed, and we recommended that the agency
revise the estimate.[Footnote 12] Although OPM agreed to produce a
reliable program cost estimate, the agency has not yet done so. Until
OPM fully completes each of the steps, the agency increases the risk
that it will produce an unreliable estimate and will not have a sound
basis for measuring program performance and formulating retirement
modernization program budgets.
OPM Is Not Prepared to Perform Reliable Earned Value Management for
Retirement Modernization:
OMB and OPM policies require major IT programs to use EVM to measure
and report program progress.[Footnote 13] EVM is a tool for measuring
program progress by comparing the value of work accomplished with the
amount of work expected to be accomplished. Such a comparison permits
actual performance to be evaluated, based on variances from the planned
cost and schedule, and future performance to be forecasted.
Identification of significant variances and analysis of their causes
helps program managers determine the need for corrective actions.
Before EVM analysis can be reliably performed, developing a credible
cost estimate is necessary. In addition to developing a cost estimate,
an integrated baseline review must be conducted to validate a
performance measurement baseline and attain agreement of program
stakeholders (e.g., agency and contractor officials) before reliable
EVM reporting can begin. The establishment of a baseline depends on the
completion of a work breakdown structure, an integrated master
schedule, and budgets for planned work.
Although the agency plans to begin reporting on the restructured
program's progress using EVM in April 2009, the agency is not yet
prepared to do so because initial steps have not been completed and are
dependent on decisions about the program that have not been made.
Specifically,
* the agency has not yet developed a reliable cost estimate for the
program; such an estimate, which is critical for establishing reliable
EVM, is not expected to be complete until July 2009;
* the agency does not plan to conduct an integrated baseline review to
establish a reliable performance measurement baseline before beginning
EVM reporting; and:
* the work breakdown structure and integrated master schedule that
agency officials report they have developed may not accurately reflect
the full scope and schedule because key program documentation, such as
the concept of operations, has not been completed.
This situation resembles the state of affairs that existed in January
2008, when we reported that OPM's EVM was unreliable because an
integrated baseline review had not been conducted to validate the
program baseline.[Footnote 14] At that time we recommended, among other
things, that the agency establish a basis for effective use of EVM by
validating a program performance measurement baseline through a program-
level integrated baseline review. Although the agency stated that it
agreed, it did not address this recommendation.
Until the agency has developed a reliable cost estimate, performed an
integrated baseline review, and validated a performance measurement
baseline that reflect its program restructuring, the agency is not
prepared to perform reliable EVM. Engaging in EVM reporting without
first performing these fundamental steps could again render the
agency's assessment unreliable.
OPM Has Not Established Processes Needed to Effectively Develop and
Manage Retirement Modernization Requirements:
Well-defined and managed requirements are a cornerstone of effective
system development and acquisition. According to recognized guidance,
disciplined processes for developing and managing requirements can help
reduce the risks of developing a system that does not meet user and
operational needs.[Footnote 15] Such processes include (1) developing
detailed requirements that have been derived from the organization's
concept of operations[Footnote 16] and are complete and sufficiently
detailed to guide system development and (2) establishing policies and
plans, including defining roles and responsibilities, for managing
changes to requirements and maintaining bidirectional requirements
traceability.[Footnote 17]
OPM's retirement modernization requirements processes include some, but
not all, of the elements needed to effectively develop and manage
requirements. The agency began an effort to better develop its
retirement modernization requirements in November 2008. This effort was
in response to the agency's recognition that its over 1,400
requirements lacked sufficient detail, were incomplete, and required
further development. The agency intends to complete this requirements
development effort in April 2009. However, the requirements will not be
derived from OPM's concept of operations because the agency is revising
the concept of operations expected to be completed by April 2009, to
reflect the program restructuring. Further, OPM documentation indicates
that the agency has not yet determined the level of detail to which
requirements should be developed. Additionally, agency officials stated
that OPM is developing a requirements development process for
retirement modernization. With respect to requirements management, OPM
developed an organizational charter that outlined roles and
responsibilities for supporting efforts to manage requirements.
However, the agency does not yet have a requirements management plan.
OPM's prior experience with DBTS illustrates the importance of
effective requirements development and management. According to
RetireEZ program officials, insufficiently detailed requirements,
poorly controlled requirements changes, and inadequate requirements
traceability were factors that contributed to DBTS not performing as
expected. Moreover, these requirements development and management
weaknesses were identified, and recommendations for improvement were
made by OPM's independent verification and validation contractor before
DBTS deployment. However, the agency has not yet corrected these
weaknesses.
Until OPM fully establishes requirements development and management
processes, the agency increases the risk that it will (1) identify
requirements that are neither complete nor sufficiently detailed and
(2) not effectively manage requirements changes or maintain
bidirectional traceability, thus further increasing agency risk that it
will produce a system that does not meet user and operational needs.
OPM Is Not Positioned to Effectively Manage Retirement Modernization
Testing:
Effective testing is an essential component of any program that
includes developing systems. Generally, the purpose of testing is to
identify defects or problems in meeting defined system requirements and
satisfying user needs. To be effectively managed, testing should be
planned and conducted in a structured and disciplined fashion that
adheres to recognized guidance and is coordinated with the requirements
development process.[Footnote 18] Beginning the test planning process
in the early stages of a program life cycle can reduce rework later in
the program. Early test planning in coordination with requirements
development can provide major benefits. For example, planning for test
activities during the development of requirements may reduce the number
of defects identified later and the costs related to requirements
rework or change requests. Further, planning test activities early in a
program's life cycle can inform requests for proposals and help
communicate testing expectations to potential vendors.
OPM has not begun to plan test activities in coordination with
developing its requirements for the RetireEZ program. According to OPM
officials, the agency intends to begin its test planning by revising
the previously developed DBTS test plans after requirements have been
developed. However, the agency has not yet added test planning to its
project schedule.
Early test planning is especially important to avoid repeating the
agency's experience during DBTS testing when it identified more defects
than it could resolve before system deployment. In January 2008, we
reported that an unexpectedly high number of defects were identified
during testing; yet, the deployment schedule had increased the risk of
not resolving all defects that needed to be corrected before deploying
DBTS. According to the RetireEZ program officials, the failure to fully
address these defects contributed to the limited number of federal
employees who were successfully processed by the system when it was
deployed in February 2008.
If it does not plan test activities early in the life cycle of
RetireEZ, OPM increases the risk that it will again deploy a system
that does not satisfy user expectations and meet requirements (i.e.,
accurately calculate retirement benefits) because of its potential
inability to address a higher number of defects than expected.
Moreover, criteria used to develop requests for proposals and
communicate testing expectations to potential vendors could be better
informed if the agency plans RetireEZ test activities early in the life
cycle.
OPM Is Not Providing Effective Oversight for Retirement Modernization:
GAO and OMB guidance calls for agencies to ensure effective oversight
of IT projects throughout all life-cycle phases. Critical to effective
oversight are investment management boards made up of key executives
who regularly track the progress of IT projects such as system
acquisitions or modernizations. These boards should maintain adequate
oversight and track project performance and progress toward predefined
cost and schedule goals, as well as monitor project benefits and
exposure to risk. Another element of effective IT oversight is
employing early warning systems that enable management boards to take
corrective actions at the first sign of cost, schedule, and performance
slippages.
OPM's Investment Review Board was established to ensure that major
investments are on track by reviewing their progress and determining
appropriate actions when investments encounter challenges. Despite
meeting regularly and being provided with information that indicated
problems with the retirement modernization, the board did not ensure
that the investment was on track, nor did it determine appropriate
actions for course correction when needed. For example, from January
2007 to August 2008 the board met and was presented with reports that
described problems the retirement modernization program was facing,
such as the lack of an integrated master schedule and earned value data
that did not reflect the "reality or current status" of the program.
However, meeting minutes indicate that no discussion or action was
taken to address these problems. According to a member of the board,
OPM guidance regarding how the board is to communicate recommendations
and corrective actions when needed for the investments it is
responsible for overseeing has not been established.
In addition, OPM established an Executive Steering Committee to oversee
retirement modernization. According to its charter, the committee is to
provide strategic direction, oversight, and issue resolution to ensure
that the program maintains alignment with the mission, goals, and
objectives of the agency and is supported with required resources and
expertise. However, the committee was inactive for most of 2008 and,
consequently, did not exercise oversight of the program during a
crucial period in its development. For example, from January 2008 until
October 2008, the committee discontinued its formal meetings, and as a
result, it was not involved in key program decisions, including the
deployment of DBTS. Further, a member of the committee noted that OPM
guidance for making recommendations and taking corrective actions also
has not been provided.
The ineffectiveness of the board and the inactivity of the committee
allowed program management weaknesses in the areas of cost estimation,
EVM, requirements management, and testing to persist and raise concerns
about OPM's ability to provide meaningful oversight as the agency
proceeds with its retirement modernization. Without fully functioning
oversight bodies, OPM cannot monitor modernization activities and make
the course corrections that effective boards and committees are
intended to provide.
Conclusions:
OPM's retirement modernization initiative is in transition from a
program that was highly dependent on the success of a major contract
that no longer exists, to a restructured program that has yet to be
fully defined. Although the agency has been able to partially implement
a database of retirement information and improvements to customer
service, it remains far from implementing six other key capabilities.
Recognizing that much work remains, OPM has undertaken steps to
restructure the retirement modernization program, but it has not yet
produced a complete description of its planned program, including
fundamental information about the program's scope, implementation
strategy, lines of responsibility and authority, management processes,
and schedule. Further, OPM's retirement modernization program
restructuring does not yet include definitions of results-oriented
goals and measures against which program performance can be objectively
and quantitatively assessed.
In addition, OPM has not overcome managerial shortcomings in key areas
of program management, including areas that we have previously
reported. Specifically, the agency is not yet positioned to develop a
reliable program cost estimate or perform reliable EVM, both of which
are critical to effective program planning and oversight. Nor has OPM
overcome weaknesses in its management of system testing and defects,
two activities that proved problematic as the agency was preparing to
deploy the RetireEZ system that subsequently was terminated. Adding to
these long-standing concerns are weaknesses in OPM's process to
effectively develop and manage requirements for whatever system or
service it intends to acquire or develop. Finally, these weaknesses
have been allowed to persist by entities within the agency that were
ineffective in overseeing the retirement modernization program.
As a consequence, the agency is faced with significant challenges on
two fronts: defining and transitioning to its restructured program, and
addressing new and previously identified managerial weaknesses. Until
OPM addresses these weaknesses, many of which were previously
identified by GAO and others, the agency's retirement modernization
initiative remains at risk of failure. Institutionalizing effective
planning and management is critical not only for the success of this
initiative, but also for that of other modernization efforts within the
agency.
Recommendations for Executive Action:
To improve OPM's effort toward planning and implementing its retirement
modernization program by addressing management weaknesses, we recommend
that the Director of the Office of Personnel Management provide
immediate attention to ensure the following six actions are taken:
* Develop a complete plan for the restructured program that defines the
scope, implementation strategy, lines of responsibility and authority,
management processes, and schedule. Further, the plan should establish
results-oriented (i.e., objective, quantifiable, and measurable) goals
and associated performance measures for the program.
* Develop a reliable cost estimate by following the best practice steps
outlined in our Cost Estimating and Assessment Guide, including
definition of the estimate's purpose, development of an estimating
plan, definition of the program's characteristics, and determination of
the estimating structure.
* Establish a basis for reliable EVM, when appropriate, by developing a
reliable program cost estimate, performing an integrated baseline
review, and validating a performance measurement baseline that reflects
the program restructuring.
* Develop a requirements management plan and execute processes
described in the plan to develop retirement modernization requirements
in accordance with recognized guidance.
* Begin RetireEZ test planning activities early in the life cycle.
* Develop policies and procedures that would establish meaningful
program oversight and require appropriate action to address management
deficiencies.
Agency Comments and Our Evaluation:
The Director of the Office of Personnel Management provided written
comments on a draft of this report. (The comments are reproduced in
appendix II.) In the comments, OPM agreed with our recommendations and
stated that it had begun to address them. To this end, the Director
stated that the agency had, among other actions, begun revising its
retirement modernization plans, developing a new program cost estimate,
planning for accurate EVM reporting, incorporating recognized guidance
in requirements management planning, and planning test activities
during requirements development. If the recommendations are properly
implemented, they should better position OPM to effectively manage its
retirement modernization initiative. The agency also provided comments
on the draft report regarding our description of the federal retirement
application process, as well as our characterizations of OPM's EVM and
requirements management capabilities vis-à-vis the retirement
modernization program. In each of these instances, we made revisions as
appropriate.
We are sending copies of this report to the Director of the Office of
Personnel Management, appropriate congressional committees, and other
interested parties. In addition, the report is available at no charge
on the GAO Web site at [hyperlink, http://www.gao.gov.
If you or your staffs have questions about this report, please contact
me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. Key contributors to this report are listed in
appendix III.
Signed by:
Valerie C. Melvin:
Director, Information Management and Human Capital Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
As requested, the objectives of our study were to (1) assess the status
of the Office of Personnel Management's (OPM) efforts toward planning
and implementing the RetireEZ program and (2) evaluate the
effectiveness of the agency's management of the modernization
initiative.
To assess the status of OPM's efforts toward planning and implementing
the RetireEZ program, we:
* reviewed and analyzed program documentation, including program
management plans, briefing slides, and project status documentation, to
identify planned retirement modernization capabilities and determine to
what extent these capabilities have been implemented;
* evaluated the agency's documentation about restructuring the program
and analyzed the extent to which the documentation describes current
and planned RetireEZ program activities;
* identified and evaluated the agency's program goals and measures and
compared them to relevant guidance to determine the extent to which the
goals and measures are described in results-oriented terms;
* supplemented agency program documentation and our analyses by
interviewing agency and contractor officials, including the OPM
Director, Chief Information Officer, Chief Financial Officer, Director
of Modernization, Associate Director for Human Resources Products and
Services Division, and executives from Hewitt Associates and Northrop
Grumman Corporation; and:
* observed retirement operations and ongoing modernization activities
at OPM and contractor facilities in Washington, D.C.; Boyers,
Pennsylvania; and Herndon, Virginia.
To determine the effectiveness of OPM's management of the retirement
modernization initiative, we evaluated the agency's management of
program cost estimating, earned value management (EVM), requirements,
test planning, and oversight and compared the agency's work in each
area with recognized best practices and guidance. Specifically,
* to evaluate whether OPM effectively developed a reliable program cost
estimate, we analyzed the agency's program documentation and determined
to what extent the agency had completed key activities described in our
Cost Estimating and Assessment Guide;
* to assess OPM's implementation of EVM, we reviewed program progress
reporting documentation and compared the agency's plans for restarting
its EVM-based progress reporting against relevant guidance, including
our Cost Estimating and Assessment Guide;
* regarding requirements management, we evaluated OPM's processes for
developing and managing retirement systems modernization requirements
and compared the effectiveness of those processes against recognized
guidance;
* to determine the effectiveness of the agency's test planning for the
retirement modernization, we reviewed program activities and test plans
against best practices and evaluated the extent to which the agency has
begun planning for these activities; and:
* we reviewed and analyzed documentation from program oversight
entities and evaluated the extent to which these entities took actions
toward ensuring the RetireEZ program was being effectively overseen.
We also evaluated OPM's progress toward implementing our open
recommendations and interviewed OPM and contractor officials as noted.
We conducted this performance audit at OPM headquarters in Washington,
D.C., the Retirement Operations Center for OPM in Boyers, Pennsylvania,
and contractor facilities in Herndon, Virginia, from May 2008 through
April 2009, in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Comments from the Office of Personnel Management:
United States Office Of Personnel Management:
The Director:
Washington, DC 20415:
"Our mission is to ensure the Federal Government has an effective
civilian workforce."
[hyperlink, www.opm.gov]
[hyperlink, www.usajobs.gov]
April 15, 2009:
Valerie C. Melvin:
Director, Information Management and Human Capital Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Ms. Melvin:
Thank you for the opportunity to provide comments in response to the
Government Accountability Office's (GAO) draft report entitled Office
of Personnel Management: Retirement Modernization Planning and
Management Shortcomings Need to Be Addressed (GAO-09-529). The Office
of Personnel (OPM) agrees with the six recommendations outlined in the
draft report and has already begun to address them.
As you are aware, the primary objectives of retirement business
transformation are: 1. to migrate from a paper-based environment to an
electronic system of retirement data management; 2. to improve
retirement processing and post-retirement processing efficiency; and 3.
to enhance customer service. To that end, the Program Management Office
(PMO), Executive Steering Committee (ESC), and program stakeholders
continue to refine the retirement systems modernization program five-
phased implementation plan to ensure each phase and its initiatives
directly align to retirement systems modernization end-state
capabilities. Progress continues on the fundamental requirements of the
plan - automating data collection, conversion of data from paper
records to electronic media and the re-orientation of agency staff
concerning their critical role in improving data quality and improving
the retirement experience. These critical, foundational steps are
needed to prepare the way for retirement modernization going forward.
Progress has also been made in the establishment of basic performance
measures and their monthly reporting. These measures are related to the
fundamental requirement for improvements and allow the agency to ensure
that current investments result in better services and reductions in
future cost increases.
As OPM's new Director, I consider one of my first and most important
responsibilities to get this program fixed and working so that the
retirement experience for government employees is as simple and
seamless as current technology allows.
To ensure success moving forward, I will solicit the views of experts
at Federal agencies that have undertaken similarly ambitious efforts to
modernize federal business processes and systems. Their views and
experiences may be useful to OPM in shaping the technology way ahead
for retirement systems modernization. We will discuss best practices
and pitfalls to avoid with agencies that changed their business models
and supporting systems to deliver services that are citizen-focused,
agile, responsive and accessible. This infusion of input from agencies
with expertise and experience with similarly large systems, combined
with a planned request for information to the private sector will
enhance OPM's ability to incorporate innovative approaches to our
efforts to modernize and improve retirement services for federal
retirees.
In addition, OPM has engaged agencies to be part of the solution, i.e.
to improve the quality and completeness of retirement applications
submitted to OPM. With them as full participants, they are far more
likely to buy in to the concept of shared responsibility for the
retirement process. Awareness and appreciation has increased to the
point that it is now clearly recognized as a joint endeavor among the
employee's agency at retirement, the employee, and OPM. One example of
this was a Retirement Symposium OPM hosted for more than 175 human
resource and payroll employees from throughout the federal government
on January 28th, 2009. In addition to the Symposium, OPM continues to
work one-on-one with agencies to improve their individual processes
based on the data collected on their retirement applications in the
2008 quality audit. Following an initial quality audit review meeting,
agencies developed and submitted to OPM action plans to improve quality
based on feedback from OPM quality audits. OPM is developing plans for
regular retirement application quality audits to ensure continued
feedback to agencies to help them make improvement on their retirement
application submissions.
OPM has reached out to over 40 organizations by conducting kick-off
meetings and working sessions on a series of activities designed to
trigger data gathering on the agency's systems, data, and business
structure; discussions on secure electronic data transfer; and data
requirements. The bottom line is that OPM has established partnerships
with the agencies to ensure their support and buy-in moving forward.
I agree with the characterization that meaningful oversight is needed
and that is the first issue addressed this past November with
designation of the new Project Manager, Deputy Project Manager and a
reconvened Executive Steering Committee. I want to assure you, however,
that the shortcomings identified within retirement systems
modernization are not indicative of all acquisition programs at OPM.
Moreover, it is important to emphasize that, late in 2007 and early in
2008, there was a determination to directly align the day-to-day
oversight of the program under the Director to ensure critical
milestones could be closely monitored and met. Unfortunately, the
desire to achieve success within an overly ambitious timeframe resulted
in shortcuts that proved to be somewhat short-sighted. Nonetheless,
this agency takes GAO's findings very seriously and is addressing each
of the six recommendations with specific actions as outlined below:
I. Complete Project Plan:
OPM recognizes the value of strong program management plans and through
implementation of previous GAO recommendations, the retirement systems
modernization program developed a robust set of program planning and
management documentation. The program recognizes that the current
baselined versions of the program documents do not fully reflect the
state of the program, as they were based on a single Defined Benefits
Technology Solution (DBTS) with supporting implementing vendors. OPM
began the process of updating the documents in December 2008.
Program documentation for retirement systems modernization includes a
program management plan (PMP) that defines the scope and objectives of
retirement systems modernization. It also describes the overall
governance structure for retirement systems modernization, including
lines of responsibility and authority, and documents the complete set
of management processes for the program. This set of management
processes includes planning and earned value management, funds
management, contract management, configuration management, and
integration with OPM's Information Technology System Manager (ITSM)
system development methodology. Also included are risk management,
requirements management, quality management, communication and security
management. In a number of these areas, such as, risk management and
requirements management, further details are provided in separate
documents referenced in the PMP.
A host of other program documents are fundamentally linked to the PMP
and provide further structure to the program, such as the program
concept of operations (CONOPS), the Work Breakdown Structure (WBS) and
Integrated Master Schedule. Examples of these linkages are the
inclusion of significant milestones from the program's Integrated
Master Schedule in the PMP, and the program's five phase implementation
approach which is detailed in the CONOPS and described in the PMP.
The PMP and associated documents are currently undergoing a thorough
revision based on a change in program approach and updates to the
processes and documents described above. The table below provides the
list of documents that are undergoing revisions:
Program Management Documentation: Concept of Operations;
Date of Projected Completion: 2/20/07;
Current Version of Update: May 2009 (in development).
Program Management Documentation: Configuration Management Process
Guide;
Date of Projected Completion: 6/25/07;
Current Version of Update: April 2009 (completed).
Program Management Documentation: Program Management Plan;
Date of Projected Completion: 12/4/07;
Current Version of Update: June 2009 (in development).
Program Management Documentation: Requirements Management Plan;
Date of Projected Completion: 6/25/07;
Current Version of Update: May 2009 (in coordination).
Program Management Documentation: Risk Management Plan;
Date of Projected Completion: 12/4/07;
Current Version of Update: May 2009 (in coordination).
Program Management Documentation: Program Cost Estimate;
Date of Projected Completion: 12/19/08;
Current Version of Update: June 2009 (in development).
Program Management Documentation: Test Management Plan;
Date of Projected Completion: 10/16/07;
Current Version of Update: June 2009 (in development).
Program Management Documentation: Work Breakdown Structure (WBS);
Date of Projected Completion: 3/3/09;
Current Version of Update: March 2009 (completed).
Program Management Documentation: WBS Dictionary;
Date of Projected Completion: 7/17/07;
Current Version of Update: May 2009 (in development).
[End of table]
As outlined above, updates to the PMP are ongoing and are not likely to
be completed until the other underlying documents are finalized, due to
the linkages among them. I anticipate this process to be completed by
June 2009. OPM is currently developing goals which tie to the
capabilities the program is working to make a reality. Establishing
results-oriented program goals and measures requires coordinating with
multiple stakeholders within OPM. This ensures harmonization with
overall agency strategic and operational goals, which are developed
under their own timetables, and as a result, are anticipated to take
longer.
When the updates currently underway are completed, and with the input
we are seeking from other agencies and the private sector, I am
confident that the retirement systems modernization program will have a
solid foundation to guide its efforts and be properly positioned to
deliver modernized retirement capabilities and systems.
II. Reliable Cost Estimate:
OPM recognizes the importance of a comprehensive, well-documented,
accurate and credible cost estimate for managing the retirement systems
modernization program. Previous cost estimates were based primarily on
the value of vendor, firm-fixed price contracts (which comprised the
vast majority of program costs), rather than a build-up of program
costs. This was deemed to be the most accurate approach at the time.
However, given the substantial changes to the program in the past year,
and recognition of the benefits of a more rigorous approach to cost
estimation, the weaknesses in previous estimates are being actively
addressed. Specifically, OPM is using the GAO Cost Estimating and
Assessment Guide (published in March 2009) and on-going program efforts
to develop a comprehensive CONOPS and the program integrated master
schedule (IMS) as a guide to developing a comprehensive cost estimate
for the program.
The cost estimation work is part of a broader business case analysis
that is re-examining all aspects of the program and engaging
stakeholders throughout OPM to ensure that program replanning provides
the proper foundation for the program. The GAO Cost Estimating and
Assessment Guide are being used during every stage of the business case
analysis and will complete each of the twelve recommended steps. OPM
has made considerable progress in the initial stages of developing the
cost estimate, including:
* Define estimate purpose The purpose of the estimate is to inform
budget planning for the program, inform the justification for continued
investment in the program, and provide the basis for measuring program
performance. The level of detail has been established at level 4 in the
program work breakdown structure (WBS), and the overall scope of the
estimate matches the scope laid out in the draft program concept of
operations (CONOPS) and is consistent with OPM's 5 phase implementation
approach.
* Develop estimating plan - The cost estimating team was established in
mid-February and has developed the master schedule and associated
timeline for completion of the estimate. The estimating approach is
being tailored to specific program initiatives. Preparations to conduct
the independent cost estimate are underway.
* Define program characteristics The program has been developing
documentation such as an updated CONOPS and has developed an RFI, the
results of which will help to inform the technical baseline
description. Additionally, interviews with program stakeholders have
collected a wealth of information on program needs, the current
environment, alternatives for going forward, and the associated risks.
* Determine estimating structure - The estimating structure for the
cost estimate is the updated program WBS which was approved by the
program change control board in early March. The WBS divides program
work into phases and initiatives and work packages detail the full
system development lifecycle for each initiative. Leveraging the
approved WBS as the structure for the cost estimation will ensure an
end-to-end cost estimate for the program that reflects dependencies
across initiatives. Development of the program WBS dictionary is
underway to ensure clear understanding of each element of the WBS and
appropriate classification of work and resources within the program's
Integrated Master Schedule (IMS). Specific technical components of each
initiative are being outlined in the CONOPS and will provide additional
detail for the cost estimate.
The work underway will position us to establish a credible,
comprehensive, and defensible cost estimate for retirement systems
modernization in the coming months. OPM is working with internal and
external stakeholders to ensure all parties are in agreement with the
approach for developing the cost estimate. It is clear that a good cost
estimate is a living artifact and that processes must be developed to
update the cost estimate throughout the life of the program in order to
maintain its accuracy over time.
III. Earned Value Management:
OPM agrees with GAO on the importance of Earned Value Management (EVM)
for measuring and reporting program progress. As a measure of the
importance OPM places on reliable EVM reporting, the deficiency of
quality EVM data was a major factor in the decision to stop work on the
DBTS contract. Retirement system modernization has invested
considerable resources in EVM processes and has required vendors to
provide detailed EVM data for measuring program progress. Cessation of
work on the DBTS contract rendered it impossible to continue program-
level EVM reporting.[Footnote 19] Given these deficiencies, OPM was
unable to conduct an integrated baseline review as recommended by GAO
in its January 2008 report.
OPM recognizes the value of a reliable cost estimate and is currently
developing one in order to establish a new program baseline upon which
to report future progress. In the interim, OPM is restarting EVM
reporting based on an updated Integrated Master Schedule (IMS) as a
means to capture program status information and to acclimate program
vendors and business owners to the process of providing and statusing
program EVM data. This is a significant effort requiring frequent
meetings with initiative owners and other OPM stakeholders to decompose
work packages into constituent tasks and identifying cross-program
dependencies. OPM recognizes that the value of the resulting data will
be limited until the cost estimate is complete and the program can
formally establish the performance measurement baseline (PMB). As such,
OPM will be working to baseline the program throughout the cost
estimation process outlined above. Program EVM status for FY2009 and
FY2010 will be statused and reported on a monthly basis while OPM
completes the cost estimation and establishes a comprehensive, credible
PMB. OPM plans to execute an integrated baseline review as soon as
reasonably possible after completing the cost estimate and establishing
the PMB.
OPM has invested considerable effort in creating the artifacts
necessary for accurate EVM reporting, having developed an IMS with over
1,000 activities. At the same time, OPM has updated and refined other
program artifacts, such as the program CONOPS. Given the concurrent
development of these artifacts, OPM has worked to ensure that updates
in one document are applied across the full set of program artifacts,
through continuous communication within the project team, thus
minimizing gaps or conflicting information.
In addition, OPM is finalizing a briefing on the EVMS to share with
program stakeholders and initiative owners in the next month. This
presentation will review the expectations for EVM statusing and
reporting and the importance of robust EVM. Additionally, the PMO will
provide training to federal and vendor initiative owners who will be
expected to provide status on a monthly basis and will be responsible
for developing variance reports.
IV. Requirements Management:
OPM supports the implementation of recognized best practices to develop
and manage detailed retirement modernization requirements. Previous
requirements development efforts focused on tailoring requirements to
the Defined Benefits Technology Solution (DBTS), resulting in
requirements that were oriented to commercial-off-the-shelf software,
but which lacked specific detail about complex federal retirement
business processes. As GAO noted, lacking an adequate degree of
specificity, previous OPM requirements efforts were unable to trace
software specifications and functionality to the complex federal
retirement business processes the technology solutions were designed to
support. This lack of traceability was one factor that contributed to
the unsuccessful DBTS technology solution. Current requirements
management efforts are based on industry best practices and technology
standards and clearly assign responsibilities across the organization.
They also establish repeatable procedures for all phases of
requirements process and ensure the management of the requirements
baseline through standard Configuration Management (CM) and
Configuration Control Board (CCB) processes.
Leveraging Carnegie Mellon's Software Engineering Institute (SEI)
Capability Maturity Model Integration (CMMI)-Development v1.2 and
Institute of Electrical and Electronics Engineers (IEEE) Standard
1233 - 1998, IEEE Guide for Developing System Requirements
Specifications, OPM has developed a requirements management framework
and procedures which will ensure future clarity, traceability, and
accuracy. The CMMI-Development framework provides standard process
steps for Requirements Management. Also, OPM's Information Technology
Systems Manager (ITSM) incorporates both CMMI and IEEE requirements
standards as part of OPM's System Development Life Cycle. These steps
have been incorporated into the Retirement Requirements Management
process as follows.
* Develop Business Requirements - The operational concepts captured in
the Concept of Operations (CONOPS) will be articulated as business
processes, data and requirements. These requirements are not technical
but represent the operational user's views of the business processes.
Retirement systems cannot be developed from these requirements as they
lack technical specificity. Once these requirements are documented and
validated by the stakeholders, they will be further deconstructed into
technical system requirements.
* Develop System Specifications - User business requirements need to be
translated into product requirements articulated in technical terms
before system design can begin. The system specifications transform the
user requirements into technical language, detailed enough to support
system design. The system requirements are then organized by system
components. An integral part of the development of these system
specifications is the elicitation of interface requirements for
connecting related systems with each other so that they are
interoperable and can exchange data.
* Testing - Test procedures are developed to verify each system
specification and that system function meets the business need stated
in the requirements. OPM is in the process of implementing a Test
Management Plan to ensure that test procedures verify that developed
systems meets the documented system and business requirements. The test
procedures will be tracked as part of the requirements management
process in order to ensure that the requirements they represent can be
tested and verified.
* Requirements Management and Traceability - The goal of Requirements
Management is to ensure that requirements are elicited, developed and
tracked in a systematic manner and managed through standard
Configuration Management and Configuration Control Board processes.
Requirements traceability provides a method to link business
requirements and technical specifications in order to validate that the
system design meets the user's business requirements.
OPM acknowledges that previous requirements management efforts were
geared specifically to the implementation of a single commercial system
and did not describe complex federal retirement business processes and
requirements to the necessary level of detail. Thus, the CMMI-
Development (Requirements Management) Framework and processes will
enable OPM to develop business requirements and specifications, based
on an approved Concept of Operations, that reliably produce modem
systems and system components that meet the Retirement program's
business needs.
V. Test Planning:
OPM recognizes that effective testing, based on established standards
and best practices, is an essential component in the delivery of
federal Retirement capabilities and systems. System testing can
identify defects early enough in the development process to allow for
corrections before deployment, verify that the developed system matches
system requirements and specifications, and validate that the system
meets user needs. In the case of Retirement Modernization testing,
basic test management plans and processes were in place to guide and
evaluate system testing. OPM test management processes generally
conformed to established software engineering standards. However, a
number of factors impacted the timely execution of testing and
correction of system defects. These factors contributed to a deployed
system which did not fully deliver required Retirement Modernization
capabilities.
The Retirement Modernization program issued a Retirement System
Modernization Test Approach and User Acceptance Testing (UAT) Execution
Plan in 2007 with updates in early 2008. In line with testing best
practices described in Institute of Electrical and Electronics
Engineering (IEEE) standard 829, IEEE Standard for Software Test
Documentation, the OPM testing approach included test scope and
methodology, timeline, test entry and exit criteria, guidance for
defect tracking and provisions for independent validation and
verification of test results.
As GAO notes, the project test plans were issued midway through the
development cycle, rather than at the inception of the process, thus
compressing the time allocated for testing into an extremely narrow
window. Complicating the shortened testing timeline, system
requirements and system specifications that drove development did not
capture, at a detailed level, a number of highly complex federal
retirement business processes required to fully deliver clean data
required for successful annuity calculations. Additionally, interface
specifications and reporting requirements were still in the development
phase three months before the system deployed, leaving the delivered
system unable to pass data to legacy systems critical to retiree and
annuitant services. Due to the compressed testing schedule, system
component testing was conducted in parallel or concurrently vice
sequentially, which did not allow developers to identify
interoperability issues or defects among the various system components.
As the testing cycle proceeded, using structured scenarios, system
defects were identified and documented and defect tracking and
assessments provided as the Test Plans required. Given the compressed
test window and the complexity of implementing a commercial system for
the federal sector, the system deficiencies could not be effectively
resolved prior to the system deployment date. The DBTS was deployed
with reduced initial operating capability, but with the hope that
further rapid development would follow. The system was deployed and
subsequently manual intervention was required to augment retirement
calculations.
OPM acknowledges that planning test activities early in the retirement
systems modernization lifecycle in conjunction with the requirements
development process will potentially reduce future system defects.
While the DBTS Test Approach and User Acceptance Test Execution Plan
were tailored specifically to the DBTS solution, OPM will release an
updated Retirement Systems Modernization Test Management Plan,
consistent with best practices and industry standards, and fully
correlated with the Requirements Management Plan. Retirement systems
modernization test activities will be included in the Project
Management Plan, and test activities will be fully integrated with the
requirements development process. OPM is confident that planning and
integrating test management activities with requirements development
will preclude the level of defects experienced in previous development
efforts and result in retirement systems modernization capabilities
that are fully consistent and supportive of federal retirement
services.
VI. Policy/Procedures for Program Oversight:
OPM has established different types of oversight groups to ensure that
acquisition programs remain on track to meet the requirements, cost,
and schedule aligned to the selected programs. For retirement systems
modernization, OPM leadership initially determined that an Executive
Steering Committee would be the proper oversight group for the
retirement system modernization program. Over the course of time,
however, it was determined that the program office would report
directly to the OPM Director to streamline the process and to
accelerate the program. Therefore, the GAO report is correct in saying
that the ESC was suspended during the period of January 2008 to October
2008.
With the change in leadership at OPM in August 2009, the new Acting
Director who headed the agency at that time quickly established a
senior level working group (Sunrise Group) to oversee the management of
the retirement modernization program. To quickly get the retirement
program back on track, the Sunrise Group established a series of
actions and each stakeholder took responsibility for progress on their
respective actions. Based on this oversight, many actions quickly
occurred, including increasing the number of personnel at the call
center, engagements with the shared service centers, and the imaging of
documents at the Retirement Operations Center, to name a few. After an
extensive Federal search a program manager and a deputy program manager
were selected to take responsibility for the day-to-day operation of
the program. One of the first duties of the program manager was to
craft a revised charter for the Executive Steering Committee to take
over direct oversight of the program. The committee is made up of
senior level (SES) stakeholders that meet on monthly basis, in addition
to the weekly meetings that continue.
OPM also has an Information Technology (IT) Investment Review Board
ORB) which was established according to Clinger-Cohen Act requirements
to advise the agency head on issues related to the selection and
evaluation of IT investments. The Board, which includes as its members
senior executives of the agency, met consistently from 2005 to evaluate
investments and recommend approval or disapproval of funding. In
addition, the Board members received independent assessments of
investment status, including the status of retirement system
modernization. Board members, who met routinely with the agency
Director in their capacities as her direct reports, provided feedback
and advice to the Director, influenced by the independent assessments
as well as the project manager's reports provided at meetings of the
Board. Because the Clinger-Cohen Act assigns responsibility for capital
planning and investment control and other requirements of the Act to
the agency head, OPM's Investment Review Board advised the Director,
but the IRB was not established to decide how the agency would resolve
concerns and risks associated with the retirement system modernization
program that were brought to its attention. Such decisions were made by
the Director per the Clinger-Cohen Act.
In addition, OPM specifically submits the following comments for GAO's
consideration:
1. "Employing agency's" should be added to clarify the first sentence
on page 7 of the draft report, thus changing it to read, "Once an
employee submits an application, the employing agency's human resources
office provides retirement counseling services to the employee and
augments the retirement application with additional paperwork.
2. Regarding the statement at the bottom of page 7 of the draft report,
"Payroll offices are required to submit the package to OPM within 30
days of the retiree's separation date.", we recommend clarifying that
this is currently only a timeliness standard for this information. This
standard is being augmented by the Agency Retirement Record Improvement
Plan which emphasizes the shared responsibility between OPM and
agencies to have complete and accurate information on which to base the
adjudication.
3. Regarding the statement in the 9th line of the 2nd paragraph on page
8, "Retirement processing includes functions such as determining
retirement eligibility, inputting data into benefit calculators, and
providing customer service," we wish to emphasize that the referenced
customer service is in relation to the initial adjudication. This does
not include the customer service group included in the call centers
that provide service to existing annuitants and survivors.
4. Regarding the heading on page 23, we recommend modifying the section
header to refer to retirement systems modernization to be consistent
with the content of this section and the scope of the report. In this
section of the report, the authors refer to the application of Earned
Value Management (EVM) techniques in the retirement systems
modernization program. However, OPM has successfully managed several IT
investments using EVM techniques. Those projects and the agency's EVM
program as a whole were not addressed in this study and are not the
subject of this report. Therefore, OPM requests that you change the
section header to more explicitly articulate your concerns about the
implementation of EVM in the retirement systems modernization program,
since the concerns you have raised are related to retirement system
modernization rather than the agency as a whole.
5. Regarding the section title in the middle of page 24, this title
similarly extrapolates from the retirement modernization analysis to an
agency-wide issue. OPM recommends that this wording be changed to
reflect areas of concern specifically related to requirements
management for retirement modernization. In addition, as referenced
above, OPM does have an established system development life cycle
(SDLC) process called IT Systems Manager (ITSM) that has clear,
established processes for development and management of requirements.
6. Similarly, OPM recommends modifying the statement in the middle of
page 25 that currently reads, "Additionally, agency officials stated
that OPM is developing a requirements development process but did not
specify when it will be finalized." to indicate that the requirements
process under development is specifically related to retirement
modernization. OPM has a requirements development process included in
the IT Systems Manager (ITSM) that has been in existence since 2003.
Thank you for this opportunity to comment on GAO's draft report. I am
excited about meeting the challenges that lie ahead and look forward
GAO's guidance in addressing them.
Sincerely,
Signed by:
John Berry:
Director:
[End of section]
Appendix III: GAO Staff Contact and Acknowledgments:
GAO Contact:
Valerie C. Melvin (202) 512-6304 or melvinv@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, key contributions to this
report were made by Mark T. Bird, Assistant Director; Barbara S.
Collier; Neil J. Doherty; David A. Hong; Thomas J. Johnson; Rebecca E.
LaPaze; Lee A. McCracken; Teresa M. Neven; Melissa K. Schermerhorn;
Donald A. Sebers; and John P. Smith.
[End of section]
Footnotes:
[1] GAO, Office of Personnel Management: Improvements Needed to Ensure
Successful Retirement Systems Modernization, GAO-08-345 (Washington,
D.C.: Jan 31, 2008).
[2] The Social Security Administration is responsible for administering
Social Security, and the Federal Retirement Thrift Investment Board
administers the defined contribution system known as the Thrift Savings
Plan. Defined benefit plans calculate benefit amounts in advance of
retirement based on factors such as salary level and years of service,
and defined contribution plans calculate benefit amounts based on how
the amount is invested by the employee and employer.
[3] OPM, An Analysis of Federal Employee Retirement Data: Predicting
Future Retirements and Examining Factors Relevant to Retiring from the
Federal Service (March 2008).
[4] OPM reported in November 2008 that it has made improvements to this
process and is currently providing retirees with interim payments that
are about 90 percent of the monthly payment which they are entitled.
[5] OPM Retirement Insurance Service, FERS Automated Process System
Information Technology Resources Board Review (Washington, D.C., July
16, 1996). Executive Order 13011, Section 5 (July 1996), established
that the Information Technology Resources Board was to provide
independent assessments to assist in the development, acquisition, and
management of selected major information systems and to provide
recommendations to agency heads and OMB as appropriate. Executive Order
13011 was revoked by Executive Order 13403 (May 2006).
[6] GAO, Office of Personnel Management: Retirement Systems
Modernization Program Faces Numerous Challenges, GAO-05-237
(Washington, D.C.: Feb. 28, 2005).
[7] GAO-08-345.
[8] GAO, Comments on the Office of Personnel Management's February 20,
2008 Report to Congress Regarding the Retirement System Modernization,
GAO-08-576R (Washington, D.C.: Mar. 28, 2008).
[9] Institute of Electrical and Electronics Engineers (IEEE), IEEE/EIA
Guide for Information Technology, IEEE/EIA 12207.1-1997 (April 1998).
[10] OMB, Circular No. A-11, Preparation, Submission, and Execution of
the Budget (Washington, D.C., Executive Office of the President, June
2008); Circular No. A-130 Revised, Management of Federal Information
Resources (Washington, D.C., Executive Office of the President, Nov.
28, 2000); and Capital Programming Guide: Supplement to Circular A-11,
Part 7: Planning, Budgeting, and Acquisition of Capital Assets
(Washington, D.C., Executive Office of the President, June 2006).
[11] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, GAO-09-3SP (Washington,
D.C.: March 2009).
[12] GAO-08-345.
[13] OMB issued policy guidance (M-05-23) to agency chief information
officers on improving technology projects that includes requirements
for reporting performance to OMB using EVM (August 2005). OPM, Earned
Value Management Policy (June 2005).
[14] GAO-08-345.
[15] Carnegie Mellon Software Engineering Institute, Capability
Maturity Model® Integration for Development, Version 1.2 (Pittsburgh,
Pa., August 2006), and Software Acquisition Capability Maturity Model®
(SA-CMM®) version 1.03, CMU/SEI-2002-TR-010 (Pittsburgh, Pa., March
2002); and the Institute of Electrical and Electronic Engineers (IEEE),
1362-1998, IEEE Guide for Information Technology--System Definition--
Concept of Operations Document (New York, N.Y.,1998).
[16] A concept of operations is a user-oriented document that describes
the characteristics of a proposed system from the users' viewpoint.
[17] Maintaining bidirectional requirement traceability means that
system-level requirements can be traced both backward to higher level
business or operational requirements, and forward to system design
specifications and test plans.
[18] See for example, Carnegie Mellon Software Engineering Institute,
Software Acquisition Planning Guidelines, CMU/SEI-2005-HB-006
(Pittsburgh, Pa., December 2005); IEEE Standard 829-2008, Standard for
Software and System Test Documentation (July 2008); and IEEE Standard
1044-1993, Standard Classification for Software Anomalies (December
1993).
[19] Project level EV reporting for the program's PDCC vendor continued
without interruption and continues to be used to track project
performance.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
