Records Management
National Archives and Records Administration's Acquisition of Major System Faces Risks Gao ID: GAO-03-880 August 22, 2003Increasingly, government records involve documents that are electronically created and stored. In support of its mission to manage and archive these records and ensure access to the "essential evidence" that they contain, the National Archives and Records Administration (NARA) is acquiring an advanced Electronic Records Archives (ERA). GAO was asked to determine, among other things, how the ERA program's system acquisition policies, plans, and practices conform to industry standards and how well NARA is meeting the ERA program's cost and schedule.
The ERA program's acquisition policies, plans, and practices do not consistently conform to industry standards. In developing the plans and policies to guide its acquisition of the ERA system, NARA elected to follow recognized industry standards set forth by the Institute of Electrical and Electronics Engineers (IEEE). However, key policy and planning documents are missing elements that are required by the standards. For example, one key document is the concept of operations, which should describe the characteristics of a proposed system from the users' viewpoint. The ERA Concept of Operations does not include several key elements required by the IEEE standard, including a complete description of the proposed system. Because these policy and planning documents form the basis of the acquisition, such shortcomings could result in serious long-term risks to the cost, schedule, and performance of the ERA program. NARA cannot adequately track the cost and schedule of the ERA program. A comprehensive schedule with an appropriate work breakdown structure is a prerequisite to program tracking, as it allows managers to measure how well the program is achieving its cost and schedule goals. To achieve upcoming major milestones, the program must successfully complete a complex series of tasks. However, the program schedule omits significant tasks and activities; for example, it does not include the process to reengineer the agency's life cycle business processes, which will be crucial to defining requirements. In addition, the schedule lacks a work breakdown structure, which would allow accurate estimates of the resources and time required for each work activity. If NARA cannot track how well the program is meeting cost and schedule, the risk is increased that funds may not be used efficiently or effectively, quality problems may limit the usefulness of the resulting system, and the system may not be delivered according to established milestones.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-03-880, Records Management: National Archives and Records Administration's Acquisition of Major System Faces Risks
This is the accessible text file for GAO report number GAO-03-880
entitled 'Records Management: National Archives and Records
Administration's Acquisition of Major System Faces Risks' which was
released on August 22, 2003.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Transportation, Treasury and
Independent Agencies, Committee on Appropriations, House of
Representatives:
On January 2, 2004, this document was revised to add various
footnote references missing in the text of the body of the document.
August 2003:
RECORDS MANAGEMENT:
National Archives and Records Administration's Acquisition of Major
System Faces Risks:
GAO-03-880:
GAO Highlights:
Highlights of GAO-03-880, a report to the Chairman, Subcommittee on
Transportation, Treasury and Independent Agencies, Committee on
Appropriations, House of Representatives
Why GAO Did This Study:
Increasingly, government records involve documents that are
electronically created and stored. In support of its mission to manage
and archive these records and ensure access to the ’essential
evidence“ that they contain, the National Archives and Records
Administration (NARA) is acquiring an advanced Electronic Records
Archives (ERA). GAO was asked to determine, among other things, how
the ERA program‘s system acquisition policies, plans, and practices
conform to industry standards and how well NARA is meeting the ERA
program‘s cost and schedule.
What GAO Found:
The ERA program‘s acquisition policies, plans, and practices do not
consistently conform to industry standards. In developing the plans
and policies to guide its acquisition of the ERA system, NARA elected
to follow recognized industry standards set forth by the Institute of
Electrical and Electronics Engineers (IEEE). However, key policy and
planning documents are missing elements that are required by the
standards. For example, one key document is the concept of operations,
which should describe the characteristics of a proposed system from
the users‘ viewpoint. The ERA Concept of Operations does not include
several key elements required by the IEEE standard, including a
complete description of the proposed system. Because these policy and
planning documents form the basis of the acquisition, such
shortcomings could result in serious long-term risks to the cost,
schedule, and performance of the ERA program.
NARA cannot adequately track the cost and schedule of the ERA program.
A comprehensive schedule with an appropriate work breakdown structure
is a prerequisite to program tracking, as it allows managers to
measure how well the program is achieving its cost and schedule goals.
To achieve upcoming major milestones (some of which are shown in the
figure), the program must successfully complete a complex series of
tasks. However, the program schedule omits significant tasks and
activities; for example, it does not include the process to reengineer
the agency‘s life cycle business processes, which will be crucial to
defining requirements. In addition, the schedule lacks a work
breakdown structure, which would allow accurate estimates of the
resources and time required for each work activity. If NARA cannot
track how well the program is meeting cost and schedule, the risk is
increased that funds may not be used efficiently or effectively,
quality problems may limit the usefulness of the resulting system, and
the system may not be delivered according to established milestones.
What GAO Recommends:
To reduce the risks associated with NARA‘s efforts to design and
acquire ERA, GAO recommends that the U.S. Archivist direct the NARA
Chief Information Officer to take a range of actions, including
revising key planning documents and developing a schedule that is
based on a comprehensive work breakdown structure (including
associated costs and other resources).
In comments on the draft report, the Archivist of the United States
accepted our recommendations and provided an update on NARA‘s efforts
to implement them. The Archivist also provided additional information
on the ERA acquisition schedule.
www.gao.gov/cgi-bin/getrpt?GAO-03-880.
To view the full product, including the scope and methodology, click
on the link above. For more information, contact Linda Koontz at (202)
512-6240 or koontzl@gao.gov.
[End of section]
Contents:
Letter:
Recommendations for Executive Action:
Agency Comments:
Appendixes:
Appendix I: National Archives and Records Administration's Acquisition
of Electronic Records Archives:
Appendix II: Comments from the National Archives and Records
Administration:
Abbreviations:
ERA: Electronic Records Archives:
IEEE: Institute of Electrical and Electronics Engineers:
IT: information technology:
NARA: National Archives and Records Administration:
Letter August 22, 2003:
The Honorable Ernest J. Istook, Jr.
Chairman
Subcommittee on Transportation, Treasury and Independent Agencies
Committee on Appropriations
House of Representatives:
Dear Mr. Chairman:
The National Archives and Records Administration (NARA) has initiated
the Electronic Records Archives (ERA), a project to acquire a major
information system to maintain and provide access to permanent federal
records independent of the technological state of the art and the
varieties of record formats. NARA's goal is for this system to preserve
and provide access to any kind of electronic record, so that the agency
can carry out its mission into the future. However, as we have reported
previously,[Footnote 1] acquiring a major information technology (IT)
system like ERA is a significant challenge for a relatively small
organization such as NARA, which has no previous experience in
acquiring major information systems.
Our objectives were to determine:
1. the status of NARA's efforts to establish organizational
capabilities for acquiring major information systems,
2. how the ERA project's system acquisition policies, plans, and
practices conform to industry standards, and:
3. how well NARA is meeting the ERA project's cost and schedule goals.
To achieve these objectives, we reviewed agency information technology
policies and practices, and we obtained and analyzed ERA program
documents on system acquisition, project management, and cost and
schedule. We evaluated ERA documents and practices by the standards
selected by the program to guide the ERA acquisition, including
specifically those of the Institute of Electrical and Electronics
Engineers (IEEE). We also interviewed NARA information resources
management and ERA program officials. We performed our work from July
2002 to May 2003 in accordance with generally accepted government
auditing standards.
On May 15, 2003, we provided your staff with a briefing on the results
of our study, which included procurement-sensitive information. The
slides from that briefing--with procurement-sensitive information
removed--are included as appendix I to this report. The purpose of this
report is to provide the published briefing slides to you and to
officially transmit our recommendations to the Archivist of the United
States.
In summary, our briefing made three points:
* To establish its capabilities for acquiring major information
systems, NARA has made progress in implementing the key management
areas of IT investment management, enterprise architecture, and IT
security. However, these capabilities are not yet completely
established, and NARA has more work to do to implement our prior
recommendations in this area.[Footnote 2] Specifically, while NARA
continues to develop an enterprise architecture, it does not plan to
complete its target architecture[Footnote 3] in time to influence the
ERA system definition and requirements. Furthermore, while NARA has
completed some elements of an information security program, several key
areas have not yet been addressed, such as (1) individual system
security plans and (2) security certification and accreditation of its
information systems.[Footnote 4] Without strong IT management
capabilities, NARA increases its risk of failing to achieve cost,
schedule, and performance objectives for its information systems,
including ERA.
* The ERA program has developed policies, plans, and practices to guide
and manage its acquisition of the ERA system. In many cases, however,
these do not conform to the chosen standards or to applicable federal
acquisition guidance. In developing its plans and policies, NARA
elected to follow recognized industry standards set forth by IEEE.
However, key policy and planning documents are missing elements that
are required by the standards. For example, one key document is the
concept of operations, which should describe the characteristics of a
proposed system from the users' viewpoint. The ERA Concept of
Operations does not include several key elements required by the IEEE
standard, including a complete description of the proposed systems. In
addition, key ERA staff positions are unfilled, including positions
that NARA determined are needed to carry out system acquisition tasks.
Without adequate policy and planning documents--which form the basis of
the acquisition--and adequate staff to carry out these policies and
plans, NARA increases the long-term risks to the acquisition.
* Finally, NARA cannot adequately track the cost and schedule of the
ERA project because the schedule does not include all program tasks and
lacks a work breakdown structure.[Footnote 5] In addition, NARA has not
used earned value management--a performance-based technique that allows
managers to track the budget against the schedule--to track the ERA
cost and schedule programwide. Without the ability to track cost and
schedule effectively, NARA increases the risk that ERA funds will not
be used efficiently or effectively, quality problems will limit the
usefulness of the ERA system, and the ERA system will not be delivered
according to established milestones.
In light of the challenges NARA faces in acquiring ERA, NARA will face
significant difficulties unless it addresses the weaknesses described
above.
Recommendations for Executive Action:
To reduce the risks associated with NARA's efforts to design and
acquire the Electronic Records Archives, we recommend that the U.S.
Archivist direct the NARA Chief Information Officer to address
weaknesses in the acquisition policies, plans, and practices by:
* revising the ERA Life Cycle document and associated procedures and
practices to conform to IEEE standards;
* revising the ERA Concept of Operations to conform to IEEE standards,
including a complete description of the current and proposed systems;
* revising the ERA Acquisition Strategy to conform to IEEE standards
and the Federal Acquisition Regulation;
* revising the ERA Risk Management Plan to provide processes and
procedures specific to the ERA program;
* revising the ERA Quality Assurance Plan to conform to appropriate
industry standards, establishing a vigorous, independent ERA quality
assessment process, and providing the staffing resources necessary to
ensure that quality assessment duties are performed effectively; and:
* filling key vacant ERA positions.
Further, we recommend that the U.S. Archivist direct the NARA Chief
Information Officer to immediately address weaknesses in tracking cost
and schedule by:
* developing an ERA schedule that is based on a comprehensive work
breakdown structure (including associated costs and other resources)
and establishes dependencies between successor and predecessor tasks;
and:
* using earned value management to capture and monitor progress for the
entire ERA program.
Agency Comments:
In providing written comments on a draft of this report (which are
reprinted in app. II), the Archivist of the United States indicated
that NARA is acting to implement our recommendations and provided an
update on the status of the agency's efforts to do so. In addition, the
Archivist provided a clarification regarding the ERA acquisition
schedule, stating that there will be two to three releases for each of
the increments in the schedule.
We are sending copies of this report to the Chairmen and Ranking
Minority Members of the Subcommittee on Transportation, Treasury and
Independent Agencies, House Committee on Appropriations; the
Subcommittee on Transportation, Treasury and General Government, Senate
Committee on Appropriations; the Subcommittee on Technology,
Information Policy, Intergovernmental Relations and the Census, House
Committee on Government Reform; and the Subcommittee on Oversight of
Government Management, the Federal Workforce and the District of
Columbia, the Senate Committee on Governmental Affairs. We are also
sending copies to the Archivist of the United States. We will also make
copies available to others upon request. In addition, the report will
be available at no charge on the GAO Web site at [Hyperlink,
www.gao.gov.] www.gao.gov.
Should you have any question on matters contained in this report,
please contact me at (202) 512-6240 or by E-mail at [Hyperlink,
koontzl@gao.gov] koontzl@gao.gov. Other key contributors to this report
were Timothy Case, Barbara Collier, Mirko Dolak, and Elena Epps.
Sincerely yours,
Linda D. Koontz
Director, Information Management Issues:
Signed by Linda D. Koontz:
[End of section]
Appendixes:
Appendix I: National Archives and Records Administration's Acquisition
of Electronic Records Archives:
[See PDF for image]
[End of figure]
[End of section]
Appendix II: Comments from the National Archives and Records
Administration:
National Archives and Records Administration:
8601 Adelphi Road College Park. Maryland 20740-6001:
Ms. Linda D. Koontz:
Director, Information Management Issues General Accounting Office:
441 G Street, NW Washington, DC 20548:
Dear Ms. Koontz:
Thank you for the opportunity to review and comment on the draft report
entitled Records Management: National Archives and Records
Administration's Acquisition of Major System Faces Risks (GAO-03-880).
For NARA to carry out its mission into the future we must be successful
implementing the Electronic Records Archives (ERA) system. We
appreciate your insight into the significant challenges that we face as
a relatively small organization in acquiring major information systems.
NARA accepts the recommendations for executive action outlined in the
report, and we are already moving to implement them. We would like to
take this opportunity to update you on the status of those efforts.
Acquisition Policies, Plans, and Practices. We are conducting
verification and validation reviews of all documents as they are
updated to ensure they conform to Institute of Electrical and
Electronics Engineers (IEEE) standards. The expected completion dates
for the documents in the recommendations are:
* ERA Life Cycle: September 30, 2003 (this document will be incorporated
into the Program Management Plan based on guidance from Circular A-
119):
* ERA Concept of Operations: August 5, 2003:
* ERA Acquisition Strategy: August 5, 2003:
* ERA Risk Management Plan: September 2, 2003:
* ERA Quality Assurance Plan: completed July 23, 2003 (now titled
Quality Management Plan):
ERA Staffing. We are in the process of recruiting for the 10 remaining
unfilled ERA positions. Provided we can attract quality candidates for
these positions, we expect to have all positions filled by the end of
the calendar year.
Schedule and Work Breakdown Structure. We have underway a large effort
to develop detailed work breakdown structure and cost and schedule
baselines for the program. We expect to have those baselines in place
by September 2003.
Earned Value Management. We are aligning our program control activities
to comply with the ANSI-748A standard for earned value management
mandated by OMB in the FY 2005 capital planning process. We are
implementing a program-wide scheduling and earned value management
analysis tool that will provide program control capabilities not
currently available. The tool will be in place by January 2004.
We also have made progress since May on in a number of other areas
covered in your report.
IT Investment Management. We are updating our policy for the third and
final phase of IT investment management: select. The policy will be
finalized in August 2003.
Enterprise Architecture. We are revising the Enterprise Architecture
document that was delivered to OMB as part of our Exhibit 300 in
September 2002. Version 2.0 will be completed by September 2003 and
will be submitted as part of our Exhibit 300 for the FY 2005 budget
request. This Enterprise Architecture supports all NARA IT programs
including ERA.
IT Security Program. We are scheduled to complete the certification and
accreditation of all IT systems by September 30, 2003. We are following
the National Institute of Standards and Technology draft standard:
Guidelines for the Security Certification and Accreditation for Federal
Information Technology Systems (800-37). This effort will result in the
development of individual risk assessments and security plans for each
system. The risk assessments will provide input to the overall IT
Security Risk Management Program.
Finally, we want to clarify one point made about the ERA acquisition
schedule. The report indicates that the system will be developed in
increments and the first increment will take two years to complete.
Although this is a true statement, it is incomplete: there will be two
to three releases for each increment.
Again, thank you for this opportunity, and we look forward to
continuing to work with you throughout the ERA acquisition process.
Sincerely,
JOHN W. CARLIN
Archivist of the United States:
Signed by John W. Carlin:
[End of section]
(310371):
FOOTNOTES
[1] U.S. General Accounting Office, Information Management: Challenges
in Managing and Preserving Electronic Records, GAO-02-586 (Washington,
D.C.: June 17, 2002).
[2] U.S. General Accounting Office, Information Management: Challenges
in Managing and Preserving Electronic Records, GAO-02-586 (Washington,
D.C.: June 17, 2002).
[3] A target architecture is one aspect of an overall enterprise
architecture. An enterprise architecture describes (in useful models,
diagrams, and narrative) the mode of operation for an enterprise, such
as an agency or mission area. It provides a perspective on enterprise
operations both for the current ("as is") operating environment and for
the target ("to be") environment. More specifically, the target
environment is the business and technology environment that is planned
to result from aligning technology investments with the strategic goals
of the enterprise (including requisite changes to the operations,
organization, and management of both the automated and manual processes
of the enterprise). An enterprise architecture also includes a
transition plan for sequencing from the current to the target
environment.
[4] Under OMB policy, responsible federal officials are required to
make a security determination (called accreditation) to authorize
placing IT systems into operation. In order for these officials to make
sound, risk-based decisions, a security evaluation (known as
certification) of the IT system is needed.
[5] A work breakdown structure provides descriptions of all work
activities for a given project that are detailed enough to expose risk
factors and allow accurate estimates of resource requirements and
schedule duration for each work activity. Each major work activity
should include standard elements such as assigned personnel, resource
budgets, estimated task duration, and dependencies among work
activities. An adequate work breakdown structure is a prerequisite to
program tracking, allowing managers to measure how well a program is
achieving its cost and schedule goals.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
