Environmental Protection Agency
EPA Needs to Complete a Strategy for Its Library Network to Meet Users' Needs
Gao ID: GAO-10-947 September 30, 2010
The Environmental Protection Agency's (EPA) library network provides agency staff and the public with access to environmental information. A 2006 attempt by EPA to reorganize its network by consolidating libraries and making more materials and services available online caused concern among users, and in 2007, EPA put a moratorium on its reorganization plans. Congress requested that GAO report on the reorganization and has again requested a follow-up on these issues. Accordingly, GAO reviewed (1) the status of EPA's overall strategy for its library network, (2) the status of EPA's plan to reopen the libraries it closed and other actions planned or taken, (3) EPA's efforts to digitize printed documents to make them electronically available, and (4) EPA's efforts to communicate with staff and other stakeholders about its library network. GAO reviewed regulations and agency funding and inventory documents and interviewed EPA staff and contractors, as well as independent library professionals. GAO also assessed the reliability of EPA's data on library holdings and from EPA's staff survey on library use and needs.
Although EPA has taken a number of steps to meet the needs of library users, it has not completed a plan identifying an overall strategy for its library network, with implementation goals and a timeline of what it intends to accomplish. Scheduled for completion in 2008, the strategic plan was to provide EPA staff and the public a detailed view of EPA's library operations and future direction. The draft outline of the strategic plan, however, is largely a placeholder list of current and planned EPA activities. For example, while it emphasizes the central role to be played by electronic library resources, the draft outline does not contain goals or a timeline for completing an inventory of holdings or digitizing those holdings. The draft outline also does not set out details of how funding decisions are to be made. Given the current economic environment, without a completed strategic plan, including a detailed strategy for acquiring, deploying, and managing funding, EPA may find itself hard-pressed to ensure that the network can meet its users' needs. The agency has reopened libraries closed during reorganization, although about half the network's 10 regional libraries are operating with reduced hours. EPA has also developed standards for the regional and headquarters libraries' use of space, on-site collections, staffing, and services. The agency has also hired a national library program manager to carry out day-to-day activities and bring focus and cohesion to the network. Working closely with EPA management and library staff, the national library program manager, who is responsible for library network strategic planning, has set in motion a number of actions meant to improve library network operation and communication, including working closely with internal and external advisory boards and creating a library policy and related procedures. EPA has resumed digitizing some of its libraries' documents, although it has not inventoried the network's holdings. The agency is digitizing documents in three phases. Phase 1 was completed in January 2007, phase 2 is scheduled for completion in December 2010, and planning has begun for phase 3. Because EPA has not taken a complete inventory of its library holdings, however, it cannot determine which documents, or how many, will need to be digitized and, consequently, cannot accurately estimate the total cost of digitization or how long it will take. Since we reported on the library network reorganization in 2008, EPA has taken steps to communicate with staff and other stakeholders about its library network, including providing information about the libraries and soliciting information from library users. EPA has also made improvements to the main Internet gateway to the network, making more documents available electronically and providing better access to electronic documents and services. Nevertheless, because EPA's 2009 survey of the information needs and library use of its staff had methodological flaws--similar to those GAO identified in 2008--the agency is unlikely to obtain accurate information that would enable it to make appropriate decisions on the corrective actions that would best address library users' needs. GAO recommends, among other actions, that EPA complete its strategic plan for the library network and ensure that survey methods provide reliable data on which to base decisions. With clarifications, EPA concurred with our recommendations.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director:
John B. Stephenson
Team:
Government Accountability Office: Natural Resources and Environment
Phone:
(202) 512-6225
GAO-10-947, Environmental Protection Agency: EPA Needs to Complete a Strategy for Its Library Network to Meet Users' Needs
This is the accessible text file for GAO report number GAO-10-947
entitled 'Environmental Protection Agency: EPA Needs to Complete a
Strategy for Its Library Network to Meet Users' Needs' which was
released on November 1, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
September 2010:
Environmental Protection Agency:
EPA Needs to Complete a Strategy for Its Library Network to Meet
Users' Needs:
GAO-10-947:
GAO Highlights:
Highlights of GAO-10-947, a report to congressional requesters.
Why GAO Did This Study:
The Environmental Protection Agency‘s (EPA) library network provides
agency staff and the public with access to environmental information.
A 2006 attempt by EPA to reorganize its network by consolidating
libraries and making more materials and services available online
caused concern among users, and in 2007, EPA put a moratorium on its
reorganization plans. Congress requested that GAO report on the
reorganization and has again requested a follow-up on these issues.
Accordingly, GAO reviewed (1) the status of EPA‘s overall strategy for
its library network, (2) the status of EPA‘s plan to reopen the
libraries it closed and other actions planned or taken, (3) EPA‘s
efforts to digitize printed documents to make them electronically
available, and (4) EPA‘s efforts to communicate with staff and other
stakeholders about its library network. GAO reviewed regulations and
agency funding and inventory documents and interviewed EPA staff and
contractors, as well as independent library professionals. GAO also
assessed the reliability of EPA‘s data on library holdings and from
EPA‘s staff survey on library use and needs.
What GAO Found:
Although EPA has taken a number of steps to meet the needs of library
users, it has not completed a plan identifying an overall strategy for
its library network, with implementation goals and a timeline of what
it intends to accomplish. Scheduled for completion in 2008, the
strategic plan was to provide EPA staff and the public a detailed view
of EPA‘s library operations and future direction. The draft outline of
the strategic plan, however, is largely a placeholder list of current
and planned EPA activities. For example, while it emphasizes the
central role to be played by electronic library resources, the draft
outline does not contain goals or a timeline for completing an
inventory of holdings or digitizing those holdings. The draft outline
also does not set out details of how funding decisions are to be made.
Given the current economic environment, without a completed strategic
plan, including a detailed strategy for acquiring, deploying, and
managing funding, EPA may find itself hard-pressed to ensure that the
network can meet its users‘ needs.
The agency has reopened libraries closed during reorganization,
although about half the network‘s 10 regional libraries are operating
with reduced hours. EPA has also developed standards for the regional
and headquarters libraries‘ use of space, on-site collections,
staffing, and services. The agency has also hired a national library
program manager to carry out day-to-day activities and bring focus and
cohesion to the network. Working closely with EPA management and
library staff, the national library program manager, who is
responsible for library network strategic planning, has set in motion
a number of actions meant to improve library network operation and
communication, including working closely with internal and external
advisory boards and creating a library policy and related procedures.
EPA has resumed digitizing some of its libraries‘ documents, although
it has not inventoried the network‘s holdings. The agency is
digitizing documents in three phases. Phase 1 was completed in January
2007, phase 2 is scheduled for completion in December 2010, and
planning has begun for phase 3. Because EPA has not taken a complete
inventory of its library holdings, however, it cannot determine which
documents, or how many, will need to be digitized and, consequently,
cannot accurately estimate the total cost of digitization or how long
it will take.
Since we reported on the library network reorganization in 2008, EPA
has taken steps to communicate with staff and other stakeholders about
its library network, including providing information about the
libraries and soliciting information from library users. EPA has also
made improvements to the main Internet gateway to the network, making
more documents available electronically and providing better access to
electronic documents and services. Nevertheless, because EPA‘s 2009
survey of the information needs and library use of its staff had
methodological flaws”similar to those GAO identified in 2008”the
agency is unlikely to obtain accurate information that would enable it
to make appropriate decisions on the corrective actions that would
best address library users‘ needs.
What GAO Recommends:
GAO recommends, among other actions, that EPA complete its strategic
plan for the library network and ensure that survey methods provide
reliable data on which to base decisions. With clarifications, EPA
concurred with our recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-10-947] or key
components. For more information, contact John Stephenson at (202) 512-
3841 or stephensonj@gao.gov.
[End of section]
Contents:
Letter:
Background:
EPA Has Not Completed a Strategic Plan for Its Library Network
Identifying an Overall Strategy for the Network:
Since 2008, EPA Has Reopened Closed Libraries and Taken Other Actions:
EPA Has Resumed Digitizing Unique EPA Documents but Has Not
Inventoried Its Holdings:
EPA Has Taken Steps to Communicate with Staff and Other Stakeholders
about Its Network, but Its Staff Survey Was Flawed:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Environmental Protection Agency:
Appendix III: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: EPA Library Operations Before, During, and After
Reorganization:
Table 2: Library Network Procedures for Implementing the May 2009
Policy:
Figure:
Figure 1: EPA's Library Network:
Abbreviations:
EPA: Environmental Protection Agency:
NEPIS: National Environmental Publications Internet Site:
NSCEP: National Service Center for Environmental Publications:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 30, 2010:
Congressional Requesters:
In 1971, soon after its creation, the Environmental Protection Agency
(EPA) established a library network to provide agency staff and the
public with access to environmental information. Comprising libraries
and repositories at EPA headquarters and in regional EPA offices,
research centers, and laboratories across the nation, this network
houses information on a broad range of subjects, including science,
environmental protection and management, and environmental laws. In
2006, partly in anticipation of fiscal year 2007 budget reductions,
EPA began a reorganization effort meant to consolidate libraries in
the network and make more materials and services available online. The
reorganization plan proposed closing libraries; dispersing, disposing
of, or digitizing some collections; cutting operating hours at certain
libraries; and altering librarian services. In 2006, EPA closed 5 out
of the 26 libraries it operated at the time.
As EPA was implementing this reorganization, Congress, professional
library associations, labor unions, and other library users raised
several concerns. In response to congressional and other concerns, in
January 2007 EPA placed a moratorium on its reorganization activities,
directing its staff to make no further changes in library services
until library policy and procedures were adopted. During the
moratorium, 5 libraries remained closed; no additional closures were
made; there were no additional reductions in hours of operation,
services, or resources; and the disposal of library materials and
document digitization stopped.
At your request, we reported on the reorganization in 2008.[Footnote
1] In our report, we found that EPA had not followed its own
recommended steps to prepare for reorganizing the libraries. Neither
did the agency develop procedures to fully inform stakeholders, such
as library users and library professionals, of the final configuration
of the library network; rather, communication with stakeholders varied
from library to library. We also found that EPA lacked an effective
strategy or central leadership to ensure the continuity of library
services after reorganization, and each library decided on its own
whether to close its doors and how to disperse or dispose of its
materials. Moreover, EPA did not specifically allot funds to help
closing libraries manage their collections, instead leaving each
program or regional office to use its annual funding for closure costs.
In light of continuing congressional and public concern over EPA's
plans for its library network, you asked us to follow up on these
issues. Accordingly, this report reviews (1) the status of EPA's
overall strategy for its library network, (2) the status of EPA's plan
to reopen the libraries it closed and other actions planned or taken,
(3) EPA's efforts to digitize printed documents to make them
electronically available, and (4) EPA's efforts to communicate with
staff and other stakeholders about its library network.
To address these objectives, we reviewed relevant EPA funding and
inventory documents, policies, plans, and guidance, as well as related
regulations and requirements pertinent to the library network and
efforts to improve its operations. We focused our review on EPA's
Office of Environmental Information's headquarters library in
Washington, D.C.; the 10 regional EPA libraries; and the two Office of
Administration and Resources Management libraries, in Cincinnati,
Ohio, and Research Triangle Park, North Carolina. We compared library
operations before, during, and after the reorganization; obtained and
reviewed the library network's policy and procedures; reviewed the
agency's draft outline of a strategic plan for the library network;
obtained and reviewed documents on EPA's digitization process; and
reviewed EPA's efforts to communicate with and solicit input from
users. We interviewed EPA librarians and library managers in selected
EPA libraries, as well as EPA officials knowledgeable about EPA's
library network and budget. We also interviewed management officials
from the federal employees' union representing EPA staff and spoke
with representatives from EPA's regional science councils, which
consist of EPA scientists and technical specialists. We further sought
information from library professionals, including representatives from
the American Library Association and from contractors involved in
digitizing EPA documents. After limited testing and discussions with
EPA officials, we determined that EPA's data on library funding and on
the number of digitized documents and those scheduled to be digitized
were not sufficiently reliable for our purposes. Because these data
were the only data available, however, we used them to discuss in
broad terms EPA's funding history and the status of its digitization
efforts, noting the data's limitations as appropriate. Appendix I
describes our scope and methodology in greater detail.
We conducted this performance audit from October 2009 through
September 2010, in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
Background:
EPA was established in December 1970 to consolidate a variety of
federal research, monitoring, standard-setting, and enforcement
activities into one agency to ensure the protection, development, and
enhancement of the total environment.[Footnote 2] To help accomplish
its mission, EPA in 1971 established a library network that came to
comprise 26 libraries located across the country. The library network
functions as a collection of independent local libraries, catering
primarily to the needs of local EPA staff and walk-in public patrons.
The libraries are funded and managed by several different regional and
program offices at EPA (see figure 1). EPA defines network libraries
as those of its libraries with an official membership presence in the
global Online Computer Library Center system.
Figure 1: EPA's Library Network:
[Refer to PDF for image: illustrated U.S.map]
Region: 1;
Regional library: Boston, Massachusetts;
Research laboratory library: Narragansett, Rhode Island.
Region: 2;
Regional library: New York, New York;
Regional library: Philadelphia, Pennsylvania.
Region: 3;
Library and repository: Washington, DC;
Specialty library[A]: Washington, DC;
Specialty library[A]: Fort Meade, Maryland.
Region: 4;
Regional library: Atlanta, Georgia;
Library and repository: Research Triangle Park, North Carolina;
Research laboratory library: Research Triangle Park, North Carolina;
Research laboratory library: Athens, Georgia;
Research laboratory library: Gulf Breeze, Florida.
Region: 5;
Regional library: Chicago, Illinois;
Library and repository: Cincinnati, Ohio;
Research laboratory library: Ann Arbor, Michigan;
Research laboratory library: Duluth, Minnesota.
Region: 6;
Regional library: Dallas, Texas;
Research laboratory library: Ada, Oklahoma.
Region: 7;
Regional library: Kansas City, Kansas.
Region: 8;
Regional library: Denver, Colorado;
Specialty library[A]: Denver, Colorado.
Region: 9;
Regional library: San Francisco, California;
Research laboratory library: Las Vegas, Nevada.
Specialty library[A]:
Region: 10;
Regional library: Seattle, Washington;
Research laboratory library: Corvallis, Oregon.
Sources: EPA; Map Resources (map).
[A] EPA identifies as specialty libraries its Legislative Reference
Library, Office of General Counsel Law Library, and Headquarters and
Chemical Library, all of which are in Washington, D.C. The other
specialty libraries are its National Enforcement Investigations Center
Environmental Forensics Library in Denver, Colorado, and the
Environmental Science Center Library at Ft. Meade, Maryland.
[End of figure]
The combined EPA library network collection contains a wide range of
general information on environmental protection and management and on
basic and applied sciences, as well as extensive coverage of topics
related to the statutory mandates that EPA must meet. Several of the
libraries maintain collections focused on special topics to support
specific regional or program office projects. The libraries thus
differ in function, scope of collections, extent of services, and
public access. In addition to its physical locations and holdings, the
EPA network provides staff and public access to its collections
through the following:
* its online library system, a Web-based database of all of EPA
library holdings, also known as EPA's online "card catalog";
* interlibrary loans to another library within the network or to other
libraries;
* a Web site combining two databases--EPA's National Service Center
for Environmental Publications (NSCEP) and its National Environmental
Publications Internet Site (NEPIS)--which provide an online gateway
for access to available print and digital documents, respectively; and:
* desktop access for staff to online journals, the Federal Register,
news, databases of bibliographic information, and article citations.
In addition, librarians are available in each library to catalog and
maintain collections and to assist EPA staff and the public with
research.
In 2003, EPA began studying options for operating its library network
in the future. In August 2006, the agency issued a reorganization
plan, titled EPA FY 2007 Library Plan: National Framework for the
Headquarters and Regional Libraries. The focus of this plan was a
reorganization of the headquarters library and the 10 regional
libraries, all of which received substantial funding from EPA's Office
of Environmental Information. The 2007 library plan identified a new
model for library services, which consisted of three components: a
coordinated library network, instead of stand-alone operations; more
electronic delivery of services; and maintenance of existing essential
services. During implementation of this plan, EPA closed the Chicago,
Dallas, and Kansas City regional libraries and closed its headquarters
library to physical access, although the headquarters library remained
as one of three repositories for storing print collections.[Footnote
3] Another library located at EPA headquarters within the Office of
Prevention, Pesticides, and Toxic Substances (referred to as the
"chemical library") was not subject to budget reductions and was not
discussed in the reorganization plan, but, like the headquarters
library, was also closed to physical access. EPA also reduced or
eliminated the library staff at the closed libraries. Several other
libraries reduced their operating hours, and some libraries disposed
of their materials or dispersed them to other EPA libraries or to non-
EPA libraries. EPA also began to digitize EPA documents not currently
in NEPIS, beginning with documents in the libraries being closed.
EPA's reorganization plan also discussed how the closed libraries were
to handle their collections, directing them primarily to disperse the
collections to other libraries.
EPA's implementation of its reorganization plan caused widespread
concern among its staff, the public, interested parties, and Congress.
In response to these concerns, congressional committees directed $1
million in funding to restore the libraries recently closed or
consolidated, asked us to review EPA's reorganization plan and its
implementation, and directed EPA to prepare a report regarding actions
to restore publicly available libraries. In addition, EPA in January
2007 imposed a moratorium on its reorganization efforts.
Until April 2007, EPA's library network operations had been guided by
EPA's Information Resources Management Policy Manual, which stated
that the library network was to provide EPA staff with access to
information to carry out the agency's mission and that the libraries
were to provide state agencies and the public with access to the
library collection. The Policy Manual also defined the role of a
national library program manager, who was to have responsibility for
coordinating major activities of the library network, although not
budget authority for the libraries. EPA replaced this manual in April
2007 with an interim library policy and, in May 2009, with its final
library policy. The final May 2009 policy also defined key roles and
responsibilities, including those of the national library program
manager and those of "federal library managers," who were to have
first-line responsibility for operating the physical libraries and
providing services.
EPA lifted the moratorium in June 2009, following implementation of
its May 2009 policy and 6 of 12 proposed procedures for the library
network. After we issued our report in February 2008,[Footnote 4]
Congress held hearings on EPA's library network reorganization
efforts, which were followed by the release of EPA's March 2008 report
addressed to Congress, in which EPA stated that it would reopen the
closed libraries by September 30, 2008.[Footnote 5] In our 2008 report
on EPA's library network reorganization, we assessed the
reorganization effort against our past work on key practices and
implementation steps to assist mergers and organizational
transformations.[Footnote 6] These key practices include ensuring that
top leadership drives the transformation, establishing a coherent
mission and integrated strategic goals to guide the transformation,
and setting implementation goals and a timeline to show progress from
day one.
EPA Has Not Completed a Strategic Plan for Its Library Network
Identifying an Overall Strategy for the Network:
Although it has been preparing a strategic plan for its library
network for 3 years, EPA has not completed a plan identifying its
overall network strategy, with implementation goals and a timeline for
what it seeks to accomplish. In our 2008 report, we stated that EPA
was developing a library strategic plan for 2008 and beyond, which was
to detail library services for staff and the public and lay out a
vision for the library network's future. EPA has had a draft outline
of this strategic plan since July 2007. We reported that in October
2007, EPA's Office of Environmental Information asked local unions
throughout the agency to comment on a draft of the plan.
The draft outline of the strategic plan envisions the library network
as "the premier environmental library network that provides timely
access to information and library services to its employees and the
public" and proposes to realize this vision by increasing emphasis on
electronic resources and using new information technologies.[Footnote
7] The draft outline of the strategic plan also lists several
principles as the foundation for present and future directions of the
library network: setting overall goals and objectives and a direction
for implementation; periodic review of the plan to evaluate progress
and update strategies to respond to new opportunities or challenges;
soliciting input from internal and external stakeholders; and
developing the plan in a transparent manner by reporting progress and
soliciting input from interested parties. According to EPA officials,
since 2007, EPA has been in the process of assessing library users'
needs, which is to be completed before they believe they can finish
the strategic plan. EPA officials have stated that a working group led
by the national library program manager is to resume work on the plan
later in 2010.
The draft outline of the strategic plan is largely a list of current
and planned EPA activities--primarily placeholders to be completed.
For example, under the heading, "Digitization," the text states that
the digitization procedures will outline the methods to be used by EPA
libraries to prepare and send EPA documents for digitization and
inclusion in NEPIS; no goals or timeline for implementing these
activities--which we previously reported are among the key components
of successful organizational transformations--are given.[Footnote 8]
We have found that an organization's transformation is strengthened
when it publicizes implementation goals and a timeline to build
momentum and show progress. Despite an emphasis on the central role to
be played by electronic library resources, the draft outline of the
strategic plan briefly describes procedures for packing and shipping
documents to be digitized, without describing actions to be taken to
digitize holdings or target dates for accomplishing those actions. EPA
holds an enormous amount of environmental information, including
publications generated by its program offices, as well as research
publications generated under contracts, grants, and cooperative
agreements. A large portion of this information exists only in print
form. As part of its vision for the library network, according to the
draft outline, EPA is seeking to convert this information into a
digital format to make it more widely available and readily accessible
to users. Yet the draft outline of the strategic plan does not
describe criteria for deciding what documents should be digitized, for
deciding whether or how to digitize copyrighted documents of value, or
for scheduling the funding needed and a time frame for completing the
digitization. Without such criteria, EPA cannot ensure that it is
digitizing the most valuable or important documents or providing users
with information most relevant to them.
Furthermore, although the draft outline of the strategic plan includes
a placeholder for a section describing a funding model for the
network, it contains no detail. Under the heading, "Funding Model,"
the text states that this section in the plan will address how EPA
will ensure that the network libraries have "adequate funding" and
will discuss how funding decisions are made, along with the Office of
Environmental Information's role in the funding process. But the draft
outline of the strategic plan does not define what constitutes
adequate funding, although inadequate funding has been a concern for
the library network since fiscal year 2007. Until then, library
spending had remained relatively stable, ranging from a high of $9.2
million in fiscal year 2002 to a low of $8.2 million in fiscal year
2006.[Footnote 9] In fiscal year 2007, when EPA's budget was reduced,
library spending was $6.3 million.
The draft outline of the strategic plan also does not set out the
details of how funding decisions are made. Setting out details for how
such decisions are to be made, to ensure that they are informed and
transparent, is especially important because of the decentralized
nature of the library network. The library network's funding remains
subject to uncertainty in the future because the several different
program and regional offices responsible for EPA's libraries generally
decide how much to spend on their libraries out of funding available
in larger accounts that support multiple activities. EPA's Office of
Environmental Information, the primary source of funding for the
regional libraries,[Footnote 10] typically provides funding through
each region's support budget and generally gives regional management
officials discretion on how to distribute this funding among the
library and other support services, such as information technology,
utilities, and mail room support. EPA officials told us that, starting
in fiscal year 2010, they are increasing the amount of funding
allocated to the libraries in the regions. The regions also obtain a
much smaller portion of their library funding from other program
offices. For example, the Superfund program office funds the storage
and maintenance of information on the National Priorities List, EPA's
list of the most seriously contaminated sites in the United States.
The extent to which other program offices provide funding to the
regional libraries varies. Thus, without a detailed strategy for how
decisions are made to acquire, deploy, and manage funding resources
across the library network, EPA may find it difficult, particularly in
an era of declining budgets and competing national priorities, to
achieve its vision for the library network and to fully meet the needs
of library users.
Moreover, although the draft outline of the strategic plan contains a
section on communication among network libraries, it makes no mention
of a strategy or a formal outreach plan to ensure that EPA
communicates with and obtains feedback from users about improvements
to its library network in a way that would allow it to measure whether
such improvements are meeting users' needs. The section lists
communication methods EPA is using, such as monthly network
teleconferences among staff and federal managers. In addition, the
section identifies comment cards, questionnaires, and a Web presence
for how it solicits users' feedback, but there is no mention of how
EPA plans to assess feedback on what is important to users or what
improvements are working well or poorly. EPA has another draft
document, titled "EPA Library Network Communication Strategies," whose
purpose is to establish procedures by which libraries in the network
are to communicate both internally and externally. Most of this
document focuses on communication within the library network itself,
explaining how the library network is coordinated and detailing
mechanisms for internal communication, including annual meetings for
library network staff and federal library managers. One of the final
sections in this procedures document lists several means of
communicating externally, including Web sites and various local
options for libraries to reach out to local patrons, such as tours,
signs, comment cards, and online feedback mechanisms. Beyond listing
such mechanisms, however, this document, like the draft outline of the
strategic plan, does not lay out a systematic communication strategy
with communication and feedback performance goals that can be measured
to determine progress. Without such a strategy, communication with
library users is likely to remain piecemeal and reactive.
Since 2008, EPA Has Reopened Closed Libraries and Taken Other Actions:
EPA has reopened all of the libraries it closed in 2007 and has taken
other actions to improve library operations. In its 2008 report
addressed to Congress,[Footnote 11] the agency stated its commitment
to have libraries in each region and at headquarters open to the
public. EPA also committed to reestablishing on-site libraries for its
staff and the public in the three regions where the libraries had been
closed and at the combined headquarters and chemical library. EPA
reopened all five closed libraries by September 2008, although the
agency had to find new space for two of the three closed regional
libraries and their collections, and all three of these regional
libraries are operating on reduced schedules. Each of the reopened
libraries was staffed with a professional librarian and required to
maintain a collection of core reference materials and additional
library resources to meet local needs and to ensure that staff had
access to core library services and the public had access to the
library and its collections. With the reopening of the closed
libraries, one other regional library that had not been closed also
began operating on reduced schedules, as compared with its hours
before reorganization (see table 1). As of September 2010, about half
of the 10 regional libraries were operating 4 days a week, rather than
5 days as they were before EPA's reorganization efforts--a reduction
in hours due largely to funding constraints, according to library
officials. All of the libraries are staffed with at least one full-
time or part-time librarian, with several libraries having more than
one librarian or additional library staff.
Table 1: EPA Library Operations Before, During, and After
Reorganization:
Program office: Office of Environmental Information;
Library, location: Headquarters library, Washington, D.C.[A];
Operating status of library:
Before reorganization: Open 5 days, 40 hours;
During reorganization: Closed to physical access;
After reorganization: Open 5 days, 40 hours.
Program office: Office of Prevention, Pesticides, and Toxic Substances;
Library, location: Chemical library, Washington, D.C.;
Operating status of library:
Before reorganization: Open 5 days, 35 hours;
During reorganization: Closed to physical access;
After reorganization: Combined with headquarters library.
Program office: Regional office;
Library, location: Region 1 library, Boston, Massachusetts;
Operating status of library:
Before reorganization: Open 5 days, 35 hours;
During reorganization: Reduced hours of operation 3 days, 18 hours;
After reorganization: Open 4 days, 24 hours.
Program office: Regional office;
Library, location: Region 2 library, New York, New York;
Operating status of library:
Before reorganization: Open 4 days, 28 hours;
During reorganization: Reduced hours of operation 3 days, 12 hours;
After reorganization: Open 4 days, 28 hours.
Program office: Regional office;
Library, location: Region 3 library, Philadelphia, Pennsylvania;
Operating status of library:
Before reorganization: Open 5 days, 45 hours;
During reorganization: Open 5 days, 45 hours;
After reorganization: Open 5 days, 45 hours.
Program office: Regional office;
Library, location: Region 4 library, Atlanta, Georgia;
Operating status of library:
Before reorganization: Open 5 days, 42.5 hours;
During reorganization: Open Some services provided by the library in
Cincinnati, Ohio 5 days, 42.5 hours;
After reorganization: Open 5 days, 42.5 hours.
Program office: Regional office;
Library, location: Region 5 library, Chicago, Illinois;
Operating status of library:
Before reorganization: Open 5 days, 25 hours;
During reorganization: Closed to physical access;
After reorganization: Open 4 days, 24 hours.
Program office: Regional office;
Library, location: Region 6 library, Dallas, Texas;
Operating status of library:
Before reorganization: Open 5 days, 40 hours;
During reorganization: Closed to physical access;
After reorganization: Open 5 days, 30 hours.
Program office: Regional office;
Library, location: Region 7 library, Kansas City, Kansas;
Operating status of library:
Before reorganization: Open 5 days, 40 hours;
During reorganization: Closed to physical access;
After reorganization: Open 4 days, 24 hours.
Program office: Regional office;
Library, location: Region 8 library, Denver, Colorado;
Operating status of library:
Before reorganization: Open 5 days, 40 hours;
During reorganization: Open 5 days, 40 hours;
After reorganization: Open 5 days, 40 hours.
Program office: Regional office;
Library, location: Region 9 library, San Francisco, California;
Operating status of library:
Before reorganization: Open 5 days, 24 hours;
During reorganization: Reduced hours of operation 4 days, 24 hours;
After reorganization: Open 4 days, 24 hours.
Program office: Regional office;
Library, location: Library, location: Region 10 library, Seattle,
Washington;
Operating status of library:
Before reorganization: Open 5 days, 40 hours;
During reorganization: Reduced hours of operation 5 days, 25.5 hours;
After reorganization: Open 5 days, 40 hours.
Source: GAO analysis of EPA data.
Note: This table shows only those libraries whose operations were
affected or potentially affected by reorganization. Days and hours of
operation shown in the table are for both the public and EPA staff;
the regional libraries in Atlanta, Denver, and Seattle are open 4 to
10 hours per week longer for EPA staff than for the public. While the
libraries in Regions 5, 6, and 7 were closed, services for these
libraries were provided by another network library.
[A] Library also serves as a repository.
[End of table]
In addition to reopening the closed libraries, according to EPA
officials we spoke with, EPA developed standards for the regional and
headquarters libraries' use of space, on-site collections, staffing,
and services. These standards specify, for example, that the libraries
make adequate space available for in-person interactions between
library staff and users, that on-site collections and materials should
address local and regional needs, that staff and the public have
certain minimum hours of access per week (at least 4 days per week on
a walk-in or appointment basis in the regional libraries and at least
5 days per week on a walk-in or appointment basis in the headquarters
library), and that the libraries provide interlibrary loans and
reference or research assistance.
One of the key actions taken by EPA in May 2007 was to hire a national
library program manager, a position that had been vacant since 2005.
[Footnote 12] Housed in EPA's Office of Information Analysis and
Access, within the Office of Environmental Information, the national
library program manager is charged with carrying out day-to-day
activities of the library network and with bringing focus and cohesion
to the network. Part of this charge involves agencywide responsibility
for public information access, including strategic planning for the
library network, and participating in policy formulation regarding
access to EPA's public information. To fulfill this leadership role,
EPA officials said, the national library program manager is to work
closely with the management of EPA's Office of Environmental
Information to set in motion a number of actions meant to improve
library network operation and communication. To communicate with and
gather feedback internally, the national library program manager
initiated monthly teleconferences and annual meetings for all library
managers and staff. Seeking to get the most out of the experience and
knowledge of these library managers, librarians, and staff, the
national library program manager established internal working groups
to research improvement activities, address concerns, and recommend
improvements. For example, the national library program manager
established working groups on digitization, staff information needs,
and development of the final May 2009 library policy and related
procedures (see table 2). In addition, the national library program
manager serves as the EPA-appointed representative in working with
outside library professionals, specifically an external board of
advisors created by the Federal Library and Information Center
Committee, which advises EPA on future directions for the library
network.[Footnote 13]
Table 2: Library Network Procedures for Implementing the May 2009
Policy:
Procedure: EPA library materials dispersal;
Purpose: Establish agencywide procedures for libraries to retain,
reduce, disperse, or dispose of their library contents when
appropriate;
Status, approval date: Approved May 15, 2009.
Procedure: EPA library usage statistics;
Purpose: Establish agencywide procedures by which the libraries
collect statistics on the services they provide EPA staff and the
public;
Status, approval date: Approved May 15, 2009.
Procedure: Digitization processes for EPA libraries;
Purpose: Establish agencywide procedures to identify, prepare, and
send EPA documents to NSCEP for digitization so they may be added to
the NEPIS database (formalizes existing steps for digitizing
documents);
Status, approval date: Approved May 15, 2009.
Procedure: EPA library reference and research services;
Purpose: Establish agencywide procedures to provide reference and
research services to EPA staff and the public;
Status, approval date: Approved May 15, 2009.
Procedure: EPA repository library management;
Purpose: Establish agencywide procedures to ensure that official EPA
documents and other materials that are hard to replace but
infrequently used are available in perpetuity in their originally
published format;
Status, approval date: Approved May 15, 2009.
Procedure: EPA library facility management;
Purpose: Establish agencywide procedures to provide, make use of, and
manage the facility allotted for library operations;
Status, approval date: Approved May 15, 2009.
Procedure: Interlibrary loan service;
Purpose: Establish agencywide procedures to borrow or acquire
materials not available in the local collections for EPA staff and the
public;
Status, approval date: In draft, due to be approved before the end of
2010.
Procedure: Cataloging;
Purpose: Establish agencywide procedures for libraries to catalog
materials in the collections and provide access to remote electronic
documents;
Status, approval date: In draft, due to be approved before the end of
2010.
Procedure: Public access;
Purpose: Establish agencywide procedures to improve public access to
EPA documents and environmental information;
Status, approval date: In draft, due to be approved before the end of
2010.
Procedure: Collection development;
Purpose: Establish agencywide standard methods for libraries to
acquire, organize, and manage materials in their local collections;
Status, approval date: In draft, due to be approved before the end of
2010.
Procedure: Disaster response and emergency;
Purpose: Establish agencywide procedures by which the libraries
prepare for and respond to disasters and provide continuing operations
during and after a disaster;
Status, approval date: In draft, due to be approved before the end of
2010.
Procedure: Library network communications strategies;
Purpose: Establish agencywide procedures by which the libraries in the
network communicate, using a range of established mechanisms, with
other EPA libraries, organizations, and the public;
Status, approval date: In draft, due to be approved before the end of
2010.
Source: EPA.
[End of table]
EPA Has Resumed Digitizing Unique EPA Documents but Has Not
Inventoried Its Holdings:
EPA has restarted its process of digitizing some of its libraries'
holdings, but because the agency has not completed an inventory of its
holdings, it does not know the total number of documents to be
digitized. According to EPA data, which our limited testing found to
be insufficiently reliable, EPA had digitized 16,175 documents from
its libraries as of January 2010. Creating an electronic copy of a
document by means of digitization is relatively simple--essentially
the same scanning process for making photocopies--although it can be
time-consuming and expensive if the document contains special features
such as foldout pages, cannot be taken apart, or needs to be digitized
at a high level of resolution or in color. After the resulting
electronic files are uploaded to EPA's Web databases, the
administrator of EPA's online library catalog is to ensure that links
to the digitized documents are included in the bibliographic records
for each document. According to EPA officials, the present
digitization effort will expand NEPIS, EPA's sole electronic archive
of published material, which, according to EPA officials, contains
40,000 publicly accessible digital documents as of June 2010, up from
4,000 in 1996.
According to EPA documents, the digitization process is to take place
in three phases:
* Phase 1 covered unique EPA documents held by the libraries that were
closed under the reorganization plan.[Footnote 14] EPA data show that
15,260 documents were digitized during this phase, which was completed
in January 2007.
* Phase 2 is to cover all remaining unique EPA documents except those
larger than 11 by 17 inches. According to EPA officials, this phase is
scheduled for completion in December 2010 and should produce 10,102
additional digitized documents, bringing the total number of digitized
library documents available to the public to over 25,000.
* Phase 3 is to include EPA documents of which more than one copy
exists in the library network, plus unique EPA documents larger than
11 by 17 inches. As of July 2010, EPA was beginning to plan this phase.
As of September 2010, the total estimated cost for digitizing EPA's
library holdings remained unclear, in large part because EPA has not
completed an inventory of its holdings and has therefore not
determined the total number of documents that need to be digitized.
When it began digitizing documents from the closed libraries in 2006
under phase 1, EPA estimated that the project would cost $80,000--
$78,000 for scanning and $2,000 for uploading the digital files to the
Web databases--although, according to agency officials, the agency did
not track the actual costs. For phase 2 digitization, which began in
fiscal year 2009, EPA estimated the cost at about $327,000. EPA has
not estimated the cost or a completion date for its final, phase 3
digitization effort, in part because the agency is still cataloging
all its library holdings in a single database so it can inventory all
the documents that need to be digitized. One regional librarian we
spoke with, for example, told us that about 2,000 documents in the
regional library's catalog were not in EPA's online library system,
and it is still unknown which or how many of these documents will need
to be digitized. Without a complete catalog or inventory of its
holdings, EPA cannot determine which documents, or how many, will need
to be digitized and, consequently, cannot accurately estimate the
total cost of digitization or how long it will take. According to the
national library program manager, an EPA workgroup is currently
drafting a new cataloging procedure for the libraries and expects the
procedure to be approved and implemented before the end of 2010. This
procedure requires each network library to inventory its collection on
a regular basis, either the entire collection every 3 years or one-
third of the collection each year.
In addition, EPA library officials observed that a significant number
of the documents in EPA libraries are copyrighted, and to date EPA
does not plan to digitize them. EPA, like other federal agencies,
often contracts with entities in the private sector to do work. In
addition, EPA provides financial assistance in the form of grants and
cooperative agreements to various recipients, such as state, local,
and tribal governments; educational institutions; hospitals; and
nonprofit organizations. Such assistance is documented in an
assistance agreement. Both contracts and assistance agreements may
result in the production of copyrighted documents. In the case of
contracts, federal regulations provide that when a contractor is
permitted to assert a copyright in any document(s) produced, the
government has a license to display the copyrighted work publicly,
which includes posting it on a Web site accessible to the public.
[Footnote 15] In the case of works produced under assistance
agreements, on the other hand, the government has a right to
reproduce, publish, or otherwise use a copyrighted work for federal
purposes, but EPA's Office of General Counsel has determined that
inclusion in EPA's online public library would not constitute a
federal government purpose.[Footnote 16]
On the advice of EPA's general counsel, EPA's digitization workgroup
has recommended digitizing documents created under contract but not
those created under EPA's assistance agreements.[Footnote 17]
According to EPA's grant awards database, these agreements have
resulted in more than 21,000 grants valued at over $40 billion in
taxpayer dollars. Some of these grants led to publications, resulting
in a substantial body of publicly funded written material. According
to EPA's general counsel, EPA may digitize such documents so that
staff and others may use them for federal government purposes but may
not disseminate them for other purposes. EPA may also seek permission
from copyright holders to digitize and disseminate online copyrighted
documents produced under assistance agreements, although some costs
may be associated with obtaining such permissions--tracking down
copyright holders after years, or even decades, have passed, for
example--further complicating any estimation of total digitization
costs. An alternative practice has been in use by the Federal Library
and Information Network,[Footnote 18] the business arm of the Federal
Library and Information Center Committee: permission to use
copyrighted material produced under assistance agreements is sought at
the time an agreement is established. If the prospective copyright
holder grants permission, then a statement to that effect is
incorporated into the assistance agreement, incurring minimal, if any,
additional costs. Without permission from copyright holders, however,
documents prepared under EPA assistance agreements, using taxpayer
dollars, will remain unavailable online to the public.
EPA Has Taken Steps to Communicate with Staff and Other Stakeholders
about Its Network, but Its Staff Survey Was Flawed:
EPA has taken steps to communicate with staff and other stakeholders
about its library network--including providing information about the
libraries as well as soliciting information from library users--but a
2009 survey about its staff's information needs was flawed. In
general, EPA staff and external stakeholders told us the agency is
doing a better job of communicating with them and soliciting input on
the operations and future direction of the library network,
particularly at the local level. Representatives from EPA's employees'
unions and regional science councils stated that communication about
the library and its services--such as new resources, training, and
open houses--is primarily done at the regional level, either through e-
mail or the region's intranet page. Although staff have not been
directly solicited for feedback, according to the representatives, no
outstanding issues regarding the libraries have been raised, except
that a few representatives said they would like to see the libraries
open 5 days and 40 hours per week.
To keep library managers and staff engaged in improving library
operations, EPA has adopted a number of techniques to communicate with
them and solicit their input. These techniques have allowed EPA to
gather staff input for policies and procedures, operational issues,
and Web page improvements. Examples include the following:
* The national library program manager holds monthly network
teleconferences with library managers and staff on matters of interest
to the entire network or on operational topics, such as the library
policy and procedures.
* The national library program manager also holds ad hoc
teleconferences with library managers elsewhere in the network to
discuss their libraries' needs.
* Managers and staff use mailing lists to communicate with one another
about daily library operations or requests for assistance.
* For the last 3 years, EPA has held an annual network meeting in
different locations for library managers and staff to foster
collaboration, provide training, and share information about the
network. At the last meeting, in October 2009, participants discussed
ways to address results of the 2009 staff survey, prepared for the
next round of digitization, and discussed ways to improve library
services. The next annual meeting is scheduled for March 2011.
* EPA and the union representing EPA staff agreed to create a union-
management advisory board with six members--three union
representatives and three from EPA management.[Footnote 19] The board
reviews and makes recommendations on library network policy and
procedures and will review the library network strategic plan once it
is completed.
* In December 2009, EPA instituted a pilot program, an "ask a
librarian" live chat. Ten libraries are participating in the program,
which lets users contact a librarian through an electronic link to
request services. As of July 2010, the "ask a librarian" pilot was
available only to EPA staff.
To begin to realize its vision of effectively implementing new
information technologies and making documents readily available
electronically, EPA in 2007 engaged a contractor to review the user-
friendliness of the combined Web page, or gateway, to the NSCEP and
NEPIS online databases.[Footnote 20] The review identified ways to
improve the site's effectiveness and overall functionality for users.
EPA implemented many of these improvements.[Footnote 21] For example,
the gateway home page now describes the purpose of each database
(NSCEP for print materials and NEPIS for electronic documents) and
what types of publications they contain, noting that they contain only
EPA publications. EPA made several changes to the document display
page as well, such as placing a navigation bar at the top and bottom
of the document with large icons and providing a button that allows
users to obtain a copy in one of three formats. The display page
allows users to put a copy of frequently used documents in a holding
area for later retrieval. Work is also under way to integrate Google
search capabilities into this gateway, as well as the ability to
refine the precision of searches with user-friendly "clouds" of
related keywords. In addition, EPA has added easy ways for users to
offer feedback, which EPA may then incorporate to make improvements.
For example, the navigation bar on the document display page now
includes a "report an error" button, and every page has a "contact us"
link. Furthermore, the left frame of the site contains a link to a
customer satisfaction survey, and the site also has a separate page
for user feedback.
EPA has also made outreach efforts to library professionals outside
EPA--primarily by presenting and exhibiting at professional library
association trade shows and conferences, attending external training,
visiting other federal national libraries, and interacting with its
external board of advisors. Ties with the external board have been a
particularly important part of EPA's response to concerns over the
agency's library reorganization. From June 2007 through February 2010,
the national library program manager met with the board approximately
20 times, working with it on the full range of key issues, from policy
development to funding models to communication with stakeholders. The
external board also advised EPA on the development of a survey to
assess the information needs of EPA staff.
One of the principles in the draft outline of the strategic plan is
soliciting feedback from internal and external stakeholders about
their information needs. To solicit such feedback from staff, EPA in
early 2008, under the direction of the national library program
manager, engaged a contractor to conduct interviews, hold focus
groups, and conduct a Web-based survey. The survey was made available
for approximately 1 month in 2009 via a secure Web site only to EPA
staff (about 17,000 individuals), not all of whom were library users.
After the survey was completed, the contractor conducted a series of
focus groups and one-on-one interviews with EPA managers, focusing on
significant issues identified in the Web-based survey; according to
EPA officials, these in-person discussions were to help ensure that a
comprehensive perspective of user needs was captured. EPA officials
stated that the Web-based survey results corroborated what the agency
learned in an earlier survey, done in 2004 to 2005. On the basis of
the survey results, focus group discussions, and management
interviews, the contractor developed recommendations for EPA's
consideration. EPA received the results of the survey and discussions,
along with the contractor's recommendations, in August 2009 and has
assigned working groups of library staff to review the findings and
suggest how EPA could address them.
We found, however, that this survey had flaws, similar to those we
identified in the 2004 to 2005 survey and discussed in our 2008
report, which greatly reduce its usefulness.[Footnote 22] First, in
both the earlier and the 2009 surveys, the response rate was 14
percent, far lower than the 80 percent response rate that Office of
Management and Budget guidance recommends for a survey to increase the
likelihood that it adequately represents a universe of respondents.
[Footnote 23] Neither EPA nor the contractor for the 2009 survey
analyzed the results for the nonresponse bias that may occur at
response rates lower than 80 percent, particularly if the group of
respondents differs significantly in relevant ways from
nonrespondents.[Footnote 24] Thus, EPA cannot be assured that either
survey accurately described staff needs for information or their uses
of the library. Second, respondents to both the 2009 and the earlier
survey were self-selected, a methodology that often leads to biased
samples, since the traits that affect a person's decision to
participate in the survey--such as strong opinions or substantial
knowledge--may be correlated with traits that affect survey responses;
the result is an unrepresentative sample of possible respondents. The
risk of potential bias through self-selection is increased by the fact
that neither EPA nor the contractor for the 2009 survey instituted any
safeguards to prevent respondents from submitting more than one survey
each. Thus, there is no assurance that the survey results are unbiased
and reflect a broad range of EPA staff perspectives and experiences.
Third, in neither survey did EPA gather views from or determine the
needs of other significant users of EPA libraries, such as state and
local environmental organizations or the public at large.[Footnote 25]
Although EPA officials told us that EPA is planning to assess the
needs of public patrons, an assessment that does not correct the
methodological weaknesses we found in EPA's two previous surveys of
its staff is unlikely to produce results that accurately reflect the
needs of public patrons.
Conclusions:
In the 4 years since EPA issued a reorganization plan for its library
network, the agency has taken a number of steps to better communicate
with, and meet the needs of, library users. EPA's lack of a completed
strategic plan identifying its overall strategy for the network,
however, leaves unclear how the agency will translate into reality its
vision of a "premier environmental library" with an "emphasis on
electronic resources." Steps the agency has taken, including hiring a
national library program manager and establishing a uniform policy and
some procedures for the libraries, have led to some improvements in
library services and will undoubtedly enhance network cohesion. But
without a completed strategic plan that contains implementation goals
and timelines, neither EPA nor users of its libraries can have a clear
view of what EPA plans to do, when EPA plans to do it, and whether
EPA's actions will ultimately meet users' needs. In particular,
without a strategy for acquiring, deploying, and efficiently
allocating library funding, the library network could have difficulty
maintaining high-quality service in the digital age. Moreover, EPA's
approach to digitizing copyrighted works in the future--as well as the
fact that the agency has not yet inventoried all library holdings--
could, if not revisited, detract significantly from the utility of the
library network. Specifically, unless EPA revisits its decision not to
digitize documents prepared with taxpayer dollars under assistance
agreements, it may be missing opportunities to make these documents
more readily available to users, including other federal users, who
need them to better carry out their work. Finally, improvements to the
library network's Internet gateway offer new means of seeking feedback
from library patrons about their use of and need for library services.
Nevertheless, EPA does not have a valid method for assessing those
library users' needs. If, in future assessments of users' needs, EPA
fails to correct the flawed methods of its previous staff surveys, the
agency is unlikely to obtain accurate information that would enable it
to make appropriate decisions on the corrective actions that would
best address those needs.
Recommendations for Executive Action:
To ensure that EPA's library network continues to meet its users'
needs, we recommend that the Administrator of EPA take the following
six actions:
* Complete EPA's strategic plan for the library network, including
implementation goals and timelines. In so doing, EPA should outline
details for how funding decisions are to be made, to ensure that they
are informed and transparent.
* Complete an inventory of the library network's holdings to identify
what items remain to be digitized.
* For assistance agreements already in place, EPA should digitize
documents produced under the agreements and make them available to
federal employees and other authorized users for federal government
purposes.
* In future assistance agreements, make explicit that EPA can include
in the agency's public online database, without obtaining prior
permission from the copyright holder, any documents produced under the
agreements.
* For future assistance agreements where EPA cannot make such an
arrangement, EPA should digitize documents produced under the
agreements and make them available to federal employees and other
authorized users for federal government purposes.
* Ensure that the data analysis protocols used for conducting surveys
of users' needs--including sampling procedures and response rates--are
sufficiently sound methodologically to provide reliable information on
which to base decisions and allocate resources efficiently.
Agency Comments and Our Evaluation:
We provided EPA with a draft of this report for review and comment.
With clarifications, the agency concurred with our recommendations.
EPA acknowledged that the planning document available on the agency's
Web site--which our report refers to as the draft outline of the
strategic plan--has provided more of a working agenda than a strategic
plan to guide the rebuilding of the library program. In responding to
our recommendations, EPA wrote that it believes it now has enough
information to develop an effective strategic plan for the library
network and that it is time to complete and publish a formal plan
identifying an overall network strategy, with implementation goals and
a timeline for future accomplishments. The agency stated that it is
moving forward with the strategic plan, which it aims to complete in
fiscal year 2011. In addition, EPA said it will address the inventory
of library holdings, completing a schedule for cataloging the
inventory by November 1, 2010, and striving to complete the cataloging
by September 30, 2011. The agency further agreed to take the necessary
steps to ensure that any future assessments of users' needs employ
methodologically sound data protocols and provide reliable information.
Regarding our recommendations on the digitization of copyrighted
documents produced under assistance agreements, EPA said it would
address the feasibility and legality of digitizing products resulting
from such agreements. For future assistance agreements, the agency
said it will develop options for gaining advance permission to
digitize products from these agreements and take these options to
senior agency managers by mid-2011 for consideration and action. For
existing assistance agreements, however, EPA wrote that, because of
legal and technical constraints, it does not plan to digitize products
produced under existing agreements. In further clarifying the agency's
written comments, EPA officials told us that because the documents
produced under existing assistance agreements are copyrighted, the
agency cannot include them in its public online database. In the
agency's view, EPA would therefore need to develop a forum for
disseminating the documents to EPA staff and determine whether other
federal employees needed access to the documents for federal
government purposes. EPA officials also said that digitizing these
documents was constrained by several factors, including agency
priorities for which documents are to take precedence and efforts to
identify which of the many types of assistance programs are likely to
produce documents of most value to EPA staff. We have clarified the
wording of our recommendations to eliminate any implied reference that
EPA should make the copyrighted documents available to the general
public in its online database, and we maintain that making copyrighted
documents resulting from assistance agreements available solely for
federal government purposes is permissible, feasible, and desirable.
EPA's written comments appear in appendix II. EPA also provided
technical corrections, which we incorporated.
As agreed with your offices, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the report date. At that time, we will send copies of this report
to the appropriate congressional committees, Administrator of EPA, and
other interested parties. in addition, this report will be available
at no charge on the GAO Web site at [hyperlink, http://www.gao.gov].
If you or your staff members have any questions on this report, please
contact me at (202) 512-3841 or stephensonj@gao.gov. Contact points
for our Offices of Congressional Relations and Public Affairs may be
found on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix III.
Signed by:
John B. Stephenson:
Director, Natural Resources and Environment:
List of Requesters:
The Honorable Barbara Boxer:
Chairman:
Committee on Environment and Public Works:
United States Senate:
The Honorable Bart Gordon:
Chairman:
Committee on Science and Technology:
House of Representatives:
The Honorable Henry A. Waxman:
Chairman:
The Honorable John Dingell:
Chairman Emeritus:
Committee on Energy and Commerce:
House of Representatives:
[End of section]
Appendix I: Scope and Methodology:
To complete our work, we reviewed relevant Environmental Protection
Agency (EPA) funding and inventory documents, policies, plans,
guidance, and procedures, as well as related regulations and
requirements pertinent to the library network and efforts to improve
its operations. We limited our review to the 26 libraries belonging to
EPA's library network, that is, libraries that are members of the
Online Computer Library Center system. We focused on EPA's
headquarters library, the 10 regional libraries funded in part by
EPA's Office of Environmental Information, and the Office of
Administration and Resources Management libraries in Cincinnati, Ohio
(which is responsible for EPA's digitization and Web site
maintenance), and Research Triangle Park, North Carolina. We compared
library operations before, during, and after attempted reorganization
in fiscal year 2007; obtained and reviewed library network policy and
procedures; reviewed the agency's draft outline of a strategic plan
for the library network; obtained and reviewed documents on EPA's
digitization process; and reviewed EPA's efforts to communicate with
and solicit input from users. We interviewed EPA librarians and
library managers in selected EPA libraries, as well as Office of
Environmental Information officials knowledgeable about EPA's library
network and budget; when possible, we corroborated information
provided to us during interviews with relevant documentation. We also
interviewed management officials from the federal employees' union
representing EPA staff and spoke with representatives from EPA's
regional science councils, which consist of EPA scientists and
technical specialists. We further sought information from library
professionals, including representatives from the Library of Congress;
the National Agriculture Library; and, through visits and interviews,
from Lockheed Martin and Integrated Solutions and Services,
contractors involved in digitizing EPA documents.
In addition, we obtained information on library funding from each of
the 26 libraries in the network from fiscal year 2002 to fiscal year
2010. Because EPA does not specifically track funding for the
libraries, the information provided contained a mix of outlays for
some fiscal years and budget authority for other fiscal years. In
addition, the information provided by each of the libraries reflected
only spending by the library and not funding sources. For example, a
large portion of funding for regional office libraries comes from the
Office of Environmental Information, but these libraries also receive
funding from other EPA program offices, such as Superfund. Also,
funding data from the libraries contained a mix of funding for
contract support; library staff salaries; and acquisition costs for
books, journals, and other materials. We interviewed EPA budget
officials to assess data reliability and performed a limited test to
verify the accuracy and completeness of the data provided by the
libraries. On the basis of this test and discussions with EPA
officials, we concluded that the data were not reliable enough to
include in our report.
We also obtained data on the number of EPA and other documents that
have already been digitized and the number still to be digitized.
After limited testing and discussions with EPA officials, we
determined that EPA's data on library funding and on the number of
digitized documents and those scheduled to be digitized were not
sufficiently reliable for our purposes. Because these data were the
only data available, however, we used them to some extent, noting
their limitations in our report as appropriate. We also reviewed
documents about EPA's digitization process, guidance on what documents
should or should not be digitized, and digitization contracts, and we
discussed the contents of these documents with EPA and digitization-
contractor officials. We also discussed EPA's future digitization
plans with Office of Environmental Information officials.
In addition, we assessed EPA's survey of library users, examining the
adequacy of the survey methodology, including response rate, sampling
methodology, security measures, survey questions, and processes. To
determine the adequacy of the response rate to EPA's survey, we
followed an 80 percent response rate as a criterion, as Office of
Management and Budget guidance recommends and we apply in our own
surveys to increase the likelihood of sufficiently representing a
universe of respondents. For surveys with response rates lower than 80
percent, we also perform an analysis to determine the existence of
nonresponse bias. To generate its survey sample, however, EPA relied
on self-selection, using a Web site to make the survey available to
approximately 17,000 EPA staff; the response rate achieved was 14
percent. We performed a limited nonresponse analysis of EPA's survey
data and determined that some staffing categories were represented in
proportions different from those found in the population of EPA staff.
Given the 14 percent response rate to EPA's survey, the nonrandom
methodology that generated the sample, and the results of our limited
analysis for nonresponse bias, we found EPA's survey results to be
inadequate for EPA's purpose of obtaining a representative view of EPA
library users. We also interviewed local union representatives from
headquarters and some of EPA's regional offices. Furthermore, we
interviewed regional science council representatives from some of the
regional offices. The science councils are located in each regional
office and consist of EPA scientists and technical specialists. To
determine the extent to which EPA communicated with and solicited
views from outside stakeholders, we interviewed representatives from
several professional library associations and other external
stakeholder groups, such as the American Library Association, the
Library of Congress, the Federal Library and Information Center
Committee, and the Union of Concerned Scientists. We also reviewed
information EPA provided to the public via the EPA Web site or, when
applicable, Federal Register notices.
We conducted this performance audit from October 2009 through
September 2010, in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives.
[End of section]
Appendix II: Comments from the Environmental Protection Agency:
United States Environmental Protection Agency:
Office Of Environmental Information:
Washington, D.C. 20460
[hyperlink, http://www.epa.gov]
September 10, 2010
Mr. John B. Stephenson:
Director:
Natural Resources and Environment:
United States Government Accountability Office:
Washington, DC 20548:
Re: FPA Comments on the Government Accountability Office's (GAO) draft
report to Congress entitled EPA Needs to Complete a Strategy for Its
Library Network to Meet Users' Needs (GAO-l0-947):
Dear Mr. Stephenson:
This letter provides the U.S. Environmental Protection Agency's (EPA)
comments on GAO's draft report entitled EPA Needs to Complete a
Strategy for Its Library Network to Meet Users Needs (GA0-10-947). EPA
appreciates the opportunity to review and comment on this report to
Congress.
EPA's focus over the past two years has been to restore confidence in
our library network to deliver high quality library services to our
staff and the public and to fulfill the commitments made in the
Agency's February 14, 2008 letter to you. EPA believes that we have
made great strides in all of the areas identified by the GAO in 2008,
and we appreciate that our progress has been noted by GAO and others,
including highly regarded members of the library profession.
In considering your comments related to our strategic plan, we
acknowledge that the current planning document available on our web
site has provided more of a working agenda to guide the rebuilding of
our library program. The outline and draft documentation has served as
a living document to guide our library staff toward solid service
delivery and enable them to explore innovative ways to benefit science
and policy making at EPA. We believe that we have sufficient input and
direction from our library users' survey, the exceptional
collaboration and cooperation of the library community in meetings and
conferences, the library support contractor and the Library Advisory
Board to move forward.
We agree, however, that having reached a new level of maturity in this
library network, it is time to complete and publish a formal plan that
identifies an overall network strategy, with implementation goals and
timeline for future accomplishments. The Agency agrees to move forward
with the strategic plan and to include the comments noted in the
enclosure, which will be the major focus of our library team in FY
2011.
In addition to the strategic plan effort, we will address the
inventory of library holdings, the feasibility and legality of
digitizing products received by the Agency as the result of assistance
agreements, and the design of any future user surveys in our follow-up
to this report.
In closing, EPA staff have benefited from this review and the
thoughtful dialogue around the development of this report. We have
addressed the specific recommendations individually, as an enclosure
with this letter, to be included in the final report. Thank you again
for the opportunity to comment. If you would like to discuss these
matters further, please contact me at 202-564-6665 or Mr. Robin
Gonzalez, Director of the Office of Information Analysis and
Access at 202-566-0600.
Sincerely,
Signed by:
Malcolm D. Jackson:
Assistant Administrator and Chief Information Officer:
Enclosure:
cc:
Craig E. Hooks, Assistant Administrator, Office of Administration and
Resources Management:
David McIntosh, Associate Administrator, Office of Congressional and
Intergovernmental Relations:
Barbara J. Bennett, Chief Financial Officer:
Scott Fulton, General Counsel:
Assistant Regional Administrators:
[End of letter]
Enclosure:
September 10, 2010:
Mr. John B. Stephenson:
Response to GAO Recommendations: EPA National Library Network:
GAO Recommendation:
Complete EPA's strategic plan for the library network, including
implementation goals and timelines. In doing so, EPA should outline
details for how funding decisions are to be made, to ensure that they
are informed and transparent.
EPA Response:
EPA has made progress towards the goal of developing a strategic plan
for the library network. The library network has engaged in many
activities, both within and outside the agency. which has provided
valuable information to inform the strategic planning process. These
activities include developing and promulgating a library network
policy and related procedures to establish standard operations across
the network; conducting extensive and regular outreach with internal
and external stakeholders, professional associations and federal
partners about user information needs and library best practices;
engaging with members of the library network through teleconferences
and in-person meetings to discuss the business and goals of the
network and related library services; enhancing library collections in
response to user needs; exhibiting and participating in national
library conferences as well as internal EPA conferences where users
informally express their needs; implementing new services in response
to user feedback, including live chat reference and critical
electronic resources; planning for and implementing Phase II of the
digitization project; and gathering user feedback through a
centralized customer satisfaction feedback form. Additionally, EPA
conducted a formal user needs assessment in FY2009. The findings,
which correlate strongly with our ongoing, informal information
gathering and communications, provide a valuable information framework
upon which to base EPA's library strategic plan. EPA will develop a
strategic plan for the library network using the needs assessment
findings alongside the additional data gathered from users and other
activities. The network has established a core work group of federal
librarians who have committed to participate in the strategic planning
process. This group will work together with the National Program
Manager and other stakeholders to complete the strategic plan in FY11.
GAO Recommendation:
Complete an inventory of the library network's holdings to identify
what items remain to be digitized.
EPA Response:
EPA agrees with this recommendation and has undertaken steps necessary
to inventory library holdings. The library network has identified
cataloging of library materials as a priority for individual
libraries. The AWBERC Library in Cincinnati is providing cataloging
services for network libraries at no cost. To date, 18 network
libraries have taken advantage of this service, resulting in the
cataloging of more than 8,000 additional items. This process will
enable EPA to more accurately estimate the remaining items to be
digitized. EPA will complete a schedule for cataloging the inventory
by November I, 2010 and will strive to complete the cataloging by
September 30, 2011. In addition, the library network has developed
library procedures that outline requirements for verifying the
completeness of collection inventories on a regular basis and is
implementing the procedures.
GAO Recommendation:
For assistance agreements already in place, EPA should digitize
documents produced under the agreements to make them more readily
available to EPA employees and other users for federal government
purposes.
EPA Response:
EPA recognizes that digitizing documents produced under assistance
agreements would provide a useful resource to the Agency at large;
however, the "federal purpose" license does not allow EPA to
unilaterally make a financial assistance recipient's copyrighted
material available to the public. As we have developed our
digitization plans, we have focused on the unique documents held in
network libraries, after which the Agency will be in a position to
consider documents produced under assistance agreements.
Due to legal and technical constraints, EPA does not plan to digitize
products from ongoing assistance agreements. As noted below, we will
develop options for gaining permission from future agreement
recipients to digitize products from their agreements.
GAO Recommendation:
In future assistance agreements, make explicit that EPA can include in
the agency's public online database, without obtaining prior
permission from the copyright holder, any documents produced under the
agreements.
EPA Response:
Over the next six months, the Office of Environmental Information
(0EI), working with the Office of Administration and Resources
Management (OARM) and the Office of General Counsel (OGC) will develop
options for gaining advance permission to digitize products from
future assistance agreements. These options will be taken to Senior
Agency managers in mid-2011 for their consideration and action.
GAO Recommendation:
For future assistance agreements where EPA cannot make such an
arrangement, EPA should digitize documents produced under the
agreements to make them more readily available to EPA employees and
other users for federal government purposes.
EPA Response:
As noted above, this issue will be among the options evaluated by OEI,
OARM and OGC.
GAO Recommendation:
Ensure that the data analysis protocols used for conducting surveys of
users' needs ” including sampling procedures and response rates ” are
sufficiently sound methodologically to provide reliable information on
which to base decisions and allocate resources efficiently.
EPA Response:
EPA recognizes the concerns of GAO related to the methodology used in
the recent needs assessment. We believe that we have sufficient
information from the survey and our continuing feedback from users,
the Library Advisory Board, and the library network staff to enable us
to develop an effective strategic plan for the library network. The
needs assessment survey was supplemented by a series of six focus
group sessions with key user group segments and seven one-on-one in-
depth interviews with management level stakeholders. These additional
venues focused on the deeper issues and learnings from the
quantitative survey and were done to help ensure a comprehensive
perspective of user needs. Additionally, the findings of the needs
assessment correlate with other data gathered on an ongoing basis from
library users, further suggesting we have valid information on which
to base such a plan. EPA will take the steps necessary to ensure that
any future assessments are done with data analysis protocols that are
methodologically sound and provide reliable information for future
decision making.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
John Stephenson, (202) 512-3841 or stephensonj@gao.gov:
Staff Acknowledgments:
In addition to the contact above, Ed Kratzer, Assistant Director;
Ellen W. Chu; Pamela Davidson; Les Mahagan; John C. Martin; Ben
Shouse; and Jeannette Soares made key contributions to this report.
[End of section]
Footnotes:
[1] GAO, Environmental Protection: EPA Needs to Ensure That Best
Practices and Procedures Are Followed When Making Further Changes to
Its Library Network, [hyperlink,
http://www.gao.gov/products/GAO-08-304] (Washington, D.C.: Feb. 29,
2008).
[2] Reorganization Plan No. 3 of 1970, 35 Fed. Reg. 15623 (Dec. 2,
1970) (5 U.S.C. Appendix 1).
[3] A repository library is a central place that collects and
preserves EPA documents and other materials deemed of value to the
library network. EPA has designated three libraries as repository
libraries, each with a subject specialty: the Headquarters Library for
waste, pesticides, regional or local environmental topics deemed
important to staff, and other topics not included elsewhere; the
Research Triangle Park Library in Research Triangle Park, North
Carolina, for air and health effects; and the Andrew W. Breidenbach
Environmental Research Center in Cincinnati, Ohio, for water and risk
assessment materials.
[4] [hyperlink, http://www.gao.gov/products/GAO-08-304].
[5] Environmental Protection Agency, EPA National Library Network
Report to Congress (Washington, D.C., 2008). EPA prepared and issued
this report in response to a directive in the explanatory statement
accompanying the fiscal year 2008 Consolidated Appropriations Act.
[6] GAO, Results-Oriented Cultures: Implementation Steps to Assist
Mergers and Organizational Transformations, [hyperlink,
http://www.gao.gov/products/GAO-03-669] (Washington, D.C.: July 2,
2003).
[7] Environmental Protection Agency, "Draft Annotated Outline for the
EPA Library Strategic Plan," Washington, D.C., July 25, 2007.
[8] [hyperlink, http://www.gao.gov/products/GAO-03-669].
[9] These spending estimates, expressed in constant 2010 dollars, came
from individual libraries in EPA's network and were provided to us by
EPA. Aside from limited testing of the data from one region, we did
not independently verify the reliability of the estimates.
[10] Over the past decade, the Office of Environmental Information has
provided from 41 to 47 percent of the library network's funding,
except for fiscal years 2007 and 2008, when, because of reductions in
its budget, the office provided 34 percent. In 2008, library spending
was $7.3 million, reflecting $1 million added by Congress, and in
fiscal year 2009, EPA increased library funding to $8.3 million.
[11] Environmental Protection Agency, EPA National Library Network
Report.
[12] In our 2008 report (GAO-03-804, 36), we observed that it was
essential that top leadership drive transformation of the library
network and that without a national manager for the library network,
EPA had no official providing essential oversight and guidance.
[13] This board of advisors was established in response to a request
from EPA for assistance in improving current and future library
operations. Created in 1965, the Federal Library and Information
Center Committee comprises the directors of the four national
libraries--Library of Congress, National Library of Medicine, National
Library of Education, and National Agriculture Library--and
representatives of cabinet-level executive departments, as well as
legislative, judicial, and independent federal agencies with major
library programs; it is chaired by the Librarian of Congress. The
committee's mission is to foster excellence in federal library and
information services through interagency cooperation and to provide
guidance and direction for the Federal Library and Information Network.
[14] EPA defines a unique document as a document published by EPA or
on behalf of EPA that exists in only one copy within EPA's National
Library Network and that is not already electronically available in
NEPIS.
[15] Copyright in works arising under contracts is governed by Subpart
27.4 of the Federal Acquisition Regulation (FAR) (48 C.F.R. §§ 27.400-
27.409). FAR Data Rights Clause No. 52.227-14, Rights in Data--General
(48 C.F.R. § 52.227-14), is found in most EPA research and development
contracts.
[16] 40 C.F.R. § 31.34 (copyrights in works produced under grants and
cooperative agreements to state and local governments); 40 C.F.R. §
30.36 (copyrights in works produced under grants and cooperative
agreements to institutions of higher education, hospitals, and other
nonprofits).
[17] Environmental Protection Agency, Digitization of EPA Library
Materials (Washington, D.C., 2009).
[18] The Federal Library and Information Network is the business
subsidiary of the Federal Library and Information Center Committee and
serves federal libraries and information centers as their purchasing,
training, and resource-sharing consortium.
[19] In July 2008, after a February 2008 arbitration ruling, EPA
management and representatives of its employees' union entered into a
memorandum of agreement to resolve concerns surfaced during the
reorganization.
[20] Environmental Protection Agency, Target User Accessibility
Review: NSCEP/NEPIS: EPA's Gateway to Free Digital and Paper
Publications (Washington, D.C., 2008).
[21] These efforts were aided by discussions with staff from other
federal libraries, including the National Agricultural Library and the
Library of Congress.
[22] [hyperlink, http://www.gao.gov/products/GAO-08-304].
[23] Office of Management and Budget, Standards and Guidelines for
Statistical Surveys, Washington, D.C., 2005, and Office of Management
and Budget, Guidance on Agency Survey and Statistical Information
Collections, Washington, D.C., 2006.
[24] According to the Office of Management and Budget's guidance, a
low response rate may indicate an error known as nonresponse bias,
which can significantly lower the accuracy of survey results. Such
bias may occur if survey participants differ substantially and
systematically from nonparticipants in ways that might influence their
responses to survey questions. Similarly, surveys with few respondents
may also be biased because they may fail to capture the true range of
variability in the surveyed population.
[25] According to EPA estimates, 20 to 40 percent of the reference
requests received by regional libraries come from the public.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: