Agency for Toxic Substances and Disease Registry
Policies and Procedures for Public Health Product Preparation Should Be Strengthened Gao ID: GAO-10-449 April 30, 2010The Agency for Toxic Substances and Disease Registry (ATSDR) has faced concerns related to the quality of some of the public health products it publishes. ATSDR investigates community exposures related to certain hazardous chemical sites and releases; assesses associated health effects; and recommends actions to stop, prevent, or minimize harmful effects. ATSDR publishes many types of products, including public health assessments, health consultations, exposure investigations, and health study reports. GAO was asked to examine the extent to which ATSDR's policies and procedures for product preparation, including work initiation, product development, and review and clearance, provide reasonable assurance of product quality. GAO reviewed ATSDR policies and procedures and interviewed agency officials and employees.
The policies and procedures that ATSDR has established for public health product preparation lack some of the critical controls to provide reasonable assurance of product quality. To provide reasonable assurance that agency objectives are being met, federal internal control standards call for agencies to establish policies and procedures, assess risks associated with achieving agency objectives, ensure effective information sharing throughout the organization, monitor agency activities, and establish key areas of authority and responsibility for management and staff. While ATSDR has established some policies and procedures to guide the preparation of its public health products, the policies and procedures do not establish how information is to flow between management and staff during initiation. Absent such policies and procedures, ATSDR generally relies on various meetings to inform management and staff about new work. The agency is also implementing a new database, which may improve information flow. Furthermore, ATSDR does not comprehensively evaluate and categorize the risk of work being initiated. While the agency used to officially classify some hazardous chemical sites as "high-priority" or "focus sites," and require any products resulting from those sites to undergo a higher level of review and clearance, it no longer does so. Because ATSDR does not comprehensively assess and categorize the risk of work being initiated at the agency, management cannot ensure that they have consistently managed the risk related to new work. Additionally, many of ATSDR's policies and procedures that guide product development do not clearly define management roles and responsibilities and do not consistently require that management monitor the development of key components of these products. These deficiencies may lead management to be unclear about their responsibilities, and may result in problems that occur during product development not being identified or addressed until review and clearance, if at all. For example, ATSDR and Institute of Medicine reports show that because scientific concerns were not identified during development of an ATSDR report regarding chemical releases in the Great Lakes region, the document underwent several years of review, and a final report was not issued until more than 4 years after the first draft was written. Moreover, because some review and clearance policies do not reflect current practices, ATSDR staff cannot rely on these policies to accurately or consistently determine review and clearance procedures. Furthermore, review and clearance policies and procedures direct management and staff to use discretion to identify products that require higher levels of review, rather than making this determination through a comprehensive risk assessment process. While ATSDR policy sets out criteria for when additional review may occur, such as when a document could have a high degree of visibility, there is no required point during a product's preparation when management and staff collectively determine whether a product meets the criteria, and whether additional review is warranted. Thus, the agency cannot ensure that all products consistently receive the appropriate level of review.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Cynthia A. Bascetta Team: Government Accountability Office: Health Care Phone: (202) 512-7207GAO-10-449, Agency for Toxic Substances and Disease Registry: Policies and Procedures for Public Health Product Preparation Should Be Strengthened
This is the accessible text file for GAO report number GAO-10-449
entitled 'Agency for Toxic Substances and Disease Registry: Policies
and Procedures for Public Health Product Preparation Should Be
Strengthened' which was released on May 20, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
April 2010:
Agency for Toxic Substances and Disease Registry:
Policies and Procedures for Public Health Product Preparation Should
Be Strengthened:
GAO-10-449:
GAO Highlights:
Highlights of GAO-10-449, a report to congressional requesters.
Why GAO Did This Study:
The Agency for Toxic Substances and Disease Registry (ATSDR) has faced
concerns related to the quality of some of the public health products
it publishes. ATSDR investigates community exposures related to
certain hazardous chemical sites and releases; assesses associated
health effects; and recommends actions to stop, prevent, or minimize
harmful effects. ATSDR publishes many types of products, including
public health assessments, health consultations, exposure
investigations, and health study reports. GAO was asked to examine the
extent to which ATSDR‘s policies and procedures for product
preparation, including work initiation, product development, and
review and clearance, provide reasonable assurance of product quality.
GAO reviewed ATSDR policies and procedures and interviewed agency
officials and employees.
What GAO Found:
The policies and procedures that ATSDR has established for public
health product preparation lack some of the critical controls to
provide reasonable assurance of product quality. To provide reasonable
assurance that agency objectives are being met, federal internal
control standards call for agencies to establish policies and
procedures, assess risks associated with achieving agency objectives,
ensure effective information sharing throughout the organization,
monitor agency activities, and establish key areas of authority and
responsibility for management and staff. While ATSDR has established
some policies and procedures to guide the preparation of its public
health products, the policies and procedures do not establish how
information is to flow between management and staff during initiation.
Absent such policies and procedures, ATSDR generally relies on various
meetings to inform management and staff about new work. The agency is
also implementing a new database, which may improve information flow.
Furthermore, ATSDR does not comprehensively evaluate and categorize
the risk of work being initiated. While the agency used to officially
classify some hazardous chemical sites as ’high-priority“ or ’focus
sites,“ and require any products resulting from those sites to undergo
a higher level of review and clearance, it no longer does so. Because
ATSDR does not comprehensively assess and categorize the risk of work
being initiated at the agency, management cannot ensure that they have
consistently managed the risk related to new work.
Additionally, many of ATSDR‘s policies and procedures that guide
product development do not clearly define management roles and
responsibilities and do not consistently require that management
monitor the development of key components of these products. These
deficiencies may lead management to be unclear about their
responsibilities, and may result in problems that occur during product
development not being identified or addressed until review and
clearance, if at all. For example, ATSDR and Institute of Medicine
reports show that because scientific concerns were not identified
during development of an ATSDR report regarding chemical releases in
the Great Lakes region, the document underwent several years of
review, and a final report was not issued until more than 4 years
after the first draft was written.
Moreover, because some review and clearance policies do not reflect
current practices, ATSDR staff cannot rely on these policies to
accurately or consistently determine review and clearance procedures.
Furthermore, review and clearance policies and procedures direct
management and staff to use discretion to identify products that
require higher levels of review, rather than making this determination
through a comprehensive risk assessment process. While ATSDR policy
sets out criteria for when additional review may occur, such as when a
document could have a high degree of visibility, there is no required
point during a product‘s preparation when management and staff
collectively determine whether a product meets the criteria, and
whether additional review is warranted. Thus, the agency cannot ensure
that all products consistently receive the appropriate level of review.
What GAO Recommends:
GAO recommends that ATSDR develop policies and procedures that direct
management to assess the risk level of work when it is initiated and
reevaluate the risk level throughout product preparation to ensure it
remains appropriate, and that ATSDR revise its policies and procedures
to include guidance about management‘s roles and responsibilities in
monitoring product development. ATSDR stated that it has begun to
incorporate GAO‘s recommendations.
View GAO-10-449 or key components. For more information, contact
Cynthia A. Bascetta at (202) 512-7114 or bascettac@gao.gov.
[End of section]
Contents:
Letter:
Background:
ATSDR's Policies and Procedures for Public Health Product Preparation
Lack Some Critical Controls to Provide Reasonable Assurance of Quality:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Comments from the Agency for Toxic Substances and Disease
Registry:
Appendix II: GAO Contact and Staff Acknowledgments:
Figures:
Figure 1: ATSDR Organizational Structure:
Figure 2: ATSDR Regions and Cooperative Agreement Partners:
Abbreviations:
ATSDR: Agency for Toxic Substances and Disease Registry:
CDC: Centers for Disease Control and Prevention:
CERCLA: Comprehensive Environmental Response, Compensation, and
Liability Act of 1980:
DHAC: Division of Health Assessment and Consultation:
DHS: Division of Health Studies:
DRO: Division of Regional Operations:
DTEM: Division of Toxicology and Environmental Medicine:
EPA: Environmental Protection Agency:
FEMA: Federal Emergency Management Agency:
HHS: Department of Health and Human Services:
NCEH: National Center for Environmental Health:
NIEHS: National Institute of Environmental Health Sciences:
NIH: National Institutes of Health:
NPL: National Priorities List:
OD: Office of the Director:
PHAGM: Public Health Assessment Guidance Manual:
SARA: Superfund Amendments and Reauthorization Act of 1986:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
April 30, 2010:
The Honorable Bart Gordon:
Chairman:
Committee on Science and Technology:
House of Representatives:
The Honorable Brad Miller:
Chairman:
The Honorable Paul C. Broun:
Ranking Member:
Subcommittee on Investigations and Oversight:
Committee on Science and Technology:
House of Representatives:
The Honorable F. James Sensenbrenner:
House of Representatives:
The Agency for Toxic Substances and Disease Registry (ATSDR)[Footnote
1] has faced concerns related to the quality of some of the public
health products it publishes. ATSDR is responsible for investigating
community exposures related to certain hazardous chemical sites and
releases; assessing associated health effects; recommending actions to
stop, prevent, or minimize harmful effects; and conducting health
studies.[Footnote 2] In conducting these activities, the agency
publishes many types of public health products, including public
health assessments, health consultations, health study reports, and
exposure investigations. In 1991, an expert panel we convened
conducted a detailed evaluation of the quality of a sample of 15 ATSDR
public health assessments and found that there were deficiencies with
these products.[Footnote 3] More recently, reports by the Institute of
Medicine[Footnote 4] and ATSDR's Board of Scientific Counselors
[Footnote 5] have identified various concerns such as the
appropriateness and quality of the data used in ATSDR's products, the
methodology and design of the studies, clearance policies, and the use
of external peer review and response to review comments.
Some members of the Congress have also expressed concern about the
quality of ATSDR's public health products. On April 1, 2008, the U.S.
House of Representatives' Subcommittee on Investigations and
Oversight, Committee on Science and Technology, held a hearing to
examine ATSDR's handling of the preparation of a health consultation
on the formaldehyde levels in trailers that the Federal Emergency
Management Agency (FEMA) provided to victims of Hurricanes Katrina and
Rita. The health consultation, which was published in February 2007,
raised congressional concerns about the quality of ATSDR's products
and the involvement of agency leadership in the issuance of a flawed
product.[Footnote 6] In response to the hearing and further
examination of ATSDR's role in the FEMA trailers health consultation,
the subcommittee issued a report in September 2008 to express its
heightened concern that ATSDR was issuing public health products of
poor quality.[Footnote 7] On March 12, 2009, the subcommittee held a
second hearing--The Agency for Toxic Substances and Disease Registry
(ATSDR): Problems in the Past, Potential for the Future?--which
focused on its continued concern about the quality of ATSDR's products.
ATSDR has responded to these concerns, noting that multiple factors
have posed challenges for the agency, including limitations in the
ability of available science to answer community questions about the
effect of chemical exposures, limitations in ATSDR's ability to
collect data related to exposures, and reductions since 2004 in the
number of ATSDR staff and resources available to conduct the agency's
mission. In his testimony for the March 2009 congressional hearing,
ATSDR's director at that time[Footnote 8] noted that the agency
intended to reexamine its approach to carrying out its mission in
light of these challenges, and had convened a National Conversation on
Public Health and Chemical Exposures which includes government,
community groups, and industry to create an agenda for revitalizing
the public health approach to chemical exposures, which would include
future direction for ATSDR.[Footnote 9]
You have expressed interest in ensuring the quality of ATSDR's public
health products. In this report we examine the extent to which ATSDR's
policies and procedures for product initiation, development, and
review and clearance provide reasonable assurance of public health
product quality.
To do our work, we reviewed ATSDR's policies and procedures and
interviewed officials to identify guidance related to the preparation
of public health products. Preparation of public health products
encompasses (1) initiation, which includes a decision by the agency to
begin work on a public health product and the assignment of staff to
prepare the product, (2) development, which includes management
approval to proceed with the development of a product and the actual
drafting of the public health product, and (3) review and clearance,
which is the process by which a product is internally or externally
reviewed and disseminated as a final public health product.[Footnote
10] We focused our review of ATSDR's policies and procedures on those
related to public health assessments, health consultations, exposure
investigations, and health study reports[Footnote 11] because these
products are considered core public health products by ATSDR and
concerns have been raised about the quality of products such as these,
in which ATSDR identifies potential exposures to hazardous chemicals
and assesses associated health effects. Throughout this report, we use
the phrase "public health products" to refer solely to those products
on which we focused our review: public health assessments, health
consultations, exposure investigations, and health studies. We
compared the policies and procedures ATSDR uses to guide the
preparation of its public health products to the standards described
in the Standards for Internal Control in the Federal Government
[Footnote 12],[Footnote 13] and the related Internal Control
Management and Evaluation Tool.[Footnote 14] We did not evaluate
ATSDR's policies and procedures on human capital, financial
management, or scientific and technical risk assessment. Additionally,
we did not review ATSDR products to assess their quality. Accordingly,
we do not express any view about their accuracy, completeness, or
scientific credibility.
In addition, to gain a better understanding of ATSDR and the policies
and procedures related to product preparation, we conducted a series
of small group interviews with ATSDR team leads[Footnote 15] and
nonmanagement employees in ATSDR's Headquarters with responsibilities
involving the preparation of public health products. Each small group
interview consisted of a group discussion to capture general themes
and opinions related to the policies and procedures. We conducted a
total of six small group interviews in ATSDR Headquarters, with no
more than eight employees in each interview. These small group
interviews included one meeting with team leads from various ATSDR
divisions; one meeting each with employees from the Division of
Toxicology and Environmental Medicine (DTEM), which participates in
preparing health consultations and provides technical expertise during
emergencies, and employees from the Division of Health Studies (DHS),
which is primarily responsible for the preparation of health study
reports; and one meeting with technical project officers who oversee
the work of cooperative agreement partners.[Footnote 16] Additionally,
we held two small group interviews with employees from the Division of
Health Assessment and Consultation (DHAC), which is primarily
responsible for the preparation of public health assessments, health
consultations, and exposure investigations. For the small group
interview with team leads, we selected employees who were identified
as team leads on the October 2009 personnel roster as well as
employees ATSDR management identified who performed the duties of team
leads. For all other small group interviews with nonmanagement
employees, we randomly selected individuals to interview from a
population of nonmanagement employees in each division obtained from
the October 2009 personnel roster.[Footnote 17] We submitted all the
names of the randomly selected individuals to ATSDR to ensure that
these individuals did not perform management duties and to coordinate
their availability for the interviews. Some employees who were
selected to participate in the small group interviews were not able to
attend. In total, we interviewed 33 team leads and nonmanagement
employees. To encourage the candor of the individuals who participated
in the small group interviews, we did not record their names in our
notes from those interviews and agreed not to share our notes with
ATSDR management. Additionally, at the conclusion of each of the six
small group interviews we administered a short questionnaire to the
participants to collect additional information about their
perspectives on the policies and procedures that guide their work, and
on improving public health product quality at the agency. Of the 33
questionnaires we distributed, we received 30 completed
questionnaires. The views expressed by these employees cannot be
generalized to all employees working within these divisions or in
these roles.
We also conducted six on-site small group interviews. We interviewed
employees in 3 of ATSDR's 10 Division of Regional Operations (DRO)
offices, and employees in 3 of 30 cooperative agreement partner
offices.[Footnote 18] We chose the three regions that issued the
greatest number of public health assessments and health consultations
in fiscal year 2008. Those regions were Region 5 (Illinois, Indiana,
Michigan, Minnesota, Ohio, and Wisconsin), Region 10 (Alaska, Idaho,
Oregon, and Washington), and Region 4 (Alabama, Florida, Georgia,
Kentucky, Mississippi, North Carolina, South Carolina, and Tennessee).
We interviewed the three cooperative agreement partners that were
located in the same states as our selected regional offices--the
Washington State Department of Health, the Illinois Department of
Public Health, and the Georgia Division of Public Health.
We also conducted interviews with officials, experts, and researchers
outside ATSDR to gain an understanding of ATSDR's relationship with
other agencies, to get their perspectives on ATSDR's work, and to
learn about the policies and procedures used by other prominent
scientific research organizations. We conducted interviews with
federal officials from the Centers for Disease Control and Prevention
(CDC); the Environmental Protection Agency (EPA), which ATSDR advises
about the health aspects of hazardous waste sites or spills; and the
National Institutes of Health's (NIH) National Institute of
Environmental Health Sciences (NIEHS), with which ATSDR collaborates
on various matters related to environmental health science. We also
conducted interviews with officials from two national scientific
research organizations, the National Academies and the National
Science Foundation. We interviewed two experts in environmental health
science who had experience working with ATSDR, and we spoke with two
advocacy organizations that work with communities that have been
affected by environmental health problems.
We conducted this performance audit from April 2009 to April 2010, in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
ATSDR investigates community exposures related to chemical sites and
releases; works with federal, tribal, state, and local agencies to
identify potential exposures; assesses associated health effects; and
recommends actions to stop, prevent, or minimize these harmful
effects, among other things.
ATSDR History:
ATSDR was established by the Comprehensive Environmental Response,
Compensation, and Liability Act of 1980 (CERCLA), which created what
is known as the Superfund program to clean up the nation's most
dangerous hazardous waste sites.[Footnote 19] CERCLA established ATSDR
to carry out Superfund's health-related activities, including the
establishment of a national registry of and the provision of medical
care and testing to persons exposed to toxic substances and the
provision of survey and screening programs to determine the
relationship between such exposure and illness.[Footnote 20] In 1985,
ATSDR was formally organized to begin to carry out its
responsibilities under the law. The Superfund Amendments and
Reauthorization Act of 1986 (SARA) broadened ATSDR's responsibilities
to include, among other things, public health assessments,
establishment and maintenance of toxicologic databases, information
dissemination, and health education. SARA required that ATSDR conduct
a public health assessment at each site proposed for or on the
National Priorities List (NPL),[Footnote 21] and authorized ATSDR to
perform public health assessments upon petition by an individual or
physician and to conduct additional follow-up health studies if
needed.[Footnote 22]
Sources of ATSDR Work and Types of Public Health Products:
ATSDR may initiate work to prepare public health products for several
reasons. Work can be necessitated pursuant to SARA by a site's
proposal to or listing on the NPL, requested by an ATSDR partner such
as EPA, negotiated as part of a work plan for federal facilities,
petitioned by individuals or physicians, or generated internally by
ATSDR officials. Once work is initiated, ATSDR may prepare any of
several different types of products, including the following:
* Public health assessments evaluate data and information on the
release of hazardous substances into the environment in order to
assess any past, current, or future impact on public health, develop
health advisories or other recommendations, and identify studies or
actions needed to evaluate and mitigate or prevent human health
effects.
* Health consultations review available information or collect new
data to respond to a specific health question or request for
information about a potential environmental hazard. Health
consultations are focused on a specific exposure issue and provide
guidance on the specific health-related question.
* Health studies are epidemiological research conducted to investigate
and characterize the association between exposure to chemicals in the
environment and health problems of people who have been exposed to
chemicals.
* Exposure investigations collect and analyze site-specific
environmental or biological samples to determine whether individuals
have been exposed to hazardous substances. Exposure investigations are
often designed to examine individuals most likely to be exposed to
hazardous substances, rather than a sample of individuals from the
exposed community that would provide information about the community
as a whole.[Footnote 23]
The time required to complete ATSDR public health products varies, and
may depend on the nature and complexity of the work site. For example,
some public health assessments and most health studies take one or
more years to complete, whereas some health consultations are
completed within weeks. ATSDR also prepares emergency response
products--most of which are completed within hours or days--which are
intended to help interpret the implications of exposure data. These
"real-time" investigations of health exposures include technical
assistance and health consultations, and are often initiated in
response to requests from agencies such as EPA or state health or
environmental departments.
ATSDR Organizational Structure:
Although ATSDR was established within the Department of Health and
Human Service's (HHS) Public Health Service, the Director of CDC
serves as the Administrator of ATSDR, and CDC performs many
administrative functions for ATSDR, such as human capital and
financial management services. ATSDR is located within CDC's Office of
Noncommunicable Diseases, Injury and Environmental Health. In 2003,
ATSDR's administrative functions were combined with those of CDC's
National Center for Environmental Health (NCEH). ATSDR and NCEH share
an Office of the Director (OD), which is led by a director and deputy
director.[Footnote 24] ATSDR has four divisions, each of which is
divided into either branches, programs, or regional offices (see fig.
1). Each division is led by a director and deputy director, and DHAC,
DHS, and DTEM each also have an associate director for science. Each
branch or program within a division is led by a chief.
Figure 1: ATSDR Organizational Structure:
[Refer to PDF for image: Organizational chart]
Top level: CDC Director:
Second level, reporting to CDC Director:
Office of Noncommunicable Diseases, Injury and Environmental Health:
Third level, reporting to Office of Noncommunicable Diseases, Injury
and Environmental Health:
ATSDR: Office of the Director (OD)[A]:
* Communication Science;
* Community Environmental Health;
* Financial and Administrative Services;
* Information Systems;
* Policy, Planning and Evaluation;
* Program Development;
* Terrorism Preparedness and Emergency Response;
* Science.
Fourth level, reporting to ATSDR: Office of the Director (OD):
Divisions:
Division of Health Assessment and Consultation (DHAC);
Branch/Regional Offices:
* Cooperative Agreement and Program Evaluation Branch;
* Exposure Investigations and Site Assessment Branch;
* Health Promotion and Community Involvement Branch;
* Site & Radiological Assessment Branch.
Division of Health Studies (DHS);
Branch/Regional Offices:
* Geographic Research, Analysis and Services Program;
* Health Investigations Branch;
* Surveillance and Registries Branch.
Division of Regional Operations (DRO);
Branch/Regional Offices:
* ATSDR Liaison Office EPA Headquarters/Washington, DC;
* 10 Regional Offices[B].
Division of Toxicology and Environmental Medicine (DTEM);
Branch/Regional Offices:
* Applied Toxicology Branch;
* Computational Toxicology Methods and Development Laboratory;
* Environmental Medicine and Educational Services Branch;
* Prevention, Response and Medical Support Branch.
Source: GAO analysis of CDC and ATSDR information.
[A] While the National Center for Environmental Health (NCEH) and
ATSDR share the same OD, the organizations perform different
functions. Our review focused only on those public health products
produced by ATSDR. Therefore, we have not included NCEH or its
divisions in this figure.
[B] The 10 regional offices are located in Boston, New York,
Philadelphia, Atlanta, Chicago, Dallas, Kansas City, Denver, San
Francisco, and Seattle.
[End of figure]
OD: As shown in figure 1, the OD has eight functional areas:
communication science; community environmental health; financial and
administrative services; information systems; policy, planning and
evaluation; program development; science; and terrorism preparedness
and emergency response. These functional areas are responsible for
providing scientific and programmatic support for agency staff and
conducting review and clearance for public health products produced by
ATSDR divisions. Specifically, the Office of Science is responsible
for the clearance, cross-clearance,[Footnote 25] and external peer
review of ATSDR public health products. The Office of Science also
coordinates the NCEH/ATSDR Board of Scientific Counselors, which
provides advice and guidance to ATSDR's director on external peer
review of ATSDR programs and issues including program goals,
objectives, strategies, and priorities.[Footnote 26] The board's
advice and guidance are intended to assist ATSDR in ensuring
scientific quality, timeliness, utility, and dissemination of
scientific results.
DHAC: DHAC produces a number of products, including public health
assessments, health consultations, and exposure investigations. DHAC's
Cooperative Agreement and Program Evaluation Branch is charged with
supporting and overseeing the work produced by ATSDR's cooperative
agreement partners, which currently include 29 state agencies and 1
tribal government (see fig. 2). In order to become a cooperative
agreement partner, state and tribal governments must respond to a
request for applications that ATSDR posts, and have their application
reviewed, scored, and funded by ATSDR. Currently, ATSDR funds
cooperative agreement partners for a 5-year funding cycle.[Footnote
27] Through these partnerships, ATSDR provides funding and technical
support for state and tribal government employees to assess
environmental health concerns at sites within their jurisdictions and
to conduct needed public health interventions. Cooperative agreement
partners prepare public health products that are monitored, reviewed,
and cleared by ATSDR. DHAC staff work with staff in regional offices
to provide technical assistance to cooperative agreement partners and
the public, and to sponsor activities in communities that have been
exposed to hazardous chemicals.
Figure 2: ATSDR Regions and Cooperative Agreement Partners:
[Refer to PDF for image: 2 U.S. maps]
ATSDR regions: map indicates geographic borders of the 10 ATSDR
regions.
States in each ATSDR region with cooperative agreements:
Region 1:
Connecticut;
Massachusetts;
New Hampshire.
Region 2:
New Jersey;
New York.
Region 3:
Pennsylvania;
West Virginia.
Region 4:
Florida;
Georgia;
North Carolina;
Tennessee.
Region 5:
Illinois;
Michigan;
Minnesota;
Ohio;
Wisconsin.
Region 6:
Arkansas;
Louisiana;
Texas.
Region 7:
Iowa;
Missouri.
Region 8:
Colorado;
Utah.
Region 9:
Arizona (Gila River Tribe);
California.
Region 10:
Alaska;
Idaho;
Oregon;
Washington.
Source: GAO analysis of CDC and ATSDR information.
[End of figure]
DHS: DHS is responsible for conducting epidemiologic health studies,
designing and conducting surveillance programs, and establishing and
maintaining registries. The division collects information to determine
whether a chemical exposure is making people sick, and collects data
on persons identified as having been exposed to a specific contaminant
or event. DHS is also involved with the ongoing collection, analysis,
and interpretation of health data. These products are released as
ATSDR reports and are often published in the scientific literature.
DRO: DRO staffs an ATSDR regional office within each of the 10 EPA
regions, in EPA Headquarters in Washington, D.C., and in two satellite
offices in Helena, Montana, and Anchorage, Alaska (see fig. 2). These
offices are responsible for acting as regional liaisons to establish
working relationships with EPA, other federal and state agencies,
individual citizens, and community groups to maintain current and
historic knowledge of issues related to hazardous chemical sites in
their regions. Working in collaboration with DHAC, DRO staff also
prepare a number of time-sensitive products, such as health
consultations.
DTEM: DTEM assists in the production of health consultations and
provides technical assistance in response to chemical spills and acute
events. DTEM staff also work closely with DRO staff to provide real-
time public health advice in case of a chemical release. DTEM is
responsible for serving as a resource for information and assistance
on toxic substances in the environment, and prepares toxicological
profiles[Footnote 28] for hazardous substances.
Internal Control:
The Standards for Internal Control in the Federal Government provides
the overall framework for establishing guidelines for internal
control, and helps government managers achieve desired objectives.
[Footnote 29] As applied to ATSDR, this could include the preparation
of quality public health products. Internal control, which is
synonymous with management control, comprises the plans, methods, and
procedures used to meet missions, goals, and objectives. The related
Internal Control Management and Evaluation Tool assists agencies in
maintaining or implementing effective control.[Footnote 30] Internal
control is not one event, but a series of actions and activities that
occur throughout an entity's operations and on an ongoing basis. The
responsibility of good internal control rests with all managers; they
set the objectives, put the control mechanisms and activities in
place, and monitor and evaluate these mechanisms and activities.
However, all employees in the organization play important roles in
this process. Internal control includes activities such as
establishing policies and procedures, assessing risks associated with
achieving agency objectives, ensuring effective information sharing
throughout the organization, conducting ongoing monitoring of agency
activities, and establishing key areas of authority and responsibility
for agency management and staff.
The Standards note, however, that while internal control helps
government managers achieve desired objectives, it cannot provide
absolute assurance that all agency objectives will be met. There are
many factors outside the control and influence of management that can
affect an agency's ability to achieve its objectives. For example,
human mistakes, judgment errors, and acts of collusion to circumvent
control can affect meeting agency objectives. Therefore, once in
place, internal control provides reasonable assurance, not absolute
assurance, that an agency's objectives are being achieved.
ATSDR's Policies and Procedures for Public Health Product Preparation
Lack Some Critical Controls to Provide Reasonable Assurance of Quality:
The policies and procedures that ATSDR has established for public
health product preparation lack some of the critical controls to
provide reasonable assurance of product quality. The controls that
have not been incorporated involve information flow, risk assessment,
and management roles, responsibilities, and monitoring. Although the
agency has established some policies and procedures to govern
initiation of work to prepare public health products, ATSDR lacks
policies and procedures that (1) establish how information about newly
initiated work should effectively flow between all levels of
management and staff, and (2) describe how to comprehensively assess
and categorize the risk of work being initiated at the agency.
Additionally, while some policies and procedures state the roles of
staff in product development, many do not identify the roles and
responsibilities of management. Moreover, although policies and
procedures include some routine oversight of product development, they
do not consistently require that management monitor the development of
key components of these products. Finally, while ATSDR has implemented
policies and procedures governing review and clearance, some sections
of the policies and procedures do not reflect current practices, and
the policies and procedures do not ensure that all products
consistently receive appropriate review.
ATSDR's Policies and Procedures for Work Initiation Do Not Establish
and Describe Information Flow or Adequate Assessment of Risk:
Although ATSDR has established some policies and procedures to govern
the initiation of work to prepare public health products, its policies
and procedures do not establish and describe how information about
newly initiated work should flow between all levels of management and
staff. Consequently, the agency cannot be certain that all management
and staff have the information they need to do their jobs effectively.
The Standards for Internal Control in the Federal Government states
that for an entity to run and control its operations, it must have
relevant, reliable, and timely communications relating to internal as
well as external events. Information is needed throughout the agency
to achieve all of its objectives, and effective communications should
occur in a broad sense with information flowing down, across, and up
the organization.
Since ATSDR has not established policies or procedures on how
information about newly initiated work should flow between all levels
of management and staff, it generally relies on various meetings held
at different levels throughout the agency to inform management and
staff about newly initiated work. ATSDR officials stated that when
site activities are controversial or of special interest, management
is informed through weekly Issues Management meetings and Senior Staff
meetings between ATSDR's OD and division directors. The former ATSDR
director said that product initiation had an important role in the
Issues Management meetings. According to ATSDR officials, information
from these meetings is shared with mid-and lower-level management
through notes and face-to-face meetings. Officials stated that newly
initiated work may also be discussed during other regular meetings
within the divisions and branches.
In addition to a lack of policies or procedures on information flow,
for several years ATSDR has operated with fragmented databases in
which information about newly initiated work is entered and tracked,
none of which are accessible to, or ensure information flows to,
people at all levels of the agency. ATSDR previously used a tracking
system called HazDat, which was taken off line in 2007.[Footnote 31]
Since that time, ATSDR management and staff have been without an
agencywide system that is capable of providing information about newly
initiated work to people at all levels of the agency. There are
several other agency databases that contain information about newly
initiated work, but none of these systems are accessible to people at
all levels of the agency. Examples of these databases include a DHAC
Tracking and Triage Database and a Petition Database.[Footnote 32]
However, ATSDR officials told us that access to the DHAC Tracking and
Triage Database was limited to management and staff within DHAC, and
that the Petition Database was accessible by only three agency
employees.[Footnote 33] Additionally, ATSDR officials told us that no
regularly scheduled reports were generated from these databases,
although division management was provided with a weekly update on
petitions under review.[Footnote 34] ATSDR's OD also created a
database about 2 years ago to track issues discussed during the weekly
Issues Management meetings. However, while the Issues Management
database is used as a tool during the meetings, it is not accessible
to division directors or other ATSDR management and staff outside of
the meeting.
Although ATSDR has not established policies or procedures that
establish and describe information flow within the agency, it is
implementing a new agencywide system called Sequoia, which may improve
the flow of information about newly initiated work between management
and staff.[Footnote 35] While data entry into Sequoia began in 2007,
the former director of ATSDR told us that resource limitations slowed
Sequoia's development and that the use of fragmented databases was a
temporary measure until Sequoia was completed.[Footnote 36] However,
officials told us that while they expected that Sequoia would replace
other existing databases, further evaluation is needed to determine if
Sequoia could do everything required by management or if some
information will still have to be captured in separate systems. ATSDR
officials told us that Sequoia was designed to track requests, cost
recovery reimbursement, exposure data, work flow for site-specific
products, and information pertaining to other products or services
done on particular sites. Sequoia includes some major features that
were not available in previous ATSDR systems, such as providing a
centralized database that is available to all ATSDR staff for tracking
incoming work requests, and providing a system for reporting and
retrieving information on the public health impact and outcome of
public health activities. According to agency officials, in January
2010 ATSDR employees began using Sequoia for planning site and project
activities, recording the results of their investigative and community
outreach efforts, and reporting the public health accomplishments of
their activities.
ATSDR also lacks comprehensive policies and procedures for assessing
and categorizing the risk of work being initiated at the agency. The
Standards for Internal Control in the Federal Government states that
effective internal control should provide for an assessment of the
risks the agency faces from both external and internal sources and
that management needs to comprehensively identify risks and consider
all significant interactions between the agency and other parties.
Risk identification methods may include qualitative and quantitative
ranking activities, management conferences, forecasting and strategic
planning, and consideration of findings from audits and other
assessments. Risk assessment also includes deciding how to manage the
risk and what actions should be taken, and the Internal Control
Management and Evaluation Tool notes that management should formulate
an approach for risk management and decide on the internal control
activities required to mitigate those risks.[Footnote 37]
ATSDR previously incorporated some of the principles of risk
assessment when the agency officially classified sites as "high
priority" or "focus sites." ATSDR officials told us that these sites
were typically identified by senior management and staff as those
sites where chemical exposures may be of significant concern, which
may require extensive agency resources, or may involve other site
complexities. If a site was classified as a focus site, which
typically occurred as work was being initiated, any public health
products resulting from that site were required to undergo a higher
level of review during review and clearance. However, ATSDR officials
told us that they stopped using these classifications several years
ago. Instead, agency officials and employees now use terms such as
"high profile concern" or "sites of interest" to refer to those sites
that might require additional review and clearance because they have
high interest from the media or the Congress, or involve issues of
difficult or emerging science. Officials stated that these sites were
now managed through meetings such as the Issues Management meeting.
They stated that they believed that the Issues Management process
incorporated many of the principles of risk assessment by enabling
senior agency management to identify and discuss important sites each
week. Nevertheless, terms such as high profile are not official agency
designations and do not trigger any additional required management
monitoring during product development or required higher levels of
review and clearance. Additionally, while certain high profile sites
may be identified as they are initiated and discussed during Issues
Management meetings, not all new sites are being reviewed by OD and
division management to assess and categorize the risk to the agency of
the public health products resulting from the sites.
One ATSDR division, however, uses a process with elements similar to
risk assessment in the way that it prioritizes work requests. DHAC
generally uses a triage process for all ATSDR work requests requiring
DHAC staff assistance. This process categorizes work requests as high,
medium, or low priority. A request's priority level is based on three
criteria, which in order of importance are extent of exposure, public
health impact, and community and political interest, according to an
ATSDR document explaining the DHAC triage process. A triage decision
team, consisting of management-level staff from DHAC and DRO, decides
on the priority level for the work request, and that information,
along with other information about the request, is tracked in the DHAC
Tracking and Triage Database. However, this information is used only
to prioritize DHAC work requests and assign staff accordingly. This
process is not used by other ATSDR divisions, and is not an official
agency designation that triggers any additional requirements for that
site or related public health products, such as additional management
monitoring during product development or required higher levels of
review and clearance. Because ATSDR does not currently have policies
and procedures that describe how the agency is to comprehensively
assess and categorize the risk of work it initiates to prepare public
health products, management cannot ensure that it has consistently
managed the risk related to all new work, or established product
preparation procedures commensurate with the risk.
ATSDR's Policies and Procedures for Product Development Do Not Provide
for Clear Management Roles and Responsibilities or Consistent
Monitoring of Product Development:
While some of ATSDR's policies and procedures state the roles of staff
in developing public health products, many do not identify the roles
and responsibilities of management for ensuring that staff follow
those policies and procedures. The Standards for Internal Control in
the Federal Government states that management is responsible for
developing the detailed policies, procedures, and practices to fit
their agency's operations. The Standards states that the agency's
organizational structure should clearly define key areas of authority
and responsibility and establish appropriate lines of reporting.
Internal control activities include approvals and the maintenance of
related records to help ensure that management's directives are
carried out. The Internal Control Management and Evaluation Tool also
states that managers and supervisors need to know their
responsibilities for internal control and need to make control and
control monitoring part of their regular operating processes.
The ATSDR documents that provide guidance on developing products do
not clearly delineate management roles and responsibilities. The
Public Health Assessment Guidance Manual (PHAGM) is the document that
officials and employees of DHAC, DRO, and cooperative agreement
partners identified as the primary document that guides their work.
The PHAGM describes how to analyze site-specific data, make
recommendations, and develop conclusion categories.[Footnote 38] This
document is used by DHAC, DRO, and cooperative agreement staff to
develop public health assessments, health consultations, and exposure
investigations. The PHAGM guides staff in developing these products,
but it does not establish lines of reporting or detail the
responsibilities of management for monitoring product development.
Additionally, although the PHAGM states that ATSDR promotes a team
approach in conducting the public health assessment process, it does
not describe how ATSDR management fits into this team approach. And
while ATSDR officials stated that the PHAGM was not developed as a
management guide, ATSDR does not have any other documents that provide
guidance to management on their responsibilities for monitoring the
development of public health assessments and health consultations. In
addition to the PHAGM, there are a number of chemical-and exposure-
specific and technical guidance documents that are used as
supplements, as well as guidance specific to site work. These
documents give additional information to staff on specific chemicals,
how and when to use certain scientific methods, and site team
procedures. However, like the PHAGM, these documents neither establish
lines of reporting nor detail the responsibilities of management for
monitoring product development. Furthermore, while the NCEH/ATSDR
Policy: Clearance of Information Products, which guides ATSDR's review
and clearance process, states that before a product is submitted for
clearance immediate supervisors should ensure that the product is
based on sound, ethical science and ensure the quality of the product,
the policy provides no further guidance to immediate supervisors on
carrying out these responsibilities. Because there is an absence of
clearly defined lines of reporting and roles and responsibilities of
management in these documents, management at various levels may not
understand their specific responsibilities for overseeing product
development.
Although ATSDR's policies and procedures include some routine
monitoring of the development of products produced by both ATSDR staff
and staff of cooperative agreement partners, they do not consistently
require that agency management monitor the development of key
components of these products. The Standards states that internal
control should be designed to ensure that ongoing monitoring occurs in
the course of normal operations, and this includes management reviews
of actual agency performance. ATSDR's policies and procedures require
monitoring of key components of health studies, which use a detailed
study protocol to guide a health study's development. ATSDR's Guidance
for ATSDR Health Studies, which provides ATSDR staff with instructions
on how to conduct a health study, states that a study protocol helps
to ensure the quality of a health study and includes components such
as a study's objectives, methodology, and timeline for completing key
activities and milestones of a health study. At a minimum, if the
study being conducted is deemed research, a study protocol is reviewed
and approved within the appropriate division and may be sent out for
scientific peer review before the health study begins. In addition,
any health study involving human subjects must also be submitted to
and approved by an established institutional review board.[Footnote
39] The guidance explains that ongoing health study reviews are
conducted to ensure that the study protocol is being followed,
appropriate changes are made, the project remains on its established
timeline, and enhancements to study quality are made when appropriate.
Exposure investigations also use protocols that must be approved
before the project is funded. These protocols include components such
as a statement of the project's objectives as well as data analysis
methods that will be used in completing the project.
In contrast, ATSDR's policies and procedures for the development of
public health assessments and health consultations, which are among
the agency's core products, do not require management monitoring of
key components of these products. For example, the PHAGM identifies an
exposure assessment[Footnote 40] and health effects evaluation as the
two primary technical components of the public health assessment
process.[Footnote 41] Other components of the public health assessment
process include data collection, community involvement, and
development of conclusions and recommendations. However, there is no
requirement that staff's work in any of these areas be reviewed and
approved by management during the development of a product to ensure
its accuracy and appropriateness. Furthermore, DHAC and DRO officials
told us that there was no formal requirement for management to monitor
or approve key components of public health products produced by their
divisions, such as the product's methodology. When asked about
monitoring requirements, a DRO official said that identifying the
expertise needed for work at a site during the DHAC triage process
helped to ensure that staff assigned to prepare a public health
product had the skill sets required to make knowledgeable decisions on
key components of a public health product. However, while identifying
staff with the needed expertise to develop a public health product at
initiation is beneficial, it is not a substitute for ongoing
monitoring, which allows problems to be identified and addressed if
they occur during a product's development. Further, during our small
group interviews one DHAC employee expressed concern that because
there were cases where only one person was developing a product, there
would be no one to monitor that work until the product was submitted
for review and clearance.
Although ATSDR's policies and procedures for the development of public
health assessments and health consultations do not require management
monitoring of key components of these products, ATSDR officials said
they held routine meetings during which issues specific to product
development could be discussed. According to ATSDR officials, division
directors schedule routine meetings with branch chiefs, and other team
and site meetings are held. For example, according to ATSDR officials,
DHAC and DHS officials meet with their respective division branch
chiefs at least once each week to discuss projects and collaborate on
site activities. However, none of the routine meetings described have
established requirements for monitoring the development of key
components of public health products. Additionally, weekly Issues
Management and Senior Staff meetings, which are attended by senior
division management, are used to discuss the work conducted at sites.
ATSDR officials said that product development may be monitored during
these meetings. However, these meetings rely on division management to
bring problems or concerns regarding product development to the
attention of the OD, and according to ATSDR officials, Issues
Management meetings focus only on "sites of interest." Thus, while
products related to "sites of interest" may be discussed at these
meetings, current ATSDR procedures do not ensure the discussion of key
components of products for ATSDR sites not identified as "sites of
interest." Additionally, items on the agenda of the Issues Management
meeting are not prioritized to ensure that the most significant
problems associated with the development of a public health assessment
or health consultation are promptly addressed.
Because ATSDR's policies and procedures do not describe management's
role for ensuring consistent monitoring of key product components,
problems occurring during the development of ATSDR public health
products may not be identified or addressed by management until the
review and clearance phase, if at all. For example, in December 2001,
the International Joint Commission requested ATSDR's assistance in
evaluating the public health implications of the presence of hazardous
materials in the Great Lakes region. According to ATSDR and Institute
of Medicine reports, problems with ATSDR's Great Lakes report were not
identified by management until the first draft of the document was
completed in April 2004. Due to scientific concerns identified in the
document once review and clearance began, the document underwent
several years of reviews and revisions, and a final report was not
issued until December 2008.[Footnote 42]
ATSDR's Review and Clearance Policies and Procedures Do Not Always
Reflect Current Practices and Do Not Establish a Process for Ensuring
Consistent Review of All Products:
While ATSDR has implemented policies and procedures governing the
review and clearance of its public health products, some sections of
ATSDR's review and clearance policies and procedures do not reflect
current practices. The Standards for Internal Control in the Federal
Government states that management is responsible for developing
detailed policies, procedures, and practices to fit their agency's
operations and ensuring that they are built into and become an
integral part of operations. Additionally, the Internal Control
Management and Evaluation Tool calls for policies and procedures to be
regularly evaluated to ensure that they are still appropriate and
working as intended. ATSDR uses the NCEH/ATSDR Policy: Clearance of
Information Products to guide the review and clearance process.
[Footnote 43] The clearance policy includes the NCEH/ATSDR Clearance
Quick-Reference Guide, which outlines the required levels of review
and clearance for each type of public health product.[Footnote 44] The
clearance policy states that public health products may undergo
required or discretionary review. The policy requires that all public
health products be cleared through the initiating division,[Footnote
45] and many public health products require additional review, such as
review by the Office of Science. Some public health products may also
undergo additional discretionary review when the originating division
believes that a division outside of the required review process should
be consulted. However, some sections of the NCEH/ATSDR Policy:
Clearance of Information Products do not reflect current ATSDR
practices. For example, the policy "highly recommends" that all public
health products be reviewed and cleared by at least four individuals:
the immediate supervisor, the branch chief, the associate director for
science, and the division director.[Footnote 46] In addition, the
policy's NCEH/ATSDR Clearance Quick-Reference Guide indicates that all
public health assessments, health consultations, and exposure
investigations must be reviewed and cleared by the division director
or the division associate director for science. Yet according to DHAC
management and staff, the review and clearance of DHAC products
usually stops after review by branch chiefs within the division.
[Footnote 47] Additionally, because the NCEH/ATSDR Clearance Quick-
Reference Guide is several years old, it does not describe the review
and clearance requirements for new types of agency products such as
"letter health consultations," which agency officials described as an
expedited version of a health consultation.[Footnote 48] Also, the
NCEH/ATSDR Clearance Quick-Reference Guide indicates that public
health assessments, health consultations, and exposure investigations
at "high priority" sites or "focus sites" must receive additional
levels of review, but, as noted above, the agency no longer uses these
designations.
As of February 2010, the NCEH/ATSDR Policy: Clearance of Information
Products also did not reflect current practices because it did not
direct staff to use a CDC-required electronic clearance system called
Documentum.[Footnote 49] The current clearance policy was effective in
March 2006, prior to implementation of Documentum. Documentum is an
electronic tool used by ATSDR to route public health products to the
appropriate staff for review and clearance and to track the progress
of each product during the process. In November 2009, officials told
us that the agency planned to issue a revised clearance policy by the
end of 2009. Additionally, although ATSDR officials said that staff
should be having their products electronically reviewed and cleared
through Documentum, management and staff told us during interviews
that not all documents were being cleared using this system. Instead,
documents that were not being entered into Documentum were being
reviewed and cleared using a manual version of the review and
clearance process. In November 2009, one ATSDR official estimated that
only about 20 percent of DHAC documents, which include public health
assessments, health consultations, and exposure investigations, were
cleared using Documentum.[Footnote 50] However, per CDC policy, as of
January 2010 all CDC centers, including ATSDR, were required to use
Documentum to review and clear all agency products. In February 2010,
ATSDR officials stated that all ATSDR divisions were currently using
Documentum for the review and clearance of all documents but that the
agency was still working to revise and update the clearance policy.
Because some sections of ATSDR's review and clearance policies and
procedures do not reflect current agency practices, staff cannot rely
on them to accurately and consistently determine what review and
clearance procedures to follow. Additionally, because there has not
been uniform compliance with using Documentum, agency officials have
been limited in their ability to track the review and clearance
history for all of the agency's products and to ensure that the
appropriate level of review was being conducted.
In addition to not reflecting current practices, ATSDR's policies and
procedures governing product review and clearance do not establish a
process for ensuring that all products consistently receive
appropriate review. The agency's clearance policy and procedures
generally direct management and staff to use discretion to identify
products that warrant a higher level of review, rather than
determining review and clearance levels through a risk assessment
process. As stated above, the Standards for Internal Control in the
Federal Government states that effective internal control should
provide for an assessment of the risks the agency faces, and that
management needs to comprehensively identify risks and consider all
significant interactions between the agency and other parties.
However, the level of review and clearance that ATSDR products undergo
varies by product type, rather than being determined by a
comprehensive risk assessment of that particular product or site. For
example, health study reports prepared by ATSDR staff are required to
be reviewed and cleared by the originating division and ATSDR's Office
of Science, and some must also undergo external peer review. In
contrast, most public health assessments, health consultations, and
exposure investigations are not required to be reviewed and cleared by
ATSDR management any higher than DHAC branch chiefs.[Footnote 51]
Based on the discretion of management and staff, some public health
assessments, health consultations, and exposure investigations may
also be submitted for additional review if they meet certain criteria.
According to the ATSDR clearance policy, discretionary review is
warranted when management or staff determines that a document (1)
contains new or revised ATSDR policy (2) could have a high degree of
visibility or (3) contains highly sensitive information. In addition,
the DHAC Director has issued his own informal criteria to indicate
which public health assessments and health consultations should
undergo additional review beyond the branch chief level.[Footnote 52]
However, even though ATSDR and DHAC have established criteria, there
is no required point during a product's preparation where management
and staff collectively determine whether a product meets the criteria,
and if additional review is warranted. Because ATSDR does not conduct
a comprehensive risk assessment of its products or sites, and its
policies and procedures instead rely on management and staff
discretion to make these determinations, the agency cannot ensure its
products consistently receive the appropriate level of review and
clearance.[Footnote 53]
Management and staff discretion is also required in determining
whether a public health assessment or health consultation should be
submitted for external peer review. The ATSDR Peer Review Policy
describes which public health products require external peer review,
and states that all studies, results, or research that ATSDR carries
out or funds in whole or in part must be peer reviewed. However, the
policy specifically identifies public health assessments as one of the
products that ATSDR does not consider "studies, results, or research."
[Footnote 54] Because public health assessments are not required to
undergo external peer review, ATSDR officials told us that management
or staff could use their discretion to determine that a public health
assessment or health consultation should be submitted for external
peer review.[Footnote 55] According to ATSDR data, only 2 of the 282
public health assessments and health consultations that were published
in fiscal year 2008 underwent external peer review.[Footnote 56],
[Footnote 57]
During the March 2009 hearing before the House Committee on Science
and Technology's Subcommittee on Investigations and Oversight, two
participants suggested that ATSDR's public health assessments and
health consultations should be required to undergo external peer
review as a way to help ensure their quality.[Footnote 58] A 2000
National Research Council report about peer review practices at EPA
noted that peer review could promote efficiency if conducted in the
early stages of a product's development, as well as assess and
potentially improve the end products of scientific work.[Footnote 59]
However, the report also noted that peer review had limitations, in
that peer review could not substitute for technically competent work
in the development of a product and could not ensure that regulatory
policies and actions would be based on good science.
Regional employees, ATSDR team leads, and nonmanagement employees in
ATSDR Headquarters expressed mixed opinions to us about the use of
external peer review for ATSDR public health products. In responses to
a short questionnaire we administered during interviews with team
leads and nonmanagement employees, 80 percent (24 of 30) said that
external peer review would be either beneficial or sometimes
beneficial in ensuring the quality of ATSDR public health products.
Some of these employees reported that using external peer review may
increase perceptions of the objectivity, credibility, and strength of
their public health products. With regard to limitations, some
employees reported that external peer review could cause further
delays in the review and clearance process. Similarly, others
suggested that external peer review should be conducted only for very
complicated public health products or products with high levels of
community concern or congressional interest.
Conclusions:
While administrative and management controls cannot guarantee product
quality, they can help ensure the development of timely and credible
public health products at ATSDR. And although ATSDR has established
some policies and procedures to govern the preparation of its public
health products, it lacks some critical controls to provide reasonable
assurance of product quality, particularly for public health
assessments, health consultations, and exposure investigations. The
controls that have not been incorporated involve information flow,
risk assessment, management roles and responsibilities, and monitoring.
The lack of an agencywide product tracking system at ATSDR has
hindered the effective flow of information about public health
products between all levels of staff and management. It has also
limited management's ability to monitor agency work and ensure that
resources are being allocated appropriately, placing the OD in a
reactive rather than leadership position with respect to the divisions
and the public health work it manages. Once the Sequoia system becomes
fully operational, management and staff should have a greater ability
to obtain and share information about the agency's site-specific work,
but it is too soon to determine whether they will take full advantage
of Sequoia's capabilities. Furthermore, once implemented, those
capabilities require that staff and cooperative agreement partners
input data into the system as was intended.
Additionally, without conducting risk assessments for the work being
undertaken by the agency and using those risk assessments to guide
agency processes for public health product preparation, ATSDR cannot
provide reasonable assurance that its products have undergone the
appropriate level of monitoring and review. If established, a risk
assessment process could be used to determine the proper level of
scrutiny throughout the initiation, development, and review and
clearance phases, including whether or not a product should undergo
external peer review, thereby ensuring that this determination is made
consistently across the agency. Basing this process on a set of
criteria, and documenting and tracking risk assessment decisions in
agency systems, should help ensure an effective process. ATSDR has
already incorporated some of the elements of risk assessment in the
existing DHAC triage process for categorizing the priority of work
requests.
Finally, because the agency's policies lack guidance for management
about their role in monitoring product development, ATSDR cannot be
sure that management has a clear understanding of the role they are
supposed to play in supervising a product's preparation. Additionally,
ATSDR's policies and procedures for the development of public health
assessments and health consultations do not require management's
monitoring and approval of key components of a product during its
development. Without adequate monitoring by management during a
product's development, product errors may not be caught or significant
publication delays may occur during the review and clearance phase,
potentially undermining public confidence in the agency's products.
Policies and procedures alone, however, cannot ensure the quality of
ATSDR's public health products and, as noted above, internal controls
provide only reasonable assurance, not absolute assurance, that an
agency's objectives are being achieved. Issues outside of the
influence of management, such as human mistakes, judgment errors, or
acts by employees to circumvent management control, could also affect
ATSDR's product quality. Nonetheless, improving ATSDR's policies and
procedures regarding public health product preparation would help the
agency provide greater assurance to those inside and outside the
agency of the quality of these products.
Recommendations for Executive Action:
To strengthen ATSDR's policies and procedures, and ensure that they
provide reasonable assurance of public health product quality, we
recommend that the director of ATSDR take the following two actions:
* Develop policies and procedures to ensure that a risk assessment is
conducted at the time site-specific work is initiated, and that any
assigned risk level be reevaluated throughout product preparation to
ensure that it remains appropriate.
* Revise existing policies and procedures, or develop new guidance, to
provide documented direction for various levels of management on their
roles and responsibilities in the monitoring of all products prior to
review and clearance, such as requirements for management monitoring
and approval of key components of these products.
Agency Comments and Our Evaluation:
ATSDR reviewed a draft of this report and provided written comments,
which appear in appendix I. While ATSDR neither agreed nor disagreed
with our recommendations and did not address them directly, in its
comments ATSDR stated that the agency has begun to incorporate our
recommendations.
Although ATSDR did not comment directly on our recommendation that the
agency conduct a risk assessment at the time site-specific work is
initiated and reevaluate the assessment throughout product
preparation, in its comments ATSDR stated that senior management was
looking into formalizing and unifying coordination, triage, and
prioritization of all incoming requests across the agency. ATSDR also
acknowledged a need to make its prioritization process more explicit
throughout the agency. It is imperative that ASTDR formalize its
processes agencywide and ensure that its processes include a risk
assessment to determine the proper level of scrutiny a product should
receive throughout its preparation, including whether or not it should
undergo external peer review.
Related to our recommendation that ATSDR revise or develop policies
and procedures to include direction for management in monitoring
products prior to review and clearance, ATSDR noted that its process
to formalize and unify coordination, triage, and prioritization of all
incoming requests was expected to include the specification of
management and staff roles and responsibilities from initiation
through publication. It is important that ATSDR take this step in
order to help ensure that management has a clear understanding of
their responsibilities in supervising a product's preparation.
In its comments, ATSDR noted that multiple guidelines are used to
conduct its work and it uses an issues management process for agency
risk management. Our findings document these guidelines and the issues
management process, and describe their limitations in establishing
effective information flow among all levels of management and staff,
in providing a comprehensive assessment and categorization of the risk
of work being initiated at the agency, and in identifying the roles
and responsibilities of management.
ATSDR also acknowledged that it would benefit from formalizing
additional internal controls, and stated that as part of its review of
the agency clearance policy it was incorporating a way to sample
documents that were previously cleared to ensure that scientific
principles are being applied across all divisions. ATSDR also stated
that it expected Sequoia, its agencywide electronic project tracking
system, to be fully implemented by the end of the year.
Finally, ATSDR commented that we did not assess public comment as a
part of our report, which it indicated was a critical component of the
agency's quality assurance process. While we agree that public comment
provides valuable input on those products which are subject to a
public comment period, it augments but does not substitute for
thorough internal review of a product or formal, external peer review
of a product by carefully selected experts.
ATSDR also provided technical comments, which we incorporated as
appropriate.
As we agreed with your offices, unless you publicly announce the
contents of this report earlier, we plan no further distribution of it
until 30 days from the report date. At that time, we will send copies
to the Secretary of Health and Human Services and other interested
parties. The report also will be available at no charge on GAO's Web
site at [hyperlink, http://www.gao.gov].
If you or your staffs have any questions about this report, please
contact me at (202) 512-7114 or bascettac@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix II.
Signed by:
Cynthia A. Bascetta:
Director, Health Care:
[End of section]
Appendix I: Comments from the Agency for Toxic Substances and Disease
Registry:
Department Of Health & Human Services:
Office Of The Secretary:
Assistant Secretary for Legislation:
Washington, DC 20201:
April 7, 2010:
Cynthia A. Bascetta:
Director, Health Care:
U.S. Government Accountability Office:
441 G Street N.W.
Washington, DC 20548:
Dear Ms. Bascetta:
Enclosed are comments on the U.S. Government Accountability Office's
(GAO) report entitled: "Agency For Toxic Substances And Disease
Registry: Policies and Procedures for Public Health Product
Preparation Should Be Strengthened" (GA0-10-449).
The Department appreciates the opportunity to review this report
before its publication.
Sincerely,
Signed by:
Andrea Palm:
Acting Assistant Secretary for Legislation:
Enclosure:
[End of letter]
General Comments of the Department of Health and Human Services (HHS)
on the Government Accountability Office's (GAO) Draft Report Entitled:
Agency for Toxic Substances and Disease Registry: Policies and
Procedures for Public Health Product Preparation Should be
Strengthened 'GAO-10-449):
The Agency for Toxic Substances and Disease Registry (ATSDR) wishes to
thank GAO for the opportunity to review and comment on this Draft
Report. ATSDR conducts valuable work and we appreciate GAO calling
attention to areas where we can improve the documentation and
functioning of our processes and controls.
As stated in the Report, GAO "did not evaluate ATSDR's policies and
procedures on scientific and technical risk assessment...did not
review ATSDR products to assess their quality,...[and does] not
express any view about their accuracy, completeness, or scientific
credibility." However, GAO concluded that the "policies and procedures
that ATSDR has established for public health product preparation lack
some of the critical controls to provide reasonable assurance of
product quality".
In practice, ATSDR has a number of guidelines that are being utilized
regarding site work, document preparation and clearance, and
scientific practices. ATSDR provided GAO with twenty-five guidance
documents that are currently employed, including the Public Health
Assessment Guidance Manual, SOPs for Site-Specific Team Work, Guidance
for Conducting a Review of Health Outcome Data, the Check List for the
Exposure Investigation Process, and the NCEH/ATSDR Clearance Policy.
In addition, as of January 1, 2010, all documents must be cleared
through the e-clearance system, which documents proper clearance by
the appropriate officials, and provides reporting that can be used by
management.
Currently, ATSDR ensures the proper handling of projects by discussing
resources needed and justification of the allocation of resources for
projects within and among ATSDR divisions. Each division has criteria
for prioritizing work and identifying potential sites or items of
interest; agency-wide risk management is handled through an issues
management process, which includes weekly meetings with senior staff.
GAO has not assessed public comment as a part of this report, which is
a very critical component of ATSDR's quality assurance process.
ATSDR's policies and procedures require public comment on all public
health assessments, as well as many of our other documents. Through
this process, ATSDR gains valuable comments from all affected parties,
including citizens, academic experts, and industry experts.
Nonetheless, ATSDR recognizes that we would benefit from formalizing
additional internal controls. NCEH/ATSDR is currently working on
revised clearance guidelines, which will incorporate a way to sample
documents that have been cleared at the division level, in order to
ensure that consistent scientific principles are being applied across
all divisions. We also expect to fully implement the new agency-wide
Sequoia system by the end of the year, which will track projects and
compile scientific data in a centralized location. New projects and
workplans are already being entered into the Sequoia system.
Senior management is also looking into formalizing and unifying
coordination, triage, and prioritization of all incoming requests
across the agency. This process is expected to include project
management and information flow, specify management and staff roles
and responsibilities from initiation through publication, and inform
the review and clearance process.
ATSDR expends significant time and attention on our priority sites and
work products. We acknowledge a need to make our prioritization
process more explicit throughout the agency, balancing process with
the timely production of quality products.
ATSDR does important and dependable work in a challenging environment,
and we continually incorporate improvements to the way in which we
conduct our work. ATSDR has already begun to incorporate the
recommendations, and we will continue to do so as we move forward.
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
Contact:
Cynthia A. Bascetta at (202) 512-7114 or bascettac@gao.gov:
Acknowledgments:
In addition to the contact named above, key contributors to this
report were Karen Doran, Assistant Director; George Bogart; Amy C.
Leone; Roseanne Price; Mario D. Ramsey; Christina Ritchie; and Carla
Willis.
[End of section]
Footnotes:
[1] ATSDR was established within the Public Health Service of the
Department of Health and Human Services. ATSDR is supported by the
Centers for Disease Control and Prevention (CDC) and located within
CDC's Office of Noncommunicable Diseases, Injury, and Environmental
Health.
[2] ATSDR is also responsible for educating the public and health care
professionals regarding contaminant exposures and for establishing
disease registries. However, these responsibilities are not the
subject of this report.
[3] GAO, Superfund: Public Health Assessments Incomplete and of
Questionable Value, [hyperlink,
http://www.gao.gov/products/GAO/RCED-91-178] (Washington, D.C.: Aug.
1, 1991). In May 2007, we issued a report that included an expert
panel evaluation of the design of one ATSDR health study. We reported
that the expert panel found that many parameters of that study were
appropriate, but that some experts suggested potential modifications
to the study. See GAO, Defense Health Care: Activities Related to Past
Drinking Water Contamination at Marine Corps Base Camp Lejeune,
[hyperlink, http://www.gao.gov/products/GAO-07-276] (Washington, D.C.:
May 11, 2007).
[4] Institute of Medicine, Review of ATSDR's Great Lakes Report Drafts
(Letter Report) (Washington, D.C.: National Academies Press, 2008).
[5] ATSDR's Board of Scientific Counselors is an advisory committee
that provides advice and guidance to the ATSDR Director. At ATSDR's
request, the Board of Scientific Counselors convened a work group to
evaluate the agency's peer review processes. The board issued a report
in March 2009; as of March 2, 2010, the report was not available on
ATSDR's Web site.
[6] Agency for Toxic Substances and Disease Registry, Health
Consultation: Formaldehyde Sampling at FEMA Temporary Housing Units,
Baton Rouge, Louisiana (Atlanta, Ga.: February 1, 2007).
[7] Majority Staff Report, Subcommittee on Investigations and
Oversight, Committee on Science and Technology, U.S. House of
Representatives, Toxic Trailers - Toxic Lethargy: How the Centers for
Disease Control and Prevention Has Failed to Protect the Public Health
(Washington, D.C.: September 2008).
[8] As of January 15, 2010, the former ATSDR director took a new
position at CDC, and an acting director has been appointed while a
search for a permanent director is conducted. This former ATSDR
director was the agency's director during the majority of the time our
audit work was conducted.
[9] Launched in 2009 by ATSDR and its companion organization--the
National Center for Environmental Health--the National Conversation on
Public Health and Chemical Exposures includes six work groups to
research and make recommendations on cross-cutting public health and
chemical exposure issues. Final work group reports will be submitted
to a Leadership Council for inclusion in a final action agenda in the
spring of 2011.
[10] In this report we use the term preparation when referring
collectively to the phases of initiation, development, and review and
clearance of public health products.
[11] In fiscal year 2008, ATSDR issued 60 public health assessments,
222 health consultations, 10 exposure investigations, and 9 health
study reports.
[12] See GAO, Standards for Internal Control in the Federal
Government, [hyperlink,
http://www.gao.gov/products/GAO/AIMD-00-21.3.1] (Washington, D.C.:
November 1999). Internal control is synonymous with management control
and comprises the plans, methods, and procedures used to meet
missions, goals, and objectives.
[13] The Office of Management and Budget's (OMB) Circular No. A-123
also defines management's responsibility for internal control in
federal agencies. The internal control standards and the definition of
internal control used in this circular are based on GAO's Standards
for Internal Control in the Federal Government. See OMB Circular No. A-
123, (Revised): Management's Responsibility for Internal Control (Dec.
21, 2004).
[14] The Internal Control Management and Evaluation Tool is based on
the Standards for Internal Control in the Federal Government, and it
is intended to provide a systematic approach to assessing an agency's
internal control structure. It is one in a series of related documents
we have issued to assist agencies in improving or maintaining
effective operations. See GAO, Internal Control Management and
Evaluation Tool, [hyperlink, http://www.gao.gov/products/GAO-01-1008G]
(Washington, D.C.: August 2001).
[15] ATSDR team leads are located in various ATSDR divisions and can
have supervisory responsibilities, including assigning and planning
work for staff, and monitoring and reporting on work progress to
management.
[16] ATSDR cooperative agreement partners are state agencies and one
tribal government that ATSDR provides with funding and technical
support to assess environmental health concerns at sites within their
jurisdiction and to conduct or coordinate appropriate public health
interventions. Cooperative agreement partners prepare public health
products that are monitored, reviewed, and cleared by ATSDR.
[17] We excluded employees with purely administrative responsibilities
from the population of nonmanagement employees in each division.
[18] We interviewed all employees in these offices who were available
to participate on the day of the interview.
[19] Pub. L. No. 96-510, 94 Stat. 2767. Under this law, EPA has
responsibility to clean up highly contaminated waste sites and address
the threats that these sites pose to human health and the environment.
[20] Pub. L. No. 96-510, § 104(i), 94 Stat. 2778-2779.
[21] NPL is a list of seriously contaminated hazardous waste sites
that have been identified by the Superfund program.
[22] Pub. L. No. 99-499, § 110, 100 Stat. 1613, 1636-1642. SARA
requires that public health assessments include preliminary
assessments of potential risk to human health based on such factors as
the nature and extent of site contamination, the potential pathways of
human exposure, the size and susceptibility of the community, and the
effects of exposure associated with identified hazardous substances.
SARA lists two purposes for health assessments--helping to decide
whether (1) actions should be taken to reduce human exposure to a
site's hazardous substances, and (2) additional information on human
exposure and associated health risks is needed and should be acquired.
[23] According to ATSDR, exposure investigations are not generalizable
beyond the population studied.
[24] ATSDR and NCEH are collectively known as NCEH/ATSDR. While these
organizations share the same OD, they perform different functions. Our
review focused only on those public health products produced by ATSDR.
[25] Cross-clearance involves review and clearance of public health
products by other persons or divisions that may have been involved in
the production of the public health product, such as providing data or
having staff serve as coauthors, or that may be affected by the
product's content. Cross-clearance is conducted both within NCEH/ATSDR
and across other centers at CDC.
[26] The Board of Scientific Counselors, an advisory committee
chartered under the Federal Advisory Committee Act, also provides
advice and assistance to the Secretary of HHS and the Director of CDC.
[27] The opportunity for a state or tribe to become a partner under a
cooperative agreement occurs only at the beginning of the funding
cycle. During the funding cycle, current state and tribal partners are
asked to submit continuation applications and are funded based on
available funding. According to ATSDR officials, the next cooperative
agreement funding cycle, which will begin April 1, 2011, will be
adjusted to a 3-year period.
[28] Toxicological profiles summarize, interpret, and evaluate
available data and possible health effects of hazardous substances
found at NPL sites, and substances that pose the most significant
potential threat to human health as determined by ATSDR and EPA. These
products are typically developed in 2-year cycles.
[29] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1].
[30] [hyperlink, http://www.gao.gov/products/GAO-01-1008G].
[31] ATSDR officials said that they took HazDat off line because it
became outdated after CDC updated its own system and no longer
provided support for HazDat.
[32] The DHAC Tracking and Triage Database is used to track work
requests assigned to DHAC staff, including public health assessments,
health consultations, and exposure investigations authored by staff
within DHAC and cooperative agreement partners. ATSDR officials said
that petition requests from individuals or groups are evaluated by a
Petition Coordinator, a Petition Evaluation Team, and the division
director, and tracked in the Petition Database.
[33] The three ATSDR employees that have access to the Petition
Database are the Petition Coordinator, one public health analyst, and
one administrative specialist within DHAC.
[34] ATSDR stated that these databases are also used to satisfy annual
reporting requirements to the Congress and the Office of Management
and Budget.
[35] In addition to providing information about newly initiated work,
Sequoia will also be able to provide information about products in
various stages of development, as well as products that have already
been issued.
[36] ATSDR officials stated that major system development of Sequoia
was expected to be completed by September 2010, and data entry
completed by December 2010.
[37] The risk assessment process described here is a management
control process and is distinct from and not related to the risk
assessment process used by EPA at Superfund sites. EPA uses risk
assessment to characterize the nature and magnitude of health risks to
humans (e.g., residents, workers, recreational visitors) and
ecological receptors (e.g., birds, fish, wildlife) from chemical
contaminants and other stressors that may be present in the
environment.
[38] ATSDR has established five distinct conclusion categories, which
are based on the level of public health hazard that a site or
hazardous substance might pose.
[39] Institutional review boards review and monitor human subjects
research, with the intended purpose of protecting the rights and
welfare of the research subjects.
[40] An exposure assessment is the process of finding out how people
come into contact with a hazardous substance, how much of the
substance they are in contact with, and where the substance is
located. An exposure assessment reviews data collected by other
federal and state government agencies, and differs from an exposure
investigation in which ATSDR staff collect and analyze site-specific
environmental or biological samples to determine whether individuals
have been exposed to hazardous substances.
[41] The public health assessment process is the method that ATSDR
uses to evaluate the public health implications of exposures to
environmental contamination. While this process bears the name of an
ATSDR product, the public health assessment process itself may lead to
a variety of products, including the public health assessment and the
health consultation.
[42] See Institute of Medicine of the National Academies, Review of
ATSDR's Great Lakes Report Drafts (Letter Report) (Washington, D.C.:
2008); Agency for Toxic Substances and Disease Registry, ATSDR Studies
on Chemical Releases in the Great Lakes Region (Atlanta, Ga.: 2008);
and Agency for Toxic Substances and Disease Registry, Statement of
Scientific Concerns About the Draft Report, Public Health Implications
of Hazardous Substances in the Twenty-Six U.S. Great Lakes Areas of
Concern (Atlanta, Ga.: 2008).
[43] In its 2009 report, ATSDR's Board of Scientific Counselors
concluded that ATSDR's peer review process generally achieved agency
quality assurance goals, but identified six general areas of concern
and provided recommendations to address those concerns. Where the work
group had concerns or recommendations relevant to our findings, we
have included that information in this report.
[44] The ATSDR Board of Scientific Counselors report stated that this
guide was unnecessarily complex, could be simplified, and there was
evidence that not all branch managers knew about the guide or paid
attention to it.
[45] Public health products that involve coauthors from another
division or office; include content that directly pertains to relevant
policy in another division or office; include comments on the program
areas of another division or office; or include data collected and
maintained by another division or office are also required to be
reviewed and cleared by those divisions or offices. The clearance
policy refers to this process as cross-clearance.
[46] The policy states that, at a minimum, all public health products
should be reviewed and cleared by the division director or designee.
[47] Certain public health assessments and health consultations that
meet specific criteria are also required to be reviewed by the
associate director for science and the division director.
[48] Agency officials stated that letter health consultations are
subject to the same review and clearance requirements as health
consultations.
[49] The ATSDR Board of Scientific Counselors report also noted that
the use of Documentum was not included in the clearance policy and
stated that the policy should be updated.
[50] After we discussed this discrepancy with ATSDR officials, the
division director of DHAC issued a memorandum on November 20, 2009,
directing all DHAC staff to use Documentum for the review and
clearance of all DHAC documents.
[51] ATSDR officials told us that protocols for conducting an exposure
investigation are reviewed by the division associate director for
science.
[52] ATSDR officials told us that the DHAC associate director for
science must review public health assessments and health consultations
if they involve (1) a site that is categorized as an Urgent Public
Health Hazard or a Health Advisory site; (2) a site where the "health
call"--a determination of the health hazards present at the site--is
based on new, unique, or unusual approaches; (3) a high profile site
or site of interest; (4) a position that is in possible conflict with
EPA or other agencies; or (5) sites that involve nonroutine analysis.
The DHAC associate director for science is given the discretion to
also forward these documents to the division director for additional
review and clearance.
[53] As mentioned above, ATSDR previously incorporated some of the
principles of risk assessment when the agency officially classified
hazardous chemical sites as "high priority" or "focus sites," thereby
requiring products resulting from those sites to undergo specific
levels of review. However, ATSDR no longer uses these designations.
[54] This is consistent with SARA, which exempts health assessments
from required peer review. Pub. L. No. 99-499, § 110, 100 Stat. 1641.
[55] In commenting on ATSDR's external peer review policies, the ATSDR
Board of Scientific Counselors' report stated that there should be a
clear written policy on when external peer review is required and what
it constitutes.
[56] ATSDR employees told us that all products resulting from
Department of Energy sites are submitted for external peer review.
Both products that underwent external peer review in 2008 were public
health assessments conducted at Department of Energy sites. In
addition to the formal peer review completed for these two products,
ATSDR reported that it also solicited informal comments from one or
more subject-matter experts on four products before the products were
finalized in 2008.
[57] ATSDR officials told us that all public health assessments and
some health consultations are also made available to the public for
review and comment for 60 to 90 days. They stated that the agency
reviews all public health comments and provides responses to them.
[58] In 1991, we recommended that at least a sample of future ATSDR
public health assessments undergo external peer review. However, as
mentioned above, ATSDR does not currently have such a policy and
instead relies on management and staff discretion to determine which
public health assessments should be submitted for external peer review.
[59] National Research Council, Strengthening Science at the U.S.
Environmental Protection Agency: Research-Management and Peer-Review
Practices (Washington, D.C.: National Academies Press, 2000).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
