Agency for Toxic Substances and Disease Registry
Policies and Procedures for Preparing Public Health Products Should Be Strengthened Gao ID: GAO-10-741T May 20, 2010This testimony discusses the Agency for Toxic Substances and Disease Registry's (ATSDR) policies and procedures for product preparation. ATSDR investigates community exposures related to chemical sites and releases; works with federal, tribal, state, and local agencies to identify potential exposures; assesses associated health effects; and recommends actions to stop, prevent, or minimize these harmful effects. In conducting these activities, the agency publishes many types of public health products, including public health assessments, health consultations, health study reports, and exposure investigations. Recent reports by the Institute of Medicine and ATSDR's Board of Scientific Counselors have identified various concerns such as the appropriateness and quality of the data used in ATSDR's products, the methodology and design of the studies, and clearance policies. This committee has held two previous hearings that focused on its concern about the quality of ATSDR's products. In response, ATSDR has noted that multiple factors have posed challenges for the agency, including limitations in the ability of available science to answer community questions about the effect of chemical exposures, limitations in ATSDR's ability to collect data related to exposures, and reductions since 2004 in the number of ATSDR staff and resources available to conduct the agency's mission. This testimony addresses the extent to which ATSDR's policies and procedures for product initiation, development, and review and clearance provide reasonable assurance of public health product quality.
We found that the policies and procedures that ATSDR has established for public health product preparation lack some of the critical controls to provide reasonable assurance of product quality. To provide reasonable assurance that agency objectives are being met, federal internal control standards call for agencies to establish policies and procedures, assess risks associated with achieving agency objectives, ensure effective information sharing throughout the organization, monitor agency activities, and establish key areas of authority and responsibility for management and staff. We found that ATSDR's policies and procedures are deficient in the three phases of preparation of public health products: (1) initiation, which includes a decision by the agency to begin work on a public health product and the assignment of staff to prepare the product; (2) development, which includes management approval to proceed with the development of a product and the actual drafting of the public health product; and (3) review and clearance, which is the process by which a product is internally or externally reviewed and disseminated as a final public health product.
GAO-10-741T, Agency for Toxic Substances and Disease Registry: Policies and Procedures for Preparing Public Health Products Should Be Strengthened
This is the accessible text file for GAO report number GAO-10-741T
entitled 'Agency for Toxic Substances and Disease Registry: Policies
and Procedures for Preparing Public Health Products Should Be
Strengthened' which was released on May 20, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Testimony:
Before the Subcommittee on Investigations and Oversight, Committee on
Science and Technology, House of Representatives:
United States Government Accountability Office: GAO:
For Release on Delivery:
Expected at 9:00 a.m. EDT:
Thursday, May 20, 2010:
Agency for Toxic Substances and Disease Registry:
Policies and Procedures for Preparing Public Health Products Should Be
Strengthened:
Statement of Cynthia A. Bascetta:
Director, Health Care:
GAO-10-741T:
[End of section]
Mr. Chairman and Members of the Subcommittee:
I am pleased to be here today to discuss the Agency for Toxic
Substances and Disease Registry's (ATSDR) policies and procedures for
product preparation. ATSDR investigates community exposures related to
chemical sites and releases; works with federal, tribal, state, and
local agencies to identify potential exposures; assesses associated
health effects; and recommends actions to stop, prevent, or minimize
these harmful effects. In conducting these activities, the agency
publishes many types of public health products, including public
health assessments, health consultations, health study reports, and
exposure investigations. Recent reports by the Institute of Medicine
[Footnote 1] and ATSDR's Board of Scientific Counselors[Footnote 2]
have identified various concerns such as the appropriateness and
quality of the data used in ATSDR's products, the methodology and
design of the studies, and clearance policies.
This committee has held two previous hearings that focused on its
concern about the quality of ATSDR's products. In response, ATSDR has
noted that multiple factors have posed challenges for the agency,
including limitations in the ability of available science to answer
community questions about the effect of chemical exposures,
limitations in ATSDR's ability to collect data related to exposures,
and reductions since 2004 in the number of ATSDR staff and resources
available to conduct the agency's mission. My testimony is based on
our April 2010 report,[Footnote 3] which is being publicly released
today, and addresses the extent to which ATSDR's policies and
procedures for product initiation, development, and review and
clearance provide reasonable assurance of public health product
quality.
To address this question, we reviewed ATSDR's policies and procedures
and interviewed officials to identify guidance related to the
preparation of public health products. We focused our review on those
policies and procedures related to public health assessments, health
consultations, exposure investigations, and health study reports
because these products are considered to be ATSDR's core public health
products and concerns have been raised about the quality of products
such as these, in which ATSDR identifies potential exposures to
hazardous chemicals and assesses associated health effects. We
compared the policies and procedures ATSDR uses to guide the
preparation of its public health products to the standards described
in the Standards for Internal Control in the Federal Government,
[Footnote 4],[Footnote 5] and the related Internal Control Management
and Evaluation Tool.[Footnote 6]
We also interviewed employees in ATSDR's headquarters, employees in 3
of ATSDR's 10 regional offices, and employees in 3 of 30 cooperative
agreement partner offices to gain a better understanding of ATSDR and
the policies and procedures related to product preparation. Further,
we conducted interviews with officials, experts, and researchers
outside ATSDR to gain an understanding of ATSDR's relationship with
other agencies, to get their perspectives on ATSDR's work, and to
learn about the policies and procedures used by other prominent
scientific research organizations. A full description of our scope and
methodology is included in our report.
We conducted this performance audit from April 2009 to April 2010, in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
In brief, we found that the policies and procedures that ATSDR has
established for public health product preparation lack some of the
critical controls to provide reasonable assurance of product quality.
To provide reasonable assurance that agency objectives are being met,
federal internal control standards call for agencies to establish
policies and procedures, assess risks associated with achieving agency
objectives, ensure effective information sharing throughout the
organization, monitor agency activities, and establish key areas of
authority and responsibility for management and staff. We found that
ATSDR's policies and procedures are deficient in the three phases of
preparation of public health products: (1) initiation, which includes
a decision by the agency to begin work on a public health product and
the assignment of staff to prepare the product; (2) development, which
includes management approval to proceed with the development of a
product and the actual drafting of the public health product; and (3)
review and clearance, which is the process by which a product is
internally or externally reviewed and disseminated as a final public
health product.
* ATSDR's policies and procedures for work initiation do not establish
and describe an adequate assessment of risk or information flow. When
work is being initiated, we found that ATSDR lacks comprehensive
policies and procedures for assessing and categorizing the risk of
that new work. For example, ATSDR previously incorporated some of the
principles of risk assessment when the agency officially classified
some hazardous chemical sites as "high-priority" or "focus sites," and
required any products resulting from review of those sites to undergo
a higher level of review and clearance. However, it no longer does so.
Because ATSDR does not currently have policies and procedures that
describe how the agency is to comprehensively assess and categorize
the risk of work it initiates to prepare public health products,
management cannot ensure that it has consistently managed the risk
related to new work. Furthermore, since ATSDR's policies and
procedures do not establish how information about newly initiated work
should flow between management and staff, ATSDR generally relies on
various meetings to inform management and staff about new work. The
agency is implementing a new database called Sequoia, which may
improve the flow of information. However, officials told us that
further evaluation is needed to determine if Sequoia could do
everything required by management or if some information will have to
be captured in separate databases.
* ATSDR's policies and procedures for product development do not
provide for clear management roles and responsibilities or consistent
monitoring of product development. During product development, many of
ATSDR's policies and procedures do not clearly define management roles
and responsibilities and do not consistently require that management
monitor the development of key components of these products. For
example, the primary document--the Public Health Assessment Guidance
Manual--that guides the development of public health assessments and
health consultations, which are among the agency's core products,
identifies exposure assessment[Footnote 7] and health effects
evaluation as the two primary technical components of the public
health assessment process. However, there is no requirement that
staff's work in either of these areas be reviewed and approved by
management during the development of a product to ensure its accuracy
and appropriateness. Because of these deficiencies, management may be
unclear about its responsibilities, and problems that occur during
product development may not be identified or addressed until review
and clearance, if at all. For example, ATSDR and Institute of Medicine
reports show that because scientific concerns were not identified
during development of an ATSDR report regarding chemical releases in
the Great Lakes region, the document underwent several years of
review, and a final report was not issued until more than 4 years
after the first draft was written.
* ATSDR's review and clearance policies and procedures do not always
reflect current practices and do not establish a process for ensuring
consistent review of all products. We found that some review and
clearance policies do not reflect current practices. For example,
ATSDR's Clearance Quick-Reference Guide indicates that all public
health assessments, health consultations, and exposure investigations
must be reviewed and cleared by the division director or the division
associate director for science. Yet according to Division of Health
Assessment and Consultation (DHAC) management and staff, the review
and clearance of DHAC products usually stops after review by branch
chiefs within the division.[Footnote 8] Furthermore, review and
clearance policies and procedures direct management and staff to use
discretion to identify products that require higher levels of review,
rather than making this determination through a comprehensive risk
assessment process. While ATSDR policy sets out criteria for when
additional review may occur, such as when a document could have a high
degree of visibility, there is no required point during a product's
preparation when management and staff collectively determine whether a
product meets the criteria, and whether additional review is
warranted. Thus, the agency cannot ensure that all products
consistently receive the appropriate level of review.
In conclusion, while management controls alone cannot guarantee
product quality, they can help ensure the development of timely and
credible public health products at ATSDR. But ATSDR lacks some
critical controls to provide reasonable assurance of product quality,
particularly for public health assessments, health consultations, and
exposure investigations. Without assessing the risk of work being
undertaken by the agency and using those risk assessments to guide
agency processes for public health product preparation, ATSDR cannot
provide reasonable assurance that its products have undergone the
appropriate level of monitoring and review. If established, a risk
assessment process could be used to determine the proper level of
scrutiny for the initiation, development, and review and clearance
phases, thereby ensuring that this determination is made consistently
across the agency. Additionally, the agency's policies lack guidance
for management about its role in monitoring product development, and
do not require management's monitoring and approval of key components
of a product during its development. Without adequate monitoring by
management during a product's development, product errors may not be
caught or significant publication delays may occur during the review
and clearance phase, potentially undermining public confidence in the
agency's products.
Our report recommends that ATSDR develop policies and procedures to
ensure that an assessment of the risk associated with a product is
conducted at the time site-specific work is initiated, and that any
assigned risk level be re-evaluated throughout product preparation to
ensure that it remains appropriate. Our report also recommends that
ATSDR revise existing policies and procedures, or develop new
guidance, to provide documented direction for various levels of
management on their roles and responsibilities in the monitoring of
all products prior to review and clearance, such as requirements for
management to monitor and approve key components of these products.
In commenting on a draft of the report from which this testimony is
based, ATSDR neither agreed nor disagreed with our recommendations and
did not address them directly, but stated that the agency has begun to
incorporate our recommendations. Although ATSDR did not comment
directly on our recommendation that the agency conduct a risk
assessment at the time site-specific work is initiated and reevaluate
the assessment throughout product preparation, in its comments ATSDR
stated that senior management was looking into formalizing and
unifying coordination, triage, and prioritization of all incoming
requests across the agency. ATSDR also acknowledged a need to make its
prioritization process more explicit throughout the agency. Related to
our recommendation that ATSDR revise or develop policies and
procedures to include direction for management in monitoring products
prior to review and clearance, ATSDR noted that its process to
formalize and unify coordination, triage, and prioritization of all
incoming requests was expected to include the specification of
management and staff roles and responsibilities from initiation
through publication.
Mr. Chairman, this concludes my statement. I would be pleased to
respond to any questions you or other Members of the Subcommittee may
have.
For questions about this testimony, please contact Cynthia A. Bascetta
at (202) 512-7114 or bascettac@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this testimony. Individuals who made key contributions to this
testimony include Karen Doran, Assistant Director; George Bogart;
Roseanne Price; Mario D. Ramsey; Christina Ritchie; and Carla Willis.
[End of section]
Footnotes:
[1] See Institute of Medicine, Review of ATSDR's Great Lakes Report
Drafts (Letter Report) (Washington, D.C.: National Academies Press,
2008).
[2] ATSDR's Board of Scientific Counselors is an advisory committee
that provides advice and guidance to the ATSDR Director. At ATSDR's
request, the Board of Scientific Counselors convened a work group to
evaluate the agency's peer review processes. The board issued a report
in March 2009; as of May 11, 2010, the report was not available on
ATSDR's Web site.
[3] See GAO, Agency for Toxic Substances and Disease Registry:
Policies and Procedures for Public Health Product Preparation Should
be Strengthened, [hyperlink, http://www.gao.gov/products/GAO-10-449]
(Washington, D.C.: Apr. 30, 2010).
[4] See GAO, Standards for Internal Control in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]
(Washington, D.C.: November 1999).
[5] See Office of Management and Budget Circular No. A-123, (Revised):
Management's Responsibility for Internal Control (Dec. 21, 2004).
[6] See GAO, Internal Control Management and Evaluation Tool,
[hyperlink, http://www.gao.gov/products/GAO-01-1008G] (Washington,
D.C.: August 2001).
[7] An exposure assessment is the process of finding out how people
come into contact with a hazardous substance, how much of the
substance they are in contact with, and where the substance is
located. An exposure assessment reviews data collected by other
federal and state government agencies, and differs from an exposure
investigation in which ATSDR staff collect and analyze site-specific
environmental or biological samples to determine whether individuals
have been exposed to hazardous substances.
[8] DHAC is one of four ATSDR divisions.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
