National Institutes of Health
Completion of Comprehensive Risk Management Program Essential to Effective Oversight Gao ID: GAO-09-687 September 11, 2009The National Institutes of Health (NIH), an agency of the Department of Health and Human Services (HHS), is the primary federal agency for supporting medical research. The Office of the Director (OD) is the central NIH office responsible for setting policy and overseeing NIH's 27 institutes and centers (IC). Allegations involving one institute raised questions about areas of oversight by the OD. In light of these questions, GAO examined (1) how NIH makes extramural research funding decisions and OD monitoring of this process, (2) the design of selected internal controls over NIH's travel and personnel appointment processes, and (3) the design of NIH's new risk management program and the program it is replacing. To address these objectives, GAO reviewed relevant NIH policies, procedures, and supporting documentation. GAO also selected 3 institutes that varied in size for in-depth reviews.
NIH is required by law to make its extramural research funding decisions--funding provided to scientists external to NIH such as those at universities--using a dual peer review system. During the first level, initial peer review groups assess applications and assign a score to them based on their scientific merit. During the second level, advisory councils review the applications and their scores and, on the basis of this review, recommend to the ICs certain applications for funding consideration. IC directors can use their discretion and choose to fund applications based on factors in addition to scientific merit, "skipping" over applications with higher scores or making "exceptions" to fund applications with lower scores. GAO found that in fiscal year 2007, IC directors funded about 19 percent of NIH's applications for a common type of grant based on factors in addition to scientific merit. However, the NIH OD does not monitor the extent to which IC directors use such discretion when making extramural funding decisions--an action that would be consistent with federal internal control standards. The NIH OD has established policies and procedures that incorporate key internal controls into the travel and personnel appointment processes. For example, the processes require multiple levels of review and approval. However, there is not an NIH-wide process for risk-based monitoring of the effectiveness of controls. Without monitoring actual implementation of controls based on assessed risk levels, NIH does not have adequate assurance that controls are operating as intended within those areas that have been identified as posing risks to the agency's ability to achieve its mission. NIH's Management Control Program, a risk management program updated in 2004, did not comprehensively address risks to the agency's overall operations and resulted in a lack of sufficient information for effective oversight and agencywide risk management. Recognizing this, in 2006, NIH began designing a new risk management program, the Enterprise Risk Management Program. Although an improvement over the earlier program, the design of the new program does not fully address the components identified in GAO's framework for effective risk management. For example, the design does not incorporate strategic goals and objectives as a precondition for risk management, the evaluation of alternative responses to address identified risks, or documentation of the rationale for selecting a risk response. Further, NIH's new program is not yet fully implemented, despite an over 3-year effort. According to NIH officials, NIH has experienced delays because of a change in contractors, balancing staff resources with competing demands, and underestimating time needed for implementation.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-09-687, National Institutes of Health: Completion of Comprehensive Risk Management Program Essential to Effective Oversight
This is the accessible text file for GAO report number GAO-09-687
entitled 'National Institutes of Health: Completion of Comprehensive
Risk Management Program Essential to Effective Oversight' which was
released on September 22, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Ranking Member, Committee on Finance, U.S. Senate:
United States Government Accountability Office:
GAO:
September 2009:
National Institutes of Health:
Completion of Comprehensive Risk Management Program Essential to
Effective Oversight:
GAO-09-687:
GAO Highlights:
Highlights of GAO-09-687, a report to the Ranking Member, Committee on
Finance, U.S. Senate.
Why GAO Did This Study:
The National Institutes of Health (NIH), an agency of the Department of
Health and Human Services (HHS), is the primary federal agency for
supporting medical research. The Office of the Director (OD) is the
central NIH office responsible for setting policy and overseeing NIH‘s
27 institutes and centers (IC). Allegations involving one institute
raised questions about areas of oversight by the OD. In light of these
questions, GAO examined (1) how NIH makes extramural research funding
decisions and OD monitoring of this process, (2) the design of selected
internal controls over NIH‘s travel and personnel appointment
processes, and (3) the design of NIH‘s new risk management program and
the program it is replacing. To address these objectives, GAO reviewed
relevant NIH policies, procedures, and supporting documentation. GAO
also selected 3 institutes that varied in size for in-depth reviews.
What GAO Found:
NIH is required by law to make its extramural research funding
decisions”funding provided to scientists external to NIH such as those
at universities”using a dual peer review system. During the first
level, initial peer review groups assess applications and assign a
score to them based on their scientific merit. During the second level,
advisory councils review the applications and their scores and, on the
basis of this review, recommend to the ICs certain applications for
funding consideration. IC directors can use their discretion and choose
to fund applications based on factors in addition to scientific merit, ’
skipping“ over applications with higher scores or making ’exceptions“
to fund applications with lower scores. GAO found that in fiscal year
2007, IC directors funded about 19 percent of NIH‘s applications for a
common type of grant based on factors in addition to scientific merit.
However, the NIH OD does not monitor the extent to which IC directors
use such discretion when making extramural funding decisions”an action
that would be consistent with federal internal control standards.
The NIH OD has established policies and procedures that incorporate key
internal controls into the travel and personnel appointment processes.
For example, the processes require multiple levels of review and
approval. However, there is not an NIH-wide process for risk-based
monitoring of the effectiveness of controls. Without monitoring actual
implementation of controls based on assessed risk levels, NIH does not
have adequate assurance that controls are operating as intended within
those areas that have been identified as posing risks to the agency‘s
ability to achieve its mission.
NIH‘s Management Control Program, a risk management program updated in
2004, did not comprehensively address risks to the agency‘s overall
operations and resulted in a lack of sufficient information for
effective oversight and agencywide risk management. Recognizing this,
in 2006, NIH began designing a new risk management program, the
Enterprise Risk Management Program. Although an improvement over the
earlier program, the design of the new program does not fully address
the components identified in GAO‘s framework for effective risk
management. For example, the design does not incorporate strategic
goals and objectives as a precondition for risk management, the
evaluation of alternative responses to address identified risks, or
documentation of the rationale for selecting a risk response. Further,
NIH‘s new program is not yet fully implemented, despite an over 3-year
effort. According to NIH officials, NIH has experienced delays because
of a change in contractors, balancing staff resources with competing
demands, and underestimating time needed for implementation.
What GAO Recommends:
To help improve oversight, GAO made three recommendations to the
Director of NIH: (1) monitor the extent to which IC directors use
discretion in funding decisions, (2) add key components to the
Enterprise Risk Management Program, and (3) ensure implementation of
the program. HHS disagreed with the first recommendation, partially
concurred with the second recommendation, and identified a final date
for implementation of the program.
View [hyperlink, http://www.gao.gov/products/GAO-09-687] or key
components. For more information, contact Linda T. Kohn at (202) 512-
7114 or kohnl@gao.gov or Susan Ragland at (202) 512-8486 or
raglands@gao.gov.
[End of section]
Contents:
Letter:
Background:
NIH Is Required to Use a Peer Review System to Make Extramural Funding
Decisions; NIH's OD Does Not Monitor Key Decisions in which IC
Directors Exercise Their Discretion Over Funding:
Design of NIH's Travel and Personnel Appointment Processes Includes Key
Control Activities and Some Monitoring Activities but Lacks Systemic
Risk-Based Monitoring:
NIH's Management Control Program and Enterprise Risk Management Program
Do Not Fully Address Key Components of Effective Risk Management:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: NIH Organization and Mission:
Appendix III: Comments from the National Institutes of Health:
Appendix IV: GAO Contacts and Staff Acknowledgments:
Tables:
Table 1: Extramural Research R01 Grant Applications Funded in Fiscal
Years 2003 through 2007:
Table 2: GAO's Risk Management Framework:
Table 3: Overview of ICs Including Establishment Date, Mission, and
Fiscal Year 2008 Appropriation:
Figures:
Figure 1: Relationship of the Risk Management Framework Components:
Figure 2: National Cancer Institute's Fiscal Year 2007 Payline for R01
Grant Applications:
Figure 3: Organizational Structure of NIH:
Abbreviations:
CSR: Center for Scientific Review:
COSO: Committee of Sponsoring Organizations:
GPRA: Government Performance and Results Act:
HHS: Department of Health and Human Services:
IC: institutes and centers:
NCI: National Cancer Institute:
NIAAA: National Institute on Alcohol Abuse and Alcoholism:
NIDDK: National Institute of Diabetes and Digestive and Kidney
Diseases:
NIEHS: National Institute of Environmental Health Sciences:
NIH: National Institutes of Health:
OD: Office of the Director:
OMB: Office of Management and Budget:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 11, 2009:
The Honorable Charles E. Grassley:
Ranking Member:
Committee on Finance:
United States Senate:
Dear Senator Grassley:
The National Institutes of Health (NIH) is the primary federal agency
for supporting medical research in the United States. In fiscal year
2008, NIH provided $24.4 billion--83 percent of its $29.5 billion
budget--in extramural research funding, which supports scientists and
research personnel working at universities, medical schools, and other
research institutions.[Footnote 1] NIH's extramural research funding
efforts reflect its large, decentralized organization. NIH comprises 27
institutes and centers (IC) and an Office of the Director (OD). Each of
the ICs has its own budget, mission, and staff and focuses on
particular diseases or research areas, such as cancer or aging issues.
Twenty-four of the 27 ICs fund extramural research, each with a
separate appropriation,[Footnote 2] and these ICs make final decisions
on which extramural research projects to fund following a standard
process defined by law and NIH policy. As the central office at NIH,
the OD establishes NIH policy and is responsible for overseeing the
ICs, including their research funding efforts and their various
administrative functions, such as hiring personnel and approving
personnel travel. The OD's oversight responsibilities have grown over
the years. Between 1985 and 2000, 7 of the 27 ICs were created--and
these additions have helped to increase the overall complexity of
overseeing the ICs. More recently, under the American Recovery and
Reinvestment Act of 2009, NIH received $10.4 billion that NIH plans to
use in 2009 and 2010 to fund extramural and other research and support
the construction, renovation, and repair of certain research
facilities.
We and others have raised questions about the OD's ability to
effectively oversee IC activities. For example, in April 2007, we
reported that NIH had not established clear policies related to
managing conflicts of interest among senior NIH employees who have
decision-making responsibilities for NIH's research efforts, [Footnote
3] which include NIH's extramural research funding. We noted that such
policies are part of NIH's framework for ensuring the integrity of NIH-
funded research and recommended that NIH clarify them. NIH agreed with
our recommendation. In mid-2007 you raised questions over allegations
of improper travel, personnel appointments, and extramural research
funding decisions involving the director of one of NIH's ICs, the
National Institute of Environmental Health Sciences (NIEHS), which
supports research on environmental influences on the development and
progression of human disease. Similar questions prompted the House
Committee on Appropriations to request that NIH conduct a management
review of NIEHS, which found management and operational problems at the
Institute.[Footnote 4]
The above issues focus on how NIH makes extramural funding decisions
and the quality of its internal control over administrative functions
such as travel arrangements and personnel appointments. Internal
control can include the establishment of safeguards, such as
supervisory reviews, that are incorporated into agency work processes.
According to federal standards, effectively designed and implemented
internal control provides reasonable assurance that an agency's
operations are effective and efficient, its financial reporting
reliable, and that the agency complies with applicable laws and
regulations.[Footnote 5] The issues at NIH also raise broader questions
about NIH's risk management, the process whereby an agency or
organization systematically identifies risks associated with achieving
its mission or objectives; assesses the magnitude of those risks; puts
in place, when necessary, mitigating actions to address those risks;
and then monitors the effectiveness of those actions. During our
review, NIH was in the process of implementing its Enterprise Risk
Management Program, a new risk management program that is replacing the
NIH Management Control Program--the agency's previous risk management
program.
You asked us to examine NIH's oversight of the ICs. Specifically, we
agreed to provide information on NIH's extramural research funding
decisions, employee travel arrangements and hiring practices for
certain employees, and NIH's process for identifying and addressing
potential risks to its operations. In this report we:
1. describe how NIH makes extramural research funding decisions and the
extent to which the NIH's OD monitors this process,
2. review the design of selected internal controls over NIH's travel
and personnel appointment processes, and:
3. review the design of the NIH Management Control Program and the
Enterprise Risk Management Program to determine if they contain key
components of an effective risk management program.
To address these objectives, we reviewed relevant NIH policies,
procedures, and supporting documentation on (1) the process used across
NIH for making extramural research funding decisions and efforts by the
OD to monitor this process, (2) the design of key internal controls for
employee travel and Title 42 personnel appointments[Footnote 6]--
specifically, control and monitoring activities--and (3) the design of
the NIH Management Control Program and the Enterprise Risk Management
Program. We also selected 3 ICs--the National Cancer Institute (NCI),
National Institute of Diabetes and Digestive and Kidney Diseases
(NIDDK), and National Institute on Alcohol Abuse and Alcoholism
(NIAAA)--for more in-depth reviews of the process used across NIH for
making extramural research funding decisions and for more in-depth
reviews of the design of the ICs' control and monitoring activities for
travel and Title 42 personnel appointment processes. We selected these
3 ICs because they vary in size and focus on different disease-specific
research missions. We interviewed officials from the NIH OD and the
selected ICs to clarify our understanding of the process used for
making extramural research funding decisions and the OD's monitoring of
this process. We also collected data on funding decisions for each of
the 24 ICs that fund extramural research.[Footnote 7] We performed
walkthroughs[Footnote 8] at the 3 selected ICs and interviewed
officials from the NIH OD and the selected ICs to clarify our
understanding of the design of the ICs' control and monitoring
activities for travel and Title 42 personnel appointment processes. We
also interviewed officials from the NIH OD to further our understanding
of the NIH Management Control Program and the Enterprise Risk
Management Program.
As part of our review, we compared the OD's monitoring of the process
used for making extramural research funding decisions and the design of
the control and monitoring activities at the three selected ICs to
GAO's Standards for Internal Control in the Federal Government.
[Footnote 9] In reviewing the design of the NIH Management Control
Program and the Enterprise Risk Management Program, we compared these
designs to our framework for effective risk management.[Footnote 10]
The scope of our audit did not include testing the implementation of
internal control over travel and Title 42 personnel appointments.
Furthermore, we did not review the implementation of either the NIH
Management Control Program or the Enterprise Risk Management Program
because, at the time of our review, NIH did not plan to continue the
Management Control program and the Enterprise Risk Management Program
was not yet fully implemented.
Appendix I includes additional details on our scope and methodology. We
conducted this performance audit from March 2008 to September 2009, in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Background:
As the primary federal agency for supporting medical research in the
United States, NIH's mission is "science in pursuit of fundamental
knowledge about the nature and behavior of living systems and the
application of that knowledge to extend healthy life and reduce the
burdens of illness and disability." NIH is headed by a Director who is
supported by staff and program offices within the OD and 27 ICs. Each
of the ICs has its own director and staff. Each IC director reports to
the OD.[Footnote 11] Appendix II provides more information about NIH's
organizational structure.
NIH's ICs were created over time, with each having an explicit mission
focused on a particular disease, organ system, stage of development, or
cross-cutting mission, such as providing scientists and researchers
with the tools they need to understand, detect, treat, and prevent a
wide range of diseases. The first institute, NCI, was created in 1937,
and the newest institute, National Institute of Biomedical Imaging and
Bioengineering, was created in 2000.
Internal Control:
Internal control is an integral part of managing an agency.[Footnote
12] It comprises the plans, methods, and procedures used to meet
missions, goals, and objectives. Effectively designed and implemented
internal control provides management with reasonable assurance that the
following objectives are being achieved: (1) effectiveness and
efficiency of operations, (2) reliability of financial reporting, and
(3) compliance with applicable laws and regulations.[Footnote 13]
Internal control serves as the first line of defense in preventing and
detecting errors and fraud. The following five elements of internal
control provide the basis against which internal control is evaluated.
* Control Environment--Sets the tone for an organization and is the
foundation for all other standards. Management and employees should
establish and maintain an environment throughout the organization that
sets a positive and supportive attitude toward internal control and
conscientious management. Among others, control environment includes
management's integrity and ethical values, commitment to competence,
philosophy and operating style, and organizational structure.
* Risk Assessment--The identification and analysis of relevant risks
associated with achieving the objectives and forming a basis for
determining how risks should be managed. This standard includes an
assessment of the risks the agency faces from both external and
internal sources.
* Control Activities--The policies, procedures, techniques, and
mechanisms that enforce management's directives. Control activities
occur at all levels and functions of the agency and include a wide
range of diverse activities such as approvals, authorizations,
verifications, and reconciliations.
* Information and Communication--Information should be recorded and
communicated to management and others within the entity who need it and
in a form and within a time frame that enables them to carry out their
internal control and other responsibilities. In addition to internal
communications, management should ensure there are adequate means of
communicating with, and obtaining information from, external
stakeholders that may have a significant impact on the agency achieving
its goals.
* Monitoring--Includes ongoing monitoring in the course of normal
operations (e.g., regular management and supervisory activities,
comparisons, and reconciliations) and risk-based monitoring that
includes separate evaluations of controls' effectiveness whose scope
and frequency depends primarily on the assessment of risks and
effectiveness of ongoing monitoring procedures.
Risk Management:
One way to help ensure that internal control is continuously monitored
and improved is through risk management. Risk management helps agencies
to identify the most significant areas in which to place or enhance
controls.[Footnote 14] Additionally, based on the assessment of risk
that is performed as part of an overall risk management program,
agencies can determine the scope and frequency of control evaluations.
Risk management is a continuous process whereby an organization
systematically identifies risks associated with achieving its
objectives; assesses the magnitude of those risks; puts in place, when
necessary, mitigating actions to address those risks; and then monitors
the effectiveness of those actions taken. In addition, because
governmental, economic, industry, regulatory, and operating conditions
continually change, risk management provides a mechanism to identify
and deal with any special risks prompted by such changes. While risk
management programs do not provide absolute assurance regarding the
achievement of an agency's objectives, an effective risk management
program can be particularly useful in a decentralized organization to
help top management identify potential problems and allocate limited
resources using a reasonable basis (such as risk).
In 2005, GAO identified risk management as an area of increasing
concern, particularly with regard to the need for the completion of
threat and risk assessments in a variety of areas.[Footnote 15] To help
address the concern, GAO developed a framework for effective risk
management activities in the federal government based on best practices
and authoritative literature.[Footnote 16] This framework includes five
components that define a risk management program for federal agencies:
strategic goals, objectives, and constraints; risk assessment;
alternatives evaluation; management selection; and implementation and
monitoring. For the purposes of our analysis of NIH's program, we also
considered two additional components, internal environment and
information and communications, based on guidance and standards on risk
management and internal controls.[Footnote 17] Figure 1 illustrates the
interrelationship of these seven components. The components of the
framework should operate within an internal environment that supports
the other components, and pertinent information should be communicated
between and among internal and external stakeholders as well as
personnel responsible for carrying out the duties associated with each
of the components.
Figure 1: Relationship of the Risk Management Framework Components:
[Refer to PDF for image: illustration]
The illustration depicts a circle with Information and Communication at
the core, surrounded by the following components, each of which is
interrelated:
Internal Environment:
* Strategic goals,objectives,and constraints;
* Risk assessment;
* Alternatives evaluation;
* Management selection;
* Implementation and monitoring.
Source: GAO.
[End of figure]
NIH Is Required to Use a Peer Review System to Make Extramural Funding
Decisions; NIH's OD Does Not Monitor Key Decisions in which IC
Directors Exercise Their Discretion Over Funding:
NIH is required by law to use a peer review system in its process for
making extramural research funding decisions. NIH's dual peer review
system is designed to help ensure the objective evaluation of the
scientific merit of applications for extramural funding. After NIH's
peer review process is concluded, IC directors have discretion when
making final extramural funding decisions and are not required to fund
applications based strictly on the scores resulting from the evaluation
of their scientific merit. We found that in fiscal year 2007 IC
directors decided to fund about 19 percent of NIH's applications for
RO1 grants, a common type of grant, based on factors in addition to
these scores. However, NIH's OD does not monitor extramural funding
decisions in which the IC Directors exercise their discretion.
By Law, NIH Must Use a Dual Peer Review System Designed to Evaluate
Scientific Merit of Extramural Funding Applications:
NIH is required by law to use a peer review system in its process for
making extramural research funding decisions. This system comprises two
sequential levels of peer review by panels of experts in various fields
of research who help NIH identify the most promising extramural grant
applications to fund, as defined primarily by an assessment of the
applications' technical and scientific merit.[Footnote 18] According to
NIH, compared to a single level of peer review, the dual peer review
system allows for multiple reviews and therefore a more objective
evaluation of the scientific merit of grant applications.
Applications for NIH's extramural funding are received by NIH's Center
for Scientific Review (CSR), which is responsible for assigning each
application to the first level of peer review. The first level of peer
review is conducted by what are known as initial peer review groups, to
which CSR assigns applications for review, based on the applications'
proposed area of research and the initial peer review groups' area of
expertise. These initial peer review groups specialize in various
research areas such as cancer or digestive disorders and are composed
of scientists, who are often recognized as experts in their field.
[Footnote 19] Each group meets three times per fiscal year to review
grant applications.
The initial peer review groups are responsible for identifying the most
promising applications for funding, based on an assessment of the
applications' scientific merit.[Footnote 20] The groups review the
applications assigned to them and assess their scientific merit, using
criteria that require reviewers to examine such components as a grant
application's design and methodology, innovation, and scientific
significance.[Footnote 21] Using these criteria, the initial peer
review groups assign a priority score to the applications they
reviewed, which are used to rank the applications from among those in
the cohort of applications. After the applications are scored and
ranked, the information is forwarded to the appropriate IC--based on
the applications' proposed area of research--for the second level of
peer review.
Each IC that funds extramural research has its own advisory council,
which conducts the second level of peer review.[Footnote 22] Advisory
councils consist of no more than 18 voting members, two-thirds of whom
are scientists in the research areas of the IC and one-third of whom
are leaders of non-science fields.[Footnote 23] Advisory councils meet
at least three times per fiscal year.[Footnote 24] Under law and NIH
policy, the advisory councils are responsible for reviewing the
applications and their priority scores and, on the basis of this
review, recommending to the ICs certain applications for funding
consideration. The advisory council may ask for applications to be
scored a second time, such as if they have questions about whether the
scientific criteria were applied appropriately. NIH advisory council
members we interviewed noted that councils only have time to discuss a
few applications individually, so they consider many applications in
large groups, particularly in cases when no concerns are apparent about
the applications or their priority scores. Based on data we reviewed,
we found that from fiscal year 2003 to 2007, in most cases, only a
small number of applications were not recommended by advisory councils
for funding consideration. The advisory councils' recommendations
conclude NIH's peer review process.
IC Directors Have Discretion to Make Final Extramural Funding
Decisions, but NIH's OD Does Not Monitor Decisions in Which IC
Directors Exercise This Discretion:
After NIH's peer review process has been concluded, the director of
each IC is responsible for making final extramural funding decisions.
In deciding which applications to fund, the IC directors choose
applications from among those recommended for funding consideration by
the advisory council.[Footnote 25] In general, IC directors select
applications for funding based on their priority scores, which reflect
the evaluation of the applications' scientific merit by NIH's peer
review process. For each fiscal year, each IC establishes a funding
line--known as the payline--which roughly corresponds with the number
of extramural grant applications the IC will be able to fund that year.
The payline for any given year is based on projections of the total
funding available at the IC that year for grants, the average dollar
amount expected to be awarded per application, and the number of
applications received by the IC. For example, as shown in figure 2,
based on the amount of funding available for extramural grants, NCI set
its payline for R01 grants[Footnote 26]--the most common grant
category--at the 15th percentile for fiscal year 2007. This means that
NCI expected to have sufficient funding at a minimum for all of the
applications with scores in the top 15 percent. In general, IC
directors fund only those applications with priority scores above the
fiscal year's payline.
Figure 2: National Cancer Institute's Fiscal Year 2007 Payline for R01
Grant Applications:
[Refer to PDF for image: illustration]
National Cancer Institute‘s payline: 15th percentile:
Application order based on percentile ranking:
656 applications scored above the 15th percentile;
3,108 applications scored below the 15th percentile.
Source: GAO.
Note: Figure includes only applications that were recommended for
funding consideration by an initial peer review group and the advisory
council. The portion of applications that scored above the payline
appears to be greater than 15 percent because applications that were
considered for funding multiple times during the year and scored below
the 15th percentile each time are only counted in this data once.
[End of figure]
While the IC directors only fund projects recommended by their advisory
councils and typically work within the paylines, ultimately they have
discretion to make final extramural funding decisions. In particular,
directors are not required to fund applications based strictly on the
applications' priority scores or the payline. In some instances, IC
directors decide not to fund applications that scored above the fiscal
year's payline, such as when the applications duplicate research that
has already received IC funding. These applications are called "skips."
For example, of the 656 applications that scored above NCI's payline in
fiscal year 2007, 3 applications were skipped. Similarly, though the IC
directors typically do not decide to fund applications with priority
scores that fall below the fiscal year's payline, in some cases they
do. These applications are known as "exceptions." For example, of the
3,108 applications that scored below NCI's payline in 2007, 137 were
funded as exceptions. In the case of exceptions, the IC directors may
exercise their discretion and choose to fund these applications based
on factors in addition to the applications' priority scores. These
factors can include efforts to support the IC or NIH's research
priorities. When skipping applications or funding applications as
exceptions, IC directors are required under NIH policy to document the
corresponding rationale used.
In reviewing the IC data, we found that 18.5 percent of NIH's funded
R01 grant applications were funded as exceptions in fiscal year 2007,
as shown in table 1. These applications had scientific merit scores
that were below the payline for their respective ICs and thus were
funded based on factors in addition to their scientific merit scores.
This represents a substantial increase from 9.7 percent of funded
applications that were exceptions in fiscal year 2003.
Table 1: Extramural Research R01 Grant Applications Funded in Fiscal
Years 2003 through 2007:
Fiscal year: Total number of applications funded;
2003: 6,461;
2004: 6,167;
2005: 5,731;
2006: 5,408;
2007: 5,715.
Fiscal year: Number of applications funded above the payline;
2003: 5,836;
2004: 5,639;
2005: 5,159;
2006: 4,788;
2007: 4,656.
Fiscal year: Number of applications funded below the payline -
exceptions;
2003: 625;
2004: 528;
2005: 572;
2006: 620;
2007: 1,059.
Fiscal year: Percentage of applications funded below the payline -
exceptions;
2003: 9.7%;
2004: 8.6%;
2005: 10.0%;
2006: 11.5%;
2007: 18.5%.
Source: GAO analysis of NIH grants data.
[End of table]
Documentation that we reviewed from three of the ICs--NCI, NIDDK, and
NIAAA--showed that IC directors funded applications as exceptions for
various reasons. For example, IC officials cited the NIH-wide
initiative to fund new investigators as one of the most frequent
reasons for making the exceptions. In addition, IC officials told us
that they funded applications as exceptions in order to maintain a
diverse portfolio of research topics.
NIH's OD collects information on some aspects of the extramural
research process. For example, the NIH OD collects information on the
number of extramural grants funded by each IC; the percentage of
applications that receive funding; and the priority rankings, by
percentile, of funded applications. The NIH OD also targets some of
these collection efforts towards specific types of extramural grants.
For example, as part of its effort to support new investigators, the
NIH OD has been collecting data on the number of extramural grants
awarded to new investigators.
Although NIH's OD collects some information on the extramural research
process, it does not monitor key funding decisions made by IC
directors--specifically, the instances in which IC directors exercise
their discretion to make skips or exceptions to the funding payline.
Skips and exceptions represent an area of potential risk because IC
directors have latitude in making these decisions; monitoring these
decisions would be consistent with federal internal control standards.
[Footnote 27] Although ICs are required to document the rationales used
when skipping applications or funding applications as exceptions, the
ICs are not required to provide the NIH OD with this documentation, and
the NIH OD does not collect it. As a result, NIH's OD does not have
information on the number of applications skipped or funded as
exceptions and the reasons for these decisions.
Design of NIH's Travel and Personnel Appointment Processes Includes Key
Control Activities and Some Monitoring Activities but Lacks Systemic
Risk-Based Monitoring:
With regard to the design of NIH-wide travel processes, NIH OD has
established policies and procedures to help ensure that federal travel
regulations are followed with regard to issues such as premium class
travel, per diem expenses, and travel paid by third parties. The key
control and monitoring activities for travel include reviews and
approvals which take place during two stages--authorization and
voucher--of the process. During the authorization stage, the traveler
receives approval to travel based on the supervisor's approval of the
mission-relatedness of the trip and an administrative official's
approval of the method of transportation used, the cost estimates set
forth for travel expenses, and the availability of funds for
reimbursement to the traveler. During the voucher stage, the traveler's
voucher for reimbursement of travel expenses is approved based on an
administrative official's review of the voucher package which includes
the traveler's certification of the voucher and required receipts for
travel expenses.
The NIH OD has also established policies and procedures to help ensure
that Title 42 personnel appointment decisions are appropriate. The
design of NIH-wide key control activities for Title 42 personnel
appointments includes reviews and approvals which take place during
three stages--resource determination, appointment selection, and
compensation--of the process. During the resource determination stage,
the IC's selecting official identifies a hiring need and the
administrative official determines whether necessary resources are
available to meet the hiring need. During the appointment selection
stage, the IC completes the recruitment including receiving
applications, conducting candidate interviews, and making a tentative
selection. For some positions, the NIH Offices of Human Resources,
Intramural Research, and Extramural Research also play a role in
preparing the recruitment and approving the selected candidate.
[Footnote 28] During the compensation stage, final approval for the
appointment and compensation is given depending upon the position and
salary level for the candidate. Specifically, if the proposed
compensation is below the lowest third of a given position's salary
range, then the IC director makes the final approval; if the proposed
compensation is above the lowest third of a given position's salary
range but still within the range, then the IC director makes the final
approval based on a recommendation from an IC committee; and if the
proposed compensation is above the salary range or other specified
limits, then the NIH director makes the final approval based on
recommendations from an IC committee, the IC director, and an NIH-wide
committee.
Overall, we found that the design of the controls included in the NIH-
wide processes over travel and Title 42 personnel appointments included
key controls necessary to help ensure these activities were being
carried out appropriately, except in one key area related to the lack
of requirements for risk-based monitoring. While controls may appear
adequate based on written policies and procedures, without monitoring
actual implementation based on the assessed risk levels, NIH does not
have adequate assurance that controls are operating as intended within
those areas that pose risk. NIH policy did not require the ICs to
perform monitoring that includes risk-based control evaluations.
Further, although NIH policy required a flexible plan for NIH-wide
control evaluations that would generally target high-and medium-risk
areas for review, according to NIH OD officials, such reviews have not
been performed for over 3 years because they do not have staff to
perform these reviews.
At NIH OD and two of the three ICs we reviewed--NCI and NIAAA--we found
that some monitoring activities were performed over travel and Title 42
personnel appointments. However, these monitoring activities were
either not part of systemic risk assessment efforts that lead to
subsequent monitoring based on assessed risk or not performed on a
consistent and ongoing basis. Specifically,
* Because of travel issues previously identified by GAO,[Footnote 29]
HHS requires each of its operating divisions, which includes NIH, to
perform quarterly control evaluations of travel cards.[Footnote 30] As
a result of this requirement, each quarter the NIH OD selected a sample
of travel transactions from across the ICs and tested compliance with
federal travel regulations and NIH policies and procedures. For
example, during each of the first 2 quarters of fiscal year 2008, the
NIH OD found problems with about 20 percent of the 100 sample items it
tested. During the third quarter, the NIH OD found problems with about
30 percent of the 75 sample items it tested. Some of the problems found
during these quarters included over-or underpayment to travelers,
failure of travelers to take advantage of lodging tax exemptions, and
misuse of travel cards. The NIH OD required follow-up actions such as
reimbursement of overpayment amounts and issuing additional guidance.
However, these travel control evaluations were not part of a systemic
process for assessing risk over operations and subsequently monitoring
or evaluating controls based on assessed risk levels.
* In 2008, NCI and NIAAA performed control evaluations over travel, and
NCI performed a control evaluation of personnel appointments (including
those under Title 42). These control evaluations were performed in
response to prior audit findings, to prepare for upcoming audits or
reviews, or to address concerns regarding process inefficiencies.
However, they were not incorporated into the design of the processes
and therefore were not performed on a consistent and ongoing basis.
One of the three ICs we reviewed--NIDDK--had adopted its own risk-based
program which consisted of assessing the risks over operational areas,
including travel and personnel appointments, and subsequently
monitoring the controls over those operational areas. The frequency of
monitoring depended upon the risk level, and high-risk activities at
NIDDK were scheduled to be monitored more frequently than low-or medium-
risk activities. The design of NIDDK's program represents a positive
step towards an effective risk management program. Further details on a
framework for an effective risk management program are discussed in the
next section.
NIH's Management Control Program and Enterprise Risk Management Program
Do Not Fully Address Key Components of Effective Risk Management:
The design of the Management Control Program provided NIH with a
limited ability to identify and address risks to the agency's overall
operations. Recognizing the need for improvement, in 2006, the NIH OD
began redesigning its program. However, while an improvement over the
Management Control Program, the new Enterprise Risk Management Program
does not fully address all of the components of GAO's framework for
effective risk management. Further, NIH's Enterprise Risk Management
Program has not been fully implemented, despite an over 3-year effort,
and NIH had not yet established milestones for its full implementation.
NIH's Management Control Program Had Weaknesses:
NIH's Management Control Program was initially implemented in 1999 and
updated in 2004. Under the design of this program, risk assessments are
performed that relate to specific management control areas, such as
functional areas, systems, or processes (e.g., intramural research
programs) without relating those areas to potential systemic or
agencywide risks. If weaknesses are identified within the particular
area being reviewed, the Management Control Program appropriately
requires that corrective action plans be developed and implemented to
correct the weakness and that such actions be monitored after
implementation to ensure that the weakness has been corrected.
As designed, NIH's Management Control Program did not address several
of the components and related key elements included in GAO's framework
for an effective risk management program. An effective risk management
program should enable management to proactively identify, assess, and
mitigate risks. Table 2 outlines the seven components of the risk
management framework and the key elements within each of these
components.
Table 2: GAO's Risk Management Framework:
Risk management component: Strategic goals, objectives, and
constraints;
Description: Addresses what the strategic goals and objectives are
attempting to achieve and the steps needed to attain these results, and
considers the constraints under which an agency operates such as
statute, higher level policy, budget, or other factors beyond
management's control that may affect an agency's risk management plans;
Key elements: An agency's risk management program should:
* Require mission-based strategic goals and objectives, that are
clearly articulated and measurable, to be set as a pre-condition for
effective risk management. Without clearly identified strategic goals
and objectives, an agency cannot effectively identify and address
potential risks to its mission, prioritize risk, or identify criteria
against which to measure performance;
* Require agencies to identify constraints (e.g., legislative
requirements or resources) that may limit effective risk management.
Risk management component: Risk assessment;
Description: Addresses the identification and evaluation of potential
risks to an agency's ability to achieve its goals and objectives so
that management can design and implement responses to prevent or
mitigate identified risks;
Key elements: An agency's risk management program should:
* Identify potential events which may adversely affect the agency,
called risks, and evaluate the events based on likelihood of occurrence
and impact. For example, an agency may identify and evaluate potential
risks associated with economic and legislative changes, natural
disasters, and criminal or terrorist activities;
* Require continuous identification and evaluation of potential risks
since governmental, economic, industry, legislative, and operating
conditions continually change.
Risk management component: Alternatives evaluation;
Description: Addresses the identification and evaluation of alternative
ways in which the agency can act to alter either the likelihood of
occurrence or the impact of a potential risk;
Key elements: An agency's risk management program should:
* Identify alternative ways the agency can respond to prevent or
mitigate an identified risk. For example, to comply with new
legislation, an agency may need to revise existing policy and
procedures or develop new policies and procedures;
* Evaluate the alternatives identified to consider the effect on
likelihood of occurrence and impact of a potential risk;
* Evaluate the alternatives identified to consider the costs and
benefits.
Risk management component: Management selection;
Description: Addresses the selection of a response to mitigate an
identified risk based on the alternatives evaluated and management
priorities, such as management's attitude towards risk and how limited
resources will be targeted;
Key elements: An agency's risk management program should:
* Require management to select and document an alternative, such as
revising or creating a policy or procedure, for addressing an
identified risk;
* Require management to document the rationale for selecting the
alternative.
Risk management component: Implementation and monitoring;
Description: Addresses how risk responses will be applied and assessed
to improve efficiency and effectiveness. In addition, addresses how the
risk management program will be assessed to determine whether changes
are needed to improve efficiency and effectiveness;
Key elements: An agency's risk management program should:
* Implement management's selected alternative to address risk;
* Periodically assess management's selected alternative to address
risk;
* Periodically assess the efficiency and effectiveness of the entire
risk management program.
Risk management component: Internal environment;
Description: Addresses how management will establish and maintain a
positive environment that sets the tone throughout the agency and is
the foundation upon which all other components of risk management
operate;
Key elements: An agency's risk management program should:
* Include an agency's risk management philosophy to help position the
agency so that it can effectively recognize and manage risk;
* Require oversight by a high-level senior body within the agency;
* Incorporate the importance of integrity and ethical values to
increase the effectiveness of the risk management program since the
program and its results depend upon the personnel who carry out risk
activities;
* Include the way management assigns authority and responsibility to
help ensure that risk responsibilities are carried out;
* Hold managers accountable for their assigned duties in the risk
management program;
* Require management to organize its risk structure to provide a
framework for the agency to plan, execute, control, and monitor risk
activities;
* Require management to initially train its personnel to help ensure
that they have the necessary knowledge and skills to perform their
assigned tasks;
* Ensure management maintains competence of the agency's personnel by
providing for continuous training to update personnel on risk
management practices and techniques.
Risk management component: Information and communication;
Description: Addresses the need to identify and communicate pertinent
information in a form and timeframe that allows personnel to carry out
their risk management responsibilities;
Key elements: An agency's risk management program should:
* Require pertinent information to be collected from and disseminated
to relevant internal stakeholders in a form and timeframe consistent
with the agency's risk management needs;
* Require pertinent information to be collected from and disseminated
to relevant external stakeholders in a form and timeframe consistent
with the agency's risk management needs.
Sources: [hyperlink, http://www.gao.gov/products/GAO-06-91]; The
Committee of Sponsoring Organizations of the Treadway Commission,
Enterprise Risk Management--Integrated Framework (Jersey City, N.J.:
American Institute of Certified Public Accountants, September 2004);
and [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1].
[End of table]
The three components of the framework that the Management Control
Program did not address are strategic goals, objectives, and
constraints; risk assessment; and information and communication.
Specifically, the program did not do the following.
* Link the identification of potential risks with the agency's
strategic goals and objectives. The design of the Management Control
Program did not require strategic goals and objectives to be set as a
precondition for risk management. Without clearly identified strategic
goals and objectives, an agency is limited in its ability to
effectively identify and address potential risks to its mission,
prioritize risk, or identify criteria against which to measure
performance.
* Require risk assessments be performed to identify and evaluate
potential risks that could adversely affect NIH's ability to achieve
its objectives. The design of the Management Control Program called for
evaluating the risks within specific functional areas, systems, or
processes rather than assessing the risks that could adversely affect
the agency as a whole.
* Require pertinent information to be collected from and disseminated
to relevant internal and external stakeholders in a form and time frame
consistent with the agency's overall risk management needs. The design
of the Management Control Program allowed for inconsistent and
incomparable information from the ICs, which can prevent management
from effectively using the information to help ensure that agency
objectives are met.
For a number of years, NIH OD officials recognized that weaknesses
existed in the agency's Management Control Program, which resulted in a
lack of sufficient information for effective oversight and agencywide
risk management. For example, according to NIH OD officials, the
program (1) did not hold managers responsible for their assigned duties
in the risk management program, (2) did not require the ICs to
communicate information in a form that allows NIH to effectively
identify and manage risk across the agency, and (3) was not overseen by
a high-level senior body, such as the Steering Committee.[Footnote 31]
The three weaknesses NIH officials identified in the agency's
Management Control Program correspond to the following key components
of our framework for effective risk management: 1) internal
environment, 2) information and communication, and 3) internal
environment, respectively. As a result of acknowledged shortcomings, in
2006 NIH began redesigning its risk management efforts. According to
NIH OD officials, the new risk management program will improve the
ability of the NIH OD to proactively identify and mitigate risks before
they become obstacles to the NIH mission. However, as discussed later,
NIH has not fully implemented the new Enterprise Risk Management
Program and has encountered several obstacles in implementing initial
phases of the program.
NIH's Enterprise Risk Management Program, while Improved, Does Not
Fully Address Several Key Components of Effective Risk Management:
NIH began developing a new risk management program in 2006. The new
program is designed to consist of a formal six-step methodology for
managing risks.[Footnote 32] The six steps include:
* Organize - Identify and train those charged with carrying out risk
management activities, and define the risk management structure.
[Footnote 33]
* Identify and Score[Footnote 34] - Identify and score risks, review
risks for quality and accuracy, and develop the risk baseline.
* Assess - Document, analyze, and test processes and controls.
* Remediate - Develop, review, approve, and execute corrective action
plans.
* Monitor - Monitor the risk baseline.
* Report - Report risk information and results.
The design of the Enterprise Risk Management Program represents an
improvement over the 2004 NIH Management Control Program in several key
areas. Specifically, the new program will allow for improved
identification, assessment, and mitigation of risks agencywide because
it includes the following:
* Risk assessments: The new program requires the identification of
potential events that could adversely affect the agency and the
evaluation of those events based on likelihood of occurrence and
impact.
* Oversight by a high-level senior body: The design requires the
Steering Committee to oversee the new risk management program.
* Information and communication: The design requires that pertinent
information be collected from and disseminated to relevant internal
stakeholders in a form and time frame consistent with NIH's risk
management needs. For example, the program requires a consistent
methodology for identifying, assessing, and communicating risks across
NIH, which will allow for consistent, comparable information from each
of the ICs.
However, the Enterprise Risk Management Program still does not fully
address all of the components that we have identified for an effective
risk management framework. As discussed below, further consideration of
the risk management framework could significantly improve the design of
NIH's new risk management program, which, if effectively implemented,
could assist management in maintaining effective control over the
agency's decentralized and diverse activities.
Strategic Goals, Objectives, and Constraints. The Enterprise Risk
Management Program does not require the NIH OD or ICs to set mission-
based strategic goals and objectives as a precondition for risk
management. This is a critical shortcoming because although the risk
design requires risks to be assessed on the basis of their impact on
NIH's mission, there is not an NIH-wide strategic plan against which to
assess risks. Further, while some ICs and NIH OD offices have strategic
plans for their organizations, the risk management program as designed
does not call for risks to be assessed on the basis of their impact on
IC-or NIH OD office-level missions.
Alternatives Evaluation. Although the Enterprise Risk Management
Program identifies four different responses the agency can select to
prevent or mitigate identified risks (creating a new policy, procedure,
or control; revising an existing policy, procedure, or control;
streamlining or automating an existing policy, procedure, or control;
or redesigning the process), the program does not require management to
evaluate the risk responses identified to consider (1) the effect on
the likelihood of occurrence and impact of a potential risk and (2) the
costs and benefits. These types of evaluations could assist management
in making an informed decision within an environment that includes
constrained resources.
Management Selection. The design of the Enterprise Risk Management
Program does not require management to document the rationale for
selecting a particular risk response. Such documentation could help
improve accountability and facilitate analysis of the effectiveness of
actions taken.
Implementation and Monitoring. Although the design of the Enterprise
Risk Management Program requires periodic assessments of the overall
efficiency and effectiveness of the risk management program, it does
not offer any detail regarding how these assessments will be performed.
For example, the program does not provide details such as the
frequency, scope, or methodology for these reviews. Further, the design
does not require periodic assessments of implemented risk responses.
These types of monitoring activities are critical in helping management
to identify problems with the overall risk management program and to
determine whether risk responses are preventing or mitigating risks and
operating as intended.
Internal Environment. The Enterprise Risk Management Program includes
many of the elements that define this component. However, the design
could be improved by (1) incorporating the importance of ethical values
into the risk management program and (2) ensuring management maintains
the competence of its personnel by providing for continuous training to
update personnel on risk management practices and techniques.
Information and Communication. The design of NIH's Enterprise Risk
Management Program does not require the collection and dissemination of
pertinent information to relevant external stakeholders in a form and
time frame consistent with NIH's risk management needs. For example,
although the design requires annual reporting, in aggregate, to HHS on
the adequacy of internal control, it does not require communication
with other external stakeholders, such as congressional oversight
committees.
Implementation of the Enterprise Risk Management Program Has Been
Hampered by Lack of Milestones:
The design and implementation of NIH's new risk management program is
not yet completed, despite an over 3-year effort. Without a sound risk
management program NIH cannot be reasonably assured that it will be
able to effectively and proactively identify, assess, and mitigate
risks before they become problems that affect NIH's ability to achieve
its mission. During fiscal year 2008, the NIH OD implemented the first
two steps of the six steps in its new risk management program. NIH had
(1) organized the risk structure at the NIH OD and ICs, and identified
and trained personnel responsible for managing risks within NIH OD, and
(2) identified and scored risk at the NIH OD. NIH OD officials said
they planned to complete the IC-level implementation of these two steps
by the end of fiscal year 2009.
The NIH OD is responsible for the design and implementation of the new
program, and it has developed a time line with milestones for
implementing some steps of the program. However, the timeline's
milestones are not firm, and the NIH OD has revised the timeline to
accommodate delays. According to NIH OD officials, they have
experienced delays in designing and implementing the new program
because of a change in contractors, balancing limited staff resources
with competing demands, and underestimating the amount of time
necessary for implementing specific steps of the program. As of the
completion of our draft report, the NIH OD had not set a date for fully
implementing the program agencywide. However, in providing written
comments on a draft of this report, HHS indicated that the Enterprise
Risk Management Program at NIH was scheduled for full implementation
throughout NIH by June 2010. (See agency comments and our evaluation
for additional details.)
Conclusions:
While NIH's decentralized structure allows NIH to address a wide range
of research areas, it also creates significant oversight challenges.
The ICs operate largely independently--each with its own budget,
mission, and staff--making it vitally important that NIH and especially
the OD have the means to ensure that all the ICs operate in accordance
with NIH's policies and mission. With an annual budget of nearly $30
billion, plus an additional $10 billion in funding available in 2009
and 2010 through the American Recovery and Reinvestment Act of 2009,
the financial stakes are high.
We found gaps in NIH's ability to monitor key aspects of its extramural
funding process. Specifically, NIH's OD does not monitor extramural
funding decisions in which IC directors exercise their discretion to
skip applications and make exceptions, even though information on these
decisions is collected at the IC level. Without routine monitoring,
which is consistent with federal internal control standards, NIH does
not have the information to be reasonably assured that these decisions
are appropriate and support the agency's mission. Appropriate funding
decisions are critical to ensuring an effective use of taxpayer dollars
and supporting NIH's reputation as the premier federal medical research
agency in the United States. In reviewing selected administrative
operations, we also found a key weakness in the design of the controls
the NIH OD has established for oversight of travel and Title 42
personnel appointments. Without internal controls that include risk-
based monitoring of the controls' actual implementation, NIH cannot be
reasonably assured that these controls are effective and operating as
intended in areas identified as posing potential risks to NIH.
Given these issues, a comprehensive risk management program could help
ensure that such monitoring gaps are identified and addressed. NIH has
recognized the importance of risk management to its organization and
has taken steps towards implementing its new Enterprise Risk Management
Program. Specifically, NIH has organized the risk structure at the NIH
OD and ICs, identified and trained personnel responsible for managing
risks within the NIH OD, and made progress in identifying and scoring
risks at both the NIH OD and the ICs, which represent important steps.
However, the design of the Enterprise Risk Management Program lacks
several key components identified in our framework as necessary for
effective risk management and the program has not yet been fully
implemented throughout NIH.
Recommendations for Executive Action:
To ensure effective oversight of extramural funding decisions, we
recommend that the Director of NIH establish a process for routine
monitoring of the extramural funding decisions in which the IC
directors use their discretion to skip applications or fund
applications as exceptions.
To help ensure that NIH has a comprehensive program to effectively
address potential risks to the agency's mission, including those
related to the monitoring of extramural research funding decisions,
travel, and personnel appointments, we recommend that the Director of
NIH take two actions to complete the design and implementation of NIH's
Enterprise Risk Management Program:
* Add key components and related elements needed to achieve
comprehensive and effective agencywide risk management to the design of
NIH's Enterprise Risk Management Program, including:
- mission-based strategic goals and objectives as a precondition for
risk management and risks to be assessed on the basis of their impact
on the achievement of these goals and objectives;
- the evaluation of risk responses to consider the effect on the
likelihood of occurrence and impact of a potential risk and the costs
and benefits;
- the documentation of the rationale for selecting risk responses;
- additional detail regarding how the assessments of the overall
efficiency and effectiveness of the risk management program will be
performed;
- periodic assessments of implemented risk responses;
- the importance of ethical values;
- continuous training to maintain the competence of personnel carrying
out risk management duties; and:
- communication with relevant external stakeholders.
* Identify major milestones, including a final implementation date, to
help ensure that NIH completes and implements the Enterprise Risk
Management Program in a reasonable time frame.
Agency Comments and Our Evaluation:
The Department of Health and Human Services provided written comments
on a draft of this report, which are reprinted in appendix III. In
responding to our draft report, HHS disagreed with the first
recommendation and partially concurred with the second recommendation.
In response to the third recommendation, HHS provided new information.
The following sections summarize HHS's comments on each of our three
major findings and related recommendations and provide our responses.
OD Oversight of Extramural Funding Decisions:
HHS disagreed with our recommendation that the Director of NIH should
establish a process for routine monitoring of the extramural funding
decisions in which the IC directors use their discretion to skip
applications or fund applications as exceptions. In its written
comments, HHS stated that we implied an inappropriate role for the NIH
OD. Specifically, HHS said that the OD's role was not to provide input
on the scientific reasoning for making skips and exceptions, which
should be left to the judgment of the scientific officials who
understand the current trends in science and the institute research
portfolios. HHS further stressed that the ICs are required to document
the reasons for these decisions and that the documents are available
for review by the OD upon request.
Our work shows there would be benefit for the Office of the Director of
NIH, as part of its responsibility to oversee IC operations, to
routinely monitor the extent to which IC directors use their discretion
to skip applications and fund applications as exceptions. This
monitoring can be consistent with NIH's stated reliance on scientific
reasoning and the judgment of the scientific officials in making these
decisions. As we noted in our draft report, when IC directors decide to
skip applications and fund applications as exceptions, they do so by
considering factors other than the science-based priority scores
originally assigned to each application by NIH's initial peer review
groups and advisory councils. There can be good reasons for the
decision to skip an application or fund an application as an exception,
such as the desire to maintain a diverse portfolio of work. Routinely
monitoring the extent to which IC directors use their discretion to
skip applications and fund applications as exceptions would position
the Director of NIH to help ensure that these decisions are consistent
with NIH policy goals and are therefore appropriate. Such routine
monitoring would also enable the Director of NIH to identify instances
in which further review by appropriate officials may be desirable.
Further, the routine monitoring we recommended is consistent with other
efforts by the OD to monitor extramural funding decisions. As we noted
in our draft report, the NIH OD already collects certain information
related to extramural funding decisions, such as the priority rankings
of funded applications and the number of extramural grants awarded to
new investigators in response to an NIH-wide initiative. Finally, NIH
OD monitoring activities would be consistent with federal internal
control standards.
In related comments, HHS drew attention to our finding that the share
of RO1 grants awarded outside the payline (as exceptions) increased
substantially from fiscal year 2003 through fiscal year 2007, and noted
that this increase resulted largely from a corresponding increase in
the number of RO1 grants awarded to new investigators. We agree with
HHS, and noted in our draft report that our analysis of NIH's records
showed that the NIH-wide initiative to fund new investigators was one
of the most frequently cited reasons for funding an application as an
exception. HHS further stated that it would like to review our methods
for quantifying the number of extramural grants funded as exceptions.
As we indicated in the scope and methodology section of our draft
report, we based our analysis on data provided by NIH. We noted that
NIH provided us with information about the payline established by each
of the 24 ICs for each fiscal year from 2003 through 2007, and the
number of RO1 grant applications funded relative to each IC's payline
for each year.
Design of Controls Over NIH's Travel and Personnel Appointment
Processes:
HHS concurred with our finding that the design of NIH's Title 42
personnel appointment process included key control activities and some
monitoring but lacked systematic risk-based monitoring. HHS said that
it intends to incorporate risk-based monitoring into the Title 42
personnel appointment process. HHS also commented that NIH has
identified and scored the agency travel process within its Enterprise
Risk Management Program (discussed in the next section) but that it
will reassess the travel risk levels to ensure that they are
appropriate.
Design of NIH's Management Control Program and Enterprise Risk
Management Program:
In response to our recommendation that NIH add key components to the
design of its Enterprise Risk Management Program to achieve
comprehensive and effective agencywide risk management, HHS agreed with
some of our specific recommendations and disagreed with others. We
identified eight specific items in this area; HHS agreed with four,
partially agreed with one, and disagreed with three.
* HHS agreed that the design of NIH's Enterprise Risk Management
Program should be modified to include the evaluation of risk responses
to consider the effect on the likelihood of occurrence and impact of a
potential risk and the costs and benefits. HHS noted that NIH will
modify its Enterprise Risk Management Guidebook to reflect this
recommendation.
* HHS agreed that the design of NIH's Enterprise Risk Management
Program should be modified to include documentation of the rationale
for selecting risk responses. HHS noted that it appreciated the
feedback and has incorporated this element into NIH's processes and
amended the NIH Enterprise Risk Management Guidebook.
* HHS agreed that the design of NIH's Enterprise Risk Management
Program should be modified to include periodic assessment of
implemented risk responses.
* HHS agreed that the design of NIH's Enterprise Risk Management
Program should be modified to include additional detail regarding how
the assessments of the overall efficiency and effectiveness of the risk
management program will be performed. However, HHS noted that the NIH
Enterprise Risk Management Program has already undergone incremental
evaluation during implementation. HHS also noted that NIH plans to
develop a program evaluation process and conduct periodic reviews of
the program in fiscal year 2011.
* HHS partially agreed that the design of NIH's Enterprise Risk
Management Program should be modified to include communication with
relevant external stakeholders. HHS noted that NIH promptly responds to
all requests for information from external stakeholders. However, HHS
also noted that the Enterprise Risk Management Program will include
external communications as it matures.
* HHS did not agree that the design of NIH's Enterprise Risk Management
Program should be modified to include mission-based strategic goals and
objectives as a precondition for risk management and to assess risks on
the basis of their impact on the achievement of these goals and
objectives. HHS said that NIH's Enterprise Risk Management Program is
designed to identify and manage risks before they become obstacles to
the NIH mission and noted that the ICs establish their own strategic
goals and objectives. As we noted in the draft report, the design of
the program does not require the NIH OD or ICs to set mission-based
strategic goals and objectives as a precondition for risk management,
nor does the design call for risks to be assessed on the basis of their
impact on IC-or NIH OD-level missions. We continue to believe that a
clear and explicit link to strategic goals and objectives would help
ensure that risks are routinely assessed based on their potential
impact to achieving NIH's mission and would identify criteria against
which to measure performance.
* HHS did not agree that the design of NIH's Enterprise Risk Management
Program should be modified to include the importance of ethical values.
HHS said that NIH's risk management program already operates within the
context of a positive environment in which integrity and ethical values
play a key role. However, HHS said that NIH would modify the design of
the Enterprise Risk Management Program as we recommended, by amending
the Enterprise Risk Management Guidebook to include specific language
addressing the importance of ethics at NIH.
* HHS did not agree that the design of NIH's Enterprise Risk Management
Program should be modified to include continuous training to maintain
the competence of personnel carrying out risk management duties.
Nevertheless, HHS stated that NIH has provided training to over 400
individuals who hold significant risk management roles and noted that
NIH plans to develop continuous training for all employees on risk
management. Moreover, HHS said that NIH will modify the design of the
Enterprise Risk Management Program as we recommended, by modifying the
Enterprise Risk Management training plan to incorporate ongoing
training such as training updates and refreshers.
In response to our recommendation that the Director of NIH should
identify major milestones, including a final implementation date, to
help ensure timely implementation of the Enterprise Risk Management
Program, HHS identified a final implementation date of June 2010.
Although HHS asserted that NIH's Enterprise Risk Management Program is
fully functional because NIH has implemented all six steps of the
program at some level, as we noted in our draft report and as HHS
confirmed in its written comments, several elements of the program have
not been implemented across all of NIH. For example, HHS stated that
steps one and two (identify and score risks) have been implemented
across all of NIH--including the OD and the ICs--but that steps three
and four (assess and remediate risks) have been implemented at the OD
level but not across the ICs. If NIH proceeds with the actions and time
frames outlined in HHS's comments, it should meet the intent of our
recommendation.
HHS stated that the prior risk management program--which our draft
referred to as the "current" program--was discontinued in 2006. This
statement is not consistent with the information we gathered during the
time of our review nor with the policy manual posted on the NIH Web
site, which states that the Management Control Program was "temporarily
rescinded effective June 24, 2009,"--1 day after HHS received our
report for review and comment--and that replacement guidance has not
been issued. If the prior program has been discontinued and the final
implementation date for the new program is scheduled for June 2010, NIH
may have been operating without a fully functioning risk assessment
program in place, which is a key element of a system of internal
control. Although we believe our draft report correctly characterized
the status of NIH's Management Control Program and Enterprise Risk
Management Program at the time of our review, in response to HHS's
comments we revised the wording in our report to more clearly
distinguish between the new Enterprise Risk Management Program and the
Management Control Program it is replacing.
In commenting on our evaluation of the NIH Enterprise Risk Management
Program, HHS questioned the criteria we used in our evaluation. HHS
stated that it defines risk management as synonymous with internal
control and that the NIH Enterprise Risk Management Program was
developed based on the Standards for Internal Control in the Federal
Government and the Office of Management and Budget (OMB) Circular A-
123. Thus, HHS suggested that we should revise our report using
different criteria. We believe that our criteria are appropriate for
the evaluation. As noted in the draft report, GAO developed the
framework based on authoritative literature and standards, as well as
previous GAO reports and testimonies. We consulted the Government
Performance and Results Act (GPRA) of 1993; the Government Auditing
Standards, 2003 Revision; GAO's Standards for Internal Control in the
Federal Government (November 1999); guidance from OMB; the work of the
President's Commission on Risk Management; consulting papers; and the
enterprise risk management approach of the Committee of Sponsoring
Organizations (COSO) of the Treadway Commission. We also reviewed
numerous risk management frameworks from industry, government, and
academic sources.
Furthermore, our draft report noted the relationship between internal
control and risk management. Specifically, risk management is a
continuous process through which an organization identifies, assesses,
and mitigates risks, and through risk management, an organization can
identify the most significant areas in which to place or enhance
internal control. Systems of internal control may help an organization
prevent or reduce risks, such as fraud, waste, abuse, or mismanagement.
Internal control standards, therefore, provide an important tool for
use in risk management. For example, in response to our draft report,
NIH pointed out a variety of management oversight mechanisms, as
discussed below. Those mechanisms could be considered part of NIH's
internal controls, but are not part of its risk management program. We
believe that the framework we used to evaluate NIH's risk management
program was appropriate.
In addition, HHS commented that our report implied that the risk
management program is the sole management oversight mechanism at NIH
and that we failed to acknowledge other oversight bodies and functions.
We agree that NIH has many mechanisms for managerial oversight and
accountability and we cited some of the mechanisms HHS specified in the
draft report, such as oversight of travel and Title 42 personnel
appointments. However, it was beyond the scope of our report to
evaluate the full spectrum of NIH's oversight and accountability
mechanisms. Further, regardless of the number or type of the other
oversight mechanisms in place at NIH, these do not in any way diminish
NIH's need to make its risk management program fully functioning,
comprehensive, and effective.
HHS also provided us with technical comments, which we incorporated as
appropriate.
As arranged with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days
after its issue date. At that time, we will send copies of this report
to other interested congressional committees, the Secretary of HHS, and
the Director of NIH. This report will also be available on the GAO Web
site at [hyperlink, http://www.gao.gov].
If you or your staff have any questions regarding this report, please
contact Linda T. Kohn at (202) 512-7114 or kohnl@gao.gov or Susan
Ragland at (202) 512-8486 or raglands@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. Key contributors to this report are
listed in appendix IV.
Sincerely yours,
Signed by:
Linda T. Kohn, Director:
Health Care:
Signed by:
Susan Ragland, Director:
Financial Management and Assurance:
[End of section]
Appendix I: Scope and Methodology:
To gain an understanding of the process used to make extramural
research funding decisions, we reviewed the laws and regulations
governing the funding process and National Institutes of Health (NIH)
policies related to each stage of the process. We also interviewed NIH
officials with responsibility for establishing these policies and
overseeing the institutes and centers' (IC) implementation of this
process. In addition, to develop a detailed understanding of how the 24
ICs that fund extramural research implement the process, we selected 3
of the 24 ICs for a more detailed review: the National Cancer Institute
(NCI), the National Institute on Alcohol Abuse and Alcoholism (NIAAA),
and the National Institute of Diabetes and Digestive and Kidney
Diseases (NIDDK). These ICs were chosen because they vary in budget
size and focus on different disease-specific research missions. We also
included the Center for Scientific Review (CSR), which does not fund
extramural research but is responsible for implementing the initial
steps in the extramural research funding process, including receipt of
all grant applications.
At the IC level, we reviewed IC policies and guidance for implementing
the extramural research funding process and interviewed officials at
each of the 3 selected ICs plus CSR about their roles in receiving
applications, facilitating peer review of the applications, and making
final funding decisions. We also interviewed members of the NCI, NIAAA,
and NIDDK advisory councils about their role in reviewing and making
recommendations regarding extramural grant applications. In addition,
we analyzed selected data from the 24 ICs that fund extramural research
regarding funding decisions for grants in NIH's R01 category, which is
the most common of NIH's various grant categories. The R01 grant is the
original and historically oldest grant mechanism used by NIH. This type
of grant is awarded to organizations of all types (universities,
colleges, small businesses, for-profit, foreign and domestic, etc.) to
support a discrete, specified project to be performed by a named
investigator or investigators. Specifically, we requested information
about the paylines each of the 24 ICs established during fiscal years
2003 through 2007 to be used when making funding decisions. (The
payline roughly corresponds with the number of extramural grant
applications an IC will be able to fund each year and is based on
projections of the total funding available for grants at the IC that
year, the average dollar amount expected to be awarded per application,
and the number of applications coming to an IC.) We also requested data
about the number of R01 grant applications received, scored, and
recommended by the peer review groups; the total number of grant
applications funded; and the number of grant applications funded
relative to each IC's payline that year. We used the data to analyze
trends in funding decisions over the 5-year period. In order to analyze
the reasons the ICs cited when funding applications as exceptions to
the payline for R01 grants, we collected IC documentation for fiscal
years 2006 and 2007 from NCI, NIAAA, and NIDDK. Because the total
number of exception decisions made by NCI and NIDDK were large during
this time frame, we analyzed documents for a random sample of the
grants awarded as exceptions to the main payline. We also reviewed IC
documentation related to applications with priority scores above the
main payline that were not funded by NCI and NIDDK. NIAAA did not
choose to skip any applications during these fiscal years.
To ensure that the IC data were sufficiently reliable for our analyses,
we conducted detailed data reliability assessments of the data that we
used. We assessed the reliability of the IC data by reviewing existing
information about the data and the system that produced them and
interviewing agency officials knowledgeable about the data. We
determined that the data were sufficiently reliable for the purposes of
this report.
To gain an understanding of the design of control and monitoring
activities over travel and Title 42 personnel appointments, we reviewed
relevant NIH policies and guidance.[Footnote 35] To further our
understanding of control and monitoring activities, we also performed
walkthroughs of the travel and Title 42 personnel appointment processes
at three ICs--NCI, NIDDK, and NIAAA; these were the three ICs selected
for our review of the extramural research funding process.
During our walkthroughs of the travel process, we reviewed
authorizations, vouchers, and supporting receipts for travel
transactions at each of the selected ICs. During our walkthroughs of
the Title 42 personnel appointment process, we reviewed checklists
showing documents included in the appointment packages, routing slips
showing who received the appointment packages, and memos documenting
approvals for Title 42 personnel appointments at each of the selected
ICs. We interviewed key officials from the NIH OD and the ICs,
including the:
* NIH Deputy Director and the NIH Deputy Director for Management to
clarify our understanding of the differences between the roles of the
NIH OD and the ICs in the travel and personnel appointment processes
and the associated control and monitoring activities;
* NIH Director of Financial Management and the NIH Director and Deputy
Director of the Office of Human Resources to gain an understanding of
the control and monitoring activities that the NIH OD performs over
travel and Title 42 personnel appointments; and:
* IC Executive Officers (the highest level officials at the ICs that
oversee administrative activities) and other specialists within the ICs
to clarify our understanding of control and monitoring activities in
the travel and Title 42 personnel appointments at the IC level.
We compared the design of the processes to GAO's Standards for Internal
Control in the Federal Government[Footnote 36] to determine if the
processes as designed included appropriate control and monitoring
activities. While the design of control activities is based on NIH-wide
policies and procedures, monitoring activities vary at the individual
ICs. Therefore, our review of monitoring activities for travel and
Title 42 personnel appointments at these selected ICs cannot be
generalized to the other ICs. The scope of our audit did not include
testing the implementation of controls over travel and Title 42
personnel appointments.
To gain an understanding of the design of the NIH Management Control
Program, we reviewed relevant NIH policy and supporting documentation.
Specifically, we reviewed relevant NIH policies[Footnote 37] and the
NIH OD's fiscal year 2008 guidance to the ICs on reporting risk
management activities. To gain an understanding of the design of the
Enterprise Risk Management Program we reviewed NIH draft guidance.
[Footnote 38] We also reviewed the time lines for implementing the
Enterprise Risk Management Program to determine the estimated
implementation dates. We interviewed key officials from the NIH OD
including the:
* NIH Deputy Director and the NIH Deputy Director for Management to
gain a high-level understanding of how the Enterprise Risk Management
Program will address recent oversight issues at NIEHS and help NIH to
better manage its decentralized organization;
* NIH Director of Financial Management to understand the risk
activities NIH performed for fiscal year 2008 as part of the NIH
Management Control Program; and:
* NIH Director for the Office of Management Assessment--the office with
primary responsibility for designing and implementing the new risk
management program--to understand current risk activities at NIH, to
clarify the design of the new risk management program, and to further
our understanding of the implementation time line for the new risk
management program as well as the cause for delays in implementation.
We compared elements of the NIH Management Control Program and the
Enterprise Risk Management Program to our risk management framework
[Footnote 39] to determine if the designs contain the key components of
an effective risk management program. We did not review the
implementation of either the NIH Management Control Program or the
Enterprise Risk Management Program because, at the time of our review,
NIH did not plan to continue the Management Control Program and the
Enterprise Risk Management Program was not yet fully implemented.
We conducted this performance audit from March 2008 to September 2009,
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Appendix II: NIH Organization and Mission:
As the primary federal agency for supporting medical research in the
United States, the National Institutes of Health's (NIH) mission is
"science in pursuit of fundamental knowledge about the nature and
behavior of living systems and the application of that knowledge to
extend healthy life and reduce the burdens of illness and disability".
NIH is headed by a Director who is supported by 11 staff offices and 1
program office within the NIH Office of the Director (OD) and 27
institutes and centers (IC). Figure 3 depicts the organizational
structure of NIH.
Figure 3: Organizational Structure of NIH:
[Refer to PDF for image: organizational chart]
Top level:
Immediate Office of the Director;
Reporting to the Immediate Office of the Director:
Office of the Director Program Office: Division of Program
Coordination, Planning, and Strategic Initiatives;
Office of the Director Staff Offices:
* Office of Extramural Research;
* Office of Intramural Research;
* Office of Management/Chief Financial Officer;
* Office of Science Policy;
* Office of Communications and Public Liaison;
* Office of Equal Opportunity and Diversity Management;
* Office of Legislative Policy and Analysis;
* Executive Office;
* Office of the Ombudsman/Center for Cooperative Resolution;
* NIH Ethics Office;
* Office of the Chief Information Officer.
Second level, direct relationship with Immediate Office of the
Director:
National Cancer Institute;
National Eye Institute;
National Heart, Lung, and Blood Institute;
National Human Genome Research Institute;
National Institute on Aging;
National Institute on Alcohol Abuse and Alcoholism;
National Institute of Allergy and Infectious Diseases;
National Institute of Arthritis and Musculoskeletal and Skin Diseases;
National Institute of Biomedical Imaging and Bioengineering;
Eunice Kennedy Shriver National Institute of Child Health and Human
Development;
National Institute on Deafness and Other Communication Disorders;
National Institute of Dental and Craniofacial Research;
National Institute of Diabetes and Digestive and Kidney Diseases;
National Institute on Drug Abuse;
National Institute of Environmental Health Sciences;
National Institute of General Medical Sciences;
National Institute of Mental Health;
National Institute of Neurological Disorders and Stroke;
National Institute of Nursing Research;
National Library of Medicine;
John E. Fogarty International Center for Advanced Study in the Health
Sciences;
National Center for Complementary and Alternative Medicine;
National Center on Minority Health and Health Disparities;
National Center for Research Resources.
Third level, direct relationship with Immediate Office of the Director:
Clinical Center;
Center for Information Technology;
Center for Scientific Review.
Source: NIH.
[End of figure]
The ICs, which were established over time, each have an explicit
mission focused on a particular disease or organ system, an area of
human health and development, or aspects of research support.[Footnote
40] The first institute, the National Cancer Institute, was established
in 1937, and the newest institute, the National Institute of Biomedical
Imaging and Bioengineering, was established in 2000. Research funded by
NIH can be conducted by scientists in NIH laboratories and Clinical
Center--called intramural research--or by nonfederal scientists at
universities, academic health centers, hospitals, and independent
research institutions--called extramural research. Table 3 depicts a
time line of the establishment of the 27 ICs and their respective
missions and fiscal year 2008 appropriations.
Table 3: Overview of ICs Including Establishment Date, Mission, and
Fiscal Year 2008 Appropriation:
IC: National Cancer Institute;
Year established: 1937;
Mission: Conducts and supports research that will lead to a future in
which we can prevent cancer, identify cancers that do develop at the
earliest stage, eliminate cancers through innovative treatment
interventions, and biologically control those cancers that we cannot
eliminate so they become manageable, chronic diseases;
FY 2008 appropriation (in 000s): $4,830,647.
IC: Center for Scientific Review;
Year established: 1946;
Mission: Conducts initial peer reviews of the majority of research and
research-training applications submitted to NIH;
FY 2008 appropriation (in 000s): N/A[A].
IC: National Institute of Allergy and Infectious Diseases;
Year established: 1948;
Mission: Leads research that strives to understand, treat, and
ultimately prevent the myriad infectious, immunologic, and allergic
diseases that threaten millions of human lives;
FY 2008 appropriation (in 000s): $4,583,344.
IC: National Heart, Lung, and Blood Institute;
Year established: 1948;
Mission: Provides leadership for a national program in diseases of the
heart, blood vessels, lung, and blood; blood resources; and sleep
disorders;
FY 2008 appropriation (in 000s): $2,937,654.
IC: National Institute of Dental and Craniofacial Research;
Year established: 1948;
Mission: Provides leadership for a national research program designed
to understand, treat, and ultimately prevent the infectious and
inherited craniofacial-oral-dental diseases and disorders that
compromise millions of human lives;
FY 2008 appropriation (in 000s): $392,233.
IC: National Institute of Mental Health;
Year established: 1949;
Mission: Provides national leadership dedicated to understanding,
treating, and preventing mental illnesses through basic research on the
brain and behavior, and through clinical, epidemiological, and services
research;
FY 2008 appropriation (in 000s): $1,412,951.
IC: National Institute of Diabetes and Digestive and Kidney Diseases;
Year established: 1950;
Mission: Conducts and supports basic and applied research and provides
leadership for a national program in diabetes, endocrinology, and
metabolic diseases; digestive diseases and nutrition; and kidney,
urologic, and hematologic diseases;
FY 2008 appropriation (in 000s): $1,715,761.
IC: National Institute of Neurological Disorders and Stroke;
Year established: 1950;
Mission: Seeks to reduce the burden of neurological diseases by
supporting and conducting research, both basic and clinical, on the
normal and diseased nervous system, fostering the training of
investigators in the basic and clinical neurosciences, and seeking
better understanding, diagnosis, treatment, and prevention of
neurological disorders;
FY 2008 appropriation (in 000s): $1,552,113.
IC: Clinical Center;
Year established: 1953;
Mission: Provides the patient care, services, and environment needed to
initiate and support the highest quality conduct of and training in
clinical research;
FY 2008 appropriation (in 000s): N/A[A].
IC: National Library of Medicine;
Year established: 1956;
Mission: Collects, organizes, and makes available biomedical science
information to scientists, health professionals, and the public;
FY 2008 appropriation (in 000s): $322,212.
IC: National Institute of General Medical Sciences;
Year established: 1962;
Mission: Supports basic biomedical research that is not targeted to
specific diseases but rather funds studies on genes, proteins, and
cells, as well as on fundamental processes like communication within
and between cells, how our bodies use energy, and how we respond to
medicines;
FY 2008 appropriation (in 000s): $1,946,104.
IC: Eunice Kennedy Shriver National Institute of Child Health and Human
Development;
Year established: 1962;
Mission: Leads research on fertility, pregnancy, growth, development,
and medical rehabilitation that strives to ensure that every child is
born healthy and wanted and grows up free from disease and disability;
FY 2008 appropriation (in 000s): $1,261,381.
IC: National Center for Research Resources;
Year established: 1962;
Mission: Provides laboratory scientists and clinical researchers with
the environments and tools they need to understand, detect, treat, and
prevent a wide range of diseases;
FY 2008 appropriation (in 000s): $1,155,560.
IC: Center for Information Technology;
Year established: 1964;
Mission: Incorporates the power of modern computers into the biomedical
programs and administrative procedures of NIH by focusing on three
primary activities: conducting-computational biosciences research,
developing computer systems, and providing computer facilities;
FY 2008 appropriation (in 000s): N/A[A].
IC: National Eye Institute;
Year established: 1968;
Mission: Conducts and supports research that helps prevent and treat
eye diseases and other disorders of vision;
FY 2008 appropriation (in 000s): $670,664.
IC: John E. Fogarty International Center for Advanced Study in the
Health Sciences;
Year established: 1968;
Mission: Promotes and supports scientific research and training
internationally to reduce disparities in global health;
FY 2008 appropriation (in 000s): $66,912.
IC: National Institute of Environmental Health Sciences;
Year established: 1969;
Mission: Reduces the burden of human illness and dysfunction from
environmental causes by, defining how environmental exposures, genetic
susceptibility, and age interact to affect an individual's health;
FY 2008 appropriation (in 000s): $645,669.
IC: National Institute on Alcohol Abuse and Alcoholism;
Year established: 1970;
Mission: Conducts research focused on improving the treatment and
prevention of alcoholism and alcohol-related problems to reduce the
enormous health, social, and economic consequences of this disease;
FY 2008 appropriation (in 000s): $438,579.
IC: National Institute on Drug Abuse;
Year established: 1973;
Mission: Supports and conducts research across a broad range of
disciplines and rapid and effective dissemination of results of that
research to improve drug abuse and addiction prevention, treatment, and
policy;
FY 2008 appropriation (in 000s): $1,006,022.
IC: National Institute on Aging;
Year established: 1974;
Mission: Leads a national program of research on the biomedical,
social, and behavioral aspects of the aging process; the prevention of
age-related diseases and disabilities; and the promotion of a better
quality of life for all older Americans;
FY 2008 appropriation (in 000s): $1,052,830.
IC: National Institute of Arthritis and Musculoskeletal and Skin
Diseases;
Year established: 1986;
Mission: Supports research into the causes, treatment, and prevention
of arthritis and musculoskeletal and skin diseases, the training of
basic and clinical scientists to carry out this research, and the
dissemination of information on research progress in these diseases;
FY 2008 appropriation (in 000s): $511,291.
IC: National Institute of Nursing Research;
Year established: 1986;
Mission: Supports clinical and basic research to establish a scientific
basis for the care of individuals across the life span--including
managing patients during illness and recovery to reducing risks for
disease and disability; promoting healthy lifestyles; promoting quality
of life in those with chronic illness; and caring for individuals at
the end of life;
FY 2008 appropriation (in 000s): $138,207.
IC: National Institute on Deafness and Other Communication Disorders;
Year established: 1988;
Mission: Conducts and supports biomedical research and research
training on normal and disordered processes of hearing, balance, smell,
taste, voice, speech, and language that affect 46 million Americans;
FY 2008 appropriation (in 000s): $396,234.
IC: National Human Genome Research Institute;
Year established: 1989;
Mission: Supports the NIH component of the Human Genome Project, a
worldwide research effort designed to analyze the structure of human
DNA and determine the location of the estimated 30,000 to 40,000 human
genes;
FY 2008 appropriation (in 000s): $489,368.
IC: National Center on Minority Health and Health Disparities;
Year established: 1993;
Mission: Promotes minority health and leads, coordinates, supports, and
assesses NIH efforts to reduce and ultimately eliminate health
disparities among minority and other medically underserved communities.
Conducts and supports basic, clinical, social, and behavioral research;
promotes research infrastructure and training; fosters emerging
programs; disseminates information; and reaches out to minority and
other medically underserved communities;
FY 2008 appropriation (in 000s): $200,630.
IC: National Center for Complementary and Alternative Medicine;
Year established: 1999;
Mission: Explores complementary and alternative medical practices in
the context of rigorous science; trains researchers; and disseminates
authoritative information;
FY 2008 appropriation (in 000s): $122,224.
IC: National Institute of Biomedical Imaging and Bioengineering;
Year established: 2000;
Mission: Improves health by promoting fundamental discoveries, design
and development, and translation and assessment of technological
capabilities in biomedical imaging and bioengineering;
FY 2008 appropriation (in 000s): $300,233.
Source: NIH.
[A] The IC does not fund research and does not receive a separate
appropriation but rather is funded through the NIH Management Fund.
[End of table]
[End of section]
Appendix III: Comments from the National Institutes of Health:
Department Of Health & Human Services:
Office Of The Secretary:
Assistant Secretary For Legislation:
Washington, DC 20201:
July 21, 2009:
Linda T. Kohn:
Director, Health Care:
U.S. Government Accountability Office:
441 G Street N.W.
Washington, DC 20548:
Dear Ms. Kohn:
Enclosed are comments on the U.S. Government Accountability Office's
(GAO) report entitled: National Institutes Of Health: Completion of
Comprehensive Risk Management Program Essential to Effective Oversight
(GAO-09-687).
The Department appreciates the opportunity to review this report before
its publication.
Sincerely,
Signed by:
Barbara Pisaro Clark:
Acting Assistant Secretary for Legislation:
Attachment:
[End of letter]
General Comments Of The Department Of Health And Human Services (HHS)
On The Government Accountability Office's (GAO) Draft Report Entitled:
"National Institutes Of Health: Completion Of Comprehensive Risk
Management Program Essential To Effective Oversight" (GAO-09-687):
The National Institutes of Health (NIH) appreciates the review
conducted by GAO and the opportunity to provide clarifications,
corrections, and additional supporting documentation on this draft
report. NIH respectfully submits the following general comments.
Technical comments are included as a separate attachment.
GAO's Overall Conclusion. Completion of Comprehensive Risk Management
Program Essential to Effective Oversight:
GAO's draft report implies that the NIH Enterprise Risk Management
Program is the sole management oversight mechanism at NIH and fails to
acknowledge other management, governance, and oversight bodies and
functions.
As emphasized by the title of the report, it appears that GAO equates
the NIH Risk Management Program with overall governance and management
of the agency. This is a false assumption. NIH offers the following
clarification.
While the risk management program at NIH is an important tool used by
managers NIH-wide for decision-making, it is only one of many
mechanisms available to the Office of the Director (OD) for effective
oversight and accountability over the agency's 27 Institutes and
Centers (IC).
OD is responsible for a number of programs that manage risk, reasonably
ensure internal control, provide governance, and drive performance for
the agency. As an Attachment to this response, NIH has enumerated many,
but not all, of the oversight functions of NIH-wide activities that OD
performs. For example, the NIH Steering Committee, chaired by the NIH
Director and composed of 10 IC Directors, works as an efficient and
transparent forum for trans-NIH governance and streamlined decision-
making. Further, standing working groups provide recommendations to the
NIH Steering Committee on issues such as intramural and extramural
research, facilities, budget, and information technology.
Another example of an OD oversight mechanism is the NIH Ethics Advisory
Committee that provides centralized, consistent, and rigorous reviews
of requests to engage in outside activities and awards that bestow
gifts over $2,500. The Office of Extramural Research manages the Peer
Review process, promotes scientific integrity, and manages research
risks to patients. Likewise, the Office of Intramural Research manages
the Institutional Review Board process for intramural research
performed by NIH scientists.
GAO Finding: NIH's Current and Proposed Risk Management Programs Do Not
Fully Address Key Components of Effective Risk Management (p.19):
1. Clarification: NIH's Enterprise Risk Management Program is based on
widely-accepted Federal Government internal control standards.
On page 19, GAO states that the program "does not fully address all of
the components of GAO's framework for effective risk management." GAO's
report relies on a comparison of the MIT Enterprise Risk Management
Program with a risk management process published in an appendix to a
report entitled, Risk Management: Further Refinements Needed to Assess
Risks and Prioritize Protective Measures at Ports and Other Critical
Infrastructure (GAO-06-91, Dec. 2005).
In a footnote on page 7 of the report, GAO states that "risk management
does not replace, but rather incorporates and expands on internal
control." However, NIH defines risk management as synonymous with
internal control. The title "Risk Management" was used to distinguish
the new internal control program from the former internal control
program, which ended in 2006. NIH believed that the Risk Management
program would bring new enthusiasm to the internal control program from
the scientific and management communities at NIH. NIH suggests deleting
the footnote.
The criteria to which GAO compared the NIH Enterprise Risk Management
Program are not the same criteria that NIH used in developing its Risk
Management Program. Specifically, NIH used the Standards for Internal
Control in the Federal Government; the Office of Management and Budget
(OMB) Circular A-123, Management's Responsibility for Internal Control;
the GAO Internal Control and Management Evaluation Tool; and the
Guidance Manual for OMB circular A-123 Assessments published by the
Department of Health and Human Services. We believe that these
frameworks provide robust guidance to effectively manage risk and
reasonably ensure internal control. Although there is some overlap
between the guidance GAO used and the guidance NIH used, NIH chose its
criteria because it meets the OMB and HHS standards for an internal
control program.
While the fundamentals of the NIH Risk Management Program adhere to the
above frameworks, NIH tailored the content and approach of the Program
to have it work effectively in a scientific research environment. NIH
strongly believes that these adaptations are critically important to
the Program's success. For example, the NIH Program has developed a set
of standardized criteria to evaluate and prioritize risks. Development
of these criteria included input from a wide array of scientific and
management personnel at NIH to make the criteria applicable to a wide
range of risks at the agency. NIH suggests that GAO revise its report
accordingly.
2. Correction: GAO's draft report refers to a "current program" that is
in fact a retired risk management program that has not been used in two
years. The "proposed program" cited in the draft report is the NIH
Enterprise Risk Management Program currently operating at NIH.
On page 19, GAO indicates that NIH implemented its "current" program in
1999 and updated it in 2004. This statement is incorrect.
The "current" program that was in place in 1999 and 2004 and was
retired by NIH in 2006. All references to the "current" program in
GAO's report correspond to the outdated program and are neither timely
nor relevant to the activities currently being conducted and
implemented as part of the NIH Enterprise Risk Management Program. The
program identified by GAO as "proposed" is in fact the current program
that is in operation at NIH.
NIH recommends that GAO correctly refer to the retired program as well
as the Enterprise Risk Management Program that is currently being
implemented and operated at NIH. In addition, NIH recommends that GAO
move all references to the retired program (pages 19-23), including
related findings, to a background section or remove them entirely from
the report. NIH has provided technical comments on each reference made
to the retired program so that GAO may appropriately correct this
error.
3. NIH does not concur with GAO's finding that the NIII Enterprise Risk
Management Program "does not require NIH OD or ICs to set mission-based
strategic goals and objectives as a precondition for risk management;"
the report fails to acknowledge that all 27 NIH ICs have strategic
plans and that the Program is designed to "support the research mission
and vision of NIB."[Footnote 41]
On page 24 of the Strategic Goals, Objectives, and Constraints section
of the draft report, GAO states that the program "does not require NIH
OD or ICs to set mission-based strategic goals and objectives as a
precondition for risk management." NIH recommends that GAO delete this
finding and the associated recommendation.
The NIH Enterprise Risk Management Program "is designed to proactively
identify and manage risks before they become obstacles to the NIH
mission."[Footnote 42] The NIH Enterprise Risk Management Program
provides explicit guidance for all Program stakeholders to "Think
strategically. Consider the goals, objectives and mission of the
organization. Any event or condition that could prevent or inhibit the
accomplishment of the organization's goals, objectives or mission
should be documented as a risk."[Footnote 43]
The ICs establish their own scientific research strategic goals and
objectives. IC-level goals and objectives do not contradict each other,
are relevant to the broader NIH-wide mission, and include measurement
criteria. However, there is a direct link between each IC's operational
strategies and those of NIH as a whole. NIH is the steward of medical
and behavioral research for the nation. One of the four goals of the
agency is to "exemplify and promote the highest level of scientific
integrity, public accountability, and social responsibility in the
conduct of science."[Footnote 44] In realizing this goal, the NIH
Deputy Director for Management provides leadership and direction to
programs such as the Enterprise Risk Management Program.
The Program is part of a management strategy to support the scientific
research mission of NIH. The NIH Deputy Director for Management's (DDM)
Strategic Plan includes four goals:
* Goal 1 - Improving Human Capital Planning and Management;
* Goal 2 - Leveraging Information for Data Driven Decision-Making to
achieve Performance Excellence;
* Goal 3 - Employing Proactive Risk Management to Enhance Program
Performance;
* Goal 4 - Enhancing Internal Communications.
The mission of the management community is to enable NIH to pursue its
biomedical research mission of scientific discovery and advancement of
knowledge by serving as a valued partner that provides timely, high
quality, and responsive programs and services in a manner that reflects
a commitment to excellence and the preservation of public trust.
4. NIH concurs with GAO's finding and corresponding recommendation
regarding alternatives evaluation, including evaluation of risk
responses.
On page 25 under the Alternatives Evaluation section of the draft
report, GAO states that the program "does not require management to
evaluate the risk responses identified to consider (1) the effect on
the likelihood of occurrence and impact of a potential risk and (2) the
costs and benefits."
We appreciate this valuable feedback to improve our Program. As a
result of GAO's review, findings and recommendations, NIH will update
the Enterprise Risk Management Guidebook to reflect this
recommendation. The proposed risk response should be reviewed by the
Program and discussed during the Remediate phase to determine
appropriateness. The resulting assignment of the risk response will
consider alternatives based upon various factors, including cost and
benefits.
NIH is in the process of updating its policy regarding enterprise risk
management and internal controls found in Manual Chapter 1750.
Revisions to the policy will be reflected in updates to its Enterprise
Risk Management Guidebook to address ongoing monitoring of risk
strategies and potential alternatives.
5. NIH concurs with GAO's finding and corresponding recommendation
regarding documentation of management selection.
On page 25 under the Management Selection section of the draft report,
GAO states that the program does "not require management to document
the rationale for selecting a particular risk response." We appreciate
this feedback and have incorporated this clement into our processes and
amended the Enterprise Risk Management Guidebook.
6. NIH concurs with GAO's Implementation and Monitoring finding
regarding how the assessments of the overall effectiveness of the risk
management program should be performed. However, NIH offers the
following clarifications.
On page 25, within the Implementation and Monitoring section of the
draft report, GAO states that the program "does not offer any detail
regarding how assessments will be performed" and "the design does not
require periodic assessments of implemented risk responses."
The NIH Enterprise Risk Management Program has already undergone
several incremental evaluations. First, upon completion of the pilot,
the Program conducted an evaluation to gather stakeholder feedback.
This feedback was used to further refine the NIH risk management
methodology and the tools that support it. Results of the pilot
evaluation are documented in a pilot test report that was completed at
the conclusion of pilot test activities.
In addition, after the completion of the risk identification and
scoring steps in the first phase of implementation with the NIH OD, the
Program conducted an internal evaluation, Program staff evaluated the
process and outcomes and developed lessons leaned and recommendations
for improvement to guide the Program in the implementation of Phase 2
with the NIH ICs. Results of this evaluation are documented in the OD
Baseline Report.
An additional staff evaluation, documentation of lessons learned and
development of recommendations for further Program improvements is
currently underway and is expected to be finalized by August 15, 2009.
A more formal evaluation of the Program is being planned and will
likely include a follow-up of the Risk Culture Survey that was
conducted during the HHS audit in 2006. The Risk Culture Survey
examines potential strengths and weakness in the risk management and
control environment. It measures the impact of the Program on the
internal control environment at NIH, focusing on Leadership and
Strategy, Accountability and Reinforcement, People and Communication,
Risk Management and Infrastructure. These four areas include sub-
components that include ethics, tone at the top, and training.
NIH has identified target milestone periods for ongoing evaluation in
the Risk Management portion of the NIH Deputy Director for Management's
Administrative Management Strategic Plan. This includes developing a
program evaluation process and conducting periodic reviews. NIH
anticipates that by June 30, 2010, NIH will conduct a follow-up Risk
Culture Survey and will develop a program evaluation process, to
include determining the frequency, scope and methodology for the
reviews. In FY 2011, NIH will conduct reviews of the overall efficiency
and effectiveness of the NIH Enterprise Risk Management Program.
7. NIH does not concur with the finding and corresponding
recommendation regarding the Program's internal environment. NIH does
incorporate the importance of ethical values into the Program and
maintains the competence of its risk management personnel by providing
training.
NIH operates within an environment in which ethical values play a key
role:
On page 25, GAO states that the Risk Management Program could be
improved by "incorporating the importance of ethical values." The
Program already operates within the context of a positive environment
in which integrity and ethical values play a key role. NIH has a formal
code of conduct; senior management has established an ethical tone and
consistently models and enforces conscientious and competent
leadership; management takes disciplinary action whenever appropriate
or necessary and NIH has an extensive set of guidance on ethics. The
NIH Ethics Office, within the OD, works in tandem with each individual
IC Ethics Program to provide mandatory ethics training-both for new and
current employees-that exceeds federal requirements. The importance of
ethical values is being reinforced constantly through other means and
in other ways that more directly affect internal controls.
Furthermore, the report does not provide any evidence to support this
statement and does not state how this would be achieved. We ask that
the finding and recommendation be deleted from the report.
Nevertheless, NIH will amend the Enterprise Risk Management Guidebook
to include specific language addressing the importance of ethics at
NIH.
NIH maintains the competence of its risk management personnel by
providing training:
On page 25, GAO states that the program could be improved by "ensuring
management maintains the competence of its personnel by providing for
continuous training to update personnel on risk management practices
and techniques." It should be highlighted that NIH provided GAO with
documentation about training given to OD and IC staff who hold
significant risk management roles, as well as the completion dates of
training. The information shows that NIH does ensure the competence of
staff working in risk management and does not support GAO's conclusion
about the lack of training on risk management practices and techniques.
NIH does not understand how GAO reached this conclusion, and the report
does not provide an explanation or supporting evidence for this
funding.
To date, the NIH Enterprise Risk Management Program has provided
training to over 400 individuals who hold significant risk management
roles. This includes training specifically tailored for all OD Office
Directors and IC Executive Officers. In addition, specific training
focusing on the methodology and the identification of risks was
provided to OD Office staff and IC leadership staff identified by OD
Office Directors and IC Executive Officers as having important risk
management roles. Additional training data is available to GAO upon
request.
The number of individuals trained to date is a result of the NIH
Enterprise Risk Management training plan, which establishes targets for
continuous training. NIH has also developed a formal risk management
training course that is available to all NIH employees through the NIH
Training Center and is developing training for NIH Administrative
Officers. Furthermore, the DDM Strategic Plan includes a strategy for
developing role-based and general awareness risk management training to
reinforce a culture of risk awareness among leadership and staff. The
training, as well as risk management communications, help ensure that
all employees understand their role in conducting operations in a
manner that manages risk. NIH has not provided training updates and
refreshers because the program first began reaching personnel in 2008
and 2009. However, the NIH Enterprise Risk Management training plan
will incorporate these as the Program continues. Therefore, NIH
recommends that GAO change its finding and revise its recommendation
for NIH to "fully implement its plan for ongoing training."
8. NIH partially concurs with GAO's finding that the NIH Enterprise
Risk Management Program should "require the collection and
dissemination of pertinent information to relevant external
stakeholders."
On page 26 within the Information and Communication component, GAO
states that "the program does not require the collection and
dissemination of pertinent information to relevant external
stakeholders in a form and timeframe consistent with NIH's risk
management needs." NIH offers the following clarification. GAO places
emphasis on the communications with internal stakeholders cited in the
GAO framework, Risk Management. Further Refinements Needed to Assess
Risks and Prioritize Protective Measures at Ports and Other Critical
Infrastructure (GAO-06-91).
NIH promptly responds to all requests for information from external
stakeholders, including those related to internal controls and risk
management. The Risk Management Program provides NIH and HHS internal
stakeholders with timely and accurate risk management data. For
example, the Program provides the NIH Risk Management Council and NIH
Risk Management Senior Assessment Team with risk management reports,
program status updates, and updates on emerging risk management issues
such as the Recovery Act. Another example of internal risk management
communications is the Program's delivery to HHS of the NIH annual
Federal Managers' Financial Integrity Act (FMFIA) statement of
assurance attesting that NIH federal programs have effective and
efficient controls in place that meet the objectives of the FMFIA and
OMB Circular A-123.
The Program will include external communications and exchanges as it
matures.
9. NIH does not concur with GAO's finding that the Enterprise Risk
Management Program has been hampered by a lack of milestones. Although
the Program has encountered some delays, the Program maintains a
schedule of milestones.
NIH has implemented all six steps of the Enterprise Risk Management
methodology.
On pages 19 and 26, the GAO report states that the NIH program has "not
yet [been] fully implemented despite an over 3-year effort." This
statement is incorrect.
During the field work phase of its audit, GAO inquired about whether
the risk management methodology had been implemented NIH-wide. NIH
stated that as of November 19, 2008, the new risk management
methodology had not been fully implemented across NIH.
However, since November 2008, NIH has implemented steps l and 2
(Organize and Identify and Score) of the methodology across all 27 ICs,
as well as steps 3 and 4 (Assess and Remediate) of the methodology
across OD. The creation of a comprehensive NIH enterprise baseline risk
inventory is a result of the agency's implementation of the Identify
and Score steps of the methodology across NIH. Furthermore, On May 7,
2009, NIH hosted the first Risk Management Council (RMC) meeting. The
RMC provides guidance on program implementation and operations, has
oversight over the completion of risk management activities, and
reports results to the NIH Risk Management Senior Assessment Team.
Because these two risk management governance structures are
operational, NIH executed Steps 5 and 6 (Monitor and Report) of the
methodology. Therefore, NIH believes that it has executed all six steps
of its Enterprise Risk Management Program for a fully functioning
Program.
NIH began developing the Enterprise Risk Management Program in August
2007. In just two years, NIH has designed and executed its Enterprise
Risk Management Program to conform to GAO, OMB, and HHS guidance. This
represents extraordinary progress. NIH recognizes the need for
continued progress to fully implement and continuously improve the
Program. NIH defines full implementation as an initial completion of
the Program's first two steps (Organize and Identify and Score) and the
continuous operation of the remaining four steps: Assess, Remediate,
Monitor and Report. The NIH Enterprise Risk Management Program is
scheduled for full implementation by June 2010.
The Program maintains a schedule of milestones:
On pages 19 and 26, GAO's draft report indicates that the NIH program
has "been hampered by a lack of milestones." This statement is also
incorrect.
According to the GAO document, Performance Measurement and Evaluation,
GAO/GGD-98-26, page 3, a "program may be any activity, project,
function, or policy that has an identifiable purpose or set of
objectives." The NIH Enterprise Risk Management Program maintains a
schedule of milestones and defines the Program's goals and objectives
in the NIH Management Strategic Plan discussed in Response #4 of this
document. During the fieldwork phase of GAO's audit, NIH provided GAO
with a Work Breakdown Structure document that contained significant
milestones such as the following:
* Complete a pilot of the Enterprise Risk Management Program
methodology;
* Implement steps I through 4 of the methodology across OD;
* Implement steps I and 2 of the methodology across all 27 ICs;
* Conduct risk management training to individuals who hold significant
risk management roles;
* Implement both risk management governance structures: the Risk
Management Council and the Risk Management Senior Assessment Team.
The development and implementation of the Program demonstrates the
remarkable level of effort and commitment NIH has invested over the
past two years.
GAO does not provide agencies with an standard timeline for filly
implementing a Risk Management Program:
On page 29, GAO recommends that NIH complete and implement its program
in a reasonable timeframe. GAO does not define what constitutes a
"reasonable" timeframe. NIH offers the following clarification.
In the December 2005 report, Risk Management Further Refinements Needed
to Assess Risks and Prioritize Protective Measures at Ports and Other
Critical Infrastructure (GAO-06-91), pages 99 and 101, GAO acknowledges
that the "[GAO] risk management framework has been used to evaluate
activities related to security and combating terrorism" and "is
intended to be a starting point for risk management activities and will
likely evolve as processes mature and lessons are learned." While the
framework is a valuable tool for establishing a "full cycle of related
activities from strategic planning through implementation and
monitoring" and is used to "inform agency officials and decision makers
of the basic components of a risk management system," it does not
provide agencies with a standard timeline for fully implementing a risk
management program.
NIH would like to know the criteria GAO used as a basis for its
conclusion and asks that this finding be modified in the report.
GAO Finding: NIH Is Required to Use a Peer Review System to Make
Extramural Funding Decisions; NIH's OD Does Not Monitor Key Decisions
In Which IC Directors Exercise Their Discretion Over Funding Decisions
(p.9):
1. NIH does not concur with the recommendation that would require an
oversight role for the OD that is inconsistent with the IC Director's
authority to make grant award decisions.
GAO states that IC directors can use their discretion and choose to
fund applications on the basis of factors other than scientific merit,
"skipping" over applications with higher scores or making "exceptions"
to fund applications with lower scores. NIH stresses that while IC
directors authorize these actions, their decisions are not made in
isolation or without consultation, and review. The NIH OD ensures that
there is a process in place that documents these decisions and that
these documents are available upon request, should questions arise.
ICs are required to document the rationales used when skipping
applications or funding applications as exceptions. While the ICs are
not required to routinely provide the NIH OD with this documentation,
these data are available, upon request, for analysis by the NIH OD for
assessing compliance or for other purposes. The GAO recommendation
should be revised accordingly to recognize that the role of the NIH OD
for this activity is to ensure that documentation procedures are in
place, not to have input about the specific scientific reasoning about
skipping applications or funding applications as exceptions.
GAO implies a role for the OD that is not scientifically appropriate.
Specific reasons for skips and exceptions must and should rely on the
judgment of scientific officials who understand the current trends in
science, as well as the portfolios of the institute.
2. In response to Page 15, Table 1, "Extramural Research R01 Grant
Applications Funded in FY03 -FY07", NIH would like to highlight that
R01 grants awarded outside the payline from FY03 through FY07 were in
fact a result of the increase in the number of R01 grants awarded to
new investigators.
Maintaining a viable research workforce is considered essential to the
vitality of health-related research. That means new investigators must
enter the pool of NIH funded Principal Investigators at a reasonable
rate to replace those who choose to leave or leave because their
applications are no longer competitive. In some cases, an adequate
supply of new investigators is dependent on funding applications that
receive review scores outside the normal funding range, as shown in the
chart below. It should be pointed out that these applications are still
well within the range of scores that are considered to be highly
meritorious. In order to protect the viability of the extramural
workforce the NIH reaches for additional applications from New
Investigators.
During FY07 through FY 2009, the OD designed and implemented policies
to support new investigators. The policies were designed to reverse the
steady decline in the number of new investigators that started in FY03.
Annually, OD has presented guidelines to the Institute Directors. Those
guidelines arc available at [hyperlink,
http://grants.nih.gov/gtants/new_investigators/indix.htm]. Each year
the OD sets New Investigator targets for the ICs and then tracks awards
to New Investigators during the course of the year. Over the past three
fiscal years the NIH has reached the established targets. The
importance of new investigators to the continued success of the NIH
extramural programs is well understood. This information has been
clearly articulated in notices that have appeared in the NIH Guide for
Grants and Contracts and in other NIH publications and presentations.
NIH created the graph below to illustrate the increase in the raise-to-
pay awards as a result of the increase in R01 awards to new
investigators.
Figure: NIH R01 Awards Outside the Pay Line Awarded to New
Investigators:
[Refer to PDF for image: line graph]
Fiscal year: 2003;
Number of R01 Awards Outside the Pay Line: 625;
Number of R01 Awards Outside the Pay Line Awarded to New Investigators:
124.
Fiscal year: 2004;
Number of R01 Awards Outside the Pay Line: 528;
Number of R01 Awards Outside the Pay Line Awarded to New Investigators:
133.
Fiscal year: 2005;
Number of R01 Awards Outside the Pay Line: 572;
Number of R01 Awards Outside the Pay Line Awarded to New Investigators:
176.
Fiscal year: 2006;
Number of R01 Awards Outside the Pay Line: 620;
Number of R01 Awards Outside the Pay Line Awarded to New Investigators:
203.
Fiscal year: 2007;
Number of R01 Awards Outside the Pay Line: 1,059;
Number of R01 Awards Outside the Pay Line Awarded to New Investigators:
532.
[End of figure]
3. NIH would like to review GAO's method and approach for quantifying
out of order funding.
GAO Finding: Design of NIH's Travel and Personnel Appointment Processes
Include Key Control Activities and Some Monitoring Activities but Lacks
Systemic Risk-Based Monitoring (p.16):
1. NIH concurs with the finding that the design of NIH's Title 42
personnel appointment process includes key control activities and some
monitoring but lacks systemic risk-based monitoring.
On page 16, GAO states: "The NIH OD has also established policies and
procedures to help ensure that Title 42 personnel appointment decisions
are appropriate." The report continues on page 17 to say that "the
design of the controls included in the NIH-wide processes over travel
and Title 42 personnel appointments included key controls necessary to
help ensure these activities were being carried out appropriately,
except in one key area related to the lack of requirements for risk-
based management." This key area was identified as systemic monitoring
that "includes risk-based control evaluations." Further the report
states that while some monitoring was going on, it was neither a part
of a systemic risk assessment plan and was not performed on an on-going
basis.
In regard to the Title 42 personnel appointment process, NIH agrees
with the GAO finding that at the present, NIH does not have in place a
systemic monitoring program for risk-based evaluations. While
evaluations are done, as the report notes, they do not meet the test of
being part of a "systemic risk assessment plan" or are not performed on
a consistent and on-going basis. NIH will incorporate risk-based
monitoring into the Title 42 personnel appointment process. In
addition, NIH will add the Title 42 personnel appointment process as a
risk area within the NIH Enterprise Risk Management Program to ensure
that the risk is monitored and assessed.
NIH identified and scored the agency travel process as a risk area
within the NIH Enterprise Risk Management Program. Therefore, travel
risks are being monitored. NIH will reassess the travel risk levels to
ensure that they are appropriate.
Table: The National Institute of Health: Inventory of OD Oversight
Mechanisms for the GAO Governance and Oversight Review:
52 items are listed in the inventory [original copy is illegible]
Refer to PDF for information.
[End of section]
Appendix IV: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Linda T. Kohn (202) 512-7114 or kohnl@gao.gov:
Susan Ragland (202) 512-8486 or raglands@gao.gov:
Acknowledgments:
In addition to the contacts named above, Paul Caban and Jenny Grover,
Assistant Directors; Jehan Abdel-Gawad; Deyanna Beeler; Francine
Delvecchio; Patrick Frey; Krister Friday; Natalie Herzog; Cynthia
Jackson; Kelli Jones; Judy Lee; Lisa Motley; Kara Patton; Will Simerl;
Jessica Smith; and Matt Zaun made key contributions to this report.
[End of section]
Footnotes:
[1] NIH also supports intramural research, which is performed by NIH
scientists in NIH laboratories.
[2] The three centers that do not fund extramural research and do not
receive separate appropriations (Center for Scientific Review, Center
for Information Technology, and the Clinical Center) are funded through
the NIH Management Fund, which is funded using a portion of other NIH
appropriations. See 42 U.S.C. § 290.
[3] GAO, NIH Conflict of Interest: Recusal Policies for Senior
Employees Need Clarification, [hyperlink,
http://www.gao.gov/products/GAO-07-319] (Washington, D.C.: Apr. 30,
2007).
[4] H.R. Rep. No. 110-231, at 161-62 (2007); NIH Office of Management
Assessment, Management Review: National Institute of Environmental
Health Sciences (Apr. 9, 2008).
[5] GAO, Standards for Internal Control in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]
(Washington, D.C.: November 1999).
[6] Under two provisions of title 42 United States Code, NIH has
additional hiring flexibilities not permitted under title 5 authorities
related to the general schedule and senior executive service. These
flexibilities are referred to as "title 42" personnel appointments.
Specifically, title 42 authorities allow NIH to hire scientists at
salary levels comparable to those outside of the federal government. In
2008, under these authorities, NIH could hire scientists with salary
levels up to $250,000. However, maximum pay for the general schedule
was $149,000 and for the senior executive service was $172,200 in 2008.
See 42 U.S.C. § 209 (f),(g).
[7] Specifically, we reviewed funding decisions made for R01 grants,
the original grant mechanism used by NIH, which is a common type of
grant awarded to organizations of all types (universities, colleges,
small businesses, for-profit, foreign and domestic, etc.) to support a
discrete, specified project to be performed by a specific investigator
or group of investigators.
[8] A walkthrough is a method used to develop an understanding of key
processes and controls in which an auditor traces a transaction through
the organization's procedures.
[9] See [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1].
[10] See table 2 for GAO's framework for effective risk management. GAO
developed the framework based on authoritative literature and
standards, as well as previous GAO reports and testimonies. We
consulted the Government Performance and Results Act (GPRA) of 1993;
the Government Auditing Standards, 2003 Revision; GAO's Standards for
Internal Control in the Federal Government (November 1999); guidance
from the Office of Management and Budget (OMB); the work of the
President's Commission on Risk Management; consulting papers; and the
enterprise risk management approach of the Committee of Sponsoring
Organizations (COSO) of the Treadway Commission. We reviewed numerous
frameworks from industry, government, and academic sources. GAO,
"Appendix I: A Risk Management Framework" of Risk Management: Further
Refinements Needed to Assess Risks and Prioritize Protective Measures
at Ports and Other Critical Infrastructure, [hyperlink,
http://www.gao.gov/products/GAO-06-91] (Washington, D.C.: Dec. 15,
2005).
[11] The Director of NIH is appointed by the President, with Senate
confirmation. The President also appoints the director of NCI, while
the Secretary of the Department of Health and Human Services (HHS)
appoints the other IC directors.
[12] See [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1].
[13] See 31 U.S.C. § 3512(c).
[14] Risk management does not replace, but rather incorporates and
expands on internal control. Thus, risk management provides a more
robust and extensive focus to effectively identify, assess, and manage
risk.
[15] GAO, High-Risk Series: An Update, [hyperlink,
http://www.gao.gov/products/GAO-05-207] (Washington, D.C.: January
2005).
[16] See table 2. GAO developed the framework based on authoritative
literature and standards, as well as previous GAO reports and
testimonies. We consulted the Government Performance and Results Act
(GPRA) of 1993; the Government Auditing Standards, 2003 Revision; GAO's
Standards for Internal Control in the Federal Government (November
1999); guidance from the Office of Management and Budget (OMB); the
work of the President's Commission on Risk Management; consulting
papers; and the enterprise risk management approach of the Committee of
Sponsoring Organizations (COSO) of the Treadway Commission. We reviewed
numerous frameworks from industry, government, and academic sources.
GAO, "Appendix I: A Risk Management Framework" of Risk Management:
Further Refinements Needed to Assess Risks and Prioritize Protective
Measures at Ports and Other Critical Infrastructure, [hyperlink,
http://www.gao.gov/products/GAO-06-91] (Washington, D.C.: Dec. 15,
2005).
[17] The Committee of Sponsoring Organizations of the Treadway
Commission, Enterprise Risk Management--Integrated Framework (Jersey
City, N.J.: American Institute of Certified Public Accountants,
September 2004) and [hyperlink,
http://www.gao.gov/products/GAO/AIMD-00-21.3.1].
[18] See 42 U.S.C. §§ 282(b)(9); 289a(a); 289a-1(a)(2). The Secretary
of HHS promulgated regulations expanding on the use of peer review by
groups appointed by the Director of NIH and the directors of the
national research institutes.
[19] The composition of the initial peer review groups is specified in
42 C.F.R. § 52h.4 (2008). Based on these criteria, NIH staff select the
initial peer reviewers, who generally agree to participate for 4 years.
[20] See 42 C.F.R. § 52h.7 (2008).
[21] 42 C.F.R. § 52h.8 (2008) directs peer review groups to assess each
proposed research project taking into account the following criteria,
among other pertinent factors: (a) its significance, (b) the adequacy
of its approach and methodology, (c) its innovativeness and
originality, (d) the qualifications and experience of its principal
investigator and staff, (e) the scientific environment and reasonable
availability of resources for it, (f) the adequacy of its plans to
include both genders, minorities, children, and special populations as
appropriate for its scientific goals, (g) the reasonableness of its
budget and duration, and (h) the adequacy of its protections for
humans, animals, and the environment.
[22] 42 U.S.C. § 284a. Although the law setting forth the requirements
for advisory councils is specific to institutes, each center that funds
extramural research has an advisory council substantially similar to
those of the institutes. See 42 U.S.C. §§ 287a (National Center for
Research Resources), 287c-21(b) (National Center for Complementary and
Alternative Medicine), 287c-31(j) (National Center on Minority Health
and Health Disparities).
[23] Advisory councils also include ex officio members, who are
nonvoting. Voting members generally serve 4-year terms. At the NCI, the
President appoints voting advisory council members, and the members
serve 6-year terms. For all other advisory councils, the Secretary of
HHS appoints voting members.
[24] By law, the advisory councils for NCI and the National Heart,
Lung, and Blood Institute must meet at least four times per fiscal
year. 42 U.S.C. § 284a(h)(2).
[25] NIH may not approve or fund any application unless it has been
recommended for approval by a majority of the members of the initial
peer review group and a majority of the voting members of the advisory
council. The initial peer review groups recommend applications for
approval via the scoring system. 42 U.S.C. § 289a-1(a)(2).
[26] The R01 grant is the original grant mechanism used by NIH. This
type of grant is awarded to organizations of all types (universities,
colleges, small businesses, for-profit, foreign and domestic, etc.) to
support a discrete, specified project to be performed by a named
investigator or investigators.
[27] See GAO, [hyperlink,
http://www.gao.gov/products/GAO/AIMD-00-21.3.1]. The Department of
Health and Human Services, of which NIH is a component, is required to
establish and maintain an effective system of internal control,
consistent with the standards prescribed by the Comptroller General. 31
U.S.C. § 3512(c), (d).
[28] Examples of positions that would require more involvement from the
NIH-level offices include senior-level employees such as tenure-track
investigators, senior investigators, senior scientists, and senior
clinicians.
[29] GAO, Department of Health and Human Services: Controls Over Travel
Program Are Generally Effective, but Some Improvements Are Needed,
[hyperlink, http://www.gao.gov/products/GAO-03-334] (Washington, D.C.:
Feb. 21, 2003).
[30] Travel cards are a type of charge card used for official travel-
related expenses.
[31] The Steering Committee, which is chaired by the NIH director and
composed of 10 IC directors who serve on a rotating basis, is NIH's
most senior-level governing body. The Steering Committee is responsible
for addressing NIH-wide issues, other than those that relate to
science.
[32] NIH Office of Management Assessment, NIH Enterprise Risk
Management Program, Enterprise Risk Management Guidebook: A Step-By-
Step Guide (March 2009, Draft).
[33] Risk management structure is a segmentation of discrete, mission-
oriented subsets of an organization to facilitate risk management
activities at the lower level.
[34] Scoring risks includes assessing the risk based on likelihood of
occurrence and impact. Based on the assessment, risks are assigned a
points value, which allows for quantitative comparison and ranking of
risks across NIH.
[35] For travel, we reviewed NIH manual chapters 1500-01: Introduction
to Official Government Travel (Jan. 5, 2004), 1500-02: Traveler
Responsibilities (May 13, 2008), and 1500-08: Acceptance of Payment
from a Nonfederal Source to Cover Travel Expenses [Sponsored Travel]
(Jan. 23, 2006). For Title 42 personnel appointments, we reviewed Title
42 Pay Model--NIH (Dec. 21, 2004) and NIH manual chapter 2300-575-2:
Title 42 Recruitment and Retention Incentives (May 4, 2000).
[36] GAO, Standards for Internal Control in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3.1]
(Washington, D.C.: November 1999).
[37] NIH Manual Chapter 1750 - Management Control Program (Nov. 15,
2004).
[38] NIH Office of Management Assessment, NIH Enterprise Risk
Management Program, Enterprise Risk Management Guidebook: A Step-By-
Step Guide (March 2009, Draft).
[39] See table 2. GAO developed the framework based on authoritative
literature and standards, as well as previous GAO reports and
testimonies. We consulted the Government Performance and Results Act
(GPRA) of 1993; the Government Auditing Standards, 2003 Revision; GAO's
Standards for Internal Control in the Federal Government (November
1999); guidance from the Office of Management and Budget (OMB); the
work of the President's Commission on Risk Management; consulting
papers; and the enterprise risk management approach of the Committee of
Sponsoring Organizations (COSO) of the Treadway Commission. We reviewed
numerous frameworks from industry, government, and academic sources.
GAO, "Appendix I: A Risk Management Framework" of Risk Management:
Further Refinements Needed to Assess Risks and Prioritize Protective
Measures at Ports and Other Critical Infrastructure, [hyperlink,
http://www.gao.gov/products/GAO-06-91] (Washington, D.C.: Dec. 15,
2005).
[40] Prior to 1985, Congress either created ICs itself or gave others
(e.g., the Surgeon General or the Secretary of HHS) the authority to
create ICs through individual laws. Since 1985, the Secretary of HHS
has had the authority to establish, reorganize, or abolish ICs. Pub. L.
No. 99-158, 99 Stat. 820 (1985).
[41] NIH Enterprise Risk Management Guidebook, page 6.
[42] Ibid.
[43] NIH Enterprise Risk Management Guidebook, page 25.
[44] NIH website, [hyperlink, http://www.nih.gov/about].
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
