VA Health Care
VA Spends Millions on Post-Traumatic Stress Disorder Research and Incorporates Research Outcomes into Guidelines and Policy for Post-Traumatic Stress Disorder Services Gao ID: GAO-11-32 January 24, 2011In addition to providing health care to veterans, the Department of Veterans Affairs (VA) funds research that focuses on health conditions veterans may experience. According to VA, experts estimate that up to 20 percent of Operation Enduring Freedom and Operation Iraqi Freedom veterans have experienced post-traumatic stress disorder (PTSD) and demand for PTSD treatment is increasing. Because of the importance of research in improving the services that veterans receive, GAO was asked to report on VA's funding of PTSD research, and its processes for funding PTSD research proposals, reviewing and incorporating research outcomes into clinical practice guidelines (CPG)--tools that offer clinicians recommendations for clinical services but do not require clinicians to provide one service over another--and determining which PTSD services are required to be made available at VA facilities. To do this work, GAO obtained and summarized VA data on the funding of PTSD research from its medical and prosthetic research appropriation through its intramural research program. GAO also reviewed relevant VA documents, such as those for developing CPGs and those related to VA's 2008 Uniform Mental Health Services in VA Medical Centers and Clinics handbook (Handbook), which defines certain mental health services that must be made available at VA facilities. GAO also interviewed VA officials.
Based on VA data GAO obtained and summarized, GAO found that the amount of funding VA provided for intramural PTSD research increased from $9.9 million in fiscal year 2005 to $24.5 million in fiscal year 2009. From fiscal year 2005 through fiscal year 2009, intramural PTSD research funding ranged from 2.5 percent to 4.8 percent of VA's medical and prosthetic research appropriation. In addition, the number of PTSD research studies VA funded through the Merit Review Program and the Cooperative Studies Program (CSP)--VA's two primary funding mechanisms in its intramural research program--increased from 47 in fiscal year 2005 to 96 in fiscal year 2009. According to VA officials, intramural research proposals, including those on PTSD, are funded primarily according to scientific merit in both the Merit Review Program and CSP. Proposals are evaluated by a panel of reviewers and scored based on their scientific merit. Directors of VA's research and development services--offices that focus on different research areas and administer VA's intramural research program--fund proposals based on their scores, typically up to a specified percentile. The number of proposals funded may vary based on budgetary considerations and, for a small number of proposals, responsiveness to VA research priority areas. VA has a process to review and incorporate relevant research outcomes to develop CPGs for a number of topics, including PTSD. VA relies on the policies of a joint VA and Department of Defense (DOD) work group--comprised of VA and DOD officials--to ensure that systematic reviews of relevant research outcomes are conducted when issuing CPGs. In brief, a systematic review is conducted to identify the most methodologically rigorous research studies that are applicable to each clinical question contained in the CPG. A group of subject matter experts then assesses the individual research studies in order to determine the overall quality of evidence available for each particular clinical question, considers the potential benefits and harms of a clinical intervention to determine its net effect, and, based on an assessment of the overall quality of the evidence and the net effect of an intervention, develops recommendations for the CPG. According to VA officials, the decision to require that two PTSD services--cognitive processing therapy and prolonged exposure therapy--be made available at VA facilities by including them in the Handbook was based on a review of research outcomes and the availability of existing resources. Specifically, VA officials told GAO that these two services were strongly recommended in the 2004 PTSD CPG and had greater evidence supporting their effectiveness than other PTSD services. VA also told GAO that prior to the Handbook's 2008 issuance, VA had already begun investing resources in training programs for cognitive processing therapy in 2006 and prolonged exposure therapy in 2007. While VA provided some documentation regarding the decision-making process for PTSD services, VA officials explained that clinical decision-making processes are not typically expected to be documented in a formal manner. VA officials told GAO that they are currently clarifying language in the Handbook but do not plan to revise any requirements relating to PTSD services at this time. VA provided technical comments that GAO incorporated as appropriate.
GAO-11-32, VA Health Care: VA Spends Millions on Post-Traumatic Stress Disorder Research and Incorporates Research Outcomes into Guidelines and Policy for Post-Traumatic Stress Disorder Services
This is the accessible text file for GAO report number GAO-11-32
entitled 'VA Health Care: VA Spends Millions on Post-Traumatic Stress
Disorder Research and Incorporates Research Outcomes into Guidelines
and Policy for Post-Traumatic Stress Disorder Services' which was
released on January 24, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Ranking Member, Subcommittee on Health, Committee on
Veterans' Affairs, House of Representatives:
January 2011:
VA Health Care:
VA Spends Millions on Post-Traumatic Stress Disorder Research and
Incorporates Research Outcomes into Guidelines and Policy for Post-
Traumatic Stress Disorder Services:
GAO-11-32:
GAO Highlights:
Highlights of GAO-11-32, a report to the Ranking Member, Subcommittee
on Health, Committee on Veterans‘ Affairs, House of Representatives.
Why GAO Did This Study:
In addition to providing health care to veterans, the Department of
Veterans Affairs (VA) funds research that focuses on health conditions
veterans may experience. According to VA, experts estimate that up to
20 percent of Operation Enduring Freedom and Operation Iraqi Freedom
veterans have experienced post-traumatic stress disorder (PTSD) and
demand for PTSD treatment is increasing. Because of the importance of
research in improving the services that veterans receive, GAO was
asked to report on VA‘s funding of PTSD research, and its processes
for funding PTSD research proposals, reviewing and incorporating
research outcomes into clinical practice guidelines (CPG)”-tools that
offer clinicians recommendations for clinical services but do not
require clinicians to provide one service over another-”and
determining which PTSD services are required to be made available at
VA facilities. To do this work, GAO obtained and summarized VA data on
the funding of PTSD research from its medical and prosthetic research
appropriation through its intramural research program. GAO also
reviewed relevant VA documents, such as those for developing CPGs and
those related to VA‘s 2008 Uniform Mental Health Services in VA
Medical Centers and Clinics handbook (Handbook), which defines certain
mental health services that must be made available at VA facilities.
GAO also interviewed VA officials.
What GAO Found:
Based on VA data GAO obtained and summarized, GAO found that the
amount of funding VA provided for intramural PTSD research increased
from $9.9 million in fiscal year 2005 to $24.5 million in fiscal year
2009. From fiscal year 2005 through fiscal year 2009, intramural PTSD
research funding ranged from 2.5 percent to 4.8 percent of VA‘s
medical and prosthetic research appropriation. In addition, the number
of PTSD research studies VA funded through the Merit Review Program
and the Cooperative Studies Program (CSP)-”VA‘s two primary funding
mechanisms in its intramural research program-”increased from 47 in
fiscal year 2005 to 96 in fiscal year 2009.
According to VA officials, intramural research proposals, including
those on PTSD, are funded primarily according to scientific merit in
both the Merit Review Program and CSP. Proposals are evaluated by a
panel of reviewers and scored based on their scientific merit.
Directors of VA‘s research and development services”-offices that
focus on different research areas and administer VA‘s intramural
research program-”fund proposals based on their scores, typically up
to a specified percentile. The number of proposals funded may vary
based on budgetary considerations and, for a small number of
proposals, responsiveness to VA research priority areas.
VA has a process to review and incorporate relevant research outcomes
to develop CPGs for a number of topics, including PTSD. VA relies on
the policies of a joint VA and Department of Defense (DOD) work group”
comprised of VA and DOD officials”to ensure that systematic reviews of
relevant research outcomes are conducted when issuing CPGs. In brief,
a systematic review is conducted to identify the most methodologically
rigorous research studies that are applicable to each clinical
question contained in the CPG. A group of subject matter experts then
assesses the individual research studies in order to determine the
overall quality of evidence available for each particular clinical
question, considers the potential benefits and harms of a clinical
intervention to determine its net effect, and, based on an assessment
of the overall quality of the evidence and the net effect of an
intervention, develops recommendations for the CPG.
According to VA officials, the decision to require that two PTSD
services”cognitive processing therapy and prolonged exposure therapy”
be made available at VA facilities by including them in the Handbook
was based on a review of research outcomes and the availability of
existing resources. Specifically, VA officials told GAO that these two
services were strongly recommended in the 2004 PTSD CPG and had
greater evidence supporting their effectiveness than other PTSD
services. VA also told GAO that prior to the Handbook‘s 2008 issuance,
VA had already begun investing resources in training programs for
cognitive processing therapy in 2006 and prolonged exposure therapy in
2007. While VA provided some documentation regarding the decision-
making process for PTSD services, VA officials explained that clinical
decision-making processes are not typically expected to be documented
in a formal manner. VA officials told GAO that they are currently
clarifying language in the Handbook but do not plan to revise any
requirements relating to PTSD services at this time.
VA provided technical comments that GAO incorporated as appropriate.
View [hyperlink, http://www.gao.gov/products/GAO-11-32] or key
components. For more information, contact Randall B. Williamson at
(202) 512-7114 or williamsonr@gao.gov.
[End of section]
Contents:
Letter:
Background:
PTSD Research Funding Increased from Fiscal Year 2005 through Fiscal
Year 2009:
VA Intramural PTSD Research Studies Are Funded Primarily According to
Scientific Merit:
VA Has Established a Standardized Process to Review All Relevant PTSD
Research Outcomes and Incorporate Them into a PTSD CPG:
VA Reviewed Research Outcomes and Used Existing Resources in
Determining Which PTSD Services to Require Its Facilities to Provide
or Make Available to Veterans:
Agency Comments:
Appendix I: VA Research Centers and Programs That Conduct or Support
PTSD Research:
Appendix II: The Review Process for VA's Merit Review Program:
Appendix III: The Review Process for VA's Cooperative Studies Program:
Appendix IV: The Process for Developing VA/DOD Evidence-Based Clinical
Practice Guidelines:
Appendix V: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: The Office of Research and Development's Research and
Development Services:
Table 2: Department of Veterans Affairs (VA) Intramural Research Study
Funding Limits and Maximum Length of Award by Research and Development
Service:
Table 3: Department of Veterans Affairs (VA) Intramural Post-Traumatic
Stress Disorder (PTSD) Research Funding and VA's Medical and
Prosthetic Research Appropriation, Fiscal Years 2005 through 2009:
Table 4: Scientific Merit Scoring Criteria in the Department of
Veterans Affairs' (VA) Merit Review Program:
Table 5: Overview of Funding Recommendations Provided by the
Department of Veterans Affairs' (VA) Cooperative Studies Scientific
Merit Review Board:
Table 6: Department of Veterans Affairs (VA) Research Centers and
Programs That Conduct or Support Post-Traumatic Stress Disorder (PTSD)
Research:
Table 7: Scoring Guidelines for the Department of Veterans Affairs'
(VA) Merit Review Program:
Table 8: Categorization of Ratings Used by the Guideline Working Group
Established by the Department of Veterans Affairs (VA)/Department of
Defense (DOD) Evidence-Based Practice Work Group to Identify the Level
of Evidence of Research Outcomes:
Table 9: Categorization of Ratings Used by the Guideline Working Group
Established by the Department of Veterans Affairs (VA)/Department of
Defense (DOD) Evidence-Based Practice Work Group to Determine the
Overall Quality of Research:
Table 10: Categorization of Ratings Used by the Guideline Working
Group Established by the Department of Veterans Affairs
(VA)/Department of Defense (DOD) Evidence-Based Practice Work Group to
Determine the Net Effect of the Intervention as Described in the
Research:
Table 11: Grade of Evidence-Based Recommendation Assigned by the
Guideline Working Group Established by the Department of Veterans
Affairs (VA)/Department of Defense (DOD) Evidence-Based Practice Work
Group Based on the Overall Quality of Evidence and the Net Effect of
the Intervention as Described in the Research:
Figures:
Figure 1: Department of Veterans Affairs (VA) Intramural Funding for
Post-Traumatic Stress Disorder (PTSD) Research Studies and Other PTSD
Research-Related Funding, Fiscal Years 2005 through 2009:
Figure 2: Department of Veterans Affairs (VA) Intramural Post-
Traumatic Stress Disorder (PTSD) Funding for Merit Review Program
Studies and Cooperative Studies Program (CSP) Studies, Fiscal Years
2005 through 2009:
Figure 3: Number of Department of Veterans Affairs (VA) Intramural
Post-Traumatic Stress Disorder (PTSD) Research Studies Funded per
Fiscal Year, Fiscal Years 2005 through 2009:
Figure 4: Overview of Submission, Review, and Funding Process in
Department of Veterans Affairs' (VA) Merit Review Program:
Figure 5: Overview of Submission, Review, and Funding for a Department
of Veterans Affairs' (VA) Cooperative Studies Program (CSP) Proposal:
Figure 6: Summary of the Department of Veterans Affairs' (VA) Merit
Review Process for the Merit Review Program:
Figure 7: Summary of the Department of Veterans Affairs' (VA)
Cooperative Studies Program (CSP) Review Process:
Figure 8: Overview of the Process Used by the Guideline Working Group
Established by the Department of Veterans Affairs (VA)/Department of
Defense (DOD) Evidence-Based Practice Work Group to Grade Evidence-
Based Recommendations for Clinical Practice Guidelines:
Abbreviations:
CPG: clinical practice guideline:
CSP: Cooperative Studies Program:
DOD: Department of Defense:
Handbook: Uniform Mental Health Services in VA Medical Centers and
Clinics handbook:
ORD: Office of Research and Development:
PTSD: post-traumatic stress disorder:
VA: Department of Veterans Affairs:
January 24, 2011:
The Honorable Michael H. Michaud:
Ranking Member:
Subcommittee on Health:
Committee on Veterans' Affairs:
House of Representatives:
Dear Mr. Michaud:
In addition to providing health care to about 5 million veterans each
year, the Department of Veterans Affairs (VA) also funds research that
focuses on the specific health conditions that veterans may
experience. One condition that is examined in VA-funded research is
post-traumatic stress disorder (PTSD), an anxiety disorder that can
occur after a person is exposed to a life-threatening event.[Footnote
1] Veterans diagnosed with PTSD may experience problems sleeping,
maintaining relationships, and returning to their previous civilian
lives.[Footnote 2] According to VA, experts estimate that up to 20
percent of Operation Enduring Freedom and Operation Iraqi Freedom
veterans,[Footnote 3] up to 10 percent of Gulf War veterans, and up to
30 percent of Vietnam War veterans have experienced PTSD.[Footnote 4]
Consequently, demand for PTSD treatment continues to grow. VA data
show that from fiscal year 2004 through fiscal year 2008, the number
of unique veterans receiving treatment for PTSD increased by 60
percent from over 274,000 to over 442,000. In particular, the number
of Operation Iraqi Freedom and Operation Enduring Freedom veterans who
received VA treatment for PTSD increased from nearly 4,400 in fiscal
year 2004 to over 69,000 in fiscal year 2008.
Veterans diagnosed with PTSD can receive a range of treatments to
manage their symptoms, including individual and group therapy and
medication. Veterans can receive PTSD treatment on an outpatient basis
at VA facilities such as medical centers, community-based outpatient
clinics, and Vet Centers.[Footnote 5] Veterans can also receive
intensive treatment through VA medical center inpatient settings for
acute care needs and through residential rehabilitation treatment
programs for more prolonged rehabilitative care.
VA has guidance and policies in place related to PTSD services. For
example, in 2004, the joint VA/Department of Defense (DOD) Evidence-
Based Practice Work Group[Footnote 6] issued a CPG for PTSD.[Footnote
7] The PTSD CPG is an educational tool for clinicians that provides
evidence-based recommendations[Footnote 8] for PTSD services based on
a review of PTSD research outcomes. It does not require clinicians to
provide one service over another. However, in a handbook issued in
2008--Uniform Mental Health Services in VA Medical Centers and Clinics
(Handbook)--VA for the first time required certain mental health
services to be made available to veterans throughout the system.
[Footnote 9] For PTSD, the Handbook requires that cognitive processing
therapy or prolonged exposure therapy, two evidence-based
psychotherapies, be provided or made available at VA facilities.
[Footnote 10]
Through its intramural research program, VA funds studies on different
topics, such as PTSD. VA intramural studies are conducted by VA
investigators--that is, researchers who conduct VA research studies.
VA intramural research refers to research that is funded by and
conducted within VA. In fiscal year 2009, VA received an appropriation
of $510 million for medical and prosthetic research.[Footnote 11] This
appropriation funds VA's intramural research program, including
providing funding for necessary equipment and supplies.[Footnote 12]
VA also receives three medical care appropriations that support VA's
intramural research by paying some of the costs associated with this
research, for example, the salaries of VA investigators who are also
VA clinicians.[Footnote 13] According to VA, in fiscal year 2009, VA's
medical care appropriations totaled about $41 billion and VA provided
$433 million of that amount to support all research conducted at VA
facilities.[Footnote 14]
Because of the growing demand for PTSD services and the importance of
research in improving the health care services that veterans receive,
you asked us to report on VA's funding of PTSD research, VA's
processes for reviewing PTSD research proposals, VA's incorporation of
research outcomes into clinical practice, and VA's process for
determining which PTSD services it requires VA facilities to provide
or make available. This report will describe:
* how much funding VA provided for intramural PTSD research from its
medical and prosthetic research appropriations from fiscal year 2005
through fiscal year 2009,
* how VA determines which intramural PTSD research studies will be
funded,
* how VA reviews PTSD research outcomes and incorporates them into its
PTSD CPG, and:
* how VA determines which PTSD services it requires VA facilities to
provide or make available.
To describe how much funding VA provided for PTSD research from its
medical and prosthetic research appropriations through its intramural
research program from fiscal year 2005 through fiscal year 2009, we
obtained and summarized data provided by VA's Office of Research and
Development (ORD), the office that manages the Merit Review Program
and the Cooperative Studies Program (CSP)--the two primary funding
mechanisms within VA's intramural research program.[Footnote 15] VA
officials define PTSD research as "the effort to acquire generalizable
knowledge about causes, epidemiology, susceptibility and genetics,
resilience, pathophysiology, prevention and treatment of post-
traumatic stress disorder." Based on these data, for fiscal years 2005
through 2009, we calculated totals and trends for (1) overall
intramural PTSD research funding--including PTSD research studies and
other PTSD research-related funding, such as career development awards
provided to junior VA investigators to conduct PTSD studies, salaries
for VA investigators who are not VA clinicians,[Footnote 16] funding
for PTSD research conducted within ORD research centers, and PTSD
research meetings, and (2) the number and type of intramural PTSD
research studies funded. According to VA, the funding data we obtained
from VA do not include funds provided to conduct or support intramural
PTSD research from VA's medical care appropriations.[Footnote 17]
Because of this, they do not necessarily represent all the funding VA
provided for PTSD research during this time. In addition, to obtain
information about how VA funds research conducted at VA, we
interviewed officials from VA's Veterans Health Administration's
Office of Finance, ORD, and Veterans Integrated Service Networks.
[Footnote 18] We also reviewed VA documents regarding its total
medical and prosthetic research appropriations. We tested the internal
consistency and reliability of the PTSD research data by reviewing the
data for obvious outliers and performing consistency checks. We found
the data to be sufficiently reliable for the purposes of this report.
[Footnote 19]
To describe how VA determines which intramural PTSD research studies
will be funded, we interviewed VA officials responsible for managing
VA's intramural research program. We also obtained and analyzed VA
documents that describe VA's policies for submitting, reviewing, and
funding intramural research proposals. In addition, to understand the
process VA uses to evaluate intramural research proposals, we observed
a meeting in which research proposals were evaluated.
To describe how VA reviews PTSD research outcomes and incorporates
them into its PTSD CPG to inform clinical practice at VA, we
interviewed VA officials who are responsible for developing CPGs. We
also obtained and reviewed the PTSD CPG and documents related to CPG
development.
To describe how VA determines which PTSD services it requires VA
facilities to provide or make available to veterans, we interviewed VA
officials responsible for determining these requirements. We also
obtained and reviewed documents that describe the PTSD service
requirements.
We conducted this performance audit from May 2010 through November
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
VA manages its intramural research program through ORD. According to
ORD's 2009 to 2014 strategic plan, ORD has 10 research priority areas,
which are topics of research that are considered important to VA.
[Footnote 20] The research priority areas are the health care needs of
veterans who have served in Operation Enduring Freedom and Operation
Iraqi Freedom, aging-related conditions, mental health care and well-
being, chronic diseases, long-term care and caregiving, deployment-
related exposure to hazardous environmental agents, equity in care,
access in rural areas, women's health, and personalized medicine.
According to VA officials, all of these research priority areas could
include PTSD research.
VA funds intramural research through the following:
* VA's Merit Review Program: This program supports research studies
typically conducted by one VA investigator at one VA facility and is
administered by ORD's four research and development services, each of
which has a different research focus. (See table 1.) Each research and
development service is responsible for soliciting, reviewing,
selecting, and funding research proposals submitted to the service.
Table 1: The Office of Research and Development's Research and
Development Services:
Research and development service: Biomedical Laboratory;
Focus of research: Biological or physiological principles in humans or
animals;
Examples of research: Investigations of tissues, blood, or other
biologic specimens.
Research and development service: Clinical Science;
Focus of research: Human beings;
Examples of research: Clinical studies examining interventions and
effectiveness; epidemiological studies.
Research and development service: Health Services;
Focus of research: The interface of health care systems, patients, and
health care outcomes;
Examples of research: Studies examining quality, access, patient
outcomes, and costs.
Research and development service: Rehabilitation;
Focus of research: Improving the quality of life of impaired and
disabled veterans;
Examples of research: Veteran rehabilitation studies; studies focused
on identifying technical solutions for impaired and disabled veterans.
Source: GAO analysis of Department of Veterans Affairs information.
[End of table]
* VA's CSP: This program, which is administered by Clinical Science,
funds larger-scale, multi-site clinical trials and epidemiological
research studies on key diseases that impact veterans.
The Merit Review Program has research award funding limits, which are
set by VA. In some cases, intramural research awards may only be
funded for a certain number of years. See table 2 for more information.
Table 2: Department of Veterans Affairs (VA) Intramural Research Study
Funding Limits and Maximum Length of Award by Research and Development
Service:
Service: Biomedical Laboratory;
Funding limit per award per fiscal year: Pilot study[A]: N/A;
Funding limit per award per fiscal year: Full study: $150,000[B];
Maximum length of award (in fiscal years): Pilot study[A]: N/A;
Maximum length of award (in fiscal years): Full study: 4.
Service: Clinical Science;
Funding limit per award per fiscal year: Pilot study[A]: N/A;
Funding limit per award per fiscal year: Full study: $150,000[B];
Maximum length of award (in fiscal years): Pilot study[A]: N/A;
Maximum length of award (in fiscal years): Full study: 4 (5 for
clinical trials).
Service: Cooperative Studies Program;
Funding limit per award per fiscal year: Pilot study[A]: N/A;
Funding limit per award per fiscal year: Full study: No limit;
Maximum length of award (in fiscal years): Pilot study[A]: N/A;
Maximum length of award (in fiscal years): Full study: No limit.
Service: Health Services;
Funding limit per award per fiscal year: Pilot study[A]: $100,000;
Funding limit per award per fiscal year: Full study: $300,000[C];
Maximum length of award (in fiscal years): Pilot study[A]: 1;
Maximum length of award (in fiscal years): Full study: 4.
Service: Rehabilitation;
Funding limit per award per fiscal year: Pilot study[A]: $75,000;
Funding limit per award per fiscal year: Full study: $300,000;
Maximum length of award (in fiscal years): Pilot study[A]: 2;
Maximum length of award (in fiscal years): Full study: 4.
Source: GAO analysis of VA information.
[A] A pilot study is a study to establish feasibility or to develop
data, a technique, a concept, or a procedure, which is preliminary to
undertaking a full study. Two of VA's Office of Research and
Development's research and development services--Health Services and
Rehabilitation--accept pilot study proposals in addition to full study
proposals.
[B] According to VA officials, this limit may be exceeded for funding
the salaries of investigators who are not VA clinicians or for
equipment.
[C] According to VA officials, the total over all years may not exceed
$925,000.
[End of table]
In addition to individual studies conducted at VA facilities, VA has
several research centers and programs that conduct or support PTSD
research. For example, the National Center for PTSD focuses on PTSD
research. VA also has Research Enhancement Award Programs, which help
support PTSD research by providing staff and other resources to
investigators. (For more information on VA research centers and
research programs that conduct or support PTSD research, see appendix
I.) According to a VA official from the National Center for PTSD, VA
does not fund most of the PTSD research that is being conducted today.
Origin and Criteria for Intramural Research Proposals:
Intramural research proposals may be service-directed--solicited by
ORD on specific topics--or investigator-initiated--submitted by
investigators to ORD on their own initiative. Investigators submit
proposals either in response to a request for proposals on a specific
topic (for service-directed proposals) or to an open request for
proposals (for investigator-initiated proposals). For both the Merit
Review Program and CSP, proposals are typically evaluated in two
review cycles per year.[Footnote 21]
To be considered for intramural research funding:[Footnote 22]
* The proposal must be veteran-centric.
* The proposal must have received approval from the director of the
medical center and the research and development office of the medical
center where the lead investigator, known as a principal investigator,
is based.[Footnote 23]
* The principal investigator and any co-principal investigators must
demonstrate a primary professional commitment to VA, as demonstrated
by at least a 5/8 time VA appointment at the time the funding is
awarded and previous VA experience, including experience in research
and patient care.[Footnote 24]
* Research must be conducted primarily on VA premises. The principal
investigator and any co-principal investigators must have designated
research space within a VA medical center.[Footnote 25]
PTSD Research Funding Increased from Fiscal Year 2005 through Fiscal
Year 2009:
Overall intramural PTSD research funding from VA's medical and
prosthetic research appropriation increased from $9.9 million in
fiscal year 2005 to $24.5 million in fiscal year 2009. The number of
intramural PTSD research studies funded through the Merit Review
Program and CSP increased from 47 in fiscal year 2005 to 96 in fiscal
year 2009.
VA Intramural PTSD Research Funding Increased from $9.9 Million in
Fiscal Year 2005 to $24.5 Million in Fiscal Year 2009:
Based on the VA data we obtained and summarized, we found that overall
intramural PTSD research funding from VA's medical and prosthetic
research appropriation increased from about $9.9 million in fiscal
year 2005 to about $24.5 million in fiscal year 2009, or by about 150
percent (see figure 1). Overall intramural PTSD research funding
included funding for specific PTSD studies as well as for other PTSD
research-related funding, such as career development awards provided
to junior VA investigators to conduct PTSD studies, salaries for VA
investigators who are not VA clinicians,[Footnote 26] funding for PTSD
research conducted within ORD research centers, and PTSD research
meetings.
Figure 1: Department of Veterans Affairs (VA) Intramural Funding for
Post-Traumatic Stress Disorder (PTSD) Research Studies and Other PTSD
Research-Related Funding, Fiscal Years 2005 through 2009:
[Refer to PDF for image: stacked vertical bar graph]
Fiscal year: 2005;
Funding for PTSD research studies: $9.4 million;
Other PTSD-related research funding: $0.5 million;
Total: $9.9 million.
Fiscal year: 2006;
Funding for PTSD research studies: $12.7 million;
Other PTSD-related research funding: $0.6 million;
Total: $13.3 million.
Fiscal year: 2007;
Funding for PTSD research studies: $14.8 million;
Other PTSD-related research funding: $0.9 million;
Total: $15.7 million.
Fiscal year: 2008;
Funding for PTSD research studies: $19.8 million;
Other PTSD-related research funding: $1.4 million;
Total: $21.2 million.
Fiscal year: 2009;
Funding for PTSD research studies: $23.5 million;
Other PTSD-related research funding: $1.0 million;
Total: $24.5 million.
Source: GAO analysis of VA data.
[End of figure]
Of the $80.2 million provided for PTSD studies from fiscal year 2005
through fiscal year 2009, $51.3 million, or about 64 percent, was for
studies funded through the Merit Review Program. The remaining
approximately $28.9 million, or about 36 percent, was for CSP studies.
(See figure 2.)
Figure 2: Department of Veterans Affairs (VA) Intramural Post-
Traumatic Stress Disorder (PTSD) Funding for Merit Review Program
Studies and Cooperative Studies Program (CSP) Studies, Fiscal Years
2005 through 2009:
[Refer to PDF for image]
[Refer to PDF for image: stacked vertical bar graph]
Fiscal year: 2005;
Funding for Merit Review Program studies: $6.4 million;
Funding for CSP studies: $3.0 million;
Total: $9.4 million.
Fiscal year: 2006;
Funding for Merit Review Program studies: $8.3 million;
Funding for CSP studies: $4.4 million;
Total: $12.7 million.
Fiscal year: 2007;
Funding for Merit Review Program studies: $10.7 million;
Funding for CSP studies: $4.1 million;
Total: $14.8 million.
Fiscal year: 2008;
Funding for Merit Review Program studies: $12.2 million;
Funding for CSP studies: $7.5 million;
Total: $19.7 million.
Fiscal year: 2009;
Funding for Merit Review Program studies: $13.6 million;
Funding for CSP studies: $9.9 million;
Total: $23.5 million.
Source: GAO analysis of VA data.
[End of figure]
From fiscal year 2005 through fiscal year 2009, intramural PTSD
research funding ranged from 2.5 percent to 4.8 percent of VA's
medical and prosthetic research appropriation. (See table 3 for VA
intramural PTSD research funding and VA's medical and prosthetic
research appropriations from fiscal year 2005 through fiscal year
2009.) For comparison,[Footnote 27] according to a 2009 report
prepared by ORD staff for VA's National Research Advisory Council,
[Footnote 28] for fiscal year 2009, funding for intramural traumatic
brain injury research was about $14.6 million, 2.9 percent of the
medical and prosthetic research appropriation. Funding for spinal cord
injury research was $27.2 million, 5.3 percent of the medical and
prosthetic research appropriation. Funding for intramural
cardiovascular disease and stroke research was $53.1 million, 10.4
percent of the medical and prosthetic research appropriation.
Table 3: Department of Veterans Affairs (VA) Intramural Post-Traumatic
Stress Disorder (PTSD) Research Funding and VA's Medical and
Prosthetic Research Appropriation, Fiscal Years 2005 through 2009:
Fiscal year: 2005;
VA intramural PTSD research funding: $9.9 million;
VA's medical and prosthetic research appropriation: $390.2 million;
Percentage: 2.5%.
Fiscal year: 2006;
VA intramural PTSD research funding: $13.3 million;
VA's medical and prosthetic research appropriation: $412.0 million;
Percentage: 3.2%.
Fiscal year: 2007;
VA intramural PTSD research funding: $15.7 million;
VA's medical and prosthetic research appropriation: $446.5 million;
Percentage: 3.5%.
Fiscal year: 2008;
VA intramural PTSD research funding: $21.3 million;
VA's medical and prosthetic research appropriation: $480.0 million;
Percentage: 4.4%.
Fiscal year: 2009;
VA intramural PTSD research funding: $24.5 million;
VA's medical and prosthetic research appropriation: $510.0 million;
Percentage: 4.8%.
Total:
VA intramural PTSD research funding: $84.7 million;
VA's medical and prosthetic research appropriation: $2.239 billion;
Percentage: 3.8%.
Source: GAO analysis of VA data.
Note: VA intramural research is also supported by VA's medical care
appropriations, which are not represented in this table. For example,
the medical care appropriations pay for the salaries of VA
investigators who are VA clinicians (according to VA, about 70 percent
of VA investigators are VA clinicians), administrative costs such as
payroll and human resources, and logistical and infrastructure costs.
[End of table]
The Number of PTSD Studies Funded through VA's Intramural Research
Program Increased from 47 in Fiscal Year 2005 to 96 in Fiscal Year
2009:
Similarly, we found that the number of PTSD studies funded from VA's
medical and prosthetic research appropriations through VA's intramural
research program increased from fiscal year 2005 through fiscal year
2009. (See figure 3.) Specifically, in fiscal year 2005, 47 intramural
PTSD research studies were funded while in fiscal year 2009, 96
intramural PTSD research studies were funded.[Footnote 29] This
represented an increase of more than 100 percent. Of all the studies
funded each fiscal year, only a small number were CSP studies.
Figure 3: Number of Department of Veterans Affairs (VA) Intramural
Post-Traumatic Stress Disorder (PTSD) Research Studies Funded per
Fiscal Year, Fiscal Years 2005 through 2009:
[Refer to PDF for image: stacked vertical bar graph]
Fiscal year: 2005;
Cooperative Studies Program studies: 44;
Merit Review Program studies: 3;
Total: 47.
Fiscal year: 2006;
Cooperative Studies Program studies: 60;
Merit Review Program studies: 3;
Total: 63.
Fiscal year: 2007;
Cooperative Studies Program studies: 70;
Merit Review Program studies: 3;
Total: 73.
Fiscal year: 2008;
Cooperative Studies Program studies: 79;
Merit Review Program studies: 5;
Total: 84.
Fiscal year: 2009;
Cooperative Studies Program studies: 80;
Merit Review Program studies: 6;
Total: 86.
Source: GAO analysis of VA data.
[End of figure]
VA Intramural PTSD Research Studies Are Funded Primarily According to
Scientific Merit:
According to VA officials, intramural research proposals, including
those on PTSD, are reviewed and funded in VA's Merit Review Program
and VA's CSP primarily according to scientific merit.
VA's Merit Review Program:
Intramural research proposals submitted to VA's Merit Review Program
are reviewed through a series of steps prior to funding. See figure 4
for an overview of the submission, review, and funding process for
proposals submitted to the Merit Review Program. (For more detailed
information on this process, see appendix II.)
Figure 4: Overview of Submission, Review, and Funding Process in
Department of Veterans Affairs' (VA) Merit Review Program:
[Refer to PDF for image: illustration]
1) Principal investigator submits proposal electronically[A];
2) Proposal is assigned to a merit review panel based on topic;
3) Merit review panel scores proposal for scientific merit; proposals
are ranked based on scores;
4) Director funds highest-ranked proposals and may fund proposals that
respond to research priority areas.
Source: GAO analysis of VA documents and GAO interviews with VA
officials.
[A] Proposals are submitted electronically to grants.gov, the
government's central grant identification and proposal portal.
Proposals are then transferred to eRA Commons, an electronic system
for grant administration functions, for VA processing and review.
[End of figure]
First, investigators submit proposals electronically. Investigators
typically submit Merit Review Program proposals to grants.gov, the
government's central grant identification and proposal portal,
[Footnote 30] in response to a request for proposals. Submitted
proposals are then transferred to eRA Commons, an electronic system
for grant administration functions, for VA processing and review.
[Footnote 31]
Second, each proposal is assigned to a merit review panel for
evaluation. Each merit review panel reviews proposals in a specific
research topic area, and is composed of panelists, typically associate-
level professors, who are selected based on their expertise in this
area.[Footnote 32] According to VA documents, as of 2010, there were a
total of 35 merit review panels across VA's research and development
services.[Footnote 33] The merit review panels evaluate each proposal
based on its scientific merit.[Footnote 34] Panelists consider several
criteria in evaluating the overall scientific merit of a proposal.
(See table 4 for criteria used to determine scientific merit.)
Table 4: Scientific Merit Scoring Criteria in the Department of
Veterans Affairs' (VA) Merit Review Program:
Criteria: Significance;
Description: Supports/advances the health and health care of veterans
and research field in general; addresses important scientific
question/area; makes a potential contribution to scientific literature.
Criteria: Approach;
Description: Incorporates current scientific/theoretical bases;
is hypothesis-driven; uses appropriate research design/methods for
addressing hypothesis; ensures that feasibility of methods is clear.
Criteria: Innovation;
Description: Addresses new concepts, gaps, or both, in the research
area; addresses potential for impact of findings on existing field of
research, treatment paradigms, or both.
Criteria: Environment;
Description: Has appropriate knowledge/background and resources (e.g.,
equipment and staff) to ensure completion of study.
Criteria: Feasibility;
Description: Provides sufficient evidence to determine that the
proposed study can be successfully conducted and completed.
Criteria: Investigator;
Description: Ensures that investigators involved in the proposed study
are appropriately trained and have expertise in the proposed area of
research.
Source: GAO analysis of VA information.
[End of table]
Third, the merit review panelists score the proposals to determine
their rank. Each panelist provides a score to each of the proposals
reviewed by the panel. The scores are averaged to create a "priority
score" for the proposal.[Footnote 35] (See appendix II for specific
scoring guidelines given to panelists in all research and development
services.) All proposals scored by the merit review panel are then
ranked by priority score among all of the proposal scores recently
assigned by the merit review panel. The rank of the proposal is used
to determine the "percentile" of the proposal.[Footnote 36]
Finally, research and development service directors determine how many
proposals to fund. All of the proposals scored by all merit review
panels in a research and development service in the review cycle are
ranked together by their percentiles to be considered for funding.
According to VA officials, research and development service directors
typically fund up to the 25th percentile of proposals in a review
cycle, beginning with those with the most scientific merit, although
the number of proposals funded may vary depending on the budget.
According to VA, research and development service directors may also
choose to fund a small number of additional proposals at the margin
that respond to research priority areas. For example, if the fundable
range determined by a research and development service director was up
to the 25th percentile, proposals at the 26th percentile related to
research priority areas could also be considered for funding.[Footnote
37]
VA's Cooperative Studies Program:
VA intramural research proposals submitted to CSP are reviewed and
scored in a process similar to that of the Merit Review Program prior
to consideration for funding. To help develop the CSP proposal,
investigators are assisted by members of a CSP center,[Footnote 38] a
VA entity that provides guidance and support for research across
multiple sites. (See figure 5 for an overview of the process for
submitting, reviewing, and funding a CSP proposal. For more
information on ORD's CSP review process, see appendix III.)
Figure 5: Overview of Submission, Review, and Funding for a Department
of Veterans Affairs' (VA) Cooperative Studies Program (CSP) Proposal:
[Refer to PDF for image: illustration]
1) CSP center submits proposal to CSP central office[A];
2) CSP Merit Review Board scores proposals and recommends funding for
those with the most scientific merit;
3) Proposals recommended for funding are provided to the Clinical
Science Director;
4) Clinical Science Director funds proposals with the most scientific
merit.
Source: GAO analysis of VA documents and GAO interviews with VA
officials.
[A] Prior to submitting the proposal to CSP central office, the
principal investigator must have a letter of intent approved by the
Director of Clinical Science.
[End of figure]
Before submitting a research proposal, investigators submit a letter
of intent, or a preliminary outline of a proposal, to the Director of
Clinical Science to be approved for planning a CSP proposal.[Footnote
39] Based on the merit of the letter of intent, as determined by three
or more reviewers,[Footnote 40] the Clinical Science Director decides
whether to fund planning efforts to develop a CSP proposal.[Footnote
41]
When the principal investigator receives approval to begin planning
efforts, the Clinical Science Director assigns a CSP center to provide
statistical and methodological guidance to the investigator. The
director of the CSP center designates a project manager and
methodologist, such as a person with expertise in biostatistics, to
provide guidance to the principal investigator. The Clinical Science
Director, with recommendations from the principal investigator, then
forms a planning committee of additional experts to assist in
developing a CSP proposal. The planning committee develops a CSP
proposal over the course of two planning meetings.
Once a proposal is developed, the CSP center, on behalf of the
principal investigator, submits a hard copy proposal to the CSP
central office for evaluation by the Cooperative Studies Scientific
Merit Review Board.[Footnote 42] This board consists of reviewers who
have extensive experience in clinical research and the conduct of
clinical trials or epidemiology studies.[Footnote 43] Reviewers
evaluate CSP proposals based on scientific merit. According to VA, the
scientific merit of a CSP proposal is defined by the importance of the
proposal, its feasibility, the clarity and achievability of its
objectives, the adequacy of the plan of investigation, the correctness
of the technical details, and the adequacy of the safeguards for the
welfare of the patients.[Footnote 44] Based on these criteria,
reviewers discuss the general scientific merit of the proposal.
Reviewers vote on whether to unconditionally approve, conditionally
approve, reject or defer with recommendation for resubmittal, or
reject each proposal. (See table 5 for an overview of funding
recommendations provided by the board.)
Table 5: Overview of Funding Recommendations Provided by the
Department of Veterans Affairs' (VA) Cooperative Studies Scientific
Merit Review Board:
Recommendation for funding: Unconditional approval;
Definition: The proposal is approved without changes or additions to
the proposal.
Recommendation for funding: Conditional approval;
Definition: The proposal is approved pending certain changes or
additions to the proposal.
Recommendation for funding: Rejection or defer with recommendation for
resubmittal;
Definition: The proposal is considered to be worthwhile, but is in
need of major revisions. A revised proposal can be resubmitted to the
Clinical Science Director.
Recommendation for funding: Rejection;
Definition: The proposal is rejected. The principal investigator must
submit a new request for planning to submit a revised proposal.
Source: GAO analysis of a VA document.
[End of table]
After the reviewers vote, they each provide scores for a proposal
recommended for funding based on scientific merit.[Footnote 45] The
scores are then averaged to provide a priority score for a proposal.
Finally, the Clinical Science Director considers the priority scores
of all the proposals in that review cycle and selects the proposals
with the strongest priority scores for funding. According to VA
officials, the number of proposals funded may vary depending on the
budget.[Footnote 46]
VA Has Established a Standardized Process to Review All Relevant PTSD
Research Outcomes and Incorporate Them into a PTSD CPG:
The VA/DOD Evidence-Based Practice Work Group, which is responsible
for developing and updating all of VA's CPGs, has a standardized and
reproducible process to review all relevant research outcomes when
developing or updating all CPGs, including the PTSD CPG. To develop or
update a CPG, the VA/DOD Evidence-Based Practice Work Group identifies
and assigns a group of VA and DOD clinical leaders and experts who are
knowledgeable in the subject area to work on the CPG.[Footnote 47]
Generally, the process to develop or update a CPG consists of the
following steps.
* First, the assigned group of VA and DOD clinical leaders and experts
identifies "clinical questions" that will be answered in the CPG.
According to VA officials, clinical questions can be either broad or
specific. For example, the 2004 PTSD CPG contained clinical questions
regarding whether early intervention is more effective than later
intervention, and whether certain interventions, such as different
psychotherapies, are more effective than others.
* Second, in order to minimize bias, an external contractor conducts a
systematic review of relevant research and selects and summarizes the
most methodologically rigorous research studies that are applicable to
each of the clinical questions.
* Third, after receiving summaries of the studies with the highest
level of evidence, the VA and DOD group of clinical leaders and
experts rates the research using an established grading scheme that
considers:
- the level of evidence of each research study--the scope and
methodological rigor of an individual study;
- the overall quality of evidence--the overall quality of all of the
research that addresses a particular clinical question, considering
the level of evidence of all the studies considered; and:
- the net effect of an intervention--according to the collective
results of the studies considered, the intervention's benefits minus
the intervention's harms.
* Finally, the assigned group of VA and DOD clinical leaders and
experts assigns a grade to each evidence-based recommendation based on
an assessment of the overall quality of evidence and the net effect of
the intervention. (See appendix IV for a detailed description of the
process used to develop evidence-based VA/DOD CPGs.)
The process for conducting a systematic review of research outcomes to
develop or update a CPG is repeated as often as is deemed necessary by
the VA/DOD Evidence-Based Practice Work Group according to its written
procedures and designated time frames.[Footnote 48] According to
VA/DOD Evidence-Based Practice Work Group documents, routine updates
to the CPGs should ideally occur approximately every 2 years. However,
updates to CPGs often do not occur every 2 years, and VA officials
told us that some CPGs are updated more frequently than others based
on availability of resources and priority areas. Additionally, VA
officials reported that a CPG will be immediately updated if any
evidence-based recommendation contained in it is identified as harmful
to patients.
According to VA, the VA/DOD Evidence-Based Practice Work Group
approved an update to the 2004 PTSD CPG on October 25, 2010, and
published the update on VA's Web site on November 17, 2010.[Footnote
49] According to VA officials, the systematic process outlined above
was used to review all relevant research outcomes and make evidence-
based recommendations for PTSD services to both develop and update the
PTSD CPG.
VA Reviewed Research Outcomes and Used Existing Resources in
Determining Which PTSD Services to Require Its Facilities to Provide
or Make Available to Veterans:
According to VA officials, the decision to require that cognitive
processing therapy and prolonged exposure therapy be made available to
veterans diagnosed with PTSD at VA facilities--as indicated in the
Handbook, which established certain requirements for mental health
services within VA--was based on a review of research outcomes and the
availability of existing resources.
* Review of research outcomes. According to VA, agency officials and
qualified subject matter experts reviewed relevant research outcomes
and the quality of the research to determine the most efficacious PTSD
treatments available when determining which PTSD services to include
in the Handbook and make available to veterans. Specifically, VA
officials told us that their decision to include cognitive processing
therapy and prolonged exposure therapy in the Handbook was influenced
by the fact that both of these had been graded as level "A" treatments
in the 2004 PTSD CPG (indicating that the intervention is always
indicated and acceptable).[Footnote 50] Furthermore, VA officials said
that these two therapies had greater evidence supporting their
effectiveness than other PTSD services also graded as level "A" in the
2004 PTSD CPG.[Footnote 51] In addition, VA officials added that their
decision was validated by the results of a VA-commissioned Institute
of Medicine study published in 2008 that reviewed the evidence for
existing PTSD treatments. According to VA, the study found that
cognitive processing therapy and prolonged exposure therapy were
considered efficacious treatments for PTSD.[Footnote 52] While the
Institute of Medicine report was released after VA had already decided
to include cognitive processing therapy and prolonged exposure therapy
in the Handbook, VA officials explained that the Institute of Medicine
report was the basis for the decision not to include other PTSD
services in the Handbook.
* Availability of existing resources. VA officials told us that prior
to issuing the Handbook in 2008, VA had already begun investing
considerable resources to implement national training programs for
cognitive processing therapy and prolonged exposure therapy in 2006
and 2007, respectively. VA officials said that they decided to
implement the national training programs because VA realized the need
to create sufficient capacity so that evidence-based PTSD treatments
could be available to veterans throughout the VA system. VA explained
that the national training programs were rolled out in advance of the
Handbook's issuance as part of the implementation of VA's
Comprehensive Veterans Health Administration Strategic Plan for Mental
Health Services, which called for rapid implementation of evidence-
based treatments. VA did this to ensure that it had the capacity to
provide cognitive processing therapy and prolonged exposure therapy to
all veterans with PTSD for whom these treatments were clinically
appropriate.[Footnote 53] VA officials said that they were able to
begin implementing national training programs for cognitive processing
therapy in 2006 and prolonged exposure therapy in 2007 because VA had
qualified instructors to administer the programs and money available
to fund them.[Footnote 54]
Unlike the written and standardized process that the VA/DOD Evidence-
Based Practice Work Group established to develop CPGs, VA does not
have a formal written process or framework to explain its decision for
including cognitive processing therapy and prolonged exposure therapy
in the Handbook.[Footnote 55] VA officials explained that they
followed a process when choosing cognitive processing therapy and
prolonged exposure therapy, but added that clinical decision-making
processes are not typically expected to be documented in a formal
manner.
VA officials told us that they plan to assess the implementation of
the Handbook and will update PTSD requirements in it as needed or as
new information or unexpected obstacles arise in the future.[Footnote
56] VA officials stated that they are currently clarifying the
language regarding some of the requirements, but do not plan to revise
any of the requirements relating to PTSD services at this time.
Agency Comments:
We provided a draft of this report to VA and received technical
comments, which we incorporated into our report as appropriate.
We are sending a copy of this report to the Secretary of Veterans
Affairs. The report also is available at no charge on the GAO Web site
at [hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-7114 or williamsonr@gao.gov. Contact points
for our Offices of Congressional Relations and Public Affairs may be
found on the last page of this report. GAO staff members who made key
contributions to this report are listed in appendix V.
Sincerely yours,
Signed by:
Randall B. Williamson:
Director, Health Care:
Enclosures - V:
[End of section]
Appendix I: VA Research Centers and Programs That Conduct or Support
PTSD Research:
In addition to post-traumatic stress disorder (PTSD) research studies
that are conducted by individual Department of Veterans Affairs (VA)
investigators, or researchers, VA also funds a number of research
centers or programs that conduct or support PTSD research. See table 6
for a description of these VA research centers and programs.
Table 6: Department of Veterans Affairs (VA) Research Centers and
Programs That Conduct or Support Post-Traumatic Stress Disorder (PTSD)
Research:
Office of Mental Health Services:
VA research centers/programs: The National Center for PTSD;
Description: Advances the clinical care and social welfare of veterans
through research, education, and training in the science, diagnosis,
and treatment of PTSD and stress-related disorders;
Established by Congress? Yes;
Number of centers or programs with a primary focus on PTSD: 1[A].
VA research centers/programs: Centers of Excellence;
Description: Research the causes and treatments of mental disorders
and use the dissemination of education to implement new knowledge into
routine VA clinical practices;
Established by Congress? Yes;
Number of centers or programs with a primary focus on PTSD: 2.
VA research centers/programs: Mental Illness Research, Education, and
Clinical Centers;
Description: Research the causes and treatments of mental disorders
and use education to implement knowledge into routine clinical
practice in VA;
Established by Congress? Yes;
Number of centers or programs with a primary focus on PTSD: 4.
Office of Research and Development:
VA research centers/programs: Centers of Excellence;
Description: Create or maintain a core program of investigators to
support and facilitate the development of research studies and train
and mentor new investigators;
Established by Congress? No;
Number of centers or programs with a primary focus on PTSD: 8.
VA research centers/programs: Quality Enhancement Research Initiative
Program;
Description: Enhances the quality and outcomes of VA health care by
implementing research findings and evidence-based recommendations into
routine clinical practice;
Established by Congress? No;
Number of centers or programs with a primary focus on PTSD: 1[B].
VA research centers/programs: Research Enhancement Award Programs;
Description: Support groups of VA investigators that address specific
medical problems of veterans to promote innovative research, train new
medical research investigators, and foster new research collaborations
among investigators;
Established by Congress? No;
Number of centers or programs with a primary focus on PTSD: 1.
Source: GAO analysis of VA information.
[A] The National Center for PTSD, headquartered in White River
Junction, Vermont, has seven divisions in five locations across the
country.
[B] Although the Mental Health Quality Enhancement Research Initiative
(one of nine Quality Enhancement Research Initiative Centers) focuses
on depression and schizophrenia, Office of Research and Development
officials stated that within the past few years it has become more
involved with PTSD research.
[End of table]
[End of section]
Appendix II: The Review Process for VA's Merit Review Program:
Research proposals submitted to the Department of Veterans Affairs'
(VA) Merit Review Program are evaluated in merit review panels that
each review proposals in a specific research topic area. Each merit
review panel is comprised of panelists, typically associate-level
professors, who are selected based on their expertise in the area.
Panelists are responsible for scoring proposals based on scientific
merit to provide funding recommendations. See figure 6 for a detailed
description of the Merit Review Program's process for reviewing
research proposals and table 7 for the Merit Review Program's scoring
guidelines.
Figure 6: Summary of the Department of Veterans Affairs' (VA) Merit
Review Process for the Merit Review Program:
[Refer to PDF for image: illustration]
1. The portfolio manager, an official responsible for managing a
specific area of research, assigns three panelists as the primary,
secondary, and tertiary panelists for each research proposal. These
panelists are responsible for reviewing the proposal prior to the
panel meeting, and individually submitting a preliminary score and
summary of the proposal into eRA Commons, an electronic system for
grant administration functions, for VA processing and review. The
preliminary scores range from 1.0 and 5.0 (with scores closer to 1.0
being more meritorious).
2. The primary, secondary, and tertiary panelists present their
preliminary scores to the entire panel[A] and respond to feedback
provided by the other panelists about the proposal.
3. Based on the discussion at the panel meeting, the primary,
secondary, and tertiary panelists can change their preliminary scores.
The primary, secondary, and tertiary panelists must announce to the
panel the final score each plans to give the proposal. The final
scores also range from 1.0 and 5.0 (with scores closer to 1.0 being
more meritorious).
4. The portfolio manager asks if any panelist plans to score the
research proposal 0.3 points above or below the range of the final
scores provided by the primary, secondary, and tertiary panelists. If
any panelist intends to score outside of this range, the panelist must
discuss this decision with the rest of the panel and explain reasons
for doing so.
5. All panelists on the review panel then submit a final score for the
research proposal. The average of the final scores multiplied by 100
is known as the priority score. In addition, the primary panelist
writes a summary of the discussion of the panel meeting and submits
the summary into eRA Commons.
Source: GAO analysis of VA documents and GAO interviews with VA
officials.
[A] According to VA's policy, panelists who have a conflict of
interest with a given proposal are required to leave the meeting
before the proposal is discussed and do not score the proposal.
According to VA, a conflict of interest exists when a reviewer has an
interest in a research proposal that is likely to bias his or her
evaluation of it. Panelists are expected to inform the portfolio
manager if they have a conflict of interest.
[B] In some instances, research proposals are streamlined, meaning
that they are evaluated only by the primary, secondary, and tertiary
panelists, and are not discussed by the entire merit review panel.
Proposals may be streamlined if the average of the primary, secondary,
and tertiary panelists' preliminary scores falls in the upper half of
the range of scores, indicating less merit. According to VA officials,
proposals with a preliminary score of lower than 2.2, indicating very
good or excellent, are also streamlined for two research and
development services. (See table 7 for merit review process scoring
guidelines.) In these instances, the administrator of the merit review
panel meeting asks the panelists if anyone has an objection to
streamlining the proposal. If a panelist objects, the proposal is
discussed in the merit review panel meeting and it is not streamlined.
Streamlined proposals with preliminary scores that fall in the upper
half of the range of scores, indicating less merit, are not considered
for funding.
[End of figure]
Table 7: Scoring Guidelines for the Department of Veterans Affairs'
(VA) Merit Review Program:
Scoring range and grade: 1.0 - 1.5 Excellent;
Description: Proposed research addresses important scientific area
that lacks needed knowledge. Hypothesis(es) is clearly stated and
research design/methodology is appropriate. Research is innovative,
representing state-of-the-art science. Potential findings may have a
vital role in advancing the health and health care of veterans and
research in general. Resources listed suggest a very high probability
of the study's completion.
Scoring range and grade: 1.6 - 2.2 Very good;
Description: Proposed research addresses important scientific area.
Hypothesis(es) is clearly stated and the research design/methodology
is appropriate with a few minor exceptions. Potential findings may
have an important role to the health and health care of veterans and
research in general. Resources listed suggest a high probability of
the study's completion.
Scoring range and grade: 2.3 - 2.8 Good;
Description: Proposed research addresses a valid area of
investigation. Hypothesis(es) is clearly stated, but research
design/methodology contains key flaws that should be corrected.
Potential findings may contribute to the health and health care of
veterans and research in general. Resources listed suggest that the
study could be completed.
Scoring range and grade: 2.9 - 3.4 Fair;
Description: Proposed research requires further preliminary data to
warrant investigation as a viable area of research. Hypothesis(es) is
not clear, research design/methods contain significant flaws, or both.
It is not clear how potential findings would contribute to the health
and health care of veterans and research in general. It is unclear
whether the resources listed are sufficient to ensure study completion.
Scoring range and grade: 3.5 - 5.0 Poor;
Description: Proposed research does not appear to address an important
scientific question/area. Hypothesis(es) is not clearly stated,
research design/methodology is inappropriate or contains uncorrectable
flaws, or both. Design/methodological limitations hinder any
significant conclusions that would contribute to the health and health
care of veterans, research in general, or both. Resources listed do
not suggest that the study will be completed.
Source: GAO analysis of VA documents.
[End of table]
[End of section]
Appendix III: The Review Process for VA's Cooperative Studies Program:
Research proposals submitted to the Department of Veterans Affairs'
(VA) Cooperative Studies Program (CSP) are reviewed and scored by the
Cooperative Studies Scientific Merit Review Board. Reviewers on the
board are chosen based on their expertise in clinical or
epidemiological research. They typically serve 4-year terms, and ad
hoc members can be added depending on specific expertise that may be
needed to review a proposal. According to VA, as of October 2010,
there were six reviewers on the board.
During the research proposal review process, the study team--which
includes the lead researcher (referred to as the principal
investigator) and a methodologist, such as a person with expertise in
biostatistics--has an interactive discussion with the board regarding
the proposal.[Footnote 57] Reviewers evaluate CSP proposals based on
scientific merit and provide scores to reflect their funding
recommendations. See figure 7 for a detailed description of the review
process for CSP research proposals.
Figure 7: Summary of the Department of Veterans Affairs' (VA)
Cooperative Studies Program (CSP) Review Process:
[Refer to PDF for image: illustration]
1. The Deputy Director of CSP assigns three individuals on the
Cooperative Studies Scientific Merit Review Board to be the initial
reviewers for a CSP proposal. One of the initial reviewers is
designated as the primary reviewer. All three initial reviewers
prepare written reviews of the proposal based on scientific merit
prior to the board meeting.[A]
2. A first board meeting is held to summarize and discuss the key
critiques of the written reviews provided by the initial reviewers.
The primary reviewer is responsible for introducing the proposal to
the board. The study team, which includes the principal investigator
and a methodologist, such as a person with expertise in biostatistics,
is not present at this meeting.
3. A second board meeting is held with the study team.[B] At the
beginning of this meeting, the principal investigator is provided a
summary of the critiques discussed in the first board meeting. The
study team is then given 15 minutes to provide a summary of the
proposal, including its importance to VA, and to address the
critiques.[B]
4. After the study team responds to the board, the board and the study
team engage in an interactive discussion regarding the proposal.
5. After the interactive discussion, the study team is dismissed and
the board votes on whether to unconditionally approve, conditionally
approve, reject or defer with recommendation for resubmittal, or
reject funding of the proposal.[D] For proposals that are
conditionally and unconditionally approved, each member of the board
assigns a numeric rating from 10 to 50, with lower scores indicating
more meritorious proposals, based on scientific merit. The average of
the scores for each proposal is considered the priority score.[E]
Source: GAO analysis of VA documents and GAO interviews with VA
officials.
[A] According to a VA document, the primary reviewers are typically a
biostatistician and a clinician. Reviewers are asked to comment on the
importance of the proposal, its feasibility, the clarity and
achievability of its objectives, the adequacy of the plan of
investigation, the correctness of the technical details, the adequacy
of the safeguards for the welfare of the patients, and any other
pertinent features of the proposal. According to VA, these criteria
are components of scientific merit.
[B] For CSP proposals that involve economic analysis, a health
economist is also present at the board meeting to discuss the
proposal. In addition, the principal investigator may ask the Clinical
Science Director to allow consultants to be present at the board
meeting if review of the proposal requires expertise in a specific
area.
[C] According to VA, if there are any economic issues raised relating
to the proposal, the study team will be given an additional 5 minutes
to address these issues with a health economist.
[D] According to a VA document, unconditional approval means a
proposal is approved by the board without changes and is recommended
for funding. Conditional approval means that the proposal is approved
by the board pending certain changes or additions to the proposal.
Reject or defer with recommendation for resubmittal means the board
considers the proposal to be worthwhile, but it is need of major
revisions. In this case, if the principal investigator chooses to
submit a revised proposal, the Clinical Science Director may waive the
requirement to submit a letter of intent. If the board rejects the
proposal, the principal investigator must submit a new letter of
intent to resubmit a revised proposal.
[E] After the board scores the proposal, a human rights committee
typically provides a general assessment of the protection of patients'
rights and welfare as described in the proposal. For studies with
unique ethical considerations, this review may occur prior to the
review by the Cooperative Studies Scientific Merit Review Board.
[End of figure]
[End of section]
Appendix IV: The Process for Developing VA/DOD Evidence-Based Clinical
Practice Guidelines:
In 1999, the Department of Veterans Affairs (VA) and the Department of
Defense (DOD) formed the VA/DOD Evidence-Based Practice Work Group to
issue joint VA/DOD clinical practice guidelines (CPG)--tools that
provide guidance and evidence-based recommendations to clinicians
regarding the most effective interventions and services for a variety
of health care topics.[Footnote 58] To develop or update a CPG, the
VA/DOD Evidence-Based Practice Work Group has a standardized process
to ensure that systematic reviews of relevant research outcomes are
conducted in order to formulate evidence-based recommendations for
prevention, assessment, and treatment services.
To develop or update a CPG, the VA/DOD Evidence-Based Practice Work
Group identifies two clinical leaders--one from VA and one from DOD--
who then help identify not more than 15 to 20 other experts in the
subject area to form a "guideline working group."[Footnote 59] A
member of the VA/DOD Evidence-Based Practice Work Group is also
selected to be an evidence chaperone for each CPG to ensure that
conformity to prevailing standards for conducting high-quality
systematic reviews is upheld.[Footnote 60]
To determine the scope of the CPG, the guideline working group, the
evidence chaperone, and a facilitator are responsible for identifying
clinical questions that are to be answered by a systematic review of
relevant research outcomes.[Footnote 61] According to VA officials,
clinical questions can be both broad and specific. For example, the
2004 post-traumatic stress disorder CPG contained clinical questions
regarding whether early intervention is more effective than later
intervention and whether certain interventions, such as different
psychotherapies, are more effective than others.[Footnote 62]
According to VA, in order to answer these clinical questions, an
external evidence center--an entity that conducts systematic reviews
of research on a variety of topics--is contracted to collect and
review all relevant research (including, but not limited to, VA-and
DOD-sponsored research) to assess its applicability to each clinical
question under consideration using explicit and reproducible methods.
[Footnote 63] The evidence center then focuses its review on the best
available research, that is, high-quality, methodologically rigorous
studies that address health issues that impact VA and DOD populations
and consider the effectiveness as well as the harms and benefits of
the intervention at issue. According to VA officials, the evidence
center provides summaries of only the best available research to the
guideline working group for review.[Footnote 64]
After receiving the summaries, the guideline working group reviews the
research in sequential steps using an established rating scheme
developed by the U.S. Preventive Services Task Force to formulate
evidence-based recommendations.[Footnote 65] See figure 8 for an
overview of the steps that the guideline working group uses to
formulate evidence-based recommendations.
Figure 8: Overview of the Process Used by the Guideline Working Group
Established by the Department of Veterans Affairs (VA)/Department of
Defense (DOD) Evidence-Based Practice Work Group to Grade Evidence-
Based Recommendations for Clinical Practice Guidelines:
[Refer to PDF for image: illustration]
1) Relevant research is assessed by its level of evidence.
2) Overall quality of evidence that addresses a particular clinical
question is assessed.
3) The net effect of the intervention is assessed (benefits minus
harms).
4) Evidence-based recommendations are graded (based on overall quality
of evidence + net effect of the intervention).
Source: GAO analysis of VA/DOD documents and GAO interviews with VA
officials.
[End of figure]
Level of evidence. First, the guideline working group reviews the
summaries to identify the level of evidence, or the level of
methodological rigor. For example, research studies that have the
highest quality are categorized as "I" (indicating at least one
properly done randomized controlled trial), while research studies of
the lowest quality are categorized as "III" (indicating that the
research reflects the opinion of respected authorities, descriptive
studies, case reports, and expert committees). (See table 8.)
Table 8: Categorization of Ratings Used by the Guideline Working Group
Established by the Department of Veterans Affairs (VA)/Department of
Defense (DOD) Evidence-Based Practice Work Group to Identify the Level
of Evidence of Research Outcomes:
Rating: I;
Description: At least one properly done randomized controlled trial.
Rating: II-1;
Description: Well-designed controlled trial without randomization.
Rating: II-2;
Description: Well-designed cohort or case-control analytic study,
preferably from more than one source.
Rating: II-3;
Description: Multiple time series evidence with/without intervention,
dramatic results of uncontrolled experiment.
Rating: III;
Description: Opinion of respected authorities, descriptive studies,
case reports, and expert committees.
Source: VA/DOD document.
[End of table]
Overall quality of research. After determining the level of evidence
of individual research studies, the guideline working group makes a
determination regarding the overall quality of all of the research
that addresses a particular clinical question. The overall quality
takes into account the number, quality, and size of all of the
individual research studies together as well as the consistency of the
results between research outcomes to determine the collective overall
strength of the research. Based on this review, the guideline working
group determines the overall quality of the evidence to be good, fair,
or poor.[Footnote 66] (See table 9.)
Table 9: Categorization of Ratings Used by the Guideline Working Group
Established by the Department of Veterans Affairs (VA)/Department of
Defense (DOD) Evidence-Based Practice Work Group to Determine the
Overall Quality of Research:
Rating: Good;
Description: High-grade evidence (I or II-1) linked to health outcome.
Rating: Fair;
Description: High-grade evidence (I-or II-1) linked to intermediate
outcome; Or: Moderate-grade evidence (II-2 or II-3) directly linked to
health outcome.
Rating: Poor;
Description: Level III evidence or no linkage of evidence to health
outcome.
Source: VA/DOD document.
[End of table]
Net effect of the intervention. For interventions that were supported
by studies of "fair" or "good" overall quality, the guideline working
group evaluates the benefits and the potential harms to determine the
net effect of the intervention.[Footnote 67] The net effect of an
intervention takes into account the benefits of the intervention minus
the harms to determine the overall potential clinical benefit that the
intervention may provide to patients. The net effect of the
intervention ranges from "substantial" (meaning the benefit
substantially outweighs the harm) to "zero or negative" (meaning it
has no impact or a negative impact on patients). (See table 10.)
Table 10: Categorization of Ratings Used by the Guideline Working
Group Established by the Department of Veterans Affairs
(VA)/Department of Defense (DOD) Evidence-Based Practice Work Group to
Determine the Net Effect of the Intervention as Described in the
Research:
Rating: Substantial;
Description: More than a small relative impact on a frequent condition
with a substantial burden of suffering; Or: A large impact on an
infrequent condition with a significant impact on the individual
patient level.
Rating: Moderate;
Description: A small relative impact on a frequent condition with a
substantial burden of suffering; Or: A moderate impact on an
infrequent condition with a significant impact on the individual
patient level.
Rating: Small;
Description: A negligible relative impact on a frequent condition with
a substantial burden of suffering; Or: A small impact on an infrequent
condition with a significant impact on the individual patient level.
Rating: Zero or negative;
Description: Negative impact on patients; Or: No relative impact on
either a frequent condition with a substantial burden of suffering or
an infrequent condition with a significant impact on the individual
patient level.
Source: VA/DOD document.
[End of table]
Grade of evidence-based recommendation. In the final step, the
guideline working group uses its assessment of the overall quality of
the evidence and the net effect of the intervention to grade evidence-
based recommendations. (See table 11.)
Table 11: Grade of Evidence-Based Recommendation Assigned by the
Guideline Working Group Established by the Department of Veterans
Affairs (VA)/Department of Defense (DOD) Evidence-Based Practice Work
Group Based on the Overall Quality of Evidence and the Net Effect of
the Intervention as Described in the Research:
Overall quality of evidence: Good;
The net benefit of the intervention:
Substantial: A;
Moderate: B;
Small: C;
Zero or negative: D.
Overall quality of evidence: Fair;
The net benefit of the intervention:
Substantial: B;
Moderate: B;
Small: C;
Zero or negative: D.
Overall quality of evidence: Poor;
The net benefit of the intervention:
Substantial: I;
Moderate: I;
Small: I;
Zero or negative: I.
Grade of recommendation: A;
Description of recommendation: A strong recommendation that the
intervention is always indicated and acceptable.
Grade of recommendation: B;
Description of recommendation: A recommendation that the intervention
may be useful/effective.
Grade of recommendation: C;
Description of recommendation: A recommendation that the intervention
may be considered.
Grade of recommendation: D;
Description of recommendation: A recommendation that a procedure may
be considered not useful/effective or may be harmful.
Grade of recommendation: I;
Description of recommendation: Insufficient evidence to recommend for
or against--the clinician will use clinical judgment.
Source: VA/DOD document.
[End of table]
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Randall B. Williamson, (202) 512-7114 or williamsonr@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Mary Ann Curran, Assistant
Director; Susannah Bloch; Stella Chiang; Martha R. W. Kelly; Melanie
Krause; Lisa Motley; Michelle Paluga; Rebecca Rust; and Suzanne Worth
made key contributions to this report.
[End of section]
Related GAO Products:
VA Health Care: Progress and Challenges in Conducting the National
Vietnam Veterans Longitudinal Study. [hyperlink,
http://www.gao.gov/products/GAO-10-658T]. Washington, D.C.: May 5,
2010.
VA Health Care: Status of VA's Approach in Conducting the National
Vietnam Veterans Longitudinal Study. [hyperlink,
http://www.gao.gov/products/GAO-10-578R]. Washington, D.C.: May 5,
2010.
VA Health Care: Preliminary Findings on VA's Provision of Health Care
Services to Women Veterans. [hyperlink,
http://www.gao.gov/products/GAO-09-899T]. Washington, D.C.: July 16,
2009.
DOD and VA Health Care: Challenges Encountered by Injured
Servicemembers during Their Recovery Process. [hyperlink,
http://www.gao.gov/products/GAO-07-589T]. Washington, D.C.: March 5,
2007.
VA Health Care: Spending for Mental Health Strategic Plan Initiatives
Was Substantially Less Than Planned. [hyperlink,
http://www.gao.gov/products/GAO-07-66]. Washington, D.C.: November 21,
2006.
VA Health Care: Preliminary Information on Resources Allocated for
Mental Health Strategic Plan Initiatives. [hyperlink,
http://www.gao.gov/products/GAO-06-1119T]. Washington, D.C.: September
28, 2006.
VA Health Care: VA Should Expedite the Implementation of
Recommendations Needed to Improve Post-Traumatic Stress Disorder
Services. [hyperlink, http://www.gao.gov/products/GAO-05-287].
Washington, D.C.: February 14, 2005.
VA and Defense Health Care: More Information Needed to Determine If VA
Can Meet an Increase in Demand for Post-Traumatic Stress Disorder
Services. [hyperlink, http://www.gao.gov/products/GAO-04-1069].
Washington, D.C.: September 20, 2004.
[End of section]
Footnotes:
[1] American Psychiatric Association, Diagnostic and Statistical
Manual of Mental Disorders, Fourth Edition, Text Revision (Washington,
D.C., 2000).
[2] Those diagnosed with PTSD may also suffer from other ailments,
such as depression and substance abuse.
[3] Operation Enduring Freedom, which began in October 2001, supports
combat operations in Afghanistan and other locations, and Operation
Iraqi Freedom, which began in March 2003, supports combat operations
in Iraq and other locations. In September 2010, Operation Iraqi
Freedom became known as Operation New Dawn.
[4] Estimates for veterans who have experienced PTSD vary. For
example, according to VA, regarding Operation Enduring Freedom and
Operation Iraqi Freedom veterans, one 2008 RAND study found that
approximately 14 percent of these veterans have experienced PTSD,
while another 2004 study by Hoge et al. estimated that from 6 to 20
percent of those veterans have experienced PTSD.
[5] Vet Centers offer readjustment counseling--a wide range of
psychosocial services that includes individual and group counseling
and screening and referrals for medical issues--to eligible veterans
and their families.
[6] Formed in 1999 and composed of VA and DOD officials, the VA/DOD
Evidence-Based Practice Work Group makes decisions about which
clinical practice guidelines (CPG) for specific conditions will be
developed and oversees their development. As of August 2010, the
VA/DOD Evidence-Based Practice Work Group had issued 24 CPGs on
conditions such as chronic heart failure and major depressive
disorder. Since VA and DOD issue the CPGs in a joint effort, they are
intended to be used by both VA and DOD clinicians. The VA/DOD Evidence-
Based Practice Work Group is supported by VA's Office of Quality and
Performance and the U.S. Army Medical Department's Office of Quality
Management.
[7] The PTSD CPG is formally known as the VA/DOD Clinical Practice
Guideline for the Management of Post-Traumatic Stress (January 2004).
[8] Evidence-based care refers to approaches that have consistently
been shown in controlled research to be effective for a particular
condition or conditions.
[9] Veterans Health Administration Handbook 1160.01, Uniform Mental
Health Services in VA Medical Centers and Clinics (Sept. 11, 2008).
According to the Handbook, all veterans with PTSD must have access to
cognitive processing therapy or prolonged exposure therapy. VA medical
centers and very large community-based outpatient clinics (serving
more than 10,000 unique veterans each year) must be able to provide
staff to administer such services at the facilities. Large (serving
5,000 to 10,000 unique veterans each year) and midsized (serving 1,500
to 5,000 unique veterans each year) community-based outpatient clinics
may provide these services through telemental health or contract care
when necessary. In March 2010, VA issued the Veterans Health
Administration Handbook 1160.03, Programs for Veterans with Post-
Traumatic Stress Disorder (PTSD), to provide additional information
regarding the implementation of the PTSD requirements contained in the
2008 Handbook.
[10] Psychotherapies focus on changing individuals' behaviors,
thoughts, perceptions, and emotions. Cognitive processing therapy
utilizes trauma-specific cognitive techniques to help patients move
past trauma-related thoughts and progress toward recovery. Prolonged
exposure therapy works by helping individuals approach trauma-related
thoughts, feelings, and situations that they have been avoiding
because of the distress they cause. Repeated exposure to these
thoughts, feelings, and situations can assist with reducing the
likelihood they will cause distress.
[11] VA's medical and prosthetic research appropriation pays for
necessary expenses in carrying out programs of medical and prosthetic
research and development as authorized by chapter 73 of title 38,
United States Code, the authorizing laws for the Veterans Health
Administration. See Pub. L. No. 110-329, 122 Stat. 3574, 3706 (2008).
[12] The medical and prosthetic research appropriation funds research
on different topics. It is not limited to funding research on PTSD.
[13] VA's three medical care appropriations are (1) medical services,
which provides funds for the provision of veterans' health care
services; (2) medical support and compliance, which provides funds for
expenses related to the administration of veterans' health care
services; and (3) medical facilities, which provides funds for the
operation and maintenance of VA's health care facilities. Each of
these appropriations provides support to all research conducted at VA
facilities. For example, the medical care appropriations pay for the
salaries of VA investigators who are VA clinicians (according to VA,
about 70 percent of VA investigators are VA clinicians),
administrative costs such as those for payroll and human resources,
and logistical and infrastructure costs.
[14] VA could not provide an estimate for the amount of funding VA
spent from the medical care appropriations for intramural PTSD
research. Of the $433 million, more than half went to personnel costs,
including the salaries of clinicians and other VA facility staff.
[15] The data provided by ORD reflect the funding amounts VA
investigators were authorized to use, not necessarily the amounts that
were spent. VA officials said that the authorized amounts are very
similar to the amounts that are spent.
[16] Salaries for VA investigators who are VA clinicians are paid from
the medical services appropriation, one of the medical care
appropriations.
[17] For example, VA's Veterans Integrated Service Networks, which
receive funding from the medical care appropriations and include VA's
medical facilities, may also fund research by VA investigators, but
this research funding is not considered part of VA's intramural
research program. According to ORD officials, about 95 percent of the
PTSD research funded by VA is funded by VA's intramural research
program.
[18] The management of VA's medical facilities is decentralized to 21
regional networks referred to as Veterans Integrated Service Networks.
[19] In 2010, the VA Office of Inspector General found issues with
ORD's validation of some research expenditure data reported by
investigators at the VA Maryland Health Care System facility. See
Department of Veterans Affairs, Office of Inspector General, Health
Care Inspection: Inappropriate Research and Development Data Entries
Affecting Veterans Equitable Research Allocation (VERA) Funding VA
Maryland Health Care System Baltimore, MD (Washington, D.C., Sept. 23,
2010). According to VA officials, the PTSD research funding data we
obtained for this report were generated from a different data system
than the data systems specifically examined by the VA Office of
Inspector General. In addition, the PTSD research funding data are
entered by VA officials at the central office and checked for accuracy
by VA officials at both the central office and the facilities.
[20] In 2009, VA's Office of Inspector General found that appropriated
funds for VA research addressed the broad spectrum of medical issues
with which veterans contend. See Department of Veterans Affairs,
Office of Inspector General, Healthcare Inspection: Review of the
Veterans Health Administration's Use of Appropriated Funds for
Research (Washington, D.C., 2009).
[21] According to VA, proposals may be reviewed outside of a review
cycle depending on when ORD solicits requests for proposals. In rare
cases, service-directed proposals are reviewed by ad hoc reviewers or
solely by research and development service directors.
[22] Each research and development service may have additional
requirements.
[23] The approval indicates that the medical center has agreed to
commit the resources, such as space and staff, necessary to conduct
the research study.
[24] Investigators who do not demonstrate a primary professional
commitment to VA can request a waiver of this requirement. According
to a VA document, a waiver may be granted by VA officials depending on
the circumstances of the request, such as for a retired investigator
who previously received funding through the Merit Review Program.
According to VA, waivers are granted for about 35 investigators per
year.
[25] The principal investigator and any co-principal investigators may
also have a designated research space in an approved non-VA facility
or other VA facility, such as a community-based outpatient clinic.
[26] Salaries for VA investigators who are VA clinicians are paid from
the medical services appropriation, one of the medical care
appropriations.
[27] Because of potential overlap in research areas, percentage
estimates should not be added.
[28] The National Research Advisory Council is responsible for
advising the Secretary of Veterans Affairs and the Under Secretary for
Health on matters related to VA's research. The Office of Management
and Budget requires that the National Research Advisory Council
conduct an annual assessment of VA's intramural research program.
[29] VA intramural research studies may be funded for multiple years
(see table 2). Therefore, the same intramural PTSD research study
funded in one fiscal year may also be funded the following fiscal
year. In our analysis, this study would be counted as a funded study
each fiscal year. In addition, some studies were not funded each
fiscal year of their duration. For example, a study that was conducted
from fiscal year 2006 through fiscal year 2009 may not have received
funding in fiscal year 2007.
[30] Depending on the specific requirements of the request for
proposal, a letter of intent (a preliminary outline of a research
proposal) may be required prior to the submission of a research
proposal.
[31] Electronic research administration, or eRA Commons, is managed
and supported by the National Institutes of Health to provide support
for the funding of research proposals.
[32] Merit review panelists may or may not be employed by VA.
According to VA officials, most merit review panelists have previous
experience reviewing proposals. According to a VA document, the number
of panelists on a merit review panel may vary. For example, the number
of panelists on Rehabilitation's merit review panels ranges from 16 to
59.
[33] Biomedical Laboratory and Clinical Science share the same merit
review panels.
[34] In addition, for Health Services, along with scientific merit,
reviewers take into account research priority areas of Health Services
when scoring proposals.
[35] For Health Services, prior to assigning a priority score for a
proposal, merit review panelists vote on whether to approve,
conditionally approve, or reject the proposal. Conditional approval
means the proposal is approved pending additions or changes. Only
approved and conditionally approved proposals are assigned a priority
score.
[36] According to VA, the percentile is calculated by the proposal's
rank, determined by its priority score in comparison to the scores of
the current and previous three review cycles, minus 0.5, divided by
the total number of applications being considered. For example, if a
proposal received the fifth best priority score and there were a total
of 60 proposals reviewed over the four cycles, the percentile for that
application would be 5 minus 0.5 divided by 60, equaling .075.
[37] After a proposal is approved for funding by the research and
development service director, the proposal must undergo review by an
institutional review board, an independent committee that reviews
research based on ethical considerations. In addition, depending on
the topic of the proposal, it may be reviewed by additional committees
at this time, such as a data monitoring committee.
[38] There are nine CSP centers: VA operates five CSP Coordinating
Centers to support multi-site clinical trials and four Epidemiological
Research and Information Centers to support epidemiology studies. In
addition to receiving support from a designated CSP center, CSP
proposals that involve drugs or medical devices also receive
assistance from the CSP Clinical Research Pharmacy Coordinating
Center, and CSP proposals that involve economic analysis receive
support from VA's Health Economics Resource Center.
[39] A letter of intent typically contains an outline of the proposed
research study, including justification for multiple sites where the
research will be conducted and a list of experts who can assist in the
initial planning efforts of the research study.
[40] Reviewers of letters of intent are selected based on their
expertise in areas related to the topic of the letter of intent.
[41] Planning efforts include travel funds for personnel involved in
developing the proposal. According to VA, there are two planning
meetings that involve approximately 15 people.
[42] CSP proposals are currently not submitted electronically;
however, ORD officials estimate that electronic submission of CSP
proposals through eRA Commons will begin in fall 2011.
[43] According to a VA document, members of the Cooperative Studies
Scientific Merit Review Board serve 4-year terms, and ad hoc members
can be added depending on specific expertise that may be needed to
review a proposal. As of October 2010, there were six members of the
Cooperative Studies Scientific Merit Review Board.
[44] CSP proposals are also reviewed by a human rights committee at
the designated CSP center to determine if the protection of patients'
rights and welfare is adequate. CSP proposals must be approved by a
human rights committee prior to being funded.
[45] Scores are given only to conditionally and unconditionally
approved proposals.
[46] After a CSP proposal has received approval for funding, the
proposal must undergo additional steps before study subjects are
enrolled. For example, the proposal must be reviewed by an
institutional review board, and an executive committee meets to review
the operational and monitoring aspects of the study.
[47] In addition, representatives from VA's Office of Patient Care
Services are responsible for identifying VA and DOD clinical leaders.
[48] According to VA, the development or update of a CPG typically
takes 18 months.
[49] According to VA, the update of the PTSD CPG mainly strengthens
the evidence that was already contained in the 2004 PTSD CPG.
According to a VA official, the updated PTSD CPG will include a module
on complementary alternative medicine, which was not included in the
2004 PTSD CPG.
[50] In the CPG, cognitive processing therapy is graded a level "A"
treatment specifically for female sexual assault-related PTSD.
However, VA officials told us that they also consider cognitive
processing therapy to be effective for PTSD in veterans. See appendix
IV for a detailed description of the process used to determine grades
for evidence-based recommendations.
[51] Two other psychotherapies, stress inoculation training and eye
movement desensitization and reprocessing, were also graded as level
"A" treatments for PTSD in the 2004 PTSD CPG, but were not included in
the Handbook.
[52] Institute of Medicine, Treatment of Posttraumatic Stress
Disorder: An Assessment of the Evidence (Washington, D.C., 2008). The
Institute of Medicine is an independent, nonprofit organization that
works outside of government to provide authoritative advice to
decision makers and the public. The Institute of Medicine found that
exposure therapies, including prolonged exposure therapy and elements
of cognitive processing therapy, are efficacious treatments for PTSD.
[53] VA officials reported that incorporating cognitive processing
therapy and prolonged exposure therapy into the Handbook helped ensure
that clinicians who had been trained in these therapies prior to
development of the Handbook would be able to offer and provide them in
VA facilities as needed. In May 2010, VA's Office of Inspector General
reported that as of November 17, 2009, the total number of mental
health practitioners who had completed either the cognitive processing
therapy or the prolonged exposure therapy training was 3,086. See
Department of Veterans Affairs, Office of Inspector General,
Healthcare Inspection: Progress in Implementing the Veterans Health
Administration's Uniform Mental Health Services Handbook (Washington,
D.C., 2010).
[54] To help implement VA's Comprehensive Veterans Health
Administration Strategic Plan for Mental Health Services (November
2004), VA allocated additional resources to fund mental health
strategic plan initiatives in fiscal years 2005 and 2006. According to
VA officials, a part of these additional resources was used to pay for
the national cognitive processing therapy and prolonged exposure
therapy training programs until the costs of the programs could be
integrated into VA's congressional budget submission.
[55] However, VA officials provided documentation regarding
consultation with experts and a list of VA entities that participated
in reviewing the Handbook.
[56] According to VA officials, VA policies must be recertified every
5 years to remain in effect. The Handbook must be recertified on or
before September 2013.
[57] The study team may also include a health economist, in cases
where a CSP proposal involves economic analysis. In addition, the
principal investigator may ask the Clinical Science Director to allow
other consultants to be a part of the study team, as necessary.
[58] Evidence-based care refers to approaches that have consistently
been shown in controlled research to be effective for a particular
condition or conditions. The VA/DOD Evidence-Based Practice Work Group
makes decisions about which clinical practice guidelines for specific
conditions will be developed and oversees their development. As of
August 2010, the VA/DOD Evidence-Based Practice Work Group had issued
24 CPGs on conditions such as chronic heart failure and major
depressive disorder. Since VA and DOD issue the CPGs in a joint
effort, they are intended to be used by both VA and DOD clinicians.
The VA/DOD Evidence-Based Practice Work Group is supported by VA's
Office of Quality and Performance and the U.S. Army Medical
Department's Office of Quality Management.
[59] In addition, representatives from VA's Office of Patient Care
Services are responsible for identifying VA and DOD clinical leaders.
[60] VA and DOD designees from the VA/DOD Evidence-Based Practice Work
Group, with assistance from VA's Office of Patient Care Services, are
responsible for selecting the VA and DOD clinical leaders and the
evidence chaperone. In addition, they also select another member of
the VA/DOD Evidence-Based Practice Work Group to monitor the
development of the CPG.
[61] The facilitator is also responsible for keeping the guideline
working group focused during meetings and to ensure that it focuses on
the quality of research outcomes when formulating evidence-based
recommendations in the CPG.
[62] The post-traumatic stress disorder CPG is formally known as The
VA/DOD Clinical Practice Guideline for the Management of Post-
Traumatic Stress (January 2004).
[63] According to a VA official, an external evidence center is used
to minimize bias.
[64] The evidence center provides summaries of research of lower
quality only if no higher-quality research is available.
[65] The U.S. Preventive Services Task Force was first convened by the
Department of Health and Human Services' Public Health Service in
1984, and since 1998 has been sponsored by the Department of Health
and Human Services' Agency for Healthcare Research and Quality. The
U.S. Preventive Services Task Force uses explicit criteria to grade
the scientific evidence for a broad range of clinical services and
develop evidence-based recommendations for clinicians and health
systems. While the U.S. Preventive Services Task Force updated its
criteria in May 2007, the VA/DOD Evidence-Based Practice Work Group
currently uses U.S. Preventive Services Task Force criteria developed
prior to May 2007.
[66] The overall quality of the research is considered "good" when the
research outcomes are consistent from a number of higher-level
research studies and have been conducted across a broad range of
populations, thereby providing a high degree of confidence that the
research outcomes are true. A "fair" overall quality indicates that
the research outcomes could have been a result of true effects or
biases present across some or all of the research studies. A "poor"
rating for overall quality indicates that any conclusion about the
research outcomes is uncertain because of serious methodological
shortcomings, sparse data, or inconsistent results.
[67] The net effect of the intervention of research outcomes that are
found to be of poor overall quality is not assessed because the
evidence is insufficient to make a recommendation for or against
routinely providing the intervention.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
