Veterans Disability Benefits
Clearer Information for Veterans and Additional Performance Measures Could Improve Appeal Process Gao ID: GAO-11-812 September 29, 2011In Process
According to VA data, which has only tracked DRO involvement since 2003, veterans chose a DRO review in 61 percent (534,439) of all appeals filed from 2003 to 2010. Veterans who sought assistance with their appeal from a veteran service organization or other qualified representatives were more likely to choose a DRO review than those without a representative. Without assistance, veterans may not fully understand their two appeal options. GAO found that the letter VA uses to inform veterans of their options does not highlight key deadlines or differences between the two options. According to more than half of surveyed regional office managers, most veterans could not make an informed choice on the options based just on the letter. The DRO review process has helped some veterans get additional benefits at the regional office level, but has not reduced the percentage of appeals continuing on to the Board--the primary purpose of the program. In fiscal years 2003 through 2008, 21 percent of DRO reviews resulted in a full grant of benefits compared to 17 percent of traditional reviews. A full grant of benefits ends, or resolves, the appeal at the regional level. However, appeals may also be resolved at the regional level if veterans who do not receive full grants decide not to continue their appeal to the Board. VA gave DROs the flexibility to interact informally with veterans in part so they could explain when the benefits already granted are appropriate given the law. However, while DRO reviews led to the grant of full benefits at a higher rate, a higher percentage of veterans not granted benefits through traditional review voluntarily ended their appeals. As a result, in fiscal years 2003 through 2008 the overall percentage of appeals resolved at the regional level was about the same for DRO and traditional reviews--about 70 percent for both. VA faces challenges in how to most effectively use and train DROs. Since the DRO process and position were established, DRO duties have expanded beyond reviewing appeals to performing additional tasks such as quality review. However, VA officials have not reached consensus on how to balance DROs' time among different tasks. VA has no performance goal or measure for appeal resolution at the regional level that could help it determine whether it is achieving the most effective balance between different tasks. In addition, VA headquarters offers no nationwide, standardized training for new DROs, which according to managers and DROs would be beneficial, as they often lack experience with other tasks that DROs frequently perform such as conducting hearings. Ninetythree percent of surveyed regional managers said a nationally standardized training for new DROs would be beneficial. GAO recommends VA (1) revise its appeals election letter, (2) develop an appeal resolution goal at the regional level, and (3) develop a training curriculum on DRO duties. In its comments, VA concurred fully with GAO's first and third recommendations but only partially with the second. VA expressed concerns about an appeal resolution goal, including that it could encourage the unjustified granting of benefits. GAO feels that VA's quality control process minimizes this risk.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Daniel Bertoni Team: Government Accountability Office: Education, Workforce, and Income Security Phone: (202) 512-5988GAO-11-812, Veterans Disability Benefits: Clearer Information for Veterans and Additional Performance Measures Could Improve Appeal Process
This is the accessible text file for GAO report number GAO-11-812
entitled 'Veterans Disability Benefits: Clearer Information for
Veterans and Additional Performance Measures Could Improve Appeal
Process' which was released on September 30, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Requesters:
September 2011:
Veterans Disability Benefits:
Clearer Information for Veterans and Additional Performance Measures
Could Improve Appeal Process:
GAO-11-812:
GAO Highlights:
Highlights of GAO-11-812, a report to congressional requesters.
Why GAO Did This Study:
The Department of Veterans Affairs (VA) has struggled to provide
timely reviews for veterans who appeal decisions on their disability
compensation claims. A veteran appeals to the VA regional office that
made the initial decision, and if still dissatisfied, to the Board of
Veterans Appeals (Board). An appeal to the Board adds more than 2
years, on average, to the wait for a decision on the appeal. To
resolve more appeals at the regional level and avoid waits at the
Board, VA, in 2001, established the Decision Review Officer (DRO)
review as an alternative to the traditional regional office appeal
review. A DRO is given authority to grant additional benefits after
reviewing an appeal based on a difference of opinion with the original
decision. In contrast, under the traditional review, new evidence is
generally required for a grant of additional benefits. GAO examined
(1) the extent to which veterans choose a DRO review, (2) outcomes for
DRO reviews, and (3) VA‘s challenges in managing DROs. GAO analyzed
Board data, surveyed managers in all 57 regional offices, visited 4
offices, and interviewed veterans.
What GAO Found:
According to VA data, which has only tracked DRO involvement since
2003, veterans chose a DRO review in 61 percent (534,439) of all
appeals filed from 2003 to 2010. Veterans who sought assistance with
their appeal from a veteran service organization or other qualified
representatives were more likely to choose a DRO review than those
without a representative. Without assistance, veterans may not fully
understand their two appeal options. GAO found that the letter VA uses
to inform veterans of their options does not highlight key deadlines
or differences between the two options. According to more than half of
surveyed regional office managers, most veterans could not make an
informed choice on the options based just on the letter.
The DRO review process has helped some veterans get additional
benefits at the regional office level, but has not reduced the
percentage of appeals continuing on to the Board-”the primary purpose
of the program. In fiscal years 2003 through 2008, 21 percent of DRO
reviews resulted in a full grant of benefits compared to 17 percent of
traditional reviews. A full grant of benefits ends, or resolves, the
appeal at the regional level. However, appeals may also be resolved at
the regional level if veterans who do not receive full grants decide
not to continue their appeal to the Board. VA gave DROs the
flexibility to interact informally with veterans in part so they could
explain when the benefits already granted are appropriate given the
law. However, while DRO reviews led to the grant of full benefits at a
higher rate, a higher percentage of veterans not granted benefits
through traditional review voluntarily ended their appeals. As a
result, in fiscal years 2003 through 2008 the overall percentage of
appeals resolved at the regional level was about the same for DRO and
traditional reviews”-about 70 percent for both.
VA faces challenges in how to most effectively use and train DROs.
Since the DRO process and position were established, DRO duties have
expanded beyond reviewing appeals to performing additional tasks such
as quality review. However, VA officials have not reached consensus on
how to balance DROs‘ time among different tasks. VA has no performance
goal or measure for appeal resolution at the regional level that could
help it determine whether it is achieving the most effective balance
between different tasks. In addition, VA headquarters offers no
nationwide, standardized training for new DROs, which according to
managers and DROs would be beneficial, as they often lack experience
with other tasks that DROs frequently perform such as conducting
hearings. Ninety-three percent of surveyed regional managers said a
nationally standardized training for new DROs would be beneficial.
What GAO Recommends:
GAO recommends VA (1) revise its appeals election letter, (2) develop
an appeal resolution goal at the regional level, and (3) develop a
training curriculum on DRO duties. In its comments, VA concurred fully
with GAO‘s first and third recommendations but only partially with the
second. VA expressed concerns about an appeal resolution goal,
including that it could encourage the unjustified granting of
benefits. GAO feels that VA‘s quality control process minimizes this
risk.
View [hyperlink, http://www.gao.gov/products/GAO-11-812]. For more
information, contact Daniel Bertoni at (202) 512-7215 or
bertonid@gao.gov.
[End of section]
Contents:
Letter:
Background:
Veterans More Often Choose the DRO Review, but Some May Not Fully
Understand Their Options:
The DRO Review More Often Results in Additional Benefits, but Does Not
Reduce Appeals Going to the Board:
VA Faces Challenges in Defining Role of and Training for DROs:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: VA Appeal Process Request Letter Template:
Appendix III: Comments from the Department of Veterans Affairs:
Appendix IV: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: Clear Federal Communication Criteria for VA Letter Analysis:
Table 2: Percentages of Appeals Using DRO and Traditional Reviews and
Related Odds and Odds Ratios (Unadjusted and Adjusted), by
Representation Status and Other Factors:
Table 3: Percentages of Appeals Granted Full or Partial Awards and No
Awards and Related Odds, and Odds Ratios (Unadjusted and Adjusted), by
Type of Review and Other Factors:
Table 4: Implementation of Structured Phone Interviews with Veterans:
Figures:
Figure 1: VA Disability Claim Appeal Process:
Figure 2: Number of DRO Reviews and Traditional Reviews, Fiscal Years
2003-2010:
Figure 3: Percentage of Regional Office Managers Who Said Veterans Can
Make Informed Decision Using VA Letter Alone:
Figure 4: Excerpt from VA Appeal Process Request Letter Template and
Examples of Clear Communication Analysis:
Figure 5: Percentage of Appeals Receiving Full or Partial Grants of
Benefits after Regional Office Review of the Notice of Disagreement,
Fiscal Years 2003-2010:
Figure 6: Average Days for Regional Office to Issue Decision on Notice
of Disagreement, Fiscals Years 2003-2009:
Figure 7: Percentage of Appeals Resolved at Regional Office Level,
Fiscal Years 2003-2008:
Figure 8: Regional Office Appeal Outcomes, Fiscal Years 2003-2008:
Figure 9: Percentage of Regional Office Managers Rating Specific
Activities as Very or Moderately Effective in Resolving Appeals:
Figure 10: Regional Office Managers' Estimates of Proportion of DROs'
Time Spent on Specific Tasks:
Figure 11: Percentage of Regional Managers Who Said Training on
Certain Topics Would Greatly or Moderately Enhance New DROs'
Performance:
Figure 12: Regional Offices Selected for Site Visits:
Abbreviations:
Board: Board of Veterans Appeals:
DRO: Decision Review Officer:
RVSR: Rating Veterans Service Representative:
VA: Department of Veterans Affairs:
VACOLS: Veterans Appeals Control and Locator System:
VBA: Veterans Benefits Administration:
VSO: veteran service organization:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 29, 2011:
The Honorable Patty Murray:
Chairman:
The Honorable Richard Burr:
Ranking Member:
The Honorable Daniel Akaka Member:
Committee on Veterans' Affairs:
United States Senate:
The Department of Veterans Affairs (VA) paid more than an estimated
$36 billion in disability compensation in fiscal year 2010 to over
three million veterans who incurred new or aggravated existing
disabilities during their military service. Veterans apply for
disability compensation benefits through one of VA's 57 regional
offices. Those who disagree with VA's initial decision on their claim
may appeal to the regional office that made the decision. If still
dissatisfied with the regional office's decision on their appeal, they
can continue their appeal to the Board of Veterans Appeals (Board) in
Washington, D.C.[Footnote 1] In fiscal year 2010, VA received more
than 140,000 appeals--a 45 percent increase over fiscal year 2007--and
the Board has struggled with an increasing backlog of cases.[Footnote
2] On average, veterans wait 8 months after filing an appeal to
receive a decision from the regional office, and those who continue to
the Board wait over two more years for a decision.
In an effort to resolve more appeals at the regional level, thus
shortening total wait times for veterans, and improve the quality of
appeals which go on to the Board, in 2001, VA established the Decision
Review Officer (DRO) review--an alternative review process at the
regional level. Prior to that time, regional offices reviewed initial
claim decisions and generally could only overturn them when they
received new evidence; otherwise appeals had to go to the Board for
consideration of a grant of additional benefits. Under the alternative
process, DROs (who are senior staff) have the authority to overturn an
initial disability claim decision without any new evidence based only
on difference of opinion. However, veterans still have the option of
choosing traditional review, which is VA's term for the process that
existed prior to 2001. As of July 2010, there were over 400 DROs
nationwide. At your request, this report examines aspects of the DRO
program. We evaluated (1) the extent to which veterans choose a DRO
review as opposed to a traditional review, (2) outcomes for veterans
who choose a DRO review, and (3) challenges VA faces in managing DROs.
To address these items, we reviewed relevant federal laws, VA
regulations, policy manuals, and regional office materials, as well as
previous VA studies, GAO reports, and other documents relevant to DROs
and the appeal process. We analyzed appeals management data from the
Veterans Appeals Control and Locator System (VACOLS), an electronic
Board database, for fiscal years 2003 through 2010. Although the DRO
process was established in 2001, VACOLS began tracking DRO involvement
in appeals in fiscal year 2003. VA extracted the VACOLS data for us on
April 7, 2011. We assessed the data and determined they were reliable
for the purposes of this study. We administered an online survey to
all 57 regional office managers, and conducted phone interviews with a
randomly selected but nongeneralizeable sample of 40 veterans who had
recently appealed their disability claim decision through either a DRO
or a traditional review.[Footnote 3] We conducted site visits to four
of VA's regional offices--Atlanta, Georgia; Providence, Rhode Island;
Waco, Texas; and Salt Lake City, Utah--and interviewed VA staff and
veterans service organizations (VSO) representatives who assist
veterans with their claims and appeals.[Footnote 4] We selected
offices based on several factors, including geographic location,
number of staff, timeliness of appeal processing for appeals in which
DRO was selected, and participation in a pilot study of new criteria
for assessing DRO performance.
We conducted this performance audit from July 2010 through September
2011 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives. Appendix I
contains a more detailed description of our scope and methodology.
Background:
Disability Claims Process:
To apply for disability compensation, a veteran submits a claim to
VA's Veterans Benefits Administration (VBA).[Footnote 5] A Veterans
Service Representative at one of VA's regional offices reviews the
claim and assists the veteran in gathering required evidence,
including military service records and medical treatment records from
VA facilities and private providers. A Veteran Service
Representative's responsibilities may include establishing claims
files, generating notification letters to veterans, assisting veterans
in obtaining the evidence needed to support their claims, and
assisting in processing appeals of claim decisions. A Rating Veterans
Service Representative (RVSR) then evaluates the evidence and, if the
RVSR finds the veteran to be eligible, determines the percentage
rating for purposes of compensation. Disability compensation varies
with the degree of disability and the number of a veteran's
dependents, and is paid monthly. Monthly base benefits in 2011 for an
individual range from $123 for 10 percent disability to $2,673 for 100
percent disability. A veteran may cite multiple medical issues in a
claim, for example, post-traumatic stress disorder, knee impairment,
and hearing loss. The RVSR may grant all, some, or none of the issues
in a claim. The veteran can obtain help by submitting a claim and
navigating the process with a VSO representative, private attorney, or
agent accredited by VA to assist veterans.[Footnote 6] In fiscal year
2010, VA received over one million disability compensation claims, a
46 percent increase from fiscal year 2007.
Appeal Process:
Regional office review of notice of disagreement (initial appeal
stage). If a veteran disagrees with the regional office's initial
decision on a claim, he or she appeals by submitting a written notice
of disagreement to the regional office.[Footnote 7] (See figure 1 for
a flowchart of appeals process at the VA.) A veteran may choose a
traditional review or a DRO review by responding to an appeal process
request letter sent by VA.[Footnote 8] VA sends this letter to a
veteran after receiving his or her notice of disagreement; the letter
informs the veteran of the DRO review and traditional review options,
requests the veteran choose one, and states how to obtain
representation if the veteran does not already have it. If the veteran
chooses the traditional review, the reviewer, who may be a RVSR or
DRO, examines the claim file and any new evidence that the veteran
submits and may hold a formal, transcribed hearing with the veteran.
The reviewer may overturn the original decision based only on (1) new
evidence or (2) a clear and unmistakable error made in the original
decision. However, if a veteran chooses the DRO review, a DRO conducts
a de novo review of the claim, meaning a new and complete review
without deference to the original decision, and can revise that
decision without new evidence or a clear and unmistakable error--in
other words, based on a difference of opinion.[Footnote 9] In addition
to formal hearings, DROs may hold informal conferences with the
veteran or the veteran's representative to discuss an appeal,
including why benefits already awarded are appropriate. Ultimately, in
either process, the reviewer may: (1) award a full grant, in which all
claimed benefits are awarded at the maximum level; (2) award a partial
grant, in which some benefits are granted but not necessarily at the
maximum level or for all claimed medical issues; or (3) confirm the
original decision on the claim, in which no further benefits are
granted on any issues.[Footnote 10] If a full grant is awarded, the
appeal ends. Otherwise, the regional office issues a written
explanation to the veteran in a statement of the case.
Regional office preparation of appeal for the Board. Regardless of
whether a DRO or a traditional review is chosen, if a veteran
disagrees with the regional office's appeal decision, he or she may
file what VA calls a substantive appeal to the Board, which is
processed initially by the regional office.[Footnote 11] The
substantive appeal is a document that the veteran completes to explain
the issues being appealed and why the veteran believes the VA decided
his or her case incorrectly. After receiving this document and
reviewing any additional supporting evidence provided by the veteran,
the regional office may award additional benefits. A veteran may
submit new evidence multiple times and each submission requires the
regional office to re-evaluate the claim. When the regional office
determines no further work is necessary, it certifies the appeal as
complete and transfers it to the Board.
Board review of appeal. When the Board receives the file, it may grant
or deny the claim. If the Board finds it cannot make a decision until
the regional office does additional work (e.g., requesting a more
recent medical exam) it sends, or remands, the case back to the
regional office or to the VBA Appeals Management Center in Washington,
D.C., which develops evidence and adjudicates the claim. A veteran
dissatisfied with the Board's decision can appeal, in succession, to
the U.S. Court of Appeals for Veterans Claims, to the Court of Appeals
for the Federal Circuit, and finally to the Supreme Court of the
United States.[Footnote 12]
Figure 1: VA Disability Claim Appeal Process:
[Refer to PDF for image: illustration]
Veteran chooses either a traditional review or a DRO review at
regional office:
Traditional review:
RVSR or DRO affirms original decision or grants additional benefits if
error or new evidence.
DRO review:
DRO conducts a completely new (de novo) review and can decide based on
difference of opinion in addition to error or new evidence.
Full grant?
Yes:
Appeal closed.
No:
Veteran receives a summary of the appeal decision (statement of the
case);
Veteran withdraws appeal or fails to respond: Appeal closed.
or:
Veteran files appeal to Board.
Regional office prepares appeal for Board:
* If new evidence, reviews and may make new decision;
* If no new evidence, sends appeal to Board;
Veteran receives full grant or withdraws appeal: Appeal closed.
or:
Board review of appeal:
Board remands appeal or grants or denies claim.
Source: GAO analysis of VBA process.
[End of figure]
Evolution of the DRO Position and Review Process:
VA created the DRO position and implemented the DRO review process
nationally in 2001 after a 1997-1998 pilot in 12 regional offices. In
the pilot, DROs were responsible for performing reviews of all appeals
and had the ability to make a new decision based solely on a
difference of opinion with the original decision. VA promulgated final
regulations when it implemented DRO review nationwide in 2001, which
made the process optional and required the veteran to expressly choose
it.[Footnote 13] Later in 2001, VA headquarters issued guidance to the
regional offices expanding the responsibilities of DROs to include
mentoring and training other disability claims staff and working with
regional office managers to identify error trends and training needs.
Veterans More Often Choose the DRO Review, but Some May Not Fully
Understand Their Options:
Since fiscal year 2003, when VA started tracking DRO involvement in
appeals, data show more veterans have chosen a DRO review than a
traditional review for appeals of decisions on their disability
compensation claims.[Footnote 14] From fiscal years 2003 through 2010,
veterans chose a DRO review in 534,439 appeals, or 61 percent of all
appeals filed over this time period, according to Board data.[Footnote
15] The percentage of appeals in which a DRO review was chosen
increased each year, from 54 percent of all appeals in fiscal year
2003 to 65 percent in fiscal year 2010 (see figure 2). However, there
was significant variation across VA's regional offices. For example,
during the 8-year time period, the regional office in Columbia, South
Carolina, had the lowest percentage of DRO reviews chosen--32 percent-
-and San Diego, California, the highest--87 percent. More than half of
the regional offices had the DRO review chosen in at least 50 percent
of all appeals filed.
Figure 2: Number of DRO Reviews and Traditional Reviews, Fiscal Years
2003-2010:
[Refer to PDF for image: stacked vertical bar graph]
Fiscal year: 2003;
DRO reviews: 59,206;
Traditional reviews: 50,614.
Fiscal year: 2004;
DRO reviews: 56,042;
Traditional reviews: 45,704.
Fiscal year: 2005;
DRO reviews: 59,501;
Traditional reviews: 40,478.
Fiscal year: 2006;
DRO reviews: 57,423;
Traditional reviews: 38,368.
Fiscal year: 2007;
DRO reviews: 60,989;
Traditional reviews: 36,831.
Fiscal year: 2008;
DRO reviews: 69,443;
Traditional reviews: 39,627.
Fiscal year: 2009;
DRO reviews: 81,353;
Traditional reviews: 43,422.
Fiscal year: 2010;
DRO reviews: 90,482;
Traditional reviews: 49,244.
Fiscal year: 2010;
DRO reviews: 90,482;
Traditional reviews: 49,244.
Source: GAO analysis of Board data.
[End of figure]
Our review suggests a key factor in the choice of a DRO review is
whether a veteran has assistance from a third party, such as a VSO
representative or a private attorney.[Footnote 16] Sixty-three percent
of all appeals filed by veterans with representation chose a DRO
review compared to 44 percent of those filed by veterans without
representation across all regional offices. Controlling for selected
factors that may affect the choice of a DRO review, veterans with
representation are still more likely to choose a DRO review for their
appeals.[Footnote 17] Additionally, we found that representation may
also be one of the factors that contribute to the variation in the
percentage of veterans choosing a DRO review across regional offices.
As the percentage of appeals with representation increased, the
percentage of DRO reviews also increased across regional offices.
However, representation alone does not fully explain the variation in
DRO reviews chosen across regional offices. Other factors that we did
not analyze, such as demographic factors, may contribute to the
variation across regional offices.
Representatives may recommend one appeal option over another to their
veterans. Of the 40 veterans we interviewed, 22 told us that
representatives from a VSO or state or county agency helped them
decide between the DRO and the traditional review, and 17 of the 22
said that the assistance they received was very or somewhat helpful in
deciding between the appeal options.[Footnote 18] Most VSO
representatives we interviewed said they usually recommend a DRO
review because they believe it is faster or because a DRO may be more
likely to grant the claim. Some also said they recommend a DRO review
over a traditional review because a DRO will thoroughly review all the
evidence and a veteran has the opportunity to make his or her case to
the regional office. VSO representatives at one regional office we
visited said that they recommend a DRO review if they believe, based
on experience, that the DRO likely to be assigned the case is more
open to working with them. However, some VSO representatives also said
that they recommend a traditional review under certain circumstances.
For example, if an appeal has ambiguous evidence, they may recommend a
traditional review because it may result in an appeal going to the
Board faster and because they believe the Board has more flexibility
than a DRO to grant benefits in such cases. Also, VSO representatives
at one site visit told us they generally recommend a traditional
review because in their view a DRO review takes too much time.
Of all appeals filed since fiscal year 2003, 53 percent included the
fact that they chose the DRO review option on the notice of
disagreement. According to VSO representatives we interviewed, because
veterans' representatives have power of attorney, they can choose an
appeal option at the same time they file an appeal for a veteran
rather than waiting for the veteran to receive VA's appeal process
request letter. A DRO review may be chosen on the same day an appeal
is filed in order to save time because the regional office does not
have to send the appeal process request letter and wait for the
veteran's response before beginning its review.[Footnote 19]
Veterans filing 13 percent of appeals chose an appeal option without
assistance from a representative, and these veterans may not fully
understand their options as described in the VA appeal process request
letter, some VA officials and VSO representatives told us. The VBA
manual provides a template of the letter for the regional offices,
which prepare and mail it to a veteran after receiving the notice of
disagreement.[Footnote 20] VA headquarters officials told us that
while regional offices have flexibility to add information to the
template letter, they cannot exclude any information, and headquarters
prefers that the regional offices use the template letter. (For VA's
template letter, see appendix II.) Some VA officials and VSO
representatives said veterans may not be able to make an informed
decision based on the letter alone. Specifically, 29 of 55 regional
office managers we surveyed (53 percent) answered that they believed
no or only some veterans could make an informed decision using only
the VA letter. The other regional office managers we surveyed (47
percent) said that all or most veterans could make an informed
decision using only the VA letter (see figure 3). Of the DROs we spoke
with, 10 of 17 said that in their view, no or only some veterans could
make informed decisions based on the letter alone, and most VSO
representatives at the regional offices we visited said that few or no
veterans could do so[Footnote 21]. Representatives from one VSO told
us veterans may phone after receiving the letter because they do not
understand its description of appeal options. Some managers in our
survey and VA staff we interviewed suggested changes to the template,
for example, shortening its length, using simpler language and less
jargon, using graphics or tables to aid understanding, explaining the
benefits of representation, and advising veterans they can request
informal or formal hearings if they choose a DRO review. They also
said the letter could better explain differences between DRO and
traditional reviews, particularly in the treatment of evidence and the
authority given a DRO to render a new decision without additional
evidence.
Figure 3: Percentage of Regional Office Managers Who Said Veterans Can
Make Informed Decision Using VA Letter Alone:
[Refer to PDF for image: pie-chart]
All or most veterans: 47%;
Some veterans: 38%;
Few or none of the veterans: 15%.
Source: GAO survey of managers in 57 VA regional offices.
[End of figure]
Our analysis of the template in the VBA manual, based on criteria
established for federal government communication with the public,
found the letter did not clearly explain appeal options and the appeal
process.[Footnote 22] Although we found the letter met several
criteria for clear communication by federal agencies and programs,
such as simple structure, concise headings, and avoidance of complex
language overall, we found it did not meet some other criteria,
including:
* defining terms that could be unfamiliar to the veteran, such as
"decision review officer," "de novo reviews," "clear and unmistakable
error," and "substantive appeal";
* highlighting key deadlines for responding by including them at the
beginning of the letter, so the veteran is clear that VA needs a
response within 60 days; and:
* using tables or graphics to explain appeal options and enhance
recipient's understanding (see figure 4).
Figure 4: Excerpt from VA Appeal Process Request Letter Template and
Examples of Clear Communication Analysis:
[Refer to PDF for image: illustration]
Department of Veterans Affairs:
Regional Office:
We received your written notice of disagreement with the Department of
Veterans Affairs (VA) decision of [date]. This letter describes what
happens next.
Will VA try to resolve my disagreement?[A]
This local VA office will try to resolve your disagreement through the
Post-Decision Review Process. As part of this process, you must decide
how you would like us to handle your appeal. You may choose to have a
Decision Review Officer (DRO)[B] assigned to your case or to follow
the traditional appeal process.
How does the Decision Review Officer Process work?
New decision: The DRO will then make a new decision. The DRO has the
authority to grant benefits based on clear and unmistakable error, de
novo review,[C] or the receipt of new and material evidence. You will
be notified of the decision and your appeal rights. If you are not
satisfied with the DRO's decision, you may then appeal, using the
traditional appeal process.
How does the Traditional Appeal Process work?
Statement of the Case: If we cannot grant your appeal based on the
review and an examination of any additional evidence, we will then
prepare a Statement of the Case (SOC) and send you a copy. The SOC
will include a summary of the evidence, a citation to pertinent laws,
a discussion of how those laws affect the decision, and a summary of
the reasons for the decision. If you still do not agree with that
decision and wish to continue your appeal, you need to submit a
substantive appeal[D] that your case can be sent to the Board of
Veterans‘ Appeals.[E] Instructions on how to file a substantive appeal
will be provided in our letter notifying you of the decision.
How do I select the Decision Review Officer process or traditional
appeal process?
You must notify us within 60 days from the date of this letter whether
you want to have your case reviewed by the Decision Review Officer
process or by the traditional appeal process. If we do not hear from
you within 60 days, your case will be reviewed under the traditional
appeal process.[F]
We hope we will be able to resolve your disagreement to your
satisfaction. If you have questions about the information in this
letter please call us at 1-800-827-1000.
Source: GAO.
[A] This letter helps the reader by using concise headings to organize
information.
[B] The letter does not define the ’Decision Review Officer“ (DRO).
For example, the reader does not know that the DRO is a senior expert on
disability claims or that a DRO is not involved in the original claim.
[C] The term ’clear and unmistakable error“ does not make it clear
that the DRO could grant the claim because the initial rater made an
error on the original claim.
The letter does not define a ’de novo review.“ For example, the reader
does not know that a de novo review is a completely new review of the
claim and that the DRO can make a new decision without additional
evidence.
[D] The letter does not define ’substantive appeal.“
[E] Letter does not explain the role of the Board of Veterans‘ Appeals
or who serves on the Board.
[F] One purpose of this letter is to explain that the veteran must
choose a DRO review within 60 days to receive one. Because the
deadline for choosing an appeal option is near the bottom of the
letter, a veteran who skims the letter may not recognize the urgency
of responding to the VA with a choice.
[End of figure]
One of the regional offices we visited made changes to the template
that address some of these issues, for example, including instructions
on how to request a hearing and italicizing important information,
such as the 60-day deadline for a veteran to respond with an appeal
choice.
Of the 40 veterans we interviewed, 14 remembered receiving and were
able to describe the appeal process request letter, and 9 of those
told us that the letter explained the appeal options only somewhat
clearly or not at all clearly. Five said it explained the options very
clearly.[Footnote 23] Some of the veterans we spoke with offered
suggestions on how VA could improve the letter, for example, by
providing a point of contact for veterans with questions, explaining
when they might next hear from VA about their appeal, and more
thoroughly describing the two appeal options. VA headquarters
officials said VBA management and attorneys reviewed the letter
template but they did not test it with focus groups of veterans
because VBA does not have contracts to conduct focus groups. According
to federal guidelines for clear communication with the public, testing
documents should be an integral part of an agency's plain-language
planning and writing process.
The DRO Review More Often Results in Additional Benefits, but Does Not
Reduce Appeals Going to the Board:
DRO Reviews More Often Resulted in Additional Benefits than
Traditional Reviews:
Veterans who chose a DRO review were more likely than those who chose
a traditional review to get additional disability compensation
benefits after initial review of their notices of disagreement.
[Footnote 24] From fiscal years 2003 through 2010 at least some
additional benefits were awarded at the regional level in 32 percent
of DRO reviews compared to 23 percent of traditional regional office
reviews, according to our analysis of Board data (see figure 5).
[Footnote 25] We found that having a DRO review was associated with a
greater chance of being awarded some additional benefits even after
controlling for some other factors that may affect appeal outcomes,
such as whether a veteran had a representative or the regional office
in which the appeal was filed.[Footnote 26] Veterans awarded
additional benefits at this appeal stage receive these benefits
immediately even if they continue their appeals, according to a VA
official.
Figure 5: Percentage of Appeals Receiving Full or Partial Grants of
Benefits after Regional Office Review of the Notice of Disagreement,
Fiscal Years 2003-2010:
[Refer to PDF for image: vertical bar graph]
Fiscal year: 2003;
Traditional review: 24%;
DRO review: 35%.
Fiscal year: 2004;
Traditional review: 21%;
DRO review: 33%.
Fiscal year: 2005;
Traditional review: 23%;
DRO review: 30%.
Fiscal year: 2006;
Traditional review: 24%;
DRO review: 31%.
Fiscal year: 2007;
Traditional review: 24%;
DRO review: 31%.
Fiscal year: 2008;
Traditional review: 22%;
DRO review: 31%.
Fiscal year: 2009;
Traditional review: 22%;
DRO review: 31%.
Fiscal year: 2010;
Traditional review: 22%;
DRO review: 30%.
Source: GAO analysis of Board data.
[End of figure]
DRO reviews took about a month longer than traditional reviews to
complete, or 266 days compared to 235 days, on average, for the time
period from fiscal years 2003 through 2009 (see figure 6).[Footnote
27] It is possible DRO reviews take longer because they involve a more
thorough examination of the evidence. For example, one DRO we spoke
with said such reviews take longer because they require a completely
fresh look at all evidence in the original claim. Regardless of
whether veterans choose a DRO or a traditional review, their waiting
time for a decision on the notice of disagreement is considerably less
than if they continue their appeal to the Board. For appeals filed
from fiscal years 2003 through 2007, the time from filing of the
notice of disagreement to Board decision was more than 1,000 days on
average.[Footnote 28]
Figure 6: Average Days for Regional Office to Issue Decision on Notice
of Disagreement, Fiscals Years 2003-2009:
[Refer to PDF for image: vertical bar graph]
Fiscal year: 2003;
Traditional review: 212;
DRO review: 237.
Fiscal year: 2004;
Traditional review: 215;
DRO review: 255.
Fiscal year: 2005;
Traditional review: 230;
DRO review: 247.
Fiscal year: 2006;
Traditional review: 252;
DRO review: 262.
Fiscal year: 2007;
Traditional review: 265;
DRO review: 279.
Fiscal year: 2008;
Traditional review: 264;
DRO review: 302.
Fiscal year: 2009;
Traditional review: 222;
DRO review: 273.
Source: GAO analysis of Board data.
Note: We analyzed data for fiscal years 2003 through 2009 because
about 90 percent or more of appeals filed in these years had received
a decision at the initial stage.
[End of figure]
Appeal Resolution Rate Is Similar for DRO and Traditional Reviews,
Leading to Minimal Change in the Number of Appeals to the Board:
Although veterans choosing a DRO review were more likely to gain some
additional benefits at the regional office level, in comparison to a
traditional review, DRO review does not appear to reduce the number of
appeals continuing to the Board--which was an important goal for VA in
introducing this appeal option. The resolution rate at the regional
office level for appeals in which a DRO review was selected has been
about the same as the rate for traditional reviews. Of the 593,526
appeals of disability claim decisions receiving either DRO or
traditional reviews from fiscal years 2003 through 2008 that had a
final regional office outcome, 72 percent ended at the regional office
level. The remainder continued to the Board[Footnote 29]. For appeals
in which a DRO review was selected, 71 percent ended at the regional
office level compared to 73 percent of appeals in which a traditional
review was chosen.[Footnote 30 The resolution rate for DRO and
traditional reviews was roughly the same in each year during this 6-
year period (see figure 7). Resolution rates varied across VA's
regional offices. For both DRO and traditional reviews, resolution
rates ranged from less than 50 percent to more than 80 percent in
individual offices, with the majority having rates of more than 70
percent for both types of reviews. VA officials offered several
possible explanations for the variation across offices, for example,
the extent to which VSO representatives are proactive in submitting
necessary evidence.
Figure 7: Percentage of Appeals Resolved at Regional Office Level,
Fiscal Years 2003-2008:
[Refer to PDF for image: vertical bar graph]
Fiscal year: 2003;
Traditional review: 73%;
DRO review: 71%.
Fiscal year: 2004;
Traditional review: 67%;
DRO review: 71%.
Fiscal year: 2005;
Traditional review: 73%;
DRO review: 70%.
Fiscal year: 2006;
Traditional review: 74%;
DRO review: 70%.
Fiscal year: 2007;
Traditional review: 75%;
DRO review: 71%.
Fiscal year: 2008;
Traditional review: 77%;
DRO review: 75%.
Source: GAO analysis of Board data.
Note: We analyzed data for fiscal years 2003 through 2008 because
about 90 percent or more of appeals filed in these years had been
resolved at the regional level or certified to the Board.
[End of figure]
The negligible difference between appeal resolution rates at the
regional office level for DRO and traditional reviews in part reflects
veterans' decisions on how to proceed if they do not receive a full
grant of requested benefits. Although a slightly higher percentage of
DRO reviews resulted in full grants of benefits for veterans, veterans
choosing DRO review were less likely than those choosing traditional
review to end their appeals when not granted full benefits. Board data
show that for appeals filed from fiscal years 2003 through 2008 the
most common reason for resolution at the regional office level was a
veteran's failure to return a form to continue the appeal (failure to
respond) after receiving VA's explanation of its decision and of what
steps to take to move the appeal to the Board (see figure 8)[Footnote
31]. On this front, traditional reviews were somewhat more likely than
DRO reviews to be resolved through veterans' failure to respond--41
percent versus 36 percent, respectively. The second most common reason
for appeal resolution at the regional level was a full grant of
benefits, which automatically ends the process. In contrast to the
first issue, DRO reviews were somewhat more likely than traditional
reviews to be resolved through a full grant of benefits--21 percent
versus 17 percent, respectively. Most decisions to grant full benefits
were made at the initial appeal stage, while some were made during the
stage in which the regional office prepares the appeal for the Board.
The remaining appeal resolutions were due to veterans or their
representatives withdrawing their appeals by contacting VA or to the
death of the veteran.
Figure 8: Regional Office Appeal Outcomes, Fiscal Years 2003-2008:
[Refer to PDF for image: vertical bar graph]
Regional office outcome: Traditional review;
Failure to respond: 41%;
Full grant: 17%;
Withdrawal: 12%;
Death of veteran: 3%;
Continued to Board: 27%.
Regional office outcome: DRO review;
Failure to respond: 36%;
Full grant: 21%;
Withdrawal: 12%;
Death of veteran: 2%;
Continued to Board: 29%.
Source: GAO analysis of Board data.
Note: We analyzed data for fiscal years 2003 through 2008 because
about 90 percent or more of appeals filed in these years had been
resolved at the regional level or certified to the Board.
[End of figure]
While we found no difference in how veterans responded to a partial
grant of benefits based on their choice of appeal option, we found
that veterans who chose a traditional review appeared to respond
differently than those who chose a DRO review when no additional
benefits were granted at the initial appeal stage. Board data for
fiscal years 2003 through 2008 show that when no additional benefits
were granted (i.e., the original decision was upheld), appeals in
which a traditional review was chosen were somewhat more likely than
those in which a DRO review was chosen to end at the regional office
level--57 percent versus 51 percent, respectively. By contrast, in
cases that resulted in the award of partial benefits, there was no
difference based on which appeal option had been chosen--75 percent of
both traditional appeals and DRO appeals were ended at the regional
level.[Footnote 32] Representation may be a factor in why veterans
selecting a traditional review are more likely to end their appeals
after receiving no additional benefits from the regional office. Fifty-
nine percent of appeals which received no additional benefits at the
initial appeal stage and in which the veteran had no representative
were ended by the veteran without going on to the Board, compared to
52 percent of such appeals in which the veteran had a representative.
As noted previously, veterans without representation are more likely
to choose a traditional review.
VA regional office staff told us that certain aspects of the DRO
review can be effective in resolving appeals before they continue to
the Board. Managers we surveyed rated DROs' authority to make a
completely new (de novo) decision on a claim and to hold informal
conferences with veterans and representatives as the most effective
tools for resolving appeals (see figure 9). Of the 17 DROs we
interviewed, 15 rated their authority to make a completely new
decision as very or moderately effective in resolving appeals, 10
rated informal conferences as very or moderately effective, and 7
rated formal hearings as very or moderately effective. Many DROs in
our site visits told us their authority to make a completely new
decision on a claim allows them to reverse original decisions without
submission of additional evidence by the veteran. DROs also said
informal conferences may help resolve appeals because they enable DROs
to obtain information from a veteran's representative or explain to a
representative and veteran why no additional benefits can be granted.
Figure 9: Percentage of Regional Office Managers Rating Specific
Activities as Very or Moderately Effective in Resolving Appeals:
[Refer to PDF for image: vertical bar graph]
Specific task: De Novo Review;
Very or moderately effective: 82%.
Specific task: Informal conference;
Very or moderately effective: 65%.
Specific task: Directed development[A];
Very or moderately effective: 55%.
Specific task: Formal hearing;
Very or moderately effective: 49%.
Specific task: Traditional review;
Very or moderately effective: 38%.
Source: GAO survey of managers in 57 VA regional offices.
[A] Directed development involves a DRO directing another staff member
to collect additional evidence such as medical or personnel records.
[End of figure]
While many regional staff believe DRO reviews have a greater potential
to resolve appeals, we found that the appeal resolution rate for DRO
and traditional reviews are quite similar, perhaps in part because
VA's criteria for assessing and rewarding individual DROs' performance
do not always encourage them to use their unique authorities to
resolve appeals. VA currently assesses the performance of individual
DROs using four criteria: quality of work, productivity, customer
service, and timeliness. The existing criteria do not specifically
encourage appeal resolution at the regional level, according to 60
percent of surveyed managers and 10 of the 17 DROs we interviewed. For
example, managers in one office we visited said the current criteria
encourage DROs to complete tasks that help them meet their
productivity requirement but may not necessarily lead to appeal
resolution.
The department is currently revising its criteria for assessing DRO
performance based on a 90-day pilot program in eight regional offices
conducted in fiscal year 2010, and it expects to implement new
criteria nationwide in fiscal year 2012. According to VA officials,
the revisions are intended in part to focus DROs' attention more on
resolving appeals at the earliest possible stage of the process. VA
officials told us the new criteria under development will better
encourage appeal resolution by restructuring the way work credits--
needed to meet the productivity requirement--are awarded to DROs. For
example, under the existing criteria, work credits are awarded for
holding informal conferences, but under the new criteria additional
credits would be awarded when the conference results in resolving the
appeal.[Footnote 33] In addition, VA officials said the new criteria
aim to promote appeal resolution by more explicitly promoting regular
communication with veterans' representatives. They said that even
though such communication can help resolve appeals by helping
representatives understand, for example, why further benefits cannot
be granted in a particular case, DROs have become less focused on
communication over the years due to production pressures.
VA Faces Challenges in Defining Role of and Training for DROs:
Lack of Performance Measures for Appeal Resolution Hinders VA's
Ability to Assess Use of DROs:
Since the DRO pilot program in 1997-1998, VA has expanded DRO duties
to include tasks not related to appeals. According to the estimates of
regional office managers who responded to our survey, DROs overall
tend to spend the majority of their time on appeal-related tasks, but
also spend a significant amount of time on training other staff and
performing quality reviews (see figure10). Time spent on different
tasks varies across regional offices. For example, the proportion of
time spent on conducting de novo reviews of appeals--that is, a
completely new evaluation of a claim without deference to the original
decision--ranged from 3 to 70 percent in individual offices, and on
formal training ranged from 0 to 40 percent. The four offices we
visited typically assigned individual DROs to specific
responsibilities, such as reviewing appeals, training other staff, and
conducting quality reviews of other staff's work, rather than having
each DRO perform the full range of tasks. Managers in one office said
specialization is helpful because it permits DROs to focus on training
without the distraction of appeals work. Regional office managers may
also assign DROs to different duties at different times to meet
changing office needs. For example, one office we visited had recently
assigned more DROs to training and quality review to address problems
with its RVSRs' performance, and managers said preliminary results
showed an improvement in RVSR quality scores after this reallocation
of DRO resources.
Figure 10: Regional Office Managers' Estimates of Proportion of DROs'
Time Spent on Specific Tasks:
[Refer to PDF for image: horizontal bar graph]
Task: De novo review;
Time Spent on task: 30%.
Task: Quality review and informal training;
Time Spent on task: 15%.
Task: Traditional review;
Time Spent on task: 39%.
Task: Formal training duties;
Time Spent on task: 38%.
Task: Directed development;
Time Spent on task: 37%.
Task: Formal hearings;
Time Spent on task: 37%.
Task: Certifying appeals;
Time Spent on task: 37%.
Task: Informal conferences;
Time Spent on task: 36%.
Task: Initial claims;
Time Spent on task: 35%.
Task: Remands;
Time Spent on task: 35%.
Task: Other;
Time Spent on task: 38%.
Source: GAO survey of managers in 57 VA regional offices.
Note: We asked regional office managers to estimate the proportion of
time that DROs in their office spend on specific tasks. Percentages
here reflect the average responses from regional offices about each
specific task, but not the percentage of time that all DROs nationally
spend on each task, because different regional offices have different
numbers of DROs.
[End of figure]
Since the implementation of the DRO review process in 2001, there have
been differing opinions within VA about the proper balance between
processing appeals and performing other DRO tasks. In 2006, an
internal VA study group that assessed the impact of the DRO review
process recommended that VA limit the scope of DRO duties to reviewing
appeals and have them perform de novo reviews of all appeals--not just
those in which a DRO review was selected. Their report found that
there was a sufficient number of DROs to perform de novo reviews of
all appeals if DROs' duties were limited to appeals-related tasks.
Officials told us that VA management decided not to implement these
recommendations because of concerns that there was an insufficient
number of DROs to review all appeals and a belief that other tasks are
also important DRO functions. There was also a mix of opinions on this
topic among regional office managers we surveyed and interviewed.
Almost two-thirds of those surveyed stated that eliminating the DRO
review election process and having DROs perform only de novo reviews
of all appeals would somewhat or greatly improve their effectiveness.
On the other hand, managers said it could be difficult to have their
DROs spend more time on such reviews. About 60 percent of surveyed
managers said they would prefer to allocate more DRO time to
performing de novo reviews, but some respondents noted that they
cannot allocate DROs' time optimally because of staff and resource
limitations and the need to assign DROs to other duties.
While VA has expanded the role of the DROs so they are contributing to
other VA goals beyond appeal resolution, the department has not
developed performance measures to assess whether DROs are successful
in meeting their original purpose of reducing the number of appeals to
the Board. The measures that VA uses to assess national and regional
office performance in processing appeals do not include the proportion
of appeals resolved at regional offices. Existing VA measures for
national and regional office appeals performance involve the number of
appeals awaiting a decision, number of days appeals have been pending
a decision, and percentage of appeals that are remanded from the
Board. We have previously noted that agencies should develop
performance measures that are linked to agency goals.[Footnote 34]
Linking performance measures to broader agency goals at each operating
level reinforces the importance of these goals and may help managers
identify areas where problems exist and corrective action is required.
VA already collects some data related to appeal resolution, but some
VA officials question the value of a performance measure related to
appeal resolution. The department tracks data on the number of appeals
resolved at different stages of the appeal process--for example,
through the veteran's failure to respond to the statement of the case
or after the veteran's submission of a substantive appeal to the Board-
-nationally and by regional office. It makes these data available to
regional offices. About half the regional office managers surveyed
said an appeal resolution goal at the regional office level would
somewhat or greatly enhance DROs' effectiveness. However, VA officials
told us they have not established a performance measure for appeal
resolution because this outcome is not entirely under the control of
the DRO, as a veteran has a right to continue his or her appeal to the
Board. Some VA managers and staff in the offices we visited also
expressed a concern that such a goal could have a negative impact. For
example, it could push DROs to grant benefits that are not completely
justified or to pressure veterans to withdraw their appeals. While we
understand this potential concern, we note that VA has quality review
procedures in place to ensure the quality and accuracy of DRO
decisions. Specifically, VA reviews on average five cases per month
that each DRO has worked on. VA assesses each case against several
criteria, including whether all claimed issues were addressed, whether
all applicable evidence was discussed, and whether all issues were
correctly granted or denied. Some regional office managers agreed that
setting an appeal resolution goal would not encourage DROs to grant
unjustified benefits, because, for example, existing quality review
procedures would prevent this from happening.
VA Provides No Nationwide Specialized Training for New DROs:
VA has not developed a nationwide training curriculum for DROs to help
them learn the duties of the position. The training available to DROs
from VA headquarters is the same training offered to RVSRs (staff who
evaluate initial claims). VA requires its claims processing staff,
including DROs, to take 85 hours of training annually. Headquarters
and regional office officials told us DROs typically take the same
courses as RVSRs. As part of its training for claims processors, VA
has three courses for all regional office staff who process appeals:
an orientation to the appeals team, a course on the appeal process,
and a course on how to use the Board's appeals tracking system. VA
headquarters officials told us they have not developed training
specifically for new DROs because they should already be technical
experts in evaluating claims when promoted. However, they acknowledged
that the department has not completed an analysis of DRO tasks
specifically to determine the training needs of this position.
According to generally accepted criteria, a key step in developing
successful training in the federal government is for an agency to
understand the skills that its workforce needs to achieve agency
goals. Furthermore, these criteria stress the need for agencies to
incorporate continuous, life-long learning into their training--even
experienced staff may still need additional training in certain areas.
[Footnote 35]
A number of managers and DROs in the regional offices we visited said
the skills DROs learn before promotion may not be sufficient to
perform their new tasks. Several managers told us new DROs, based on
their prior experience, are indeed technical experts in evaluating
claims; however, managers and DROs said new DROs may lack experience
with certain responsibilities, such as holding formal hearings and
informal conferences, training other staff, processing appeals, and
balancing multiple priorities. Regional managers at three of the four
offices we visited said the appeals courses offered by VA are not
sufficient to teach new DROs how to perform their duties; a manager in
one office said these courses provide a general overview and are not
primarily used to train them. DROs in the four offices we visited
learn their duties primarily on the job and by observing more
experienced colleagues, according to managers. However, some offices
have developed more formal training for DROs. Forty percent of
surveyed regional managers said their office has developed specific
training that address such topics as the appeals process, conducting
formal hearings and informal conferences, reviewing other staff's
work, and policy and regulatory changes.
Despite regional office efforts to provide targeted training, our
survey of regional managers found that 93 percent believe a nationally
standardized training program for newly promoted DROs would improve
their effectiveness, and 16 of the 17 DROs we interviewed also said a
national training program would be helpful. Managers in two regional
offices we visited said a training program developed by VA's central
office would ensure consistency in how DROs are trained across the
nation. Managers responding to our survey indicated that tailored
training on a number of topics would enhance the ability of new DROs
to perform their job duties (see figure 11).
Figure 11: Percentage of Regional Managers Who Said Training on
Certain Topics Would Greatly or Moderately Enhance New DROs'
Performance:
[Refer to PDF for image: vertical bar graph]
Training area: Training other staff;
Greatly or Moderately Enhance New DROs' Performance: 75%.
Training area: Policy and legal changes;
Greatly or Moderately Enhance New DROs' Performance: 69%.
Training area: Mentoring;
Greatly or Moderately Enhance New DROs' Performance: 64%.
Training area: Workload management;
Greatly or Moderately Enhance New DROs' Performance: 64%.
Training area: Informal conferences;
Greatly or Moderately Enhance New DROs' Performance: 62%.
Training area: Quality review;
Greatly or Moderately Enhance New DROs' Performance: 62%.
Training area: Appeals process;
Greatly or Moderately Enhance New DROs' Performance: 60%.
Training area: Formal hearings;
Greatly or Moderately Enhance New DROs' Performance: 60%.
Source: GAO survey of managers in 57 VA regional offices.
[End of figure]
A formal, centrally designed training program--especially for new
DROs--has been proposed within VA. Under this proposal, according to
one VA official involved with the management of the DROs nationally,
new DROs would receive instruction in areas such as training other
staff and communicating with veterans and their representatives. The
training might be administered in a central location for new DROs from
multiple regional offices, or developed centrally and then
administered by regional offices. However, VA does not yet have formal
plans to move ahead with the development of such a program, according
to VA officials.
Conclusions:
Review of veterans' appeals by VA's DROs has had some positive impacts
for veterans, and it is possible that more veterans would choose a DRO
review if they fully understood the distinction between the DRO and
traditional reviews. Because VA's outreach letter lacks information
and clarity that veterans need to make an informed choice between the
options, some--in particular, those without a representative--may not
be taking advantage of the DRO review process. However, looking more
broadly, VA has not achieved its original goal for the DRO review
process of reducing the number of appeals continuing to the Board and
thereby shortening the time that veterans wait for appeal decisions.
Indeed, while VA is taking steps to place a greater focus on appeal
resolution in the criteria it uses to assess the performance of
individual DROs, it has yet to establish any national or regional
office performance measures related to resolving appeals at the
regional level. Certainly, VA must balance competing demands on DROs,
senior claims processors in its regional offices, to both resolve
appeals and train and supervise less experienced staff, but without a
more strategic approach--which includes goals and measures for
resolution of appeals--VA does not know it is fully leveraging DRO
reviews and, ultimately, their effectiveness for the veterans VA
serves. Regardless of the proper balance between their different
responsibilities, DROs may not be contributing as much as they could
be, either to the goal of appeal resolution or to the goal of
developing newer claims processing staff, because they do not always
receive training specific to their duties.
Recommendations for Executive Action:
To clarify information for veterans and ensure the most effective use
of DROs, the Secretary of Veterans Affairs should direct the Veterans
Benefits Administration to take the following three actions:
* revise the sample appeals election letter in its policy manual to
define unfamiliar terms and emphasize key deadlines, and test any
revised letter's clarity with veterans before implementing it;
* establish national and regional office performance measures related
to appeal resolution at the regional level and ensure that sufficient
quality review procedures are in place to prevent DROs from granting
unjustified benefits; and:
* assess the knowledge and skills that DROs need to perform their
varied responsibilities, determine if any gaps exist in the training
currently available, and, if necessary, develop a training curriculum
or program tailored to DROs.
Agency Comments and Our Evaluation:
We provided a draft of this report to VA for review and comment. In
its written comments (see appendix III), VA concurred fully with two
of our recommendations and partially with our recommendation regarding
an appeal resolution performance measure. The department concurred
with our recommendation to revise its sample appeal election letter.
It reported that it has formed a workgroup to assess and recommend
changes to the appeal election process, potentially including changes
to the letter. VA also concurred with our recommendation to assess the
knowledge and skills that DROs need and if necessary develop a
training curriculum for them. VA said it plans to develop additional
training based on the results of skills certification tests
administered to DROs and on a job task analysis conducted to aid the
design of these certification tests.
VA partially concurred with our recommendation to establish national
and regional performance measures related to the resolution of appeals
at the regional office level. The department noted that--as explained
previously in our report--it is revising the criteria used to assess
the performance of individual DROs to place greater emphasis on
resolving appeals at the earliest stage possible, for example through
communication with veterans and their representatives. However, it
also stated that resolution of appeals at the regional level should
not be a performance measure used to assess the impact of the DROs,
for several reasons including (1) the DROs also play an important role
in training, mentoring, and quality review, so an appeal resolution
measure does not capture their full impact; (2) each veteran has a
right to continue an appeal to the Board if dissatisfied with the
regional office decision, and whether a veteran continues is beyond a
DRO's control; and (3) establishing an appeal resolution performance
measure could encourage the granting of benefits that are not
justified. We acknowledge that DROs impact VA operations beyond
appeals. Yet VA's own documents--including its 2006 review of the DRO
program--state that the department's primary goal in establishing the
DRO review process was to increase the percentage of appeals resolved
before continuing to the Board, and the changes VA is now making to
its criteria for measuring individual DRO performance indicate that
appeal resolution remains a major goal. We are not recommending that
an appeal resolution performance measure replace existing VA
performance measures related to appeals or to other VA processes on
which DROs could have an impact. Our recommendation is that VA add
such a measure to gauge whether the program is meeting its specific
intended goal and help the department assess whether further
adjustments are needed. We also acknowledge that it is not possible to
prevent all appeals from continuing to the Board; such an objective
may not even be desirable. However, based on its changes to the
criteria for individual DRO performance, VA clearly believes that
changes in incentives can encourage behaviors that are likely to
resolve more appeals early on. Finally, as VA also noted in its
comments, individual DROs must meet rigorous accuracy standards in
their appeals work, which should mitigate against the granting of
unjustified benefits. However, if VA does not believe that its
existing quality control measures are sufficient to guard against the
granting of unjustified benefits, we would encourage the department to
consider what additional quality control measures it could implement
to ensure that the addition of a new performance measure would not
have the unintended consequence of increasing the award of unjustified
benefits.
As agreed with your offices, unless you publicly announce the contents
of this report earlier, we plan no further distribution until one day
from the report date. At that time, we will send copies of this report
to the relevant congressional committees, the Secretary of Veterans
Affairs, and other interested parties. This report is also available
at no charge on GAO's website at [hyperlink, http://www.gao.gov].
If you or your staff have any questions regarding this report, please
contact me at (202) 512-7215 or bertonid@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. Staff members who made key
contributions to this report are listed in appendix IV.
Signed by:
Daniel Bertoni:
Director, Education, Workforce, and Income Security issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
We were asked to examine (1) the extent to which veterans choose a
Decision Review Officer (DRO) review as opposed to a traditional
review, (2) outcomes for veterans who choose a DRO review, and (3)
challenges the Department of Veterans Affairs (VA) faces in managing
DROs. To answer these objectives, we reviewed relevant laws and
regulations, and VA documents including procedural manuals, internal
studies, and the sample appeal election letter that VA headquarters
provides for its regional offices. We interviewed cognizant officials
from the Veterans Benefits Administration (VBA) and the Board of
Veterans' Appeals (Board). We interviewed managers, DROs, and veteran
service organization (VSO) staff in 4 regional offices, and we
conducted a web-based survey of managers in all of VA's 57 regional
offices. We obtained and analyzed administrative data from the Board
on outcomes and processing times for appeals of disability
compensation claims. Finally, to learn about veterans' perspectives on
the DRO review election process we conducted phone interviews with a
small sample of veterans who had appealed their disability
compensation claims.
We conducted this review from July 2010 to September 2011 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Analysis of VA Appeal Process Request Letter:
We analyzed the VBA's appeal process request letter as a factor that
may affect veterans' decisions about the appeal options. For our
analysis we used the sample letter in the VBA manual since VBA central
office officials said that regional offices use the sample as a
template for the letters they mail to veterans.[Footnote 36] We
selected 16 criteria for clear federal communication using information
from the Plain Language Guidelines,[Footnote 37] a previous GAO report
on VBA letter clarity,[Footnote 38] and the VBA manual (see table 1).
We reviewed the letter and came to a consensus about which criteria
were met and which were not met.
Table 1: Clear Federal Communication Criteria for VA Letter Analysis:
Clear federal communication criteria: The letter includes statements
of deadlines;
Criteria descriptions: The letter should include a statement of the
deadline for responding to the letter;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter includes contact
information;
Criteria descriptions: Contact information includes a phone number for
the veteran to call or an address that the veteran could write to;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter states the purpose;
Criteria descriptions: As part of the critical structure, letters
should identify the purpose;
Did the letter meet the criterion? No.
Clear federal communication criteria: Most important information is at
the beginning, such as the purpose;
Criteria descriptions: Most important information should be stated at
the beginning, and background information (when necessary) should be
included toward the end. The document's purpose and its bottom line
are stated at the beginning. Unnecessary content is eliminated;
Did the letter meet the criterion? No.
Clear federal communication criteria: The letter is in a logical
sequence;
Criteria descriptions: Similar ideas are grouped and are not
contradictory, steps are presented chronologically, and general
information is presented earlier than specific information;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter addresses one person
rather than a group (for example, "you" or "a veteran" instead of
"veterans");
Criteria descriptions: Singular nouns and verbs prevent confusion
about whether a statement or requirement applies to individual users
or to groups;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter's structure is simple;
there are not more than five levels of headings;
Criteria descriptions: Crafting documents with four, five, or even
more levels of headings makes it difficult for the audience to keep
track of where they are in the structure of the document;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter uses concise headings
to organize information;
Criteria descriptions: Headings should not be so long that they
overwhelm the material in the section itself. Headings with one-word
answers should be avoided;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The headings highlight short
sections of information;
Criteria descriptions: Short sections break up material so it appears
easier to comprehend;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter does not use jargon;
Criteria descriptions: Jargon refers to terms that are unfamiliar to
and undefined for the average layperson, including unexplained
acronyms and highly technical medical or legalistic terminology;
Did the letter meet the criterion? No.
Clear federal communication criteria: The letter does not use
convoluted language;
Criteria descriptions: Convoluted language refers to situations where
the sentence structure or syntax is overly complex;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter appropriately uses
examples, lists, tables, or illustrations to clarify complex concepts;
Criteria descriptions: Examples help to clarify complex concepts and
can substitute for long explanations. The more complex the concept,
the more using examples should be considered. Vertical lists highlight
a series of requirements or other information in a visually clear way.
Vertical lists help the user focus on important material but not to
over-emphasize trivial matters. Tables can make complex material
easier to understand and help the audience see relationships that are
often times hidden in dense text;
Did the letter meet the criterion? No.
Clear federal communication criteria: The letter uses bold and italics
to make important concepts stand out;
Criteria descriptions: Using emphasis, such as bold and italics, is
effective at drawing readers' attention to a short, particularly
important section, sentence, phrase, or word;
Did the letter meet the criterion? No.
Clear federal communication criteria: The letter uses minimal cross-
references (for example, referring to regulations, manual citations,
etc.);
Criteria descriptions: Numerous cross-references can confuse users and
make them less attentive to the message;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: Document design does not appear
cluttered or dense;
Criteria descriptions: There should be no lists within lists and the
text should not be fully justified so that documents do not appear
cluttered or dense. Also, the document should have five or six
sections on each printed page (about two on each typewritten page);
use lists and tables often, but not overuse them and not have lists
within lists;
and use ragged margins where possible, rather than fully justifying
text;
Did the letter meet the criterion? Yes.
Clear federal communication criteria: The letter clearly explains the
actions that the claimant can take;
Criteria descriptions: The communication should clearly state: the
actions the claimant can take, including procedures and time limits;
how the claimant can contact VBA for more information;
and, any responsibilities that the claimant might have;
Did the letter meet the criterion? No.
Source: GAO analysis.
[End of table]
Site Visits to VA Regional Offices:
We conducted site visits to 4 of VA's 57 regional offices: Atlanta,
Georgia; Providence, Rhode Island; Salt Lake City, Utah; and Waco,
Texas. We conducted these visits to learn about regional office
practices in utilizing DROs and regional staff's opinions on such
topics as how veterans choose between appeal options, factors
affecting DRO review outcomes, and challenges facing VA in utilizing
DROs. In each regional office, we interviewed regional office
managers, at least one appeals coach, multiple DROs, and local VSO
staff. In total, we interviewed 17 DROs across the 4 offices, with
varying levels of experience in their position. In some offices we
also interviewed the regional office training coordinator. We
judgmentally selected these offices to achieve variation in several
factors, including geographic location, number of staff, timeliness of
appeal processing for appeals in which DRO was selected, and
participation in a pilot study of revised criteria for assessing DRO
performance (see figure 12). What we report about these sites may not
necessarily be representative of other VA regional offices.
Figure 12: Regional Offices Selected for Site Visits:
[Refer to PDF for image: illustrated U.S. map]
Eastern area:
Location: Providence, Rhode Island;
Number of staff (rank): 60 (47th);
Processing time for DRO reviews: 224 days (25th)
Performance criteria pilot site: Yes.
Southern area:
Location: Atlanta, Georgia;
Number of staff (rank): 303 (6rd);
Processing time for DRO reviews: 475 days (54th);
Performance criteria pilot site: No.
Central area:
Location: Waco, Texas;
Number of staff (rank): 460 (2nd);
Processing time for DRO reviews: 160 days (11th);
Performance criteria pilot site: No.
Western area:
Location: Salt Lake City, Utah;
Number of staff (rank): 171 (21st);
Processing time for DRO reviews: 158 days (10th);
Performance criteria pilot site: Yes.
Sources: VA; Map Resources (map).
Note: Number of staff represents claims processing staff as of July
31, 2010. Processing time for DRO reviews represents the average
number of days from the veteran's election of DRO review to the
regional office's issuance of a statement of the case, from January 1,
2007 through August 15, 2010.
[End of figure]
Analysis of Administrative Data from the Board:
To analyze data on VA's appellate workload, including the number of
appeals in which DRO and traditional reviews were selected, the
outcomes for DRO and traditional reviews, and the processing times for
DRO and traditional reviews, we obtained record-level appeals data
extracted on April 7, 2011, from the Veterans Appeals Control and
Locator System (VACOLS). We limited our analysis to appeals of
disability compensation claims. We also limited our analysis to
original appeals, as opposed to, for example, appeals that had been
remanded by the Board. Using the record-level data, we generated
nationwide annual data on appeals filed from fiscal years 2003 through
2010. The data for each fiscal year includes all notices of
disagreement filed with the VA during that fiscal year. We looked at
data back to fiscal year 2003, even though the DRO review process was
established in 2001, because the VACOLS data element that identifies
appeals in which DRO review was selected was only added to VACOLS at
the end of 2002. For some analyses, we excluded data for appeals filed
in recent fiscal years because a high proportion of these appeals were
still pending some action by the regional office. To assess the
reliability of the record-level appeals data, we (1) reviewed
documentation on VACOLS including the data dictionary, (2) interviewed
Board officials about VACOLS and any data reliability issues, and (3)
performed electronic testing. We found the data to be sufficiently
reliable for our reporting objectives.
Statistical Analysis of Factors Affecting Choice of DRO Review:
To investigate the relationship between DRO review selection and
representation, using VACOLS data, we first examined descriptive
statistics at the regional office level. We found substantial
variation in rates of DRO election, as well as rates of representation
among veterans filing appeals, across regional offices. Using linear
multiple regression analysis, we estimated that at the regional office
level, a one percentage point increase in the proportion of veterans
with representation was associated with approximately half a
percentage point increase in the rate of DRO election across regional
offices, after controlling for fiscal year.[Footnote 39] However,
variation in representation levels across regional offices did not
fully explain variation in DRO election across offices.
Our next analysis focused on appeals level data. Descriptive data show
that, prior to controlling for other factors, veterans with
representation were more likely to choose a DRO review than those
without representation. Specifically, table 2 shows that approximately
63 percent of those veterans with representation requested a DRO
reviews, compared to approximately 44 percent of veterans with no
representation.
An alternative way to examine the likelihood of DRO election among
veterans with and without representation is the odds of requesting a
DRO review. Prior to controlling for other factors, the odds that a
veteran with no representation requesting a DRO review is defined as
the proportion of veterans requesting a DRO review divided by the
proportion requesting a traditional review, or 0.79 (44.1/55.9). In
comparison, among veterans with representation, the odds of requesting
a DRO review were 1.72 (63.3/36.7). To compare the relative proportion
of veterans with and without representation that requested a DRO
review, we can construct an odds ratio that compares the odds of DRO
selection to the reference group of veterans without representation,
which is 2.18 (1.72/0.79). In other words, prior to controlling for
other factors, the unadjusted odds that a veteran with representation
requesting a DRO review were 2.18 times that of the odds that a
veteran without representation requesting such review.
Although unadjusted odds provide useful summary information, they do
not account for multiple other factors that could also affect a
veteran's choice of review. Logistic regression analysis can be used
to estimate the odds ratio comparing the likelihood of DRO selection
among veterans with and without representation, after also controlling
for other factors that could potentially influence the choice of
review type. We estimated a logistic regression model of DRO selection
controlling for other factors including the fiscal year of the appeal,
the regional office of the appeal, whether the veteran filing the
appeal had representation, how many distinct medical issues were
associated with each appeal, and what type of body systems were
included in the issues under appeal. We were not able to control for
certain other factors, such as the timing of DRO election, because we
lacked comparative data for traditional reviews. There may be
systematic differences between veterans who decide not to appeal a
ruling and those who appeal a ruling and thus chose between DRO and
traditional reviews. Additionally, we lacked information to control
for certain factors such as a veteran's branch of service or
demographic characteristics.[Footnote 40]
After controlling for fiscal year, regional office, number of issues
in each appeal, and body systems related to the issues under appeal,
we found that appeals filed by veterans with representation were
substantially more likely than those filed by veterans without
representation to elect the DRO review process. Per the last column of
table 2, we can see that, after controlling for other factors, the
odds that an appeal filed by a veteran with representation underwent a
DRO review were approximately two times higher than the odds that an
appeal filed by a veteran without representation underwent a DRO
review. This result was statistically significant at the p
