Veterans' Disability Benefits
Expanded Oversight Would Improve Training for Experienced Claims Processors Gao ID: GAO-10-445 April 30, 2010The Veterans' Benefits Improvement Act of 2008 (P.L. 110-389) mandated that GAO evaluate the Department of Veterans Affairs VA training for disability claims processors. This report answers the following two questions: (1) How appropriate is the training provided to experienced disability claims processors? (2) How adequate is the Veterans Benefits Administration's (VBA) monitoring and assessment of this training? To address these questions, GAO conducted a web-based survey of a nationally representative sample of claims processors, interviewed VBA headquarters and regional office officials, and reviewed VBA training material, relevant federal statutes, regulations, and court cases.
VBA'sannual training requirements and the training received by experienced staff--those with more than two years experience--may not be appropriate, based on the results of GAO's survey of claims processors nationwide. Experienced claims processors had problems with five key areas: the number of hours of training required, the amount of training received on particular topics, the usefulness of some subject matter, the way training is delivered and the timing of training. GAO's survey results indicated that more than half (55 percent) of experienced claims processors found it difficult to meet VBA's 80-hour annual training requirement given their workload. In addition, based on its survey, GAO estimates that 45 percent of supervisors of experienced Rating Veterans Service Representatives (RVSR) and 53 percent of supervisors of experienced Veterans Service Representatives (VSR) thought that only some or few, if any, of the experienced staff they supervise need 80 hours of training to perform their job duties effectively. Many experienced staff also thought they received too little training on some topics and too much on others. For example, 42 percent thought they received less training than needed in how to rate claims involving special monthly compensation and 34 percent thought they received more than enough training on records management. Finally, opinions varied on how helpful the various modes of training were. Nearly all claims processors, in general, considered on-the-job experience to be the method of training best suited to their needs. An estimated 42 percent of all experienced claims processors, in general, felt that some of the training they received was delivered too late, suggesting that regional offices may not always deliver the training needed by experienced claims processors in a timely manner. According to Standards for Internal Control in the Federal Government, federal agencies must have control mechanisms in place to help ensure that all employees receive appropriate and consistent training. Under its recently revised annual training requirements, VBA delegates considerable responsibility for training experienced claims processors to each of its 57 regional offices. In particular, regional offices are responsible for ensuring that claims processors complete annual training requirements. Each office also determines what topics are covered for half of the required training hours, what material to provide on each of these topics, and how and when the training should occur. Regional offices also have considerable discretion in determining what activities qualify as training. However, VBA lacks controls to ensure that regional offices deliver required training and record completed training in a consistent manner, and does little to assess the appropriateness or consistency of the training experienced claims processors receive.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Daniel Bertoni Team: Government Accountability Office: Education, Workforce, and Income Security Phone: (202) 512-5988GAO-10-445, Veterans' Disability Benefits: Expanded Oversight Would Improve Training for Experienced Claims Processors
This is the accessible text file for GAO report number GAO-10-445
entitled 'Veterans' Disability Benefits: Expanded Oversight Would
Improve Training for Experienced Claims Processors' which was released
on April 30, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
[Note: This report was revised on September 3, 2010, to correct
estimates in figures 2 through 7, and associated text, to account for
a programming error in our survey data analyses. In the majority of
cases, corrected estimates in these figures vary by no more than 1
percentage point from those in the original report and never more than
6 percentage points. The margin of error in footnote 6 has been
increased to plus or minus 15 percentage points because questions in
the survey did not always apply to all experienced claims processors.
Text was also modified, where appropriate, to better describe
responses by new and experienced claims processors, to clarify survey
results, and to more appropriately contrast survey responses related
to different training modes. None of these changes affect this
report‘s conclusions or recommendations.]
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
April 2010:
Veterans' Disability Benefits:
Expanded Oversight Would Improve Training for Experienced Claims
Processors:
GAO-10-445:
GAO Highlights:
Highlights of GAO-10-445, a report to congressional committees.
Why GAO Did This Study:
The Veterans‘ Benefits Improvement Act of 2008 (P.L. 110-389) mandated
that GAO evaluate the Department of Veterans Affairs VA training for
disability claims processors. This report answers the following two
questions: (1) How appropriate is the training provided to experienced
disability claims processors? (2) How adequate is the Veterans
Benefits Administration‘s (VBA) monitoring and assessment of this
training? To address these questions, GAO conducted a web-based survey
of a nationally representative sample of claims processors,
interviewed VBA headquarters and regional office officials, and
reviewed VBA training material, relevant federal statutes,
regulations, and court cases.
What GAO Found:
VBA‘s annual training requirements and the training received by
experienced staff”those with more than two years experience”may not be
appropriate, based on the results of GAO‘s survey of claims processors
nationwide. Experienced claims processors had problems with five key
areas: the number of hours of training required, the amount of
training received on particular topics, the usefulness of some subject
matter, the way training is delivered and the timing of training. GAO‘
s survey results indicated that 60 percent of experienced claims
processors who believed that VBA requires 80 hours of training
annually found it difficult to meet this training requirement given
their workload. In addition, based on its survey, GAO estimates that
45 percent of supervisors of experienced Rating Veterans Service
Representatives (RVSR) and 53 percent of supervisors of experienced
Veterans Service Representatives (VSR) thought that only some or few,
if any, of the experienced staff they supervise need 80 hours of
training to perform their job duties effectively.
Many experienced staff also thought they received too little training
on some topics and too much on others. For example, 42 percent thought
they received less training than needed in how to rate claims
involving special monthly compensation and 34 percent thought they
received more than enough training on records management. Finally,
opinions varied on how helpful the various modes of training were.
Nearly all claims processors, in general, considered on-the-job
experience to be the method of training best suited to their needs. An
estimated 39 percent of all experienced claims processors, in general,
felt that the training they received was delivered too late,
suggesting that regional offices may not always deliver the training
needed by experienced claims processors in a timely manner.
According to Standards for Internal Control in the Federal Government,
federal agencies must have control mechanisms in place to help ensure
that all employees receive appropriate and consistent training. Under
its recently revised annual training requirements, VBA delegates
considerable responsibility for training experienced claims processors
to each of its 57 regional offices. In particular, regional offices
are responsible for ensuring that claims processors complete annual
training requirements. Each office also determines what topics are
covered for half of the required training hours, what material to
provide on each of these topics, and how and when the training should
occur. Regional offices also have considerable discretion in
determining what activities qualify as training. However, VBA lacks
controls to ensure that regional offices deliver required training and
record completed training in a consistent manner, and does little to
assess the appropriateness or consistency of the training experienced
claims processors receive.
What GAO Recommends:
GAO recommends that VBA (1) adopt procedures for routinely monitoring
and ensuring compliance with annual training requirements, including
more fully using its available electronic data to ensure that training
requirements are met, (2) develop clear written guidance on the types
of activities all regional offices should and should not count toward
completion of annual training requirements, and (3) develop and
implement a written strategy for routinely assessing the
appropriateness of the training regional offices provide to
experienced claims processors. In its comments, VA generally concurred
with GAO‘s conclusions and concurred with all of GAO‘s recommendations.
View GAO-10-445 or key components. For more information, contact
Daniel Bertoni at (202) 512-7215 or bertonid@gao.gov.
[End of section]
Contents:
Letter:
Background:
Experienced Claims Processors Had Concerns with Various Aspects of the
Training They Received:
VBA Does Little to Systematically Monitor or Assess Training for
Experienced Claims Processors:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Survey Methodology:
Questionnaire Development and Administration:
Population and Sample Design:
Estimation, Sampling, and Nonsampling Error:
Appendix II: Training Topics Included in GAO's Survey of VBA Claims
Processors:
Appendix III: VBA Fast Letter 10-05, Revision of Core Technical
Training Requirement Hours for VSRs and RVSRs:
Appendix IV: Comments from the Department of Veterans Affairs:
Appendix V: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: Phases in VBA's Claims Processing Improvement Model and
Responsible Teams:
Table 2: Population, Sample, and Respondents by Stratum:
Table 3: Training Topics Included in GAO's Survey of VBA Claims
Processors:
Figures:
Figure 1: Number of Nonsupervisory VBA Claims Processors, by
Experience, Fiscal Years 2004-2008:
Figure 2: Estimated Percentages of New and Experienced VSRs, RVSRs,
and Supervisory VSRs, as of May 2009:
Figure 3: Claims Processors' Views on the Ease or Difficulty of
Completing 80 Hours of Training Each Year:
Figure 4: Experienced Claims Processors' Views Regarding Sufficiency
of Training on Different Topics Since Their Employment with VBA:
Figure 5: Experienced Claims Processors' Views Regarding the Effect of
Formal Training Received in the Previous 12 Months on Their Ability to
Do Their Job:
Figure 6: Experienced Claims Processors' Views Regarding the Effect of
Training about Specific Medical Conditions on Their Ability to Do
Their Job:
Figure 7: Experienced Claims Processors' Views on the Extent to Which
Various Training Modes Helped Them Learn What They Needed to Know to
Perform Their Jobs:
Abbreviations:
CTTR: Core Technical Training Requirements:
LMS: Learning Management System:
RVSR: Ratings Veterans Service Representatives:
TBI: traumatic brain injury:
TPSS: Training Performance Support System:
VA: Department of Veterans Affairs:
VBA: Veterans Benefits Administration:
VSR: Veterans Service Representatives:
Note: This report was revised on September 3, 2010, to correct numbers
in figures 2 through 7 and text presenting survey results. Changes
were made to text and reported numbers on the highlights page, pages
2, 4, 5, 9 through 16, and 24.
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
April 30, 2010:
The Honorable Daniel K. Akaka, Chairman:
The Honorable Richard Burr, Ranking Member:
Committee on Veterans' Affairs:
United States Senate:
The Honorable Robert Filner, Chairman:
The Honorable Steve Buyer, Ranking Member:
Committee on Veterans' Affairs:
House of Representatives:
For the past several years, the Veterans Benefits Administration (VBA)
has experienced a growing inventory of claims for disability benefits.
In addition, the Department of Veterans Affairs' (VA) Office of
Inspector General recently identified serious issues with the
consistency of claims decisions in these cases across regional
offices.[Footnote 1] To help VBA manage its increasing workload and
replace the growing number of experienced claims processors who are
retiring, Congress appropriated funds for VBA to hire more than 4,000
new staff since fiscal year 2005, including 1,200 to be hired through
fiscal year 2010.
To ensure that decisions in disability compensation and pension cases
are accurate, consistent, and timely, training for newly-hired claims
processors must enable them to become fully proficient and training
for experienced claims processors must help them maintain their
knowledge and skills. In the Veterans' Benefits Improvement Act of
2008, Congress expressed its ongoing concerns in this area by
mandating a GAO evaluation of VA training for disability claims
processors.[Footnote 2] In 2008 we reported that VBA's centralized
training for new claims processors appeared well designed but that
some claims processors had raised concerns about
implementation.[Footnote 3] VBA has since evaluated its training for
new claims processors and made changes based upon that evaluation. In
this report, we focus on training for experienced claims processors.
[Footnote 4] In response to the mandate, this report answers the
following two questions: (1) How appropriate is the training provided
to experienced disability claims processors? and (2) How adequate is
VBA's monitoring and assessment of this training?
To address these questions, we conducted a Web-based survey in August
and September 2009 of a nationally representative sample of VBA
Veterans Service Representatives (VSR), Ratings VSRs (RVSR), and
supervisory VSRs to obtain information about their training
experiences and views.[Footnote 5] Seventy percent of those sampled
responded to the survey and the results were tested and determined to
be generalizable to all claims processors.[Footnote 6] We also
interviewed VBA officials from VBA's Compensation and Pension Service
and its Office of Employee Development and Training and obtained
available documentation of VBA's procedures for identifying staff
training needs and other written guidance. We spoke with managers and
training coordinators in four regional offices--Little Rock, Arkansas;
Denver, Colorado; St. Petersburg, Florida; and White River Junction,
Vermont--chosen to reflect a range of office sizes and geographic
locations. These offices do not represent all regional offices
nationwide. We also asked Veterans' Service Organizations to provide
their views on training for claims processors. To assess the adequacy
of monitoring and evaluation, we applied criteria GAO has identified
for assessing federal training programs and for assessing federal
agency internal controls.[Footnote 7] We also reviewed relevant
federal statutes, regulations, and court decisions. Additional
information on our survey methodology can be found in appendix I.
We conducted this performance audit from May 2009 to April 2010 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions
based on our audit objectives.
Background:
In 2009 the VBA employed 5,198 VSRs and 2,344 RVSRs[Footnote 8]--also
known as claims processors--that were most directly responsible for
handling more than 975,000 veterans' claims for disability
compensation and pension benefits at 57 regional offices.[Footnote 9]
VSRs perform nonrating tasks related to veterans' claims, and RVSRs
assign ratings upon which benefit amounts are based. Since 2005, VBA
has hired more than 4,000 new employees to help with its growing
inventory of disability claims.[Footnote 10] Consequently, the
proportion of claims processors with 2 years or less of experience has
nearly doubled from about one quarter of all claims processors in
fiscal year 2004 to more than one half in 2008, the most current year
for which full-year data were available (see figure 1).
Figure 1: Number of Nonsupervisory VBA Claims Processors, by
Experience, Fiscal Years 2004-2008:
[Refer to PDF for image: stacked vertical bar graph]
Year: 2004;
Experienced staff (more than 2 years on job): 3,629 (73%);
New staff (2 years or less on job): 1,342 (27%).
Total: 4,971.
Year: 2005;
Experienced staff (more than 2 years on job): 3,623 (72%);
New staff (2 years or less on job): 1,409 (28%).
Total: 5,032.
Year: 2006;
Experienced staff (more than 2 years on job): 3,476 (65%);
New staff (2 years or less on job): 1,871 (35%).
Total: 5,347.
Year: 2007;
Experienced staff (more than 2 years on job): 3,534 (57%);
New staff (2 years or less on job): 2,666 (43%).
Total: 6,200.
Year: 2008;
Experienced staff (more than 2 years on job): 3,687 (49%);
New staff (2 years or less on job): 3,837 (51%).
Total: 7,524.
Source: VBA.
[End of figure]
Note: The experience categories of more than 2 years and 2 years or
less used in this figure differ slightly from the definitions of
experienced (2 years or more) and new (less than 2 years) we used in
our analysis of survey data.
Based on our survey of claims processing staff, we estimate that at
the time of our survey,[Footnote 11] about equal proportions of VSRs
and RVSRs had less than 2 years of experience (see figure 2).
Figure 2: Estimated Percentages of New and Experienced VSRs, RVSRs,
and Supervisory VSRs, as of May 2009:
[Refer to PDF for image: horizontal bar graph]
RVSRs:
New: 1,155 (49%);
Experienced: 1,189 (51%).
VSRs:
New: 2,617 (50%);
Experienced: 2,581 (50%).
Supervisory VSRs:
New: 268 (47%);
Experienced: 309 (53%).
Source: GAO analysis of GAO survey data.
[End of figure]
To process claims accurately, consistently, and in a timely manner,
VSRs and RVSRs must perform a complex set of tasks well. When a claim
is received, a VSR reviews it and assists the veteran in gathering the
evidence, or documentation, needed to support it. Such evidence
includes, but may not be limited to, a veteran's military service and
discharge records, medical examination and treatment records, proof of
income, and certificates of marriage and birth. An RVSR then evaluates
the evidence to determine whether the claimant's medical condition(s)
constitutes a disability, and its level of severity. The RVSR assigns
a disability percentage rating to the medical condition(s), which is
used to determine the amount of benefits the veteran is eligible to
receive. A VSR calculates and authorizes, or obtains authorization
for, the amount of monthly payments, including any retroactive
payments to the claimant. VSRs and RVSRs may also perform follow-up
reviews if, for example, there is evidence a claimant's medical
condition has changed, or a court determines that a claimant was
incorrectly denied benefits. To better manage the process, VBA's
regional offices typically follow VBA's Claims Processing Improvement
Model, which organizes claims processing duties into six phases and
assigns a team of VSRs and RVSRs, as needed, to each phase. (See table
1.)
Table 1: Phases in VBA's Claims Processing Improvement Model and
Responsible Teams:
Claims processing phase: Establish the regional office's tracking
procedures for all mail, as well as process claims that require only a
brief review to determine eligibility;
Responsible team: Triage[A].
Claims processing phase: Develop evidence for disability ratings and
prepare administrative decisions;
Responsible team: Predetermination[A].
Claims processing phase: Make decisions on claims that require
consideration of medical evidence;
Responsible team: Rating[B].
Claims processing phase: Develop evidence for nonrating issues,
processes benefit awards, and notify veterans of rating decisions;
Responsible team: Postdetermination[C].
Claims processing phase: Conduct personal interviews and handle
telephone inquiries, including calls from veterans;
Responsible team: Public contact[C].
Claims processing phase: Handle requests for reconsideration of claims
in which veterans have formally disagreed with claims decisions;
Responsible team: Appeals[A].
Source: GAO analysis of information provided by VBA, as cited in GAO-
08-561.
[A] Team typically includes both VSRs and RVSRs.
[B] Team typically includes only RVSRs.
[C] Team typically includes only VSRs.
[End of table]
Although the Claims Processing Improvement Model narrows the range of
tasks VSRs and RVSRs must perform at any given time, the practice in
some offices of rotating staff among teams is, according to VBA,
beneficial in helping VSRs and RVSRs reach journey-level, by giving
them a broader range of skills and competencies to perform
effectively. In addition to learning a variety of tasks specific to
their team, claims processors must keep current with ongoing changes
in regulations and laws governing veterans' benefits.
To ensure that VSRs and RVSRs develop and maintain the knowledge and
skills needed to process disability claims accurately, consistently,
and in a timely manner, VBA sets overall annual training requirements
for claims processors and has developed the Challenge program for
newly-hired or promoted claims processors.[Footnote 12] Prior to
January 2010, VBA had two annual training requirements for all VSRs
and RVSRs intended to ensure they develop and maintain the knowledge
and skills they need to perform well. First, VBA required that claims
processors receive a total of 80 hours of training each year directly
related to processing disability claims, regardless of their
experience handling VA disability claims. Second, VBA required that:
* 60 of the 80 hours cover topics each regional office selected from
VBA's Core Technical Training Requirements (CTTR)--a list of training
topics covering all aspects of the claims process--and:
* 20 of the 80 hours cover topics determined by each regional office.
During the course of this review, VBA revised its annual training
requirements for claims processors who had either completed the
Challenge program or had been in their current position for more than
6 months. In a VBA document dated January 15, 2010, (see appendix IV)
VBA notified VA regional offices and centers that beginning in fiscal
year 2010, claims processors are required to receive 85 hours of
training annually--80 in topics directly related to processing
disability claims and an additional 5 hours in more general topics
such as cyber security and ethics.
Under the new requirements, VBA continues to require staff to complete
80 hours of training in topics directly related to processing
disability claims. However, VBA now requires that 40 of those hours be
in topics selected by VBA, 20 cover topics selected by each regional
office from the CTTR, and 20 cover topics determined by each regional
office. VBA indicated that the requirement for 40 hours of training
annually in VBA-designated topics would standardize half of the
required training for all VSRs and RVSRs.[Footnote 13] VBA is in the
process of revising the CTTR and did not provide the complete, revised
requirements for this review. VBA plans to provide regional offices
with course materials for each VBA-designated topic tailored to the
needs of claims processors with 6 months to 2 years experience, and to
claims processors with more than 2 years of experience, which VBA
refers to respectively as intermediate and journey level staff.
[Footnote 14]
In addition to its Challenge program and annual training requirements,
VBA issues Fast Letters, conducts telephone conferences, and develops
ad hoc required training on emerging issues to help ensure that
disability claims processors have the information they need to do
their job well. VBA's Fast Letters are periodic memoranda designed to
quickly communicate policy changes as they occur and before formal
training related to these changes can be developed. VBA issued 48 Fast
Letters in 2008 and 52 in 2009 on topics ranging from cost-of-living
adjustments in disability benefit amounts to rating the effects of
traumatic brain injury. According to VBA officials, its monthly or
quarterly telephone conferences with regional offices and centers
concentrate on claims processing issues it identifies through its
quality reviews or on topics such as new management priorities or
initiatives that may affect how claims processors do their jobs. VBA
officials also told us the agency periodically requires training on
emerging topics such as rating post-traumatic stress disorder and
spinal, neck, and joint injuries.
Experienced Claims Processors Had Concerns with Various Aspects of the
Training They Received:
VBA's annual training requirements and the training received by
experienced staff may not be appropriate, based on the results of our
survey of claims processors nationwide. Experienced claims processors
had concerns in five key areas: the number of hours of training
required, the amount of training received on particular topics, the
helpfulness of some subject matter, the way training is delivered, and
the timing of training.
Experienced Staff Expressed Concerns with the Amount of Training and
Their Ability to Meet Current Training Requirements:
VBA currently requires all VSRs and RVSRs to complete 80 hours of
training each year in topics directly related to disability claims
processing.[Footnote 15] Based on the results of our survey of claims
processors, many believed that 80 training hours each year was not
appropriate, particularly for experienced staff. An estimated 45
percent of supervisors of experienced RVSRs and 53 percent of
supervisors of experienced VSRs thought that only some or few, if any,
of the experienced staff they supervise need 80 hours of training.
Most of the regional office officials we interviewed also said 80
hours are too many for at least some experienced staff and one
regional official told us it would make sense to vary the required
number of hours based on the training needs of individual claims
processors.[Footnote 16] In 2008 we recommended that VBA collect and
review feedback from staff to determine if the 80-hour training
requirement was appropriate for all VSRs and RVSRs. VBA has not yet
taken steps to implement this recommendation.
Our survey results indicated that, of those claims processors who
believe that VBA requires 80 hours of training annually, 60 percent of
experienced claims processors and 44 percent of new claims processors
found it somewhat or very difficult, in general, to meet the 80-hour
annual training requirement, given their workload (see figure 3).
Moreover, a large proportion of those who supervised experienced
claims processors agreed. Sixty-one percent of experienced RVSR's
supervisors and 76 percent of experienced VSR's supervisors thought it
was somewhat or very difficult for experienced staff to complete 80
hours of training each year.
Figure 3: Claims Processors' Views on the Ease or Difficulty of
Completing 80 Hours of Training Each Year:
[Refer to PDF for image: horizontal bar graph]
New staff (2 years or less on job):
Very easy: 12%;
Somewhat easy: 5%;
Total Easy: 17%;
Somewhat difficult: 26%;
Very difficult: 18%;
Total Difficult: 44%.
Experienced staff (more than 2 years on job):
Very easy: 3%;
Somewhat easy: 8%;
Total Easy: 11%;
Somewhat difficult: 30%;
Very difficult: 30%;
Total Difficult: 60%.
Source: GAO analysis of GAO survey data.
[End of figure]
While many new staff also found it somewhat or very difficult to meet
this requirement, one VBA official told us it is easier for them to do
so because they typically receive far more than 80 hours each year
through Challenge training, which is targeted exclusively to newly
hired and promoted staff and can take more than a year to complete.
Many Experienced Staff Thought They Did not Receive the Right Amount
of Training in Some Topics:
Our survey results indicated many experienced claims processors felt
they received too little training on some topics and too much on
others (see figure 4). Specifically, 47 percent of all experienced
claims processors, in general, thought they received less than
sufficient training in developing appeals and remands. An estimated 42
percent thought they received less training than needed in how to rate
claims involving special monthly compensation and 36 percent thought
they received less training than needed in Dependency and Indemnity
Compensation.[Footnote 17] Conversely, about one-third of all
experienced claims processors thought they received more than enough
training in records management (34 percent), rating disability
compensation claims (33 percent), and calculating payment amounts
based on disability ratings (37 percent).
Figure 4: Experienced Claims Processors' Views Regarding Sufficiency
of Training on Different Topics Since Their Employment with VBA:
[Refer to PDF for image: horizontal bar graph]
Estimated percentage of experienced claims processors:
Topics: Records management;
Less than sufficient: 12%;
Sufficient: 54%;
More than sufficient: 34%.
Topics: Claims recognition/establishment;
Less than sufficient: 11%;
Sufficient: 55%;
More than sufficient: 33%.
Claims development: Service verification;
Less than sufficient: 15%;
Sufficient: 53%;
More than sufficient: 32%.
Claims development: Medical evidence;
Less than sufficient: 19%;
Sufficient: 54%;
More than sufficient: 28%.
Claims development: Income;
Less than sufficient: 27%;
Sufficient: 54%;
More than sufficient: 19%.
Claims development: Dependency;
Less than sufficient: 19%;
Sufficient: 51%;
More than sufficient: 30%.
Claims development: Special monthly compensation;
Less than sufficient: 28%;
Sufficient: 51%;
More than sufficient: 21%.
Claims development: Burial;
Less than sufficient: 29%;
Sufficient: 50%;
More than sufficient: 21%.
Claims development: Appeals and remands;
Less than sufficient: 47%;
Sufficient: 42%;
More than sufficient: 11%.
Ratings decisions: Disability pension claims;
Less than sufficient: 27%;
Sufficient: 53%;
More than sufficient: 20%.
Ratings decisions: Disability compensation claims;
Less than sufficient: 23%;
Sufficient: 44%;
More than sufficient: 33%.
Ratings decisions: Dependency and Indemnity Compensation;
Less than sufficient: 36%;
Sufficient: 45%;
More than sufficient: 19%.
Ratings decisions: Ancillary benefits and special purposes;
Less than sufficient: 36%;
Sufficient: 43%;
More than sufficient: 22%.
Ratings decisions: Special monthly compensation;
Less than sufficient: 42%;
Sufficient: 42%;
More than sufficient: 16%.
Ratings decisions: Hospitalization;
Less than sufficient: 30%;
Sufficient: 54%;
More than sufficient: 16%.
Claims processing: Payment amount;
Less than sufficient: 27%;
Sufficient: 37%;
More than sufficient: 37%.
Claims processing: Payment authorization;
Less than sufficient: 29%;
Sufficient: 41%;
More than sufficient: 30%.
Claims processing: Evaluation of disabilities based on routine future
examinations;
Less than sufficient: 24%;
Sufficient: 51%;
More than sufficient: 25%.
Claims processing: System-generated messages;
Less than sufficient: 31%;
Sufficient: 45%;
More than sufficient: 24%.
Claims processing: Due process and duty to inform;
Less than sufficient: 18%;
Sufficient: 52%;
More than sufficient: 30%.
Claims processing: Workload and time management;
Less than sufficient: 32%;
Sufficient: 47%;
More than sufficient: 21%.
Claims processing: Telephone development and customer service skills;
Less than sufficient: 21%;
Sufficient: 53%;
More than sufficient: 27%.
Claims processing: Computer skills and Web navigation;
Less than sufficient: 31%;
Sufficient: 49%;
More than sufficient: 21%.
Claims processing: Supervision;
Less than sufficient: 35%;
Sufficient: 47%;
More than sufficient: 18%.
Source: GAO analysis of GAO survey data.
Note: Some totals do not add to 100 percent due to rounding.
[End of figure]
Some claims processors wanted more training on topics outside of their
official job duties. For example, many VSRs (46 percent) indicated
they did not receive sufficient training in developing appeals and
remands and about half (53 percent) of all VSRs felt that they
received an insufficient amount of training on rating decisions
involving ancillary benefits, even though VSRs do not usually process
appeals and remands or make ratings determinations. One VBA official
told us that some VSRs try to prepare for promotion by taking training
in tasks RVSRs usually perform. In addition, some regional offices may
rotate claims processors from one team to another to help manage
workloads, so VSRs may feel the need to cross train and develop skills
that are not typically required of people on their assigned team.
Experienced Staff Had Mixed Views on How Much Their Training Helped
Them Do Their Job:
Based on our survey results, experienced staff thought training on
some topics had a moderate to very positive effect on their ability to
do their jobs and some thought training had only somewhat or no
positive effect. For example, 71 percent thought that training on how
to rate disability compensation claims had a moderate or very positive
effect on their ability to do their job (see figure 5). Similarly,
about 71 percent thought that claims development training on service
verification had a moderate or very positive effect on their ability
to do their job. In addition, 47 percent of all experienced claims
processors felt training on responding to system generated messages
had no or only some positive effect on their ability to do their jobs.
Also, about 45 percent of experienced claims processors felt courses
on supervision had no or only some positive effect on their job
performance. Experienced claims processors, in general, assessed
training on specific medical conditions, such as post-traumatic stress
disorder and spinal, neck, and joint injuries favorably (see figure 6).
Figure 5: Experienced Claims Processors' Views Regarding the Effect of
Formal Training Received in the Previous 12 Months on Their Ability to
Do Their Job:
[Refer to PDF for image: horizontal bar graph]
Topics: Records management;
No positive effect: 12%;
Somewhat positive effect: 29%;
Moderately positive effect: 29%;
Very positive effect: 29%.
Topics: Claims recognition/establishment;
No positive effect: 12%;
Somewhat positive effect: 18%;
Moderately positive effect: 38%;
Very positive effect: 32%.
Claims development: Service verification;
No positive effect: 13%;
Somewhat positive effect: 16%;
Moderately positive effect: 40%;
Very positive effect: 31%.
Claims development: Medical evidence;
No positive effect: 14%;
Somewhat positive effect: 19%;
Moderately positive effect: 36%;
Very positive effect: 31%.
Claims development: Income;
No positive effect: 18%;
Somewhat positive effect: 21%;
Moderately positive effect: 34%;
Very positive effect: 27%.
Claims development: Dependency;
No positive effect: 12%;
Somewhat positive effect: 17%;
Moderately positive effect: 38%;
Very positive effect: 33%.
Claims development: Special monthly compensation;
No positive effect: 16%;
Somewhat positive effect: 23%;
Moderately positive effect: 34%;
Very positive effect: 26%.
Claims development: Burial[A];
No positive effect: 25%;
Somewhat positive effect: 19%;
Moderately positive effect: 31%;
Very positive effect: 25%.
Claims development: Appeals and remands;
No positive effect: 24%;
Somewhat positive effect: 36%;
Moderately positive effect: 20%;
Very positive effect: 20%.
Ratings decisions: Disability pension claims;
No positive effect: 22%;
Somewhat positive effect: 15%;
Moderately positive effect: 31%;
Very positive effect: 32%.
Ratings decisions: Disability compensation claims[A];
No positive effect: 6%;
Somewhat positive effect: 23%;
Moderately positive effect: 30%;
Very positive effect: 41%.
Ratings decisions: Dependency and Indemnity Compensation;
No positive effect: 15%;
Somewhat positive effect: 19%;
Moderately positive effect: 26%;
Very positive effect: 40%.
Ratings decisions: Ancillary benefits and special purposes;
No positive effect: 15%;
Somewhat positive effect: 23%;
Moderately positive effect: 35%;
Very positive effect: 27%.
Ratings decisions: Special monthly compensation;
No positive effect: 13%;
Somewhat positive effect: 25%;
Moderately positive effect: 29%;
Very positive effect: 33%.
Ratings decisions: Hospitalization;
No positive effect: 18%;
Somewhat positive effect: 16%;
Moderately positive effect: 30%;
Very positive effect: 36%.
Claims processing: Payment amount[A];
No positive effect: 14%;
Somewhat positive effect: 22%;
Moderately positive effect: 27%;
Very positive effect: 36%.
Claims processing: Payment authorization[A];
No positive effect: 17%;
Somewhat positive effect: 22%;
Moderately positive effect: 24%;
Very positive effect: 37%.
Claims processing: Evaluation of disabilities based on routine future
examinations;
No positive effect: 10%;
Somewhat positive effect: 24%;
Moderately positive effect: 36%;
Very positive effect: 31%.
Claims processing: System-generated messages;
No positive effect: 21%;
Somewhat positive effect: 26%;
Moderately positive effect: 22%;
Very positive effect: 31%.
Claims processing: Due process and duty to inform;
No positive effect: 10%;
Somewhat positive effect: 23%;
Moderately positive effect: 37%;
Very positive effect: 31%.
Claims processing: Workload and time management;
No positive effect: 15%;
Somewhat positive effect: 20%;
Moderately positive effect: 32%;
Very positive effect: 34%.
Claims processing: Telephone development and customer service
skills[A];
No positive effect: 17%;
Somewhat positive effect: 14%;
Moderately positive effect: 36%;
Very positive effect: 34%.
Claims processing: Computer skills and Web navigation;
No positive effect: 16%;
Somewhat positive effect: 15%;
Moderately positive effect: 34%;
Very positive effect: 34%.
Claims processing: Supervision;
No positive effect: 30%;
Somewhat positive effect: 14%;
Moderately positive effect: 29%;
Very positive effect: 27%.
Source: GAO analysis of GAO survey data.
Note: Some totals do not add to 100 percent due to rounding.
[A] The margin of error for one or more of this item's response
categories is between plus or minus 15 and plus or minus 20 percentage
points at the 95 percent confidence level. Sampling errors for these
estimates are higher than for others in this report because not all
experienced claims processors received training on each topic in the
previous 12 months.
[End of figure]
Figure 6: Experienced Claims Processors' Views Regarding the Effect of
Training about Specific Medical Conditions on Their Ability to Do
Their Job:
[Refer to PDF for image: horizontal bar graph]
Medical condition: Post traumatic stress disorder;
No positive effect: 6%;
Somewhat positive effect: 22%;
Moderately positive effect: 35%;
Very positive effect: 37%.
Medical condition: Spinal, neck, and joint injuries;
No positive effect: 7%;
Somewhat positive effect: 28%;
Moderately positive effect: 28%;
Very positive effect: 36%.
Medical condition: Traumatic brain injury;
No positive effect: 20%;
Somewhat positive effect: 25%;
Moderately positive effect: 26%;
Very positive effect: 29%.
Source: GAO analysis of GAO survey data.
Note: Some totals do not add to 100 percent due to rounding.
[End of figure]
One possible explanation for experienced claims processors' negative
views regarding the effect of training in some topics may be that the
training they receive covers information they already know or tasks
they have already mastered.[Footnote 18] In discussing training needs
with us, two of the four regional training managers we spoke with said
training for experienced staff should be more advanced. Another told
us that refresher training does not address complex issues experienced
on the job and resembles basic or remedial training.
An estimated 72 percent of all experienced claims processors felt
most, almost all or all of their instructors in the last 12 months
deserved high marks for subject matter knowledge, and about 53 percent
reported that most, almost all or all of their instructors in the last
12 months demonstrated effective teaching skills. Moreover, based on
our survey, an estimated 2,373 claims processors taught at least one
course between August 2008 and September 2009, but VBA provided or
paid for instructor training for only one in four of them.
Opinions of Experienced Staff Varied on the Helpfulness of Various
Training Modes and On the Timeliness of Training:
Training for disability claims processors can be delivered in a number
of ways: formal classroom training, online instruction, video or
satellite conferences, and other instructor-led training. Disability
claims processors can access online training courses through VBA's
Training Performance Support System (TPSS). More learning resources
may also be used, such as VBA training materials, published guidance,
and technical information available on VBA's internal Web site.
Regional offices also provide individual coaching and mentoring, and
may hold weekly meetings for claims processing teams.
While regional offices and VBA can deliver training for claims
processors in a number of ways, based on our survey results,
experienced claims processors, in general, found certain training
modes and learning resources more helpful than others (see figure 7).
Nearly all experienced staff thought that on-the-job experience, to a
great or very great extent, helped them learn what they needed to know
to perform their jobs. Although most experienced staff found this
learning method helpful, it would be unreasonable to expect them to
remain proficient at their jobs without the benefit of at least some
periodic, formal training. However, relatively fewer experienced
claims processors view VBA's more formal training modes to be helpful.
For example, TPSS or other online training and video or satellite
training appeared to be among the least helpful to experienced claims
processors, even though most had received such training. Only about 20
percent said this training had, to a great or very great extent,
helped them become familiar with even the basic information needed to
handle claims.
Figure 7: Experienced Claims Processors' Views on the Extent to Which
Various Training Modes Helped Them Learn What They Needed to Know to
Perform Their Jobs:
[Refer to PDF for image: horizontal bar graph]
Training mode: On-the-job experience;
Little or no extent: 0%;
Some extent: 0%;
Moderate extent: 3%;
Great extent: 19%;
Very great extent: 77%.
VBA manuals, Web aids;
Little or no extent: 5%;
Some extent: 9%;
Moderate extent: 27%;
Great extent: 23%;
Very great extent: 37%.
Coaching from peers;
Little or no extent: 5%;
Some extent: 10%;
Moderate extent: 17%;
Great extent: 32%;
Very great extent: 36%.
Mentoring;
Little or no extent: 18%;
Some extent: 6%;
Moderate extent: 16%;
Great extent: 26%;
Very great extent: 33%.
Fast/Training Letters (management communications);
Little or no extent: 4%;
Some extent: 12%;
Moderate extent: 27%;
Great extent: 26%;
Very great extent: 31%.
Coaching from supervisors;
Little or no extent: 22%;
Some extent: 16%;
Moderate extent: 17%;
Great extent: 17%;
Very great extent: 28%.
Internet searches;
Little or no extent: 5%;
Some extent: 11%;
Moderate extent: 27%;
Great extent: 31%;
Very great extent: 26%.
Team meetings;
Little or no extent: 15%;
Some extent: 24%;
Moderate extent: 25%;
Great extent: 20%;
Very great extent: 16%.
Instructor-led classroom on-site;
Little or no extent: 10%;
Some extent: 19%;
Moderate extent: 32%;
Great extent: 23%;
Very great extent: 16%.
Non-VBA manuals/references;
Little or no extent: 15%;
Some extent: 17%;
Moderate extent: 32%;
Great extent: 20%;
Very great extent: 16%.
Instructor-led classroom off-site;
Little or no extent: 30%;
Some extent: 15%;
Moderate extent: 33%;
Great extent: 13%;
Very great extent: 10%.
Live Meeting/audio conferences;
Little or no extent: 22%;
Some extent: 31%;
Moderate extent: 26%;
Great extent: 15%;
Very great extent: 7%.
VBA video and satellite;
Little or no extent: 21%;
Some extent: 29%;
Moderate extent: 29%;
Great extent: 14%;
Very great extent: 7%.
VBA online(TPSS and other training);
Little or no extent: 31%;
Some extent: 20%;
Moderate extent: 30%;
Great extent: 13%;
Very great extent: 6%.
Source: GAO analysis of GAO survey data.
Note: Some totals do not add to 100 percent due to rounding.
[End of figure]
Survey results also indicate that regional offices may not deliver the
training needed by experienced claims processors in a timely manner.
An estimated 39 percent of all experienced claims processors, in
general, felt that the formal training they received in the last 12
months was delivered too late to help them effectively perform their
duties. For example, although the regulation governing ratings
decisions on the effects of traumatic brain injury (TBI) was changed
in October 2008, one RVSR wrote in a comment that, "TBI training is
not projected to come out until [2010.]"[Footnote 19] Another claims
processor wrote that introductory leadership training was not received
until 2 years after that claims processor was promoted to a
supervisory position.
VBA Does Little to Systematically Monitor or Assess Training for
Experienced Claims Processors:
According to Standards for Internal Control in the Federal Government,
[Footnote 20] federal agencies must have mechanisms in place to help
ensure that all employees receive appropriate training. Under its
recently revised annual training requirements, VBA delegates
considerable responsibility for training experienced claims processors
to each of its 57 regional offices. Regional offices are responsible
for ensuring that claims processors complete annual training
requirements. Each office also determines what topics are covered for
half of the required hours, what material to provide on each of these
topics, and how and when the training should occur. VBA does little,
however, to ensure that regional offices actually provide the required
training or to assess the appropriateness and consistency of the
training experienced claims processors receive.
VBA Does Not Systematically Monitor Regional Office Compliance with
its Annual Training Requirements:
It is important for federal agencies to have mechanisms in place to
ensure their employees actually receive required training. According
to our prior report on assessing employee training in federal
agencies,[Footnote 21] tracking the actual receipt of required
training calls for reliable data from a comprehensive learning
management system. Currently, VBA headquarters does not ensure that
all experienced claims processors across all regional offices receive
the number of hours and types of training required each year. Each
regional office is responsible for recording completed training hours
for each claims processor in VA's Learning Management System (LMS)--a
Web-based application intended to provide a single access point that
all VA managers and employees can use. VBA used LMS data in fiscal
year 2009 to determine the total number and percentage of VSRs and
RVSRs at each regional office who had met the annual training
requirement, and used it once in fiscal year 2010 to determine if VSRs
and RVSRs were on track to meet their fiscal year 2010 training
requirements. These data show that only one regional office met the 80-
hour requirement for VSRs and RVSRs in fiscal year 2009, and in nine
regional offices, fewer than half of VSRs and RVSRs had met the
requirement. In addition, VBA officials told us that VBA headquarters
does not use it at this time to centrally monitor the specific types
of training individual claims processors have completed. In our survey
we found that some claims processors did not receive training they
should have. Based on our survey results, an estimated 24 percent of
all RVSRs with more than 1 year of experience, who should have
received mandatory training on rating spinal, neck, and joint
injuries, never did.[Footnote 22] This evidence, together with VBA's
data indicating that regional offices are not meeting the 80-hour
requirement, underscore the importance of VBA's monitoring of training
for claims processors.
In addition to not using LMS to centrally monitor receipt of required
training, VBA lacks controls to ensure that regional offices all
record completed training in a consistent manner. Each regional office
has considerable discretion in determining what activities qualify as
training. In our interviews with regional office management, we found
evidence that they were not all defining training consistently. For
example, some regions counted time claims processors spent reading
Fast Letters as training. Others did not. This raises questions about
the reliability of the data that regional offices enter into the LMS.
VBA Does Not Systematically Assess the Appropriateness or Consistency
of Training Regional Offices Provide to Experienced Claims Processors:
It is also important for federal agencies to have mechanisms in place
to ensure that training for employees is appropriate and consistent.
According to our prior report on training for federal employees,
[Footnote 23] agencies should assess the appropriateness and
consistency of the training employees receive. However, VBA does
little to determine if all regional offices provide training that is
consistent or appropriate for experienced claims processors--that is,
(1) covers topics relevant to what they do, (2) helps them do their
job, (3) is delivered in the most useful and efficient way, and (4) is
provided when needed.
In the past, VBA has evaluated some mandatory training for RVSRs,
assessed training for newly-hired claims processors, and solicited
feedback on TPSS. VBA provided training for RVSRs in response to a
study conducted by VBA headquarters in July 2008 that found large
differences in disability ratings for lower back conditions across
regional offices.[Footnote 24] After the training was provided, VBA
conducted a follow-up study in April 2009 and found consistency in
ratings across regional offices improved from 69 percent to 82
percent. In June 2008, VBA completed an evaluation of its Challenge
training for new staff to determine if the training as implemented,
provided value to VBA, and to help identify possible improvements. The
study included (1) face-to-face interviews with 183 VBA employees
including managers, supervisors, instructors, VSRs, RVSRs, and
trainers at 16 regional offices; (2) an online survey of VSRs, RVSRs,
coaches, and training coordinators at VBA's 57 regional offices: and
(3) observations of Challenge training at 6 locations. It identified a
number of areas for improvement, and VBA officials said changes were
made based on the findings and recommendations. Finally, in June 2009,
VBA conducted a satisfaction survey of online TPSS training provided
to new staff as part of Challenge training. According to VBA
officials, about 75 percent of respondents rated the quality of TPSS
training good to excellent and felt TPSS was satisfying to use.
[Footnote 25]
Feedback from training participants, as well as supervisors,
instructors, and other stakeholders, can provide valuable information
agencies can use to assess the appropriateness and consistency of
training for employees. In the past, however, VBA has not
systematically collected feedback from experienced claims processors
on training received at the regional offices. VBA has recently
developed a training evaluation tool for claims processors. This tool
is linked to VBA's internal Web site and, as of March 1, 2010, all
VSRs and RVSRs are required to use the tool to evaluate their CTTR
courses; however the tool will not be used to evaluate non-CTTR
training provided by the regional offices.
VBA officials told us that teams from VBA headquarters periodically
visit regional offices to monitor their compliance with VBA policies
and procedures. Since July 2009, such visits have included a training
specialist who is responsible for reviewing some aspects of training
in the regional office, such as LMS training records, training
materials, and whether new claims processors have completed the last
phase of Challenge training at their regional office. The training
specialist also discusses training with the regional office training
manager and coaches. Although examining some aspects of training for
claims processors during these visits may provide VBA with some
information on the appropriateness and consistency of training
provided in a given regional office, each office is only visited about
once every 3 years.
Conclusions:
VBA claims processors perform a vital role in helping the nation serve
the needs of disabled veterans. Through its training program, VBA
strives to provide the necessary tools to enable claims processors to
perform this role well; however, VBA needs to do more to monitor and
evaluate its training for experienced claims processors. Even though
VBA devotes significant staff resources to training, it does not
sufficiently monitor compliance with the training requirements and
therefore cannot be sure that all staff are receiving the type of
training that VBA believes is essential for success on the job.
Although VA's automated LMS data system should play a key role in
monitoring the training received by staff, VBA does not fully use this
system for this purpose. Furthermore, there are questions about the
reliability of the training data regional offices record in this system.
VBA has invested considerable resources evaluating and refining
Challenge training for new claims processors. According to our survey,
however, both experienced claims processors and their supervisors have
a number of concerns regarding the training that experienced claims
processors receive. VBA has recently taken steps to standardize some
of the topics that experienced claims processors are required to train
in, which may address some of the concerns raised in our survey.
However, without a strategy for routinely assessing the
appropriateness and consistency of the training provided to
experienced claims processors--such as the number of hours required,
the relevance of VBA-designated training topics, the manner in which
training is delivered, and its timing--VBA cannot ensure the
effectiveness of its training.
Recommendations for Executive Action:
We recommend that the Secretary of Veterans Affairs direct the
Undersecretary for Benefits to take the following three actions:
1. To ensure that claims processors receive the types of training
required by VBA each year, VBA should adopt procedures for routinely
monitoring and ensuring compliance with annual training requirements,
including more fully using its LMS to ensure that claims processors
receive required CTTR and ad hoc training on emerging topics.
2. To ensure that regional offices record training activities
consistently in the LMS so that LMS data are reliable, VBA should
develop clear written guidance on the types of activities all regional
offices should and should not count toward completion of annual
training requirements.
3. To ensure that training is appropriate for experienced claims
processors, VBA should develop and implement a written strategy for
systematically assessing the content, mode and timing of training
experienced claims processors receive in regional offices and make
improvements, when indicated. Such a strategy should include a
standardized approach for:
* obtaining and analyzing feedback from experienced claims processors
on the appropriateness of non-CTTR training provided by regional
offices and:
* obtaining input from all regional office managers and training
coordinators on the appropriateness of CTTR course content.
Agency Comments and Our Evaluation:
We provided a draft of this report for review and comment. In its
comments, VA generally agreed with our conclusions and concurred with
all of our recommendations. In response to our recommendation that VBA
should adopt procedures for routinely monitoring and ensuring
compliance with annual training requirements, VA highlighted the steps
that it now takes to monitor compliance and noted that VBA considers
this recommendation to be fully implemented. However, our
recommendation is aimed at ensuring that claims processors receive not
only the amount of training required, but also the types of training
required. It is not clear that VBA's current processes will address
both. VA also provided technical comments that were incorporated into
this report as appropriate.
We are sending copies of this report to the relevant congressional
committees, the Secretary of Veterans Affairs, and other interested
parties. The report is also available at no charge on GAO's Web site
at [hyperlink, http://www.gao.gov.
If you or your staffs have any questions concerning this report,
please contact me at (202) 512-7215 or bertonid@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs
may be found on the last page of this report. Staff members who made
key contributions to this report are listed in appendix V.
Signed by:
Daniel Bertoni:
Director, Education, Workforce, and Income Security Issues:
[End of section]
Appendix I: Survey Methodology:
To obtain information about the training Veterans Benefits
Administration (VBA) disability compensation and pension claims
processors, in general, received in the 12 months preceding August
2009 and their views on the amount, content, helpfulness, mode, and
timeliness of this training, we conducted a Web-based survey of a
nationally representative sample of these claims processors from June
through August 2009. Our survey methodology is described in detail
below.
Questionnaire Development and Administration:
We developed survey questions with input from officials from the VBA
Compensation and Pension Service and Office of Employee Development
and Training. We pretested versions of the draft questionnaire with a
total of 15 claims processors in a variety of positions in VBA's
Wilmington, Delaware, and Philadelphia, Pennsylvania, regional offices
and revised the questionnaire based on their comments. Once the
questionnaire was finalized, sampled claims processors were sent an
email asking them to complete the questionnaire. It was available to
them over the Web from August 12, 2009, to September 11, 2009. To
ensure a high response rate, we e-mailed follow-up requests to
nonrespondents approximately 2 and 4 weeks after the initial email.
Population and Sample Design:
Our population for this survey consisted of 8,119 disability
compensation and pension claims processors that were employed by the
agency as of May and June 2009.[Footnote 26] VBA provided us with
lists of these claims processors that we divided into three strata
based on their position:
* Veterans Service Representatives (VSR),
* Rating Veterans Service Representatives (RVSR), and:
* Supervisory VSRs.
We drew an independent random sample from each of these strata to
enable us to project survey results to all VBA claims processors, in
general, and to claims processors in each position listed above.
[Footnote 27] Table 2 shows the size of each population, the size of
the sample drawn from each, and the number of claims processors in
each stratum that responded to the survey. Of our total sample of 595
claims processors, 414 responded to the survey for an overall response
rate of 70 percent.
Table 2: Population, Sample, and Respondents by Stratum:
Claims processor group: VSR;
Population size: 5,198;
Sample size: 209;
Number of respondents: 143.
Claims processor group: RVSR;
Population size: 2,344;
Sample size: 204;
Number of respondents: 142.
Claims processor group: Supervisory VSR;
Population size: 577;
Sample size: 182;
Number of respondents: 129.
Claims processor group: Total;
Population size: 8,119;
Sample size: 595;
Number of respondents: 414.
Source: Analysis GAO survey data and information provided by VBA.
[End of table]
Estimation, Sampling, and Nonsampling Error:
To produce estimates regarding the experiences and views of claims
processors, in general, from survey responses from a sample of claims
processors, we weighted each response from claims processors in each
stratum to statistically account for all members of that stratum.
Because estimates are based on responses from a sample, each estimate
we report has a measurable precision or sampling error. The sampling
error or margin of error surrounding an estimate is expressed as a
number of percentage points higher or lower than that estimate and the
entire range the sampling error covers is referred to as a confidence
interval. Sampling errors are calculated based on a certain confidence
level. Sampling errors for estimates we report from this survey are
based on a confidence level of 95 percent and are calculated using
methods appropriate for a stratified random sample. Unless otherwise
indicated, sampling errors for estimates in this report are never
greater than plus or minus 15 percentage points. At a 95 percent
confidence level, this means that in 95 out of 100 instances the
sampling procedures we used would produce a confidence interval (in
this case, a 30 percentage point range) containing the true population
value we estimate. All numerical estimates other than percentages have
margins of error no greater than plus or minus 18 percent of the value
of those numerical estimates, unless otherwise noted.
In addition to sampling error, estimates based on survey results are
subject to what is referred to as nonsampling error that can result
from, among other things, poorly designed survey questions and
mistakes in data entry or analysis. We took a number of steps in
developing the questionnaire and in entering and analyzing the data to
minimize nonsampling error. For example, a social science survey
specialist collaborated with GAO subject matter experts in designing
the questionnaire, it was reviewed by three high level VBA officials
and, as noted above, it was pretested with a number of different types
of claims processors in two locations. In addition, when we analyzed
the data, an independent analyst verified all computer programs.
Because this was a Web-based survey, respondents' answers to survey
questions were automatically entered into an electronic file,
eliminating the need to separately key responses into a data file,
further minimizing the potential for nonsampling error.
[End of section]
Appendix II: Training Topics Included in GAO's Survey of VBA Claims
Processors:
The following is a list of potential training topics for VBA claims
processors which we used to solicit information about the content of
training provided in our survey of VSRs, RVSRs, and supervisory RVSRs.
We based this list on VBA's Fiscal Year 2008 Core Technical Training
Requirements, VBA's Challenge Training Curriculum for new staff,
feedback received from VBA field staff during survey pretests, and
comments from officials representing VBA's Compensation and Pension
Service and Office of Employee Development and Training.
Table 3: Training Topics Included in GAO's Survey of VBA Claims
Processors:
Training topic: Records management;
Related claims processing task: Maintaining both physical claims
folders and electronic files.
Training topic: Claims recognition/ claims establishment;
Related claims processing task: Reading through a veteran's narrative,
identifying issues relevant for particular types of claims, and
establishing a computer record for the veteran's inquiry.
Training topic: Claims development: service verification;
Related claims processing task: Searching various databases for
evidence of the claimant's military service in order to determine both
the character of service and military service dates.
Training topic: Claims development: medical evidence;
Related claims processing task: Ensuring that there is sufficient
medical evidence for the ratings team to issue a rating regarding a
veteran's medical condition.
Training topic: Claims development: income;
Related claims processing task: Determining whether the veteran meets
the income thresholds for eligibility for benefits.
Training topic: Claims development: dependency;
Related claims processing task: Locating information on the claimant's
relationship and dependency, such as marital status and dissolution of
prior marriages, and parental relationships and child dependency.
Training topic: Claims development: special monthly compensation;
Related claims processing task: Determining claimants' eligibility for
special monthly compensation, based on whether the veteran has
suffered additional disability as a result of a service-connected
disability or needs special assistance.
Training topic: Claims development: burial;
Related claims processing task: Determining whether veterans are
eligible for burial benefits under certain conditions, such as their
disability occurred while in the line of duty.
Training topic: Claims development: appeals and remands;
Related claims processing task: Sending a case back for more work or
additional development is called a "remand." An appeal is a request
for a review of a Department of Veterans Affairs (VA) determination on
a claim for benefits issued by a local VA office.
Training topic: Ratings decisions: disability pension claims;
Related claims processing task: Determining if a veteran who claims
pension benefits served in a period of war, is permanently and totally
disabled for reasons not service-connected, and meets the income
thresholds for eligibility.
Training topic: Ratings decisions: disability compensation claims;
Related claims processing task: Evaluating claimed disabilities and
assigning ratings based on degree of disability.
Training topic: Ratings decisions: dependency and indemnity
compensation, and accrued benefits;
Related claims processing task: Determining whether claimants are
eligible for the Dependency and Indemnity Compensation program, which
is only for survivors of veterans whose deaths occurred while on
active duty, are service-connected, or follow from a period of
permanent and total service-connected disability.
Training topic: Ratings decisions: ancillary benefits and special
purposes;
Related claims processing task: Determining whether claimants are
eligible for ancillary benefits, which might include automobile and
adaptive equipment allowances, special housing adaptation grants, and
a clothing allowance.
Training topic: Ratings decisions: special monthly compensation;
Related claims processing task: Determining entitlement to special
monthly compensation is based on medical evidence, which raters must
evaluate.
Training topic: Ratings decisions: total disability due to
hospitalization or convalescence;
Related claims processing task: Issuing a temporary rating when a
veteran experiences a prolonged hospitalization or convalescence.
Training topic: Claims processing: payment amount;
Related claims processing task: Determining the amount of a basic
monthly payment based on the percentage on the veteran's disability.
Training topic: Claims processing: payment authorization;
Related claims processing task: Authorizing the amount of the benefit
based on veteran's disability.
Training topic: Evaluation of disabilities based on routine future
examinations;
Related claims processing task: Requesting medical re- examinations in
order to verify either the continued existence or the current severity
of a disability.
Training topic: System-generated messages (write-outs);
Related claims processing task: Processing computer-generated error
message that alert VA regional offices to complete actions that ensure
the accuracy of compensation payments and case tracking information.
Training topic: Due process and duty to inform;
Related claims processing task: Assisting veterans, as required by
law, by providing due process and informing them of how to develop
well-grounded claims, and when relevant records cannot be located.
Training topic: Workload and time management;
Related claims processing task: Accurately processing claims while
meeting daily productivity and annual training requirements.
Training topic: Telephone development and customer service skills;
Related claims processing task: Interacting in a professional manner
with the public via telephone or directly at Veterans Service Centers.
Training topic: Computer skills and Web navigation;
Related claims processing task: Operating various computer programs,
such as the Benefits Delivery Network and MAP-D (Modern Award
Processing Development), which are unique to VA, and navigating the
Internet.
Training topic: Supervision;
Related claims processing task: Developing staff, maintaining team
morale, and assigning and reviewing the work of other claims
processors, as assigned.
Source: GAO analysis of information provided by VBA and GAO survey
questionnaire.
[End of table]
[End of section]
Appendix III: VBA Fast Letter 10-05, Revision of Core Technical
Training Requirement Hours for VSRs and RVSRs:
Department Of Veterans Affairs:
Veterans Benefits Administration:
Washington, D.C. 20420:
January 15, 2010:
Director (00/21):
All VA Regional Offices and Centers:
Fast Letter 10-05:
Subject: Revision of Core Technical Training Requirement (CTTR) Hours
for Veterans Service Representatives (VSRs) and Rating Veterans
Service Representatives (RVSRs).
Purpose:
There is a critical need to ensure national training efforts are
consistent and aligned with VA's organizational goals. Veterans
Benefits Administration (VBA) field offices provide specific employee
training which enhances delivery of quality service to Veterans. Each
office is responsible for establishing training plans at the beginning
of every fiscal year and for ensuring employees complete the
designated number of Core Technical Training Requirement (CTTR) hours.
The C&P Training Staff has been working with the Systematic Technical
Accuracy Review (STAR) Quality Staff to use STAR findings to develop a
standardized national training agenda. The result is an 85-hour
curriculum, specific to level of experience, for VSRs and RVSRs. This
curriculum includes mandatory technical training courses, while
allowing flexibility for local office needs.
Background:
Technical training is content that is directly related to an
employee's function in the organization. Each position in the VBA has
technical training requirements; however, this document addresses the
FY 2010 training hour requirements for only the VSR and RVSR positions.
Procedure:
Both the VSR and RVSR positions are highly standardized positions, and
as RVSR or VSR experience levels increase, their ability and knowledge
also increases. Because of the varying level of knowledge expected at
each of the skill levels, experience level-specific training, has been
developed. This training is identified as follows: Entry Level ” in
position less than 6 months; Intermediate Level ” in position 6 to 24
months; and Journey Level ” in position over 24 months.
Beginning in FY 2010, all VSRs and RVSRs who have completed the
Challenge curriculum and/or have more than six months in their job
positions will be required to complete a minimum of 85 hours of
technical training. The C&P Training Staff will make available
specific training materials for 40 hours of mandatory technical
training. This will standardize at least half of the required training
for all VSRs and RVSRs. An additional 20 hours of technical training
must be selected from the standardized instructional content available
on the C&P Training website. The remaining 25 hours will be reserved
for station-determined issues/topics. VA mandatory training, such as
Cyber Security and Ethics, will count as part of the 25 hours of
locally directed training.
The mandatory training topics along with the associated training
materials are being posted on the C&P Training Website as they are
completed. Topics and materials for the entire 85 hours of the
curriculum will be available by the end of January 2010.
The CTTR courses will be entered as items in the VA-Learning
Management System (LMS). The LMS will be updated as additional
training materials are developed and released to the field. The
revised CUR will reflect the appropriate curriculum based on two skill
levels (Intermediate Level and Journey Level) for both the VSR and
RVSR positions.
The C&P Service is working with ED&T to develop a mandatory course
evaluation protocol for CUR training. In the near future this
evaluation protocol will be incorporated into the CUR training program
for the purpose of ensuring CUR compliance and evaluating the
effectiveness and quality of the standardized training curriculum.
Additional guidance will be provided once the mandatory course
evaluation protocol is ready for deployment.
Questions:
E-mail questions concerning this letter to VAVBAWAS/CO/C&PTraining.
Signed by:
Bradley G. Mayes:
Director:
Compensation and Pension Service:
[End of section]
Appendix IV: Comments from the Department of Veterans Affairs:
Department of Veterans Affairs:
Office of the Secretary:
April 21, 2010:
Mr. Daniel Bertoni:
Director:
Education, Workforce, and Income Security Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Bertoni:
The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, Veterans' Disability
Benefits: Expanded Oversight Would Improve Training for Experienced
Claims Processors (GAO-10-445) and generally agrees with GAO's
conclusions and concurs with GAO's recommendations to the Department.
At the initiation of this review, the Veterans Benefits Administration
(VBA) informed GAO of the Compensation and Pension (C&P) Service's
initiative to revise the Core Technical Training Requirements (CUR)
for experienced claims processors. This initiative was already in the
developmental phase when GAO began the review.
VBA has an active program to support the development of training for
experienced claims processors. This program has grown significantly
over the past six months, and VBA has made substantial progress in
using the VA Learning Management System (LMS) to manage training
activities.
In order to restructure the CUR for experienced claims processors, C&P
staff reviewed National STAR quality analyses and the results of two
studies released by the Office of Employee Development and Training.
Feedback concerning field needs and local "best practices" was also
obtained during the July 2009 national Training Managers Conference
and from regional office employees who served as centralized training
instructors during FY 2009. Since July 2009, C&P site visits to
regional offices include a member of the C&P training staff
specifically to gather and evaluate information regarding training
issues and compliance.
The GAO draft report refers frequently to the results of its survey of
VBA claims processing staff administered between August 12, 2009, and
September 11, 2009. VBA completed its training review and implemented
the CUR curriculum changes effective January 15, 2010. Prior to
completion of the review, GAO was provided detailed information
regarding the curriculum changes for claims processors. VBA will
continue to improve and expand the CTTR training curriculum as needed.
VBA continues to place a high priority on training for claims
processors. Actions to address the areas of concern identified by GAO
are already underway. The enclosure specifically addresses GAO's
recommendations. VA appreciates the opportunity to comment on your
draft report.
Sincerely,
Signed by:
John R. Gingrich:
Chief of Staff:
Enclosure:
[End of letter]
Enclosure:
Department of Veterans Affairs (VA) Comments to Government
Accountability Office (GAO) Draft Report: Veterans' Disability
Benefits: Expanded Oversight Would Improve Training for Experienced
Claims Processors (GA0-10-445):
GAO Recommendation: We recommend that the Secretary of Veterans
Affairs direct the Undersecretary for Benefits to take the following
three actions:
Recommendation 1: To ensure that claims processors receive the types
of training required by VBA each year, VBA should adopt procedures for
routinely monitoring and ensuring compliance with annual training
requirements, including more fully using its LMS to ensure that claims
processors receive required CUR training, and required ad hoc training
on emerging topics.
VA Response: Concur. Monitoring compliance with training requirements
is a collaborative effort between the Office of Field Operations
(OFO), the Office of Employee Development and Training (ED&T), and the
appropriate business line that determines the training requirements.
OFO monitors regional office (RO) compliance through area office site
visits. During area office site visits, Area Directors meet with the
RO Director and Training Manager to discuss the RO's training plan,
training compliance, current training status of employees, and
appropriate follow-up actions. Area offices review the RO's Learning
Management System (LMS) summary report to ensure proper documentation
of training and verify this through employee spot checks. Area offices
may also use LMS to conduct additional monitoring of training
compliance for ROs that have training concerns. The Area Director
addresses any concerns with the RO Director and Training Manager to
ensure proper action is taken.
Beginning in FY 2010, ED&T provides quarterly LMS summary reports
directly to OFO. These reports are issued to VBA RO leaders providing
the percentage of claims processors meeting annual training
requirements. Area Director and RO Directors conduct local analyses,
implement follow-up actions as necessary, and report results to OFO
and ED&T for review. VBA Central Office recently developed a template
for future use by all ROs to report on compliance with training
requirements. Beginning in the fourth quarter of FY 2010, ROs will
submit quarterly reports to OFO and ED&T for review to ensure
compliance with annual training requirements. VBA considers this
recommendation fully implemented.
Recommendation 2: To ensure that regional offices record training
activities consistently in the LMS so that LMS data are reliable, VBA
should develop clear written guidance on the types of activities all
regional offices should and should not count toward completion of
annual training requirements.
VA Response: Concur. During 2009, the Compensation and Pension (C&P)
Service identified targeted areas for training and began developing
national training materials to provide standardized training for
experienced claims processors. On January 15, 2010, VBA published Fast
Letter 10-05 (Attachment. A), which directed ROs to provide training
selected from a list of national training topics. VBA published
standardized training materials for claims processors and established
national LMS item numbers to ensure that regional offices record
activities consistently in LMS. The standardized item numbers are
available within LMS and on the relevant VBA website.
C&P Service is developing a methodology with criteria for use in
defining the types of locally-developed activities that should and
should not count toward completion of training requirements. These
training activities will also be recorded in LMS. This process is
expected to be implemented by September 30, 2010.
Recommendation 3: To ensure that training is appropriate for
experienced claims processors, VBA should develop and implement a
written strategy for systematically assessing the content, mode and
timing of training experienced claims processors receive in regional
offices and make improvements, when indicated. Such a strategy should
include a standardized approach for:
* obtaining and analyzing feedback from experienced claims processors
on the appropriateness of non-CTTR training provided by regional
offices, and;
* obtaining input from all regional office managers and training
coordinators on the appropriateness of CUR course content.
VA Response: Concur. In January 2010, VBA developed and published a
strategy for systematically assessing the national training required
of claims processors, Monitoring and Evaluation Plan for Core
Technical Training Requirements (CTTR) (Attachment B). During the
February 24, 2010, Associate Deputy Under Secretary for Field
Operations call, VBA announced implementation of this plan and the
mandatory course evaluation protocol for use by all employees who
receive CUR training. Employee feedback will be used to improve
training for experienced claims processors. These improvements may
include changes to the content, mode, and timing of training. VBA will
also explore the feasibility of using the evaluation tool to obtain
feedback on locally developed training and other non-CUR course
content. VBA will also consider using a similar evaluation protocol to
obtain feedback from other sources such as managers and training
coordinators. VBA expects to complete the assessment of development of
these remaining evaluation tools by September 30, 2010.
Attachment A:
Department Of Veterans Affairs:
Veterans Benefits Administration:
Washington, D.C. 20420:
January 15, 2010:
Director (00/21):
All VA Regional Offices and Centers:
Fast Letter 10-05:
Subject: Revision of Core Technical Training Requirement (CTTR) Hours
for Veterans Service Representatives (VSRs) and Rating Veterans
Service Representatives (RVSRs).
Purpose:
There is a critical need to ensure national training efforts are
consistent and aligned with VA's organizational goals. Veterans
Benefits Administration (VBA) field offices provide specific employee
training which enhances delivery of quality service to Veterans. Each
office is responsible for establishing training plans at the beginning
of every fiscal year and for ensuring employees complete the
designated number of Core Technical Training Requirement (CTTR) hours.
The C&P Training Staff has been working with the Systematic Technical
Accuracy Review (STAR) Quality Staff to use STAR findings to develop a
standardized national training agenda. The result is an 85-hour
curriculum, specific to level of experience, for VSRs and RVSRs. This
curriculum includes mandatory technical training courses, while
allowing flexibility for local office needs.
Background:
Technical training is content that is directly related to an
employee's function in the organization. Each position in the VBA has
technical training requirements; however, this document addresses the
FY 2010 training hour requirements for only the VSR and RVSR positions.
Procedure:
Both the VSR and RVSR positions are highly standardized positions, and
as RVSR or VSR experience levels increase, their ability and knowledge
also increases. Because of the varying level of knowledge expected at
each of the skill levels, experience level-specific training, has been
developed. This training is identified as follows: Entry Level ” in
position less than 6 months; Intermediate Level ” in position 6 to 24
months; and Journey Level ” in position over 24 months.
Beginning in FY 2010, all VSRs and RVSRs who have completed the
Challenge curriculum and/or have more than six months in their job
positions will be required to complete a minimum of 85 hours of
technical training. The C&P Training Staff will make available
specific training materials for 40 hours of mandatory technical
training. This will standardize at least half of the required training
for all VSRs and RVSRs. An additional 20 hours of technical training
must be selected from the standardized instructional content available
on the C&P Training website. The remaining 25 hours will be reserved
for station-determined issues/topics. VA mandatory training, such as
Cyber Security and Ethics, will count as part of the 25 hours of
locally directed training.
The mandatory training topics along with the associated training
materials are being posted on the C&P Training Website as they are
completed. Topics and materials for the entire 85 hours of the
curriculum will be available by the end of January 2010.
The CTTR courses will be entered as items in the VA-Learning
Management System (LMS). The LMS will be updated as additional
training materials are developed and released to the field. The
revised CUR will reflect the appropriate curriculum based on two skill
levels (Intermediate Level and Journey Level) for both the VSR and
RVSR positions.
The C&P Service is working with ED&T to develop a mandatory course
evaluation protocol for CUR training. In the near future this
evaluation protocol will be incorporated into the CUR training program
for the purpose of ensuring CUR compliance and evaluating the
effectiveness and quality of the standardized training curriculum.
Additional guidance will be provided once the mandatory course
evaluation protocol is ready for deployment.
Questions:
E-mail questions concerning this letter to VAVBAWAS/CO/C&PTraining.
Signed by:
Bradley G. Mayes:
Director:
Compensation and Pension Service:
[End of Attachment A]
Attachment B:
CTTR Plan:
January 2010:
Monitoring And Evaluation Plan For Core Technical Training
Requirements (CTTR):
January 2010:
Compensation and Pension Service ” Training Office:
Veterans Benefits Administration:
Table of Contents:
Purpose:
Procedure:
Background:
CTTR Requirements:
CUR Course Evaluation:
CTI-R Reporting:
Monitoring Compliance:
Evaluating the Standardized CTTR Training Curriculum:
Appendix A: CUR Course Evaluation:
Purpose:
To specify the general processes for 1) monitoring core technical
training requirement (CTTR) compliance and 2) evaluating the
effectiveness and quality of the standardized C 11'K training
curriculum.
Procedure:
Background:
CTTR Requirements:
All VSR and RVSR employees are required to complete a series of
training courses in order to satisfy their CTTR (JAW Fast Letter 10-05
and Training Letter TED).
All VSR and RVSR employees must submit an on-line Cl IR Course
Evaluation immediately upon completing each training course.
Each Regional Office (RO) is responsible for ensuring employees
complete the designated number of Cf1R hours. ROs must certify their
employees' fulfillment of training hours by entering information into
the VA - Learning Management System (LMS) for each training course
that is not automatically entered by a computer system.
CTTR Course Evaluation:
C&P Service will maintain a mandatory online CUR Course Evaluation
(see appendix A) for the purpose of ensuring CTTR compliance and
evaluating the effectiveness and quality of the standardized training
curriculum.
C&P is responsible for coordinating the promulgation of both these
requirements and how to access the online CTTR Course Evaluation.
CTTR Reporting:
C&P Service will use an online reporting system and a "dashboard" to
monitor responses to the CTTR Course Evaluation for both compliance
and evaluation as required by Fast Letter 10-05.
Monitoring Compliance:
C&P Service will monitor compliance with the training and evaluation
requirement by:
1. Verifying the completion of the prescribed number of hours of
training.
2. Comparing the number of evaluations submitted for each training
course to the number of employees certified on the LMS as having
fulfilled each training course. (Note: the ability to compare surveys
with LMS entries will be limited during the pilot year.)
3. Examining the frequency of employee response choices to particular
items on the CTTR Course Evaluation to ensure that employees are being
properly provided the prescribed training. For example, C&P service
will monitor the number of employees who indicate either that a
training course took a lot less than the allotted time to complete or
that they did not know the name of a training course, its objectives,
or the time allotted to the training.
C&P Service will monitor CUR compliance both by RO and by area. Areas
will be identified as having compliance concerns based on the number
of their ROs flagged as non-compliant.
Evaluating the Standardized CTTR Training Curriculum:
In accordance with Fast Letter 10-05, C&P Service will evaluate the
effectiveness and quality of the standardized CUR training curriculum
by analyzing employees' responses on the CUR Course Evaluation. C&P
Service will keep track of:
1. How useful employees think each course is for teaching them new
knowledge or skills.
2. How relevant they think each course is to their job.
3. How worthwhile they think the course is.
4. How well the training achieved its objectives.
5. How well the course was presented/delivered.
6. The quality of the facilities in which the training was conducted.
ED&T and TT&E Will provide technical assistance with the surveys, and
a summary of the evaluation data at intervals identified by C&P
Training.
Appendix A: CTTR Course Evaluation:
CTTR Course Evaluation:
The VBA is committed to providing quality training for its employees.
Please help us do so by providing your opinion about the training
course you just completed.
Training Course Completed: (Select an option).
Regional Office: (Select an option).
Experience Level: (Select an option).
Note: * indicates a mandatory question.
* 1. How useful was this course for teaching you new knowledge,
skills, or for refreshing your memory about important facts; policies,
procedures, or duties?
Not at All:
Minimally:
Moderately:
Mostly:
Extremely:
Not Applicable:
*2. How relevant is the content presented in this course to your job?
Not at All:
Minimally:
Moderately:
Mostly:
Extremely:
Not Applicable:
*3. If you think the content is at least minimally relevant, how
confident are you that you will be able to apply the knowledge and/or
skills to your job?
Not at All:
Minimally:
Moderately:
Mostly:
Extremely:
Not Applicable:
*4. How worthwhile is the VBA's requirement for you to complete this
course?
Not at All:
Minimally:
Moderately:
Mostly:
Extremely:
Not Applicable:
*5. How well did this course achieve its objective(s)?
Not at All:
Minimally:
Moderately:
Mostly:
Extremely:
Not Applicable:
*6. How well was the course presented?
Not at All:
Minimally:
Moderately:
Mostly:
Extremely:
Not Applicable:
7. Please offer any comments you may have about the content and/or
presentation of the course:
8. Please rate the training facilities (if your training was conducted
in a classroom setting):
Poor:
Fair:
Good:
Excellent:
Not Applicable:
9. Please offer any comments you may have about the training
facilities:
*10. Did you have any issue(s) during this course?
No issue(s) at all:
Only minor issue{s):
Fairly serious issue(s):
Very serious issue(s):
*11. Please describe the issue(s). If you selected "No issue(s) at
all," please enter, "Not applicable."
*12. How long did it take you to complete this course?
(Select an option):
13. If you took a lot more or a lot less than the allotted time to
complete the course, please describe why.
Thank you for completing this questionnaire.
Submit:
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Daniel Bertoni, (202) 512-7215 or bertonid@gao.gov:
Staff Acknowledgments:
Clarita Mrena (Assistant Director), Regina Santucci, Martin Scire,
Julie Corwin, and Lisa Fisher made significant contributions to this
report. In addition, Greg Wilmoth and Stuart Kaufman provided guidance
on research design and methodology, Dae Park assisted with statistical
methodology, Joanna Chan assisted with data analysis, Roger Thomas
provided legal counsel, Susan Aschoff helped with report preparation,
and James Bennett assisted with graphics.
[End of section]
Related GAO Products:
Veterans' Disability Benefits: Further Evaluation of Ongoing
Initiatives Could Help Identify Effective Approaches for Improving
Claims Processing. [hyperlink,
http://www.gao.gov/products/GAO-10-213]. Washington, D.C.: January 29,
2010.
Veterans' Disability Benefits: Preliminary Findings on Claims
Processing Trends and Improvement Efforts. [hyperlink,
http://www.gao.gov/products/GAO-09-910T]. Washington, D.C.: July 29,
2009.
Veterans' Benefits: Increased Focus on Evaluation and Accountability
Would Enhance Training and Performance Management for Claims
Processors. [hyperlink, http://www.gao.gov/products/GAO-08-561].
Washington, D.C.: May 27, 2008.
Veterans' Benefits: Improvements Needed in VA's Training and
Performance Management Systems. [hyperlink,
http://www.gao.gov/products/GAO-08-1126T]. Washington, D.C.: September
18, 2008.
Veterans' Benefits: Improved Management Would Enhance VA's Pension
Program. [hyperlink, http://www.gao.gov/products/GAO-08-112].
Washington, D.C.: February 14, 2008.
Veterans' Disability Benefits: Claims Processing Challenges Persist,
while VA Continues to Take Steps to Address Them. [hyperlink,
http://www.gao.gov/products/GAO-08-473T]. Washington, D.C.: February
14, 2008.
Veterans' Benefits: VA Needs Plan for Assessing Consistency of
Decisions. [hyperlink, http://www.gao.gov/products/GAO-05-99].
Washington, D.C.: November 19, 2004.
Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government. [hyperlink,
http://www.gao.gov/products/GAO-04-546G]. Washington, D.C.: March 2004.
Human Capital: Selected Agencies' Experiences and Lessons Learned in
Designing Training and Development Programs. [hyperlink,
http://www.gao.gov/products/GAO-04-291]. Washington, D.C.: January 30,
2004.
Veterans' Benefits: Improvements Needed in the Reporting and Use of
Data on the Accuracy of Disability Claims Decisions. [hyperlink,
http://www.gao.gov/products/GAO-03-1045]. Washington, D.C.: September
30, 2003.
Veterans' Benefits: Training for Claims Processors Needs Evaluation.
[hyperlink, http://www.gao.gov/products/GAO-01-601]. Washington, D.C.:
May 31, 2001.
[End of section]
Footnotes:
[1] VA, Office of Inspector General, Audit of Veterans Benefits
Administration Compensation Rating Accuracy and Consistency Reviews,
08-02073-96 (Washington, D.C., Mar. 12, 2009).
[2] Pub. L. No. 110-389 § 225(b).
[3] See GAO, Veterans Benefits: Increased Focus on Evaluation and
Accountability Would Enhance Training and Performance Management for
Claims Processors, [hyperlink, http://www.gao.gov/products/GAO-08-561]
(Washington, D.C.: May 27, 2008).
[4] While the focus is on experienced staff, we provide some
descriptive information about new staff and some survey results for
new staff for comparative purposes.
[5] For this review, we define experienced staff as those with 2 years
or more of experience processing VA disability claims. New staff
refers to staff with less than 2 years of experience.
[6] Unless otherwise indicated, the margin of error for percentage
estimates based on this survey cited in the report are within plus or
minus 15 percentage points at the 95 percent confidence level. All
numerical estimates other than percentages have margins of error no
greater than plus or minus 18 percent of the value of those numerical
estimates, unless otherwise noted.
[7] See GAO, Human Capital: A Guide for Assessing Strategic Training
and Development Efforts in the Federal Government, [hyperlink,
http://www.gao.gov/products/GAO-04-546G] (Washington, D.C.: March
2004), and Standards for Internal Control in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3] (Washington,
D.C.: November 1999).
[8] VSRs and RVSRs were employed as of May 23, 2009. Supervisory VSRs
were employed as of June 6, 2009.
[9] Disability compensation is a monthly cash payment to veterans who
are disabled by an injury or illness incurred or aggravated during
active military service. Payment amount is determined by the severity
of the disability and the number of eligible dependents in each case.
A disability pension is a monthly cash payment to low-income veterans
who are permanently and totally disabled, or elderly, and meet certain
minimum service requirements. Only veterans who are honorably
discharged from the military can receive disability compensation or
pension benefits. About 165,000--or 17 percent--of the estimated
975,000 disability claims VBA received in fiscal year 2009 were for
disability pension benefits. The remainder was for disability
compensation benefits.
[10] VBA's year-end pending inventory of claims has grown nearly 35
percent since 2005, from about 309,000 in fiscal year 2005 to about
416,000 in fiscal year 2009.
[11] The survey was administered from August 12, 2009, through
September 11, 2009 (see appendix I).
[12] Challenge consists of a uniform curriculum that is implemented in
three phases: initial orientation training provided at a participant's
regional office; centralized classroom instruction typically delivered
at VBA's Training Academy in Baltimore, Maryland; and comprehensive on-
the-job and classroom training that new claims processors receive at
their regional offices.
[13] According to one VBA official, requiring all experienced claims
processors to receive training in the same VBA-designated topics will
better target training across regional offices to address problems in
claims processing identified by headquarters' quality assurance
procedures.
[14] VBA's newly instituted categories differ from those used in our
analysis. For this review, we define new staff as those with less than
2 years in their current position, and experienced staff as those with
2 or more years in their current position.
[15] In January 2010, VBA officially added 5 hours to the 80-hour
annual training requirement on topics such as cyber security and
ethics that were already mandatory for disability claims processors.
[16] An 80-hour annual training requirement may be appropriate for
some, in particular new, staff. An estimated 70 percent (ranging from
52 to 84 percent at a 95 percent confidence level) of all supervisors
of new RVSRs and 62 percent of supervisors of new VSRs thought that
all or almost all of the new staff they supervise needed 80 hours of
training.
[17] The Dependency and Indemnity Compensation program is for
survivors of veterans whose deaths occurred on active duty, are
service-connected, or follow from a period of permanent and total
service-connected disability.
[18] This is consistent with staff views we reported in our previous
report in this area. At that time staff at three regional offices said
the training topics set by the Compensation and Pension Service are
designed for new staff and do not change much from year to year, so
experienced staff end up repeating courses. See [hyperlink,
http://www.gao.gov/products/GAO-08-561], p. 21.
[19] A presentation on TBI was available on VBA's intranet site.
[20] [hyperlink, http://www.gao.gov/products/GAO/AIMD-00-21.3].
[21] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[22] This training, delivered in September 2008, was undertaken to
clarify requirements resulting from DeLuca v. Brown, 8 Vet. App. 202
(1995), in which the court held that under federal regulations
defining joint and spine impairment severity in terms of limits on
range of motion, VA claims adjudicators must consider whether range of
motion is further limited by factors such as pain and fatigue during
"flare-ups" or following repetitive use of the impaired joint or spine.
[23] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[24] VBA began conducting inter-rater reliability studies in 2008 to
assess how consistent raters from all regional offices are on
eligibility determinations on specific types of disabilities. A sample
is selected from RVSRs and decision review officers from all regional
offices.
[25] Our survey found that about 65 percent of staff found TPSS
somewhat to very easy to use. However, only 21 percent said that TPSS
helped them, to a great or very great extent, become familiar with
basic information needed to handle claims.
[26] Veterans Service Representatives (VSRs) and Rating Veterans
Service Representatives were employed as of May 23, 2009. Supervisory
VSRs were employed as of June 6, 2009.
[27] We over sampled from each stratum to take into account sampled
claims processors who might have been incorrectly included in the
original population and sampled claims processors who would not
respond to the survey.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
