Veterans' Benefits
Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors Gao ID: GAO-08-561 May 27, 2008Faced with an increase in disability claims, the Veterans Benefits Administration (VBA) is hiring a large number of new claims processing staff. We were asked to determine: (1) What training is provided to new and experienced claims processors and how uniform is this training? (2) To what extent has VBA planned this training strategically, and how well is the training designed, implemented, and evaluated? and (3) To what extent is the performance management system for claims processors consistent with generally accepted practices? To answer the questions, GAO reviewed documents including VBA policies and training curricula; interviewed VBA central office officials; visited 4 of VBA's 57 regional offices, which were selected to achieve diversity in geographic location, number of staff, and officewide accuracy in claims processing; and compared VBA's training and performance management to generally accepted practices identified by GAO.
VBA has a standardized training curriculum for new staff and a training requirement for all staff, but does not hold staff accountable for meeting this requirement. The curriculum for new staff includes what is referred to as centralized training and training at their home offices. All claims processors must complete 80 hours of training annually, which may cover a mix of topics identified centrally and by regional offices. Individual staff members face no consequences for failing to meet the training requirement, however, and VBA has not tracked training completion by individuals. It is implementing a new system that should provide this capacity. Although VBA has taken steps to plan its training strategically, the agency does not adequately evaluate training and may be falling short in training design and implementation. VBA has a training board that assesses its overall training needs. However, the agency does not consistently collect feedback on regional office training, and both new and experienced staff GAO interviewed raised issues with their training. Some new staff raised concerns about the consistency of training provided by different instructors and about the usefulness of an on-line learning tool. Some experienced staff believe that 80 hours of training annually is not necessary, some training was not relevant for them, and workload pressures impede training. The performance management system for claims processors generally conforms to GAO-identified key practices, but the formula for assigning overall ratings may prevent managers from fully acknowledging and rewarding staff for higher levels of performance. The system aligns individual and organizational performance measures and requires that staff be given feedback throughout the year. However, VBA officials raised concerns about the formula used to assign overall ratings. Almost all staff in the offices GAO visited were placed in only two of five overall rating categories, although managers said greater differentiation would more accurately reflect actual performance differences. The Department of Veterans Affairs (VA) has not examined the ratings distribution, but acknowledges a potential issue with its formula and is considering changes.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-08-561, Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors
This is the accessible text file for GAO report number GAO-08-561
entitled 'Veterans' Benefits: Increased Focus on Evaluation and
Accountability Would Enhance Training and Performance Management for
Claims Processors' which was released on May 27, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Committee on Veterans' Affairs, House of
Representatives:
United States Government Accountability Office:
GAO:
May 2008:
Veterans' Benefits:
Increased Focus on Evaluation and Accountability Would Enhance Training
and Performance Management for Claims Processors:
GAO-08-561:
GAO Highlights:
Highlights of GAO-08-561, a report to the Chairman, Committee on
Veterans' Affairs, House of Representatives.
Why GAO Did This Study:
Faced with an increase in disability claims, the Veterans Benefits
Administration (VBA) is hiring a large number of new claims processing
staff. We were asked to determine: (1) What training is provided to new
and experienced claims processors and how uniform is this training? (2)
To what extent has VBA planned this training strategically, and how
well is the training designed, implemented, and evaluated? and (3) To
what extent is the performance management system for claims processors
consistent with generally accepted practices? To answer the questions,
GAO reviewed documents including VBA policies and training curricula;
interviewed VBA central office officials; visited 4 of VBA‘s 57
regional offices, which were selected to achieve diversity in
geographic location, number of staff, and officewide accuracy in claims
processing; and compared VBA‘s training and performance management to
generally accepted practices identified by GAO.
What GAO Found:
VBA has a standardized training curriculum for new staff and a training
requirement for all staff, but does not hold staff accountable for
meeting this requirement. The curriculum for new staff includes what is
referred to as centralized training and training at their home offices.
All claims processors must complete 80 hours of training annually,
which may cover a mix of topics identified centrally and by regional
offices. Individual staff members face no consequences for failing to
meet the training requirement, however, and VBA has not tracked
training completion by individuals. It is implementing a new system
that should provide this capacity.
Although VBA has taken steps to plan its training strategically, the
agency does not adequately evaluate training and may be falling short
in training design and implementation. VBA has a training board that
assesses its overall training needs. However, the agency does not
consistently collect feedback on regional office training, and both new
and experienced staff GAO interviewed raised issues with their
training. Some new staff raised concerns about the consistency of
training provided by different instructors and about the usefulness of
an on-line learning tool. Some experienced staff believe that 80 hours
of training annually is not necessary, some training was not relevant
for them, and workload pressures impede training.
The performance management system for claims processors generally
conforms to GAO-identified key practices, but the formula for assigning
overall ratings may prevent managers from fully acknowledging and
rewarding staff for higher levels of performance. The system aligns
individual and organizational performance measures and requires that
staff be given feedback throughout the year. However, VBA officials
raised concerns about the formula used to assign overall ratings.
Almost all staff in the offices GAO visited were placed in only two of
five overall rating categories, although managers said greater
differentiation would more accurately reflect actual performance
differences. The Department of Veterans Affairs (VA) has not examined
the ratings distribution, but acknowledges a potential issue with its
formula and is considering changes.
Figure: Fiscal Year 2007 Appraisals for Four Offices Were Concentrated
in Two Categories:
[See PDF for image]
This figure is a stacked vertical bar graph depicting the following
data:
Office: Atlanta;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 81%;
Excellent: 10%;
Outstanding: 9%.
Office: Baltimore;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 66%;
Excellent: 10%;
Outstanding: 24%.
Office: Milwaukee;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 70%;
Excellent: 2%;
Outstanding: 28%.
Office: Portland;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 72%;
Excellent: 2%;
Outstanding: 26%.
Source: VBA regional offices.
[End of figure]
What GAO Recommends:
GAO is recommending that VBA collect feedback on training provided by
regional offices and use this feedback to further improve training, and
hold staff accountable for meeting their training requirement. GAO also
recommends that the VA assess and, if necessary, adjust its process for
placing staff in overall performance categories. In its comments, VA
agreed with GAO‘s conclusions and concurred with the recommendations.
To view the full product, including the scope and methodology, click on
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-561]. For more
information, contact Daniel Bertoni (202) 512-7215 bertonid@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
VBA Has a Uniform Training Curriculum for New Claims Processors and an
Annual Training Requirement for All Claims Processors, but Staff Are
Not Held Accountable for Meeting This Requirement:
VBA Is Taking Steps to Strategically Plan Its Training for Staff, but
Does Not Adequately Evaluate Training and May Be Falling Short in
Design and Implementation:
Performance Management System for Claims Processors Generally Conforms
to Accepted Practices, but May Not Clearly Differentiate between
Performance Levels:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Fiscal Year 2007 Core Technical Training Requirements for
VSRs and RVSRs:
Appendix III: Excerpts from Fiscal Year 2007 Training Plans for Four
Regional Offices:
Appendix IV: Agency Comments:
Appendix V: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: VBA's Disability Compensation and Pension Service's Claims
Processing Teams:
Table 2: Individual Performance Elements for VSRs and RVSRs:
Table 3: Performance Elements for VSRs and RVSRs and Corresponding
Organizational Performance Measures for VBA:
Table 4: Selected Key Questions to Consider in Assessing Agency's
Training Program:
Table 5: Selected Accepted Practices for Effective Performance
Management Systems:
Table 6: Excerpt from Atlanta Regional Office Training Plan:
Table 7: Excerpt from Baltimore Regional Office Training Plan:
Table 8: Excerpt from Milwaukee Regional Office Training Plan:
Table 9: Excerpt from Portland Regional Office Training Plan:
Figures:
Figure 1: Phases of Training for New VSRs and RVSRs:
Figure 2: Excerpt from the Instructor's Guide for a Centralized
Training Lesson on Reference Materials:
Figure 3: Most Fiscal Year 2007 Training Hours Completed by Claims
Processors in the Offices We Visited Were in the Form of Classroom
Instruction:
Figure 4: Sample of VBA's Centralized Training Evaluation Form:
Figure 5: VA Overall Performance Appraisal Formula:
Figure 6: Fiscal Year 2007 Overall Performance Ratings for Claims
Processors in Four Regional Offices Were Concentrated in the
Outstanding and Fully Successful Categories:
Figure 7: Regional Offices Selected for Site Visits:
Figure 8: Four Components of the Training and Development Process:
Abbreviations:
RVSR: Rating Veterans Service Representative:
STAR: Systematic Technical Accuracy Review:
TPSS: Training and Performance Support System:
VA: Department of Veterans Affairs:
VBA: Veterans Benefits Administration:
VSR: Veterans Service Representative:
[End of section]
United States Government Accountability Office: Washington, DC 20548:
May 27, 2008:
The Honorable Bob Filner:
Chairman:
Committee on Veterans' Affairs:
House of Representatives:
Dear Mr. Chairman:
The Veterans Benefits Administration (VBA) is facing an increased
volume of claims for disability benefits related to the current
conflicts in Afghanistan and Iraq as well as the aging of veterans from
past conflicts. Between fiscal years 2000 and 2006, the number of
disability-related claims filed annually with VBA increased by almost
40 percent. As a result, VBA continues to experience challenges in
processing veterans' disability claims. As of fiscal year 2007, VBA had
approximately 392,000 disability claims pending benefit decisions, and
the average time these claims were pending was 132 days. According to
VBA, the current conflicts have also produced more claims related to
post-traumatic stress disorder and traumatic brain injury, conditions
few VBA staff have had much experience evaluating. To process the
increased volume of claims, in fiscal year 2007 the agency began hiring
a large number of new Veterans Service Representatives (VSR), who
collect evidence related to veterans' claims, and Rating Veterans
Service Representatives (RVSR), who evaluate claims and determine
benefit eligibility. It plans to add 3,100 new claims-processing staff
by the end of fiscal year 2008.
Given the increased volume of claims, the increased focus on certain
types of disabilities, and the large number of new hires, training and
performance management systems for VSRs and RVSRs now play an
especially critical role in enabling VBA to meet its organizational
claims processing goals for accuracy and productivity. Training that is
properly designed and implemented is vital both to help new staff learn
their jobs and experienced staff to update their knowledge and learn
about emerging issues. An effective performance management system would
also help VBA manage its staff on a day-to-day basis to achieve its
organizational goals. To provide Congress with information on the
training and performance management of claims processors, we were asked
to determine: (1) What training is provided to new and experienced
claims processors and how uniform is this training? (2) To what extent
has VBA developed a strategic approach to planning training for claims
processors and how well is their training designed, implemented, and
evaluated? And (3) To what extent is the performance management system
for claims processors consistent with generally accepted performance
management practices in the public sector?
To address these objectives, we collected documents and data from VBA
central office and interviewed central office staff. In addition, GAO
experts on training reviewed VBA documents related to training
curriculum, lesson plans, and course evaluations. We conducted site
visits to 4 of VBA's 57 regional offices--Atlanta, Baltimore,
Milwaukee, and Portland, Oregon. These offices were selected to achieve
diversity in geographical location, number of staff, and officewide
accuracy in claims processing. While we examined VBA-wide policies and
requirements, we primarily assessed how the training and performance
management systems are implemented at four sites. Therefore, our
results may not be representative of how these systems are implemented
across all regional offices. We assessed VBA's training and performance
management practices by comparing them to certain generally accepted
practices for federal agencies in these areas that have been identified
by GAO.[Footnote 1] We conducted this performance audit from September
2007 through May 2008 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.
(See app. I for more detailed information on our objectives, scope, and
methodology.)
Results in Brief:
VBA has a standard training curriculum for new claims processors and an
80-hour annual training requirement for all claims processors, but
staff are not held accountable for meeting this requirement. VBA's
three-stage training program for new staff is intended to deliver
training in a consistent manner. First, VBA policy states that new
staff are required to complete some orientation training, which is
provided in their home offices. Second, they are required to attend a 3-
week standardized training session, referred to as centralized
training, that provides a basic introduction to their job
responsibilities. Third, new staff are required to spend several more
months in training at their home offices, which is supposed to include
on-the-job training, instructor-led training classes that follow a
required curriculum, and use of an on-line learning tool called the
Training and Performance Support System. VBA policy states that all
claims processors are required to complete a minimum of 80 hours of
training annually, and regional offices have some discretion over what
training they provide to meet this requirement. At least 60 hours must
be selected from a list of core topics identified by VBA central
office. Regional offices may choose the topics for the remaining 20
hours based on local needs, such as to prevent errors identified in
processing claims. Each regional office develops an annual training
plan listing the courses needed, and VBA central office periodically
reviews these plans and provides feedback to regional offices. Although
VBA has a training requirement for VSRs and RVSRs, it does not have a
policy outlining consequences for individual staff who do not complete
their required training. Further, VBA does not maintain data on the
training completed by individuals, but agency officials said they are
currently implementing a new, on-line learning management system that
should enable them to do so in the future.
VBA is taking steps to strategically plan its training, but does not
adequately evaluate its training and may be falling short in some areas
of training design and implementation. VBA appears to have followed
several accepted practices in planning its training, including the
establishment of a training board that assesses VBA's overall training
needs and makes recommendations to the Undersecretary for Benefits.
Also, VBA makes some effort to evaluate its centralized training for
new staff, soliciting feedback from students with forms that are well-
constructed and well-balanced. However, VBA does not require regional
offices to collect feedback on any of the training they provide to new
and experienced staff. In fact, claims processors we interviewed raised
some issues with the training they received. For example, some new
staff reported that different instructors in centralized training
sessions sometimes taught different ways of performing the same
procedure, and that one of VBA's on-line learning tools--the Training
and Performance Support System--is too theoretical and often out-of-
date. More experienced staff had mixed opinions as to whether 80 hours
of training annually is appropriate for all staff. Also, many
experienced staff indicated that training topics are redundant from
year to year, and some told us that courses available to them are not
always relevant for their position or experience level because they are
often adapted from courses for new staff. Some staff said they struggle
to meet the annual 80-hour training requirement because of workload
pressures.
The Department of Veterans Affairs' performance management system for
VSRs and RVSRs generally conforms to accepted practices, including
aligning individual and organizational performance measures, but the
system may not clearly differentiate among staff's overall performance
levels. Several elements of VSRs' and RVSRs' performance are evaluated,
and these elements are generally aligned with VBA's organizational
performance measures. For example, VSRs and RVSRs are evaluated on
their accuracy in claims processing, and one of VBA's organizational
performance measures is accuracy in claims processing. VBA's
performance management system is also consistent with other accepted
practices, such as providing performance feedback throughout the year
and emphasizing collaboration. However, the system may not clearly and
accurately differentiate among the overall performance levels of VSRs
and RVSRs. A VA-wide formula is used to translate an employee's ratings
on all individual elements into one of five overall rating categories.
Several VBA central and regional office managers raised concerns with
this formula, saying that it is difficult for staff to be placed in
certain overall performance categories, even if staff's performance
truly does fall within one of those categories. In fact, when we
reviewed the results of VSR and RVSR appraisals at the regional offices
we visited, almost all staff were placed in either the outstanding
(highest) or fully successful (middle) categories. To the extent that
the performance appraisals do not make meaningful distinctions in
performance, staff may lack the constructive feedback they need to
improve, and managers may lack the information they need to reward top
performers and address performance issues. Although VA acknowledged
this issue and indicated that it is considering changes to the system,
no formal actions have been taken to date.
We are recommending that VBA central office collect feedback on
training provided by the regional offices, to determine whether (1) 80
hours is the appropriate amount of annual training for all staff, (2)
regional offices are providing training that is relevant for all staff,
and (3) whether any changes are needed to improve the Training and
Performance Support System. We are also recommending that VBA central
office hold individual staff accountable for meeting their training
requirement and that VA assess and, if necessary, adjust its
performance rating system for staff to make it a more meaningful
management tool. In its comments, VA agreed with our conclusions and
concurred with our recommendations.
Background:
In fiscal year 2007, the Department of Veterans Affairs (VA) paid about
$37.5 billion in disability compensation and pension benefits to more
than 3.6 million veterans and their families. Through its disability
compensation program, the VBA pays monthly benefits to veterans with
service-connected disabilities (injuries or diseases incurred or
aggravated while on active military duty). Monthly benefit amounts vary
according to the severity of the disability. Through its pension
benefit program, VBA pays monthly benefits to wartime veterans with low
incomes who are either elderly or permanently and totally disabled for
reasons not service-connected. In addition, VBA pays dependency and
indemnity compensation to some deceased veterans' spouses, children,
and parents and to survivors of servicemembers who died while on active
duty.
When a veteran submits a benefits claim to any of VBA's 57 regional
offices, a Veterans Service Representative (VSR) is responsible for
obtaining the relevant evidence to evaluate the claim. For disability
compensation benefits, such evidence includes veterans' military
service records, medical examinations, and treatment records from VA
medical facilities and private providers. Once a claim is developed
(i.e., has all the necessary evidence), a Rating Veterans Service
Representative (RVSR) evaluates the claim, determines whether the
claimant is eligible for benefits, and assigns a disability rating
based on degree of impairment. The rating determines the amount of
benefits the veteran will receive. For the pension program, claims
processing staff review the veteran's military, financial, and other
records to determine eligibility. Eligible veterans receive monthly
pension benefit payments based on the difference between their
countable income, as determined by VA, and the maximum pension amounts
as updated annually by statute.[Footnote 2] In fiscal year 2007, VBA
employed over 4,100 VSRs and about 1,800 RVSRs to administer the
disability compensation and pension programs' caseload of almost 3.8
million claims.
In 2001 the VA Claims Processing Task Force noted that VSRs were
responsible for understanding almost 11,000 separate benefit delivery
tasks, such as tasks in claims establishment, claims development,
public contacts, and appeals. To improve VBA's workload controls,
accuracy rates, and timeliness, the Task Force recommended that VA
divide these tasks among a number of claims processing teams with
defined functions. To that end, in fiscal year 2002, VBA developed the
Claims Processing Improvement model that created six claims processing
teams, based on phases of the claims process. (See table 1.)
Table 1: VBA‘s Disability Compensation and Pension Service‘s Claims
Processing Teams:
Team: Triage Team;
Summary of claims processing duties: Establishes the regional office's
tracking procedures for all mail as well as processes claims that only
require a brief review to determine eligibility.
Team: Pre-Determination Team;
Summary of claims processing duties: Develops evidence for disability
ratings and prepares administrative decisions.
Team: Rating Team;
Summary of claims processing duties: Makes decisions on claims that
require consideration of medical evidence.
Team: Post-Determination Team;
Summary of claims processing duties: Develops evidence for non-rating
issues, processes benefit awards, and notifies veterans of rating
decisions.
Team: Public Contact Team;
Summary of claims processing duties: Conducts personal interviews and
handles telephone inquiries, including calls from veterans.
Team: Appeals Team;
Summary of claims processing duties: Handles requests for
reconsideration of claims where veterans have formally disagreed with
claim decisions.
Source: VBA.
Note: The Rating Board is made up of RVSRs, the Post-Determination and
Public Contact teams are made up of VSRs, and the Pre-Determination,
Triage, and Appeals teams are made up of both RVSRs and VSRs.
[End of table]
According to one VA official, new claims processing staff generally
begin as VSRs and typically have a probationary period of about one
year.[Footnote 3] After their probationary period ends, staff can
either continue to qualify to become senior VSRs or apply for RVSR
positions.[Footnote 4] VSRs are also given the option to rotate to
other VSR claim teams to gain a broader understanding of the claims
process.
VBA Has a Uniform Training Curriculum for New Claims Processors and an
Annual Training Requirement for All Claims Processors, but Staff Are
Not Held Accountable for Meeting This Requirement:
VBA has established a standardized curriculum for training new VSRs and
RVSRs on how to process claims, and it has an 80-hour annual training
requirement for both new and experienced staff; however, it does not
hold individual staff accountable for meeting this requirement. VBA has
designed a uniform curriculum for training new VSRs and RVSRs that is
implemented in three phases--initial orientation training, a 3-week
training session referred to as centralized training, and comprehensive
on-the-job and classroom training after centralizing training. It also
requires all staff to meet an annual 80-hour training requirement. To
ensure that staff meet this requirement, each regional office must
develop an annual training plan, which can contain a mix of training
topics identified by VBA central office and by the regional office.
However, individual staff members are not held accountable for meeting
their training requirement.
Training for New Staff Is Conducted in Three Stages Using a Uniform
Curriculum:
VBA has a highly structured, three-phased program for all new claims
processors designed to deliver standardized training, regardless of
training location or individual instructors. (See fig. 1.) For example,
each topic included in this training program contains a detailed lesson
plan with review exercises, student handouts, and copies of slides used
during the instructor's presentation. Each phase in this program is
designed to both introduce new material and reinforce material from the
previous phase, according to a VBA official.
Figure 1: Phases of Training for New VSRs and RVSRs:
[See PDF for image]
This figure is an illustration of the phases of training for new VSRs
and RVSRs, as follows:
Phase I: Prerequisite Training;
* Objective: Lays foundation for future training;
* Location: Home office;
* Duration: 2-3 weeks.
Phase II: Centralized Training;
* Objective: Provides overview of VSR and RVSR claims processing tasks;
* Location: A regional office or Veterans Benefits Academy, Baltimore,
MD;
* Duration: 3 weeks.
Phase III: Additional Training;
* Objective: Provides hands-on experience and builds on previous
training;
* Location: Home office;
* Duration: 6-12 months.
Source: GAO analysis.
[End of figure]
According to VBA policy, the first phase of training for new VSRs and
RVSRs is prerequisite training. New VSRs and RVSRs begin prerequisite
training at their home regional office as soon as they begin working.
Prerequisite training lays the foundation for future training by
introducing new VSRs to topics such as the software applications used
to process and track claims, medical terminology, the system for
maintaining and filing a case folder, and the process for requesting
medical records. Although VBA specifies the topics that must be covered
during prerequisite training, regional offices can choose the format
for the training and the time frame. New VSRs and RVSRs typically spend
2 to 3 weeks completing prerequisite training in their home office
before they begin the second program phase, centralized training.
During what is referred to as centralized training, new VSRs and RVSRs
spend 3 weeks in intensive classroom training. Participants from
multiple regional offices are typically brought together in centralized
training sessions, which may occur at their home regional office,
another regional office, or the Veterans Benefits Academy in Baltimore,
Maryland. According to VBA officials in three of the four offices we
visited, bringing together VSRs and RVSRS from different regional
offices helps to promote networking opportunities, while VBA officials
from two of these offices also stated that it provides a nationwide
perspective on VBA. Centralized training provides an overview of the
technical aspects of the VSR and RVSR positions. Training instructors
should follow the prescribed schedule and curriculum dictating when and
how material is taught. For example, for a particular topic, the
instructor's guide explains the length of the lesson, the instructional
method, and the materials required; lays out the information that must
be covered; and provides exercises to review the material. (See fig. 2
for a sample of an instructor's guide from the centralized training
curriculum.) Centralized training classes have at least three
instructors, but the actual number can vary depending on the size of
the group. VBA's goal is to maintain a minimum ratio of instructors to
students.
Figure 2: Excerpt from the Instructor‘s Guide for a Centralized
Training Lesson on Reference Materials:
[See PDF for image]
This figure contains the following excerpts:
Excerpt 1: Teaching guide for instructors, page 1:
Reference Materials Lesson Plan:
Identifying and Accessing VA Laws, Regulations, and Procedural
Directives:
Prerequisite Training: Prior to this training, the trainee must have
completed the Tour of the C&P Website lesson.
Purpose of Lesson: Introduce trainees to VSR reference materials
(including VA laws, regulation, manuals and directives), as well as the
tools for accessing those references (including WARMS and the Internet)
so that by the end of the lesson they will be able to:
* Competently discuss the importance of developing a good working
knowledge of the manuals and CFRs;
* Correctly list the main topics found in M21-1, Parts 1 through 7, and
the Manual Rewrite;
* Properly cite a reference under 38 CFR, M21-1 and M21-1MR;
* Demonstrate WARMS system skills, including opening the application,
selecting an index, performing a subject search, and retrieving
reference documents;
* Demonstrate reference retrieval skills using both WARMS and the
Publications page on the C&P Intranet to successfully complete the
practical exercises in the student handouts.
Time required: 8 hours.
Instructional Method: Lectures, participatory discussion, instructor
demonstration, and group practical exercise.
Materials/Training Aids:
* Computers with Intranet access for all students;
* Reference Materials Trainee Handout;
* Reference Materials PowerPoint presentation.
Instructor: Print trainee handout (from the website) prior to class.
Excerpt 2: Page 9:
Except of the Reference Materials Lesson Plan.
Source: VBA.
[End of figure]
The first week of centralized training for VSRs focuses on key
concepts, such as security, privacy and records management;
terminology; and job tools, such as the policy manual and software
applications. The final 2 weeks of training focus on the different
roles and responsibilities of VSRs on the Pre-determination and Post-
determination teams in processing claims. To practice processing
different types of claims and processing claims from start to finish,
VSRs work on either real claims or hypothetical claims specifically
designed for training. Centralized training for new RVSRs--many of whom
have been promoted from the VSR position--focuses on topics such as
systems of the human body, how to review medical records, and how to
interpret a medical exam. According to staff in one site we visited,
RVSRs new to VBA also take VSR centralized training or its equivalent
to learn the overall procedures for processing claims.
To accommodate the influx of new staff it must train, in fiscal year
2007 VBA substantially increased the frequency of centralized training
and is increasing student capacity at the Veterans Benefits Academy.
During fiscal year 2007, VBA held 67 centralized training sessions for
1,458 new VSRs and RVSRs. Centralized training sessions were conducted
at 26 different regional offices during fiscal year 2007, in addition
to the Veterans Benefits Academy. By comparison, during fiscal year
2006, VBA held 27 centralized training sessions for 678 new claims
processors.
To implement centralized training, VBA relies on qualified regional
office staff who have received training on how to be an instructor.
According to VBA officials, centralized training instructors may be
Senior VSRs, RVSRs, supervisors, or other staff identified by regional
office managers as having the capability and the right personality to
be effective instructors. Potential instructors have certain training
requirements. First, they must complete the week-long Instructor
Development Course, which covers the ways different adults learn, the
process for developing lesson plans, and the use of different training
methods and media. During this course, participants are videotaped and
given feedback on their presentation style. In addition, each time
instructors teach a centralized training session, they are supposed to
take the 2.5 day Challenge Curriculum Course, designed to update
instructors on changes to the curriculum and general training issues.
Between October 2006 and February 2008, about 250 VSRs and RVSRs from
regional offices completed the Instructor Development Course, and VBA
officials reported that, given the influx of new VSRs and RVSRs, they
are increasing the number of times this course is offered in order to
train more instructors. Instructors can teach centralized training
sessions in their home office, another regional office, or the Veterans
Benefits Academy.[Footnote 5]
When new VSRs and RVSRs return to their home office after centralized
training, they are required to begin their third phase of training,
which is supposed to include on-the-job, classroom, and computer-based
training, all conducted by and at their regional office. In the
regional offices we visited, managers indicated that new VSRs and RVSRs
typically take about 6 to 12 months after they return from centralized
training to complete all the training requirements for new staff.
During this final phase, new claims processing staff cover more
advanced topics, building on what they learned in centralized training.
Under the supervision of experienced claims processors, they work on
increasingly complex types of real claims. On-the-job training is
supplemented in the offices we visited by regular classroom training
that follows a required curriculum of courses developed by VBA's
Compensation and Pension Service, specifically for new VSRs and RVSRs.
For example, new VSRs might complete a class in processing burial
claims and then spend time actually processing such claims. The amount
of time spent working on each type of claim varies from a couple of
days to a few weeks, depending on the complexity of the claim. On-the-
job training is also supposed to be supplemented with modules from the
Training and Performance Support System (TPSS), an interactive on-line
system that can be used by staff individually or in a group.[Footnote
6] TPSS modules provide detailed lessons, practice cases, and tests for
VSRs and RVSRs. Modules for new VSRs cover topics such as burial
benefits and medical terminology; RVSR modules cover topics such as the
musculoskeletal system, general medical terminology, and introduction
to post-traumatic stress disorder.
New and Experienced Staff Have an Annual Training Requirement, and
Regional Offices Develop Training Plans That Cover a Mix of Topics
Identified Centrally and Locally:
A policy established by VBA's Compensation and Pension Service requires
both new and experienced VSRs and RVSRs to complete a minimum of 80
hours of technical training annually, double the number VBA requires of
its employees in other technical positions.[Footnote 7] VBA officials
said this higher training requirement for VSRs and RVSRs is justified
because their jobs are particularly complex and they must work with
constantly changing policies and procedures.
The 80-hour training requirement has two parts. At least 60 hours must
come from a list of core technical training topics identified by the
central office of the Compensation and Pension Service. For example,
core topics for VSRs in fiscal year 2007 included establishing veteran
status and asbestos claims development; topics for RVSRs included due
process provisions and eye-vision issues. VBA specifies more core
topics than are necessary to meet the 60-hour requirement, so regional
offices can choose those topics most relevant to their needs. They can
also choose the training method used to address each topic, such as
classroom or TPSS training. (See app. II for the list of core technical
training topics for fiscal year 2007.) Regional offices determine the
training topics that are used to meet the remaining 20 hours, based on
local needs and input. Regional offices may select topics from the list
of core technical training topics or identify other topics on their
own.
The four regional offices we visited varied in the extent to which they
utilized their discretion to choose topics outside the core technical
training topics in fiscal year 2007. Two sites selected the required 60
hours of training from the core requirements and identified their own
topics for the remaining 20 hours. In the other two sites, almost all
the training provided to staff in fiscal year 2007 was based on topics
from the list of core requirements. An official in one regional office,
for example, said that his office used its full 20 hours to provide
training on new and emerging issues that are not covered by the core
technical training topics, as well as training to address error prone
areas. An official in another regional office said the core
requirements satisfied staff training needs in fiscal year 2007,
possibly because this regional office had a large proportion of new
staff and the core topics are focused on the needs of new staff.
Regional offices must develop training plans each year that indicate
which courses will actually be provided to staff to enable them to meet
the 80-hour training requirement. The training plan is a list of
courses that the regional office plans to offer throughout the year, as
well as the expected length and number and types of participants in
each course. In the regional offices we visited, when managers develop
their training plans, they solicit input from supervisors of VSRs and
RVSRs and typically also consider national or local error trend data.
Regional offices must submit their plans to the VBA central office at
the beginning of each fiscal year for review and feedback. Central
office officials review the plans to determine whether (1) the regional
office will deliver at least 60 hours of training on the required core
topics, (2) the additional topics identified by the regional office are
appropriate, and (3) staff in similar positions within an office
receive the same level and type of training. According to central
office officials, they provide feedback to the regional offices on
their current plans as well as guidance on what topics to include in
the next year's training plans. Regional offices can adjust their
training plans throughout the year to address shifting priorities and
unexpected training needs. For example, a regional office may add or
remove courses from the plan in response to changing trends in errors
or policy changes resulting from legal decisions. (See app. III for
excerpts from the fiscal year 2007 training plans from the regional
offices we visited.)
While regional offices have discretion over the methods they use to
provide training, the four offices we visited relied primarily on
classroom training in fiscal year 2007. In each of these offices, at
least 80 percent of the total fiscal year 2007 training hours completed
by all claims processors was in the form of classroom instruction (see
fig. 3). Officials in two of the regional offices we visited said they
used lesson plans provided by the Compensation and Pension Service and
adapted these plans to the needs of their staff; one regional office
developed its own courses. An official in one office said they
sometimes invite guest speakers, and an official in another regional
office said that classroom training is sometimes delivered as part of
team meetings. The offices we visited generally made little use of
other training methods. Only one office used TPSS for its training more
than 1 percent of the time. Two offices used self-instruction--such as
reading memos from VBA central office--for about 10 percent of their
training, and no office used videos for more than 1 percent of their
training. The central office usually communicates immediate policy and
regulatory changes through memos called Fast Letters, which may be
discussed in team meetings or may just be read by staff individually.
Figure 3: Most Fiscal Year 2007 Training Hours Completed by Claims
Processors in the Offices We Visited Were in the Form of Classroom
Instruction:
[See PDF for image}
This figure is a stacked vertical bar graph depicting the following
data:
Office: Atlanta;
Classroom training: 99%;
Self-instruction: 0;
TPSS: 0;
Video: 1%.
Office: Baltimore;
Classroom training: 80%;
Self-instruction: 1%;
TPSS: 19%;
Video: 0.
Office: Milwaukee;
Classroom training: 91%;
Self-instruction: 8%;
TPSS: 0;
Video: 1%.
Office: Portland;
Classroom training: 85%;
Self-instruction: 14%;
TPSS: 1%;
Video: 0.
Source: VBA central office and regional offices.
[End of figure]
Staff Are Not Held Accountable for Meeting Their Training Requirement:
Because the agency has no policy outlining consequences for individual
staff who do not complete their 80 hours of training per year,
individual staff are not held accountable for meeting their annual
training requirement, and at present, VBA central office lacks the
ability to track training completed by individual staff members.
According to VBA officials, however, the agency is in the process of
implementing an automated system that should allow it to track the
training each staff member completes. Officials reported that this
system is expected to be implemented during fiscal year 2008. VBA
officials reported that this system will be able to record the number
of training hours and the courses completed for each individual, staff
position, and regional office. One official said the central office and
regional office supervisors will have the ability to monitor training
completed by individual staff members, but that central office will
likely not monitor the training completed by each individual staff
member, even though it may monitor the training records for a sample of
staff members. Furthermore, despite the absence of a VBA-wide tracking
system, managers in two of the regional offices we visited reported
using locally developed tracking methods to determine the number of
training hours their staff had completed.
While individuals are not held accountable, VBA reported taking some
steps to ensure that staff complete the required number of training
hours. VBA central office periodically reviews the aggregated number of
training hours completed at each regional office to determine whether
the office is on track to meet the training requirement.[Footnote 8]
According to a VBA official, managers in offices where the staff is not
on track to complete 80 hours of training during the year can be
reprimanded by a higher-level manager, and if their staff do not meet
the aggregate training hours at the end of the fiscal year, managers
could face negative consequences in their performance assessments.
VBA Is Taking Steps to Strategically Plan Its Training for Staff, but
Does Not Adequately Evaluate Training and May Be Falling Short in
Design and Implementation:
VBA is taking steps to strategically plan its training for VSRs and
RVSRs including the establishment of a training board to assess VBA's
training needs. VBA has also made some effort to evaluate its training
for new staff, but does not require regional offices to collect
feedback from staff on any of the training they provide. Although some
regional offices collect some training feedback, it is not shared with
VBA central office. Both new and experienced staff we interviewed did,
in fact, report some problems with their training. A number of new
staff raised issues with how consistently their training curriculum was
implemented. Experienced staff differed in their assessments of the
VBA's annual training requirement, with some indicating they struggle
to meet this requirement because of workload pressures or that training
topics are sometimes redundant or not relevant to their position.
VBA Is Taking Steps to Strategically Plan Its Training:
VBA is taking steps to strategically plan its training for claims
processors, in accordance with generally accepted practices identified
by GAO. (See app. I for a detailed description of these generally
accepted practices.)
Aligning Training with the Agency's Mission and Goals:
VBA has made an effort to align training with the agency's mission and
goals. According to VBA documents, in fiscal year 2004 an Employee
Training and Learning Board (board) was established to ensure that
training decisions within the VBA are coordinated; support the agency's
strategic and business plans, goals and objectives; and are in
accordance with the policy and vision of VBA.[Footnote 9] Some of the
board's responsibilities include establishing training priorities and
reviewing regional office and annual training plans.
Identifying the Skills and Competencies Needed by the Workforce:
VBA has identified the skills and competencies needed by VBA's claims
processing workforce. VBA developed a decision tree and task analysis
of the claims process, which GAO experts in the field of training told
us made it possible to understand and map both the claims process and
the decisions associated with it that supported the development of
VBA's training curriculum.
Determining the Appropriate Level of Investment in Training and
Prioritizing Funding:
VBA is taking steps to determine the appropriate level of investment in
training and prioritize funding. According to VBA documents, some of
the board's responsibilities include developing annual training budget
recommendations and identifying and recommending training initiatives
to the Under Secretary of Benefits. VBA officials also reported
developing several documents that made a business case for different
aspects of VBA's training, such as VA's annual budget and the task
analysis of the VSR and RVSR job positions.
Considering Government Reforms and Initiatives:
According to one VBA official, the agency identifies regulatory,
statutory, and administrative changes as well as any legal or judicial
decisions that affect how VBA does business and issues guidance
letters, or Fast Letters, which can be sent out several times a year,
to notify regional offices of these changes. Also, as a result of
Congress authorizing an increase in its number of full-time employees
and VBA's succession planning efforts, VBA has increased the number of
centralized training sessions for new staff and has also increased the
number of Instructor Development Courses offered to potential
centralized training instructors. As a result, VBA is taking steps to
consider government reforms and initiatives to improve its management
and performance when planning its training.
VBA Collects Feedback on Centralized Training, but Regional Offices Do
not Always Collect Feedback on the Training They Provide:
According to accepted practices, federal agencies should also evaluate
their training programs and demonstrate how these efforts help
employees, rather than just focusing on activities or processes (such
as number of training participants or hours of training). VBA has made
some efforts to evaluate its training for claims processors. During the
3-week centralized training session for new staff, VBA solicits daily
feedback from participants using forms that experts in the training
field consider well-constructed and well-balanced. According to one GAO
expert, the forms generally employ the correct principles to determine
the effectiveness of the training and ascertain whether the instructor
effectively presented the material (see fig. 4). VBA officials told us
that they have used this feedback to improve centralized training for
new staff. Management at one regional office cited the decision to
separate training curricula for VSRs on Pre-determination teams and
VSRs on Post-determination teams as an example of a change based on
this feedback.
Figure 4: Sample of VBA‘s Centralized Training Evaluation Form:
[See PDF for image]
This figure is a sample of VBA‘s Centralized Training Evaluation Form.
as follows:
Baltimore Classroom 1 Centralized Training Topic Evaluation - 2008
Session 4:
* Please select your training topic from the list of options:
1. Rate the overall quality of the content:
Excellent:
Good:
Fair:
Poor:
2. Rate the overall effectiveness of the practical exercises:
Excellent:
Good:
Fair:
Poor:
3. Rate the overall effectiveness of the instructor(s):
Excellent:
Good:
Fair:
Poor:
4. How was the pace of the instruction?
About right:
Too slow:
Too fast:
Much too slow:
Much too fast:
5. What would have made this training more useful? Please explain:
6. Please include any additional narrative comments about the
training/instructor(s):
Submit:
Source: VBA.
[End of figure]
Although VBA evaluates centralized training, it does not require
regional offices to obtain feedback from participants on any of the
training they provide to new and experienced staff. In a previous GAO
report, VA staff told us that new training materials they develop are
evaluated before being implemented.[Footnote 10] However, none of the
regional offices we visited consistently collect feedback on the
training they conduct. Supervisors from three of the regional offices
we visited told us that they collect feedback on some of the training
their office conducts, but this feedback largely concerns the
performance of the instructor. Participants are generally not asked for
feedback on course content. Moreover, regional offices we visited that
do, to some degree, collect feedback do not share this information with
VBA.
VBA's Training Curriculum for New Staff Appears Generally Well
Designed, but Some Staff Raised Issues Concerning Its Implementation:
According to GAO experts in the training field, VBA's training
curriculum for new staff appears well designed. VBA's curriculum for
new staff conforms to adult learning principles, carefully defining all
pertinent terms and concepts, and providing abundant and realistic
examples of claims work. GAO experts also determined that VBA's
training for those who teach the curriculum for new staff was well
designed and would enable experienced claims processors to become
competent trainers because they are coached on teaching theory and have
multiple opportunities to practice their teaching skills and receive
feedback.
Many of the new staff at all four sites we visited reported that
centralized training provided them with a good foundation of knowledge
and prepared them for additional training conducted by their regional
office. Also, regional office managers from three offices we visited
told us that centralized training affords new staff the opportunity to
network with other new staff at different regional offices, which
imbues a sense of how their positions fit in the organization. However,
some staff reported that VBA's implementation of their centralized
training was not always consistent. A number of staff at three regional
offices reported that during their centralized training the instructors
sometimes taught different ways of performing the same procedures or
disagreed on claim procedures. Regional office officials told us that
while centralized training instructors attempt to teach consistently
through the use of standardized training materials, certain procedures
can be done differently in different regional offices while adhering to
VBA policy. For example, regional offices may differ on what to include
in veteran notification letters. VBA officials also told us that
centralized training conducted at the regional offices may not be as
consistent as centralized training conducted at the Veterans Benefits
Academy. According to these officials, unlike the regional offices, the
Veterans Benefits Academy has on-site training experts to guide and
ensure that instructors are teaching the curriculum consistently.
New staff also gave mixed assessments about how training was conducted
at their home office after they returned from centralized training.
While some staff at all of the regional offices we visited told us that
the additional training better prepared them to perform their jobs,
with on-the-job training identified as a useful learning tool, others
told us that the training could not always be completed in a timely
manner due to regional office priorities. Some management and staff at
two of the regional offices we visited reported that, because of
workload pressures, some of their RVSRs had to interrupt their training
to perform VSR duties. Also, a few new staff indicated that VBA's TPSS
was somewhat difficult to use.[Footnote 11] Although TPSS was developed
to provide consistent technical training designed to improve the
accuracy of claims ratings, a number of staff at all of the regional
offices we visited reported that TPSS was too theoretical. For example,
some staff said it provided too much information and no practical
exercises in applying the knowledge. Some staff also noted that certain
material in TPSS was out-of-date with policy changes such as how to
order medical examinations. Some staff at three of the regional offices
also reported that TPSS was not always useful in training staff, in
part, because TPSS does not use real cases. Three of the regional
offices reported using TPSS for less than 1 percent of their training
and VSRs at one regional office were unaware of what TPSS was.
Experienced Staff Expressed Mixed Views of the Design and
Implementation of Their Training:
At all of the regional offices we visited, staff we spoke with
generally noted that training enables them to keep up-to-date on
changes in laws and regulations as well as provides opportunities for
obtaining refresher training on claims procedures they perform
infrequently. However, regional office staff we spoke with differed in
their assessment of the 80-hour requirement. Some regional office staff
said the number of training hours required was appropriate, while
others suggested that VBA adopt a graduated approach, with the most
experienced staff being required to complete fewer hours than new
staff. VBA officials told us that, in 2007, the Compensation and
Pension Service reviewed their annual training requirements and
determined the 80-hour annual training requirement was appropriate.
However, the officials we spoke with could not identify the criteria
that were used to make these determinations. Furthermore, VBA
management does not systematically collect feedback from staff
evaluating the usefulness of the training they must receive to meet
this requirement. Consequently, when determining the appropriateness of
the 80-hour requirement, VBA has not taken into account the views of
staff to gauge the effect the requirement has on them.
Experienced staff had mixed views on training provided by the regional
office. Staff at three regional offices said the core technical
training topics set by the Compensation and Pension Service are really
designed for newer staff and do not change much from year to year, and
therefore experienced staff end up repeating courses. Also, a number of
staff at all of the regional offices we visited told us some regional
office training was not relevant for those with more experience.
Conversely, other regional office staff note that although training
topics may be the same from year to year, a person can learn something
new each time the course is covered. Some VBA officials and regional
office managers also noted that some repetition of courses is good for
several reasons. Staff may not see a particular issue very often in
their day-to-day work and can benefit from refreshers. Also, regional
office managers at one office told us that the core technical training
topics could be modified to reflect changes in policy so that courses
are less repetitive for experienced staff.
Many experienced staff also reported having difficulty meeting the 80-
hour annual training requirement due to workload pressures. Many of the
experienced staff we spoke with, at each of the regional offices we
visited, told us that there is a constant struggle between office
production goals and training goals. For example, office production
goals can affect the availability of the regional office's instructors.
A number of staff from one regional office noted that instructors were
unable to spend time teaching because of their heavy workloads and
because instructors' training preparation hours do not count toward the
80-hour training requirement. Staff at another regional office told us
that, due to workload pressures, staff may rush through training and
may not get as much out of it as they should.
Performance Management System for Claims Processors Generally Conforms
to Accepted Practices, but May Not Clearly Differentiate between
Performance Levels:
VA's performance management system for claims processors is consistent
with several accepted practices for effective performance management
systems in the public sector, but may not clearly differentiate between
staff's overall performance levels. VA's performance management system
aligns individual performance elements with broader organizational
performance measures, provides performance feedback to staff throughout
the year, and emphasizes collaboration. However, the system may not
clearly differentiate VSRs' and RVSRs' varying levels of performance.
While the system has five summary rating categories for VSRs and RVSRs,
several VBA managers told us that, because of a problem with the
formula used to convert ratings on individual performance elements into
an overall performance category, it is more difficult for staff to be
placed in certain categories than others:
Performance Management System for Claims Processors Is Generally
Consistent with Accepted Practices:
The elements used to evaluate individual VSRs' and RVSRs' performance
appear to be generally aligned with VBA's organizational performance
measures, something prior GAO work has identified as a well-recognized
practice for effective performance management systems (see app. I).
Aligning individual and organizational performance measures helps staff
see the connection between their daily work activities and their
organization's goals and the importance of their roles and
responsibilities in helping to achieve these goals. VSRs must be
evaluated on four critical elements: quality, productivity, workload
management, and customer service. RVSRs are evaluated on quality,
productivity, and customer service. In addition, VBA central office
requires regional offices to evaluate their staff on at least one non-
critical element. The central office has provided a non-critical
element called cooperation and organizational support, and although
regional offices are not required to use this particular element, all
four offices we visited did so (see table 2). For each element, there
are three defined levels of performance: exceptional, fully successful,
or less than fully successful.[Footnote 12] Table 2 refers only to the
fully successful level of performance for each element.
Table 2: Individual Performance Elements for VSRs and RVSRs:
Performance element: Critical: Quality;
How performance is evaluated: A random selection of 5 cases or phone
calls per month is reviewed for accuracy based on certain criteria, for
example whether all necessary evidence was requested, proper
notifications were sent to applicants, and accurate information was
provided in phone calls. Any case or phone call with one or more errors
is counted as one inaccurate case or call;
Standard for minimum fully successful performance (journey-level VSR):
85% accuracy;
Standard for minimum fully successful performance (journey-level RVSR):
85% accuracy.
Performance element: Critical: Productivity;
How performance is evaluated: Number of weighted actions (VSRs) or
weighted cases (RVSRs) completed per day. VSRs receive different
weights for different actions, such as 0.125 for conducting a telephone
interview or 1.50 for developing the evidence for a claim with a
special issue such as radiation. RVSRs receive different levels of
credit for processing cases with different numbers of issues to be
evaluated;
Standard for minimum fully successful performance (journey-level VSR):
8 weighted actions per day[A];
Standard for minimum fully successful performance (journey-level RVSR):
3.5 weighted cases per day[B].
Performance element: Critical: Customer service;
How performance is evaluated: Number of valid complaints about
employee's behavior from external customers or internal colleagues;
Standard for minimum fully successful performance (journey-level VSR):
No more than 3 valid complaints or incidents.
Performance element: Critical: Workload management;
How performance is evaluated: Completion of designated tasks in a
timely manner, such as obtaining the results of a medical exam within a
specified period of time;
Standard for minimum fully successful performance (journey-level VSR):
Tasks are completed in timely manner 85 percent of the time;
Standard for minimum fully successful performance (journey-level RVSR):
Not applicable.
Performance element: Non-critical[C]: Cooperation and organizational
support;
How performance is evaluated: Understanding of agency goals,
interaction with colleagues, contribution to agency goals;
Standard for minimum fully successful performance (journey-level VSR):
Interacts with colleagues professionally. Follows directions and
adheres to guidance conscientiously. Adjusts easily to different
working styles and perspectives.
Source: GAO analysis of VBA information.
Note: This table includes the levels set for journey-level VSRs and
RVSRs, who are considered experienced and fully trained in their
positions. For some elements VBA sets different performance standards
for entry-level and experienced claims processors. For example, VSRs
are typically promoted to the journey-level position after about 2
years. VBA has separate, lower performance standards in the accuracy,
productivity, and workload management elements for VSRs who are not yet
at the journey level. Also, regional offices have the option of setting
fully successful levels for their staff that are higher than the
national minimum, but not lower. This table indicates instances when
the regional offices we visited have set thresholds that are higher
than the national minimum.
[A] Milwaukee has set a fully successful level of 10 weighted actions
per day.
[B] Baltimore, Milwaukee, and Portland have set fully successful levels
of, respectively, 4, 5, and 3.8 weighted cases per day.
[C] Regional offices are required to use at least one non-critical
element. VBA central office provided regional offices with the
cooperation and organizational support element, but regional offices
are not required to use this element in particular.
[End of table]
Three critical elements in particular--quality, workload management,
and productivity--are aligned with VBA's organizational performance
measures (see table 3). According to VA's strategic plan, one key
organizational performance measure for VBA is overall accuracy in
rating disability claims. This organizational measure is aligned with
the quality element for VSRs and RVSRs, which is assessed by measuring
the accuracy of their claims-processing work. An individual performance
element designed to motivate staff to process claims accurately should,
in turn, help VBA meet its overall accuracy goal. Two other key
performance measures for VBA are the average number of days that open
disability claims have been pending and the average number of days it
takes to process disability claims. VSRs are evaluated on their
workload management, a measure of whether they complete designated
claims-related tasks within specific deadlines. Individual staff
performance in this element is linked to the agency's ability to manage
its claims workload and process claims within goal time frames.
Finally, a performance measure that VBA uses to evaluate the claims-
processing divisions within its regional offices--and that, according
to VBA, relates to the organization's overall mission--is production,
or the number of compensation and pension claims processed by each
office in a given time period. Individual VSRs and RVSRs are evaluated
on their productivity, i.e., the number of claims-related tasks they
complete per day. Higher productivity by individual staff should result
in more claims being processed by each regional office and by VBA
overall.
Table 3: Performance Elements for VSRs and RVSRs and Corresponding
Organizational Performance Measures for VBA:
Performance element for VSRs and RVSRs: Quality;
Corresponding VBA performance measure(s): Accuracy rate for ratings of
compensation claims.
Performance element for VSRs and RVSRs: Productivity;
Corresponding VBA performance measure(s): Number of compensation and
pension claims completed by the claims-processing division within a
regional office in a given time period.
Performance element for VSRs and RVSRs: Workload management[A];
Corresponding VBA performance measure(s): Average days pending for
compensation and pension claims; (average number of days since claim
was received by VBA, for all open claims); Average days to process
compensation and pension claims (average number of days from receipt of
claim to final decision).
Source: VBA and GAO analysis.
[A] Workload management element applies only to VSRs, not RVSRs.
[End of table]
The performance management system for VSRs and RVSRs also appears to be
consistent with several other accepted practices for performance
management systems in the public sector:
Providing and Routinely Using Performance Information to Track
Organizational Priorities:
Providing objective performance information to individuals helps show
progress in achieving organizational goals and allows individuals to
manage their performance during the year by identifying performance
gaps and improvement opportunities. Regional offices are supposed to
use the critical and non-critical performance elements to evaluate and
provide feedback to their staff. Supervisors are required to provide at
least one progress review to their VSRs and RVSRs each year, indicating
how their performance on each element compares to the defined standards
for fully successful performance. In the offices we visited,
supervisors typically provide some feedback to staff on a monthly
basis. For example, VSRs in the Atlanta regional office receive a memo
on their performance each month showing their production in terms of
average weighted actions per day, their accuracy percentage based on a
review of a sample of cases, and how their performance compared to the
minimum requirements for production and accuracy. If staff members fall
below the fully successful level in a critical element at any time
during the year, a performance improvement plan must be implemented to
help the staff member improve.
Connecting Performance Expectations to Crosscutting Goals:
Performance elements related to collaboration or teamwork can help
reinforce behaviors and actions that support crosscutting goals and
provide a consistent message to all employees about how they are
expected to achieve results. VSR and RVSR performance related to
customer service is evaluated partly based on whether any valid
complaints have been received about a staff member's interaction with
their colleagues. And performance related to the cooperation and
organizational support element is based on whether staff members'
interaction with their colleagues is professional and constructive.
Using Competencies to Provide a Fuller Assessment of Performance:
Competencies, which define the skills and supporting behaviors that
individuals are expected to exhibit to carry out their work
effectively, can provide a fuller assessment of an individual's
performance. In addition to elements that are evaluated in purely
quantitative terms, VBA uses a cooperation and organizational support
element for VSRs and RVSRs that requires supervisors to assess whether
their staff are exhibiting a number of behaviors related to performing
well as a claims processor.
Involving Employees and Stakeholders to Gain Ownership of the
Performance Management System:
Actively involving employees and stakeholders in developing the
performance management system and providing ongoing training on the
system helps increase their understanding and ownership of the
organizational goals and objectives. For example, VA worked with the
union representing claims processors to develop an agreement about its
basic policies regarding performance management. Also, VBA indicated
that it planned to pilot revisions to how productivity is measured for
VSRs in a few regional offices, partly so VSRs would have a chance to
provide feedback on the changes.
VA's System May Not Clearly Differentiate between Performance Levels:
Clear differentiation between staff performance levels is also an
accepted practice for effective performance management systems. Systems
that do not result in meaningful distinctions between different levels
of performance fail to give (1) employees the constructive feedback
they need to improve, and (2) managers the information they need to
reward top performers and address performance issues. GAO has
previously reported that, in order to provide meaningful distinctions
in performance for experienced staff, agencies should use performance
rating scales with at least three levels, and scales with four or five
levels are preferable because they allow for even greater
differentiation between performance levels.[Footnote 13] If staff
members are concentrated in just one or two of multiple performance
levels, however, the system may not be making meaningful distinctions
in performance.
VA's performance appraisal system has the potential to clearly
differentiate between staff performance levels. Each fiscal year,
regional offices give their staff a rating on each critical and non-
critical performance element using a three-point scale--exceptional,
fully successful, or less than fully successful. Based on a VA-wide
formula, the combination of ratings across these elements is converted
into one of VA's five overall performance levels: outstanding,
excellent, fully successful, minimally satisfactory, and unsatisfactory
(see fig. 5). Regional offices may award financial bonuses to staff on
the basis of their end-of-year performance category.[Footnote 14] Prior
to fiscal year 2006, VA used two performance levels--successful and
unacceptable--to characterize each staff member's overall performance.
To better differentiate between the overall performance levels of
staff, VA abandoned this pass-fail system in that year, choosing
instead to use a five-level scale.
Figure 5: VA Overall Performance Appraisal Formula:
[See PDF for image]
This figure is an illustration of the VA Overall Performance Appraisal
Formula. The following information is depicted:
Ratings on Critical Elements: All exceptionals; and;
Ratings on Non-Critical elements: All exceptionals; yields:
Overall Performance Category: Outstanding.
Ratings on Critical Elements: All exceptionals; and;
Ratings on Non-Critical elements: One or more fully successful(s), any
others exceptionals; yields:
Overall Performance Category: Excellent.
Ratings on Critical Elements: One or more fully successful(s), all
others exceptionals; and;
Ratings on Non-Critical elements: All fully successfuls or
exceptionals; yields:
Overall Performance Category: Fully Successful.
Ratings on Critical Elements: One or more less than fully successful(s);
yields:
Overall Performance Category: Unsatisfactory.
Source: GAO analysis of VBA information.
[End of figure]
However, there is evidence to suggest that the performance management
system for VSRs and RVSRs may not clearly or accurately differentiate
among staff's performance. VBA central office officials and managers in
two of the four regional offices we visited raised concerns with VA's
formula for translating ratings on individual performance elements into
an overall performance rating.[Footnote 15] These officials said that
under this formula it is more difficult for staff to be placed in
certain overall performance categories than others, even if staff's
performance truly does fall within one of those categories. Indeed, at
least 90 percent of all claims processors in the regional offices we
visited were placed in either the outstanding or the fully successful
category in fiscal year 2007. (Fig. 6 shows the distribution of overall
performance ratings for claims processors in each office.) [Footnote
16]
Figure 6: Fiscal Year 2007 Overall Performance Ratings for Claims
Processors in Four Regional Offices Were Concentrated in the
Outstanding and Fully Successful Categories:
[See PDF for image]
This figure is a stacked vertical bar graph depicting the following
data:
Office: Atlanta;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 81%;
Excellent: 10%;
Outstanding: 9%.
Office: Baltimore;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 66%;
Excellent: 10%;
Outstanding: 24%.
Office: Milwaukee;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 70%;
Excellent: 2%;
Outstanding: 28%.
Office: Portland;
Unsatisfactory: 0;
Minimally satisfactory: 0;
Fully successful: 72%;
Excellent: 2%;
Outstanding: 26%.
Source: VBA regional offices.
Note: These data cover VSRs, RVSRs, and some other claims processing
staff.
[End of figure]
Central and regional office managers noted that, in particular, it is
difficult for staff to receive an overall rating of excellent. Managers
in one office said there are staff whose performance is better than
fully successful but not quite outstanding, but under the formula it is
difficult for these staff to be placed in the excellent category as the
managers feel they should be. An excellent rating requires exceptional
ratings in all the critical elements and a fully successful rating in
at least one non-critical element. However, according to staff we
interviewed, virtually all staff who are exceptional in the critical
elements are also exceptional in all non-critical element(s), so they
appropriately end up in the outstanding category. On the other hand,
the overall rating for staff who receive a fully successful rating on
just one of the critical elements--even if they are rated exceptional
in all the other elements--drops down to fully successful. Managers in
one regional office commented that the system would produce more
accurate overall performance ratings if staff were given an overall
rating of excellent when they had, for example, exceptional ratings on
three of five overall elements and fully successful ratings on the
other two.
An official in VA's Office of Human Resources Management acknowledged
that there may be an issue with the agency's formula. Although neither
VBA nor VA central office officials have examined the distribution of
VSRs and RVSRs across the five overall performance ratings, VA
indicated it is considering changes to the system designed to allow for
greater differentiation in performance ratings. For example, one
possible change would be to use a five-point scale for rating
individual elements--probably mirroring the five overall performance
rating categories of outstanding, excellent, fully successful,
minimally satisfactory, and unsatisfactory--rather than the current
three-point scale. Under the proposed change, a staff member who was
generally performing at the excellent but not outstanding level could
get excellent ratings in all the elements and receive an overall rating
of excellent. This change must still be negotiated with several
stakeholder groups, according to the VA official we interviewed.
Conclusions:
In many ways, VBA has developed a training program for its new staff
that is consistent with accepted training practices in the federal
government. However, because VBA does not centrally evaluate or collect
feedback on training provided by its regional offices, it lacks the
information needed to determine if training provided at regional
offices is useful and what improvements, if any, may be needed.
Ultimately, this information would help VBA determine if 80 hours of
training annually is the right amount, particularly for its experienced
staff, and whether experienced staff members are receiving training
that is relevant for their positions. Identifying the right amount of
training is crucial for the agency as it tries to address its claims
backlog. An overly burdensome training requirement needlessly may take
staff away from claims processing, while too little training could
contribute to processing inaccuracies. Also, without collecting
feedback on regional office training, VBA may not be aware of issues
with the implementation of its TPSS, the on-line training tool designed
to ensure consistency across offices in technical training. Setting
aside the issue of how many hours of training should be required, VBA
does not hold its staff accountable for fulfilling their training
requirement. As a result, VBA is missing an opportunity to clearly
convey to staff the importance of managing their time to meet training
requirements as well as production and accuracy goals. With the
implementation of its new learning management system, VBA should soon
have the ability to track training completed by individual staff
members, making it possible to hold them accountable for meeting the
training requirement.
As with its training program for VSRs and RVSRs, the VA is not
examining the performance management system for claims processors as
closely as it should. VBA is generally using the right elements to
evaluate its claims processors' performance, and the performance
appraisals have the potential to give managers information they can use
to recognize and reward higher levels of performance. However, evidence
suggests the formula used to place VSRs and RVSRs into overall
performance categories may not clearly and accurately differentiate
among staff's performance levels. Absent additional examination of the
distribution of claims processors among overall performance categories,
VA lacks a clear picture of whether its system is working as intended
and whether any adjustments are needed.
Recommendations for Executive Action:
The Secretary of Veterans Affairs should direct VBA to:
* Collect and review feedback from staff on the training conducted at
the regional offices to determine:
* if the 80-hour annual training requirement is appropriate for all
VSRs and RVSRs;
* the extent to which regional offices provide training that is
relevant to VSRs' and RVSRs' work, given varying levels of staff
experience; and:
* whether regional offices find the TPSS a useful learning tool and, if
not, what adjustments are needed to make it more useful; and:
* Use information from its new learning management system to hold
individual VSRs and RVSRs accountable for completing whatever annual
training requirement it determines is appropriate.
The Secretary of Veterans Affairs should also examine the distribution
of claims processing staff across overall performance categories to
determine if its performance appraisal system clearly differentiates
between overall performance levels, and if necessary adjust its system
to ensure that it makes clear distinctions.
Agency Comments:
We provided a draft of this report to the Secretary of Veterans Affairs
for review and comment. In VA's written comments (see app. IV), the
agency agreed with our conclusions and concurred with our
recommendations. For example, VBA plans to consult with regional office
staff to evaluate its annual 80-hour training requirement and will
examine if staff performance ratings clearly differentiate between
overall performance levels. VA also provided technical comments that
were incorporated as appropriate.
We are sending copies of this report to the Secretary of Veterans
Affairs, relevant congressional committees, and others who are
interested. We will also provide copies to others on request. The
report is also available at no charge on GAO's Web site at [hyperlink
http://www.gao.gov].
Please contact me on (202) 512-7215 if you or your staff have any
questions about this report. Contact points for the Offices of
Congressional Relations and Public Affairs may be found on the last
page of this report. Key contributors are listed in appendix V.
Sincerely,
Singed by:
Daniel Bertoni:
Director, Education, Workforce, and Income Security Issues:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
We were asked to determine: (1) What training is provided to new and
experienced claims processors and how uniform is this training? (2) To
what extent has the Veterans Benefits Administration (VBA) developed a
strategic approach to planning training for claims processors and how
well is their training designed, implemented, and evaluated? And (3) To
what extent is the performance management system for claims processors
consistent with generally accepted performance management practices in
the public sector? To answer these questions, we reviewed documents and
data from the central office of the Department of Veterans Affairs'
Veterans Benefits Administration (VBA) and interviewed VBA central
office officials. We conducted site visits to and collected data from
four VBA regional offices, and visited the Veterans Benefits Academy.
We also interviewed officials from the American Federation of
Government Employees, the labor union that represents Veterans Service
Representatives (VSR) and Rating Veterans Service Representatives
(RVSR). We compared VBA's training and performance management systems
to accepted human capital principles and criteria compiled by GAO. We
conducted this performance audit from September 2007 through May 2008
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
Regional Office Site Visits:
We conducted site visits to 4 of VBA's 57 regional offices--Atlanta;
Baltimore; Milwaukee; and Portland, Oregon. We judgmentally selected
these offices to achieve some diversity in geographic location, number
of staff, and claims processing accuracy rates, and what we report
about these sites may not necessarily be representative of any other
regional offices or all regional offices (see fig. 7). [Footnote 17]
Figure 7: Regional Offices Selected for Site Visits:
[See PDF for image]
This figure is a map of the United States depicting the regional
offices selected for site visits. The following information is
represented:
Eastern Area:
Location: Baltimore, Maryland;
Number of staff[A]: 122.5; rank[B]: 37th;
Accuracy rate[C]: 88%; rank[B]: 32nd.
Southern Area:
Location: Atlanta, Georgia;
Number of staff[A]: 590.5; rank[B]: 3rd;
Accuracy rate[C]: 84%; rank[B]: 48th.
Central Area:
Location: Milwaukee, Wisconsin;
Number of staff[A]: 362.9; rank[B]: 12th;
Accuracy rate[C]: 92%; rank[B]: 13th.
Western Area:
Location: Portland, Oregon;
Number of staff[A]: 156.1; rank[B]: 31st;
Accuracy rate[C]: 86%; rank[B]: 44th.
Sources: VBA; Map Resources (map).
[A] Full-time equivalents as of September 2007.
[B] Rank among all 57 regional offices.
[C] Claims-processing accuracy rate for the period of August 1, 2006 to
July 31, 2007.
[End of figure]
During our site visits, we interviewed regional office managers,
supervisors of VSRs and RVSRs, VSRs, and RVSRs about the training and
performance management practices in their offices. The VSRs and RVSRs
we interviewed at the four regional offices had varying levels of
experience at VBA. Regional office managers selected the staff we
interviewed. We also observed a demonstration of VBA's on-line learning
tool, the Training and Performance Support System (TPSS), and collected
data from the regional offices on, for example, the training they
provided during fiscal year 2007.[Footnote 18] In conjunction with our
visit to the Baltimore regional office, we also visited VBA's Veterans
Benefits Academy, where we observed classes for VSRs and RVSRs and
interviewed the director of the Academy.
Assessment of VBA's Training for Claims Processors:
To determine whether VBA's training program is consistent with accepted
training practices in the public sector, we relied partly on a guide
developed by GAO that lays out principles that federal agencies should
follow to ensure their training is effective.[Footnote 19] This guide
was developed in collaboration with government officials and experts in
the private sector, academia, and nonprofit organizations; and in
conjunction with a review of laws, regulations and literature on
training and development issues, including previous GAO reports. The
guide lays out the four broad components of the training and
development process (see fig. 8).
Figure 8: Four Components of the Training and Development Process:
[See PDF for image]
This figure is an illustration of the four components of the training
and development process. The first three components are sequential, but
all three provide information the the final component. The following
information is depicted:
Planning/Front-end Analysis:
* Develop a strategic approach that establishes priorities and
leverages investments in training and development to achieve agency
results.
Design/Development:
* Identify specific training and development initiatives that, in
conjunction with other strategies, improve individual and agency
performance.
Implementation:
* Ensure effective and efficient delivery of training and development
opportunities in an environment that supports learning and change.
Evaluation:
* Demonstrate how training and development efforts contribute to
improved performance and results.
Source: GAO.
Note: The evaluation component may include the use of participant
feedback to ensure continuous improvement, as well as an assessment of
the impact of training on organizational performance. We have reported
that higher-level evaluations that attempt to measure the return on
investment in a training program may not always be appropriate, given
the complexity and costs associated with efforts to directly link
training programs to improved individual and organizational
performance.
[End of figure]
The guide also provides key questions for federal agencies to consider
in assessing their performance in each component. (See table 4 for a
sample of these questions.)
Table 4: Selected Key Questions to Consider in Assessing Agency‘s
Training Program:
Planning/Front End Analysis:
* Does the agency have training goals and related performance measures
that are consistent with its overall mission, goals, and culture?
* How does the agency identify the appropriate investment to provide
for training and development efforts and prioritize funding so that the
most important training needs are addressed first?
Design and Development;
* What criteria does the agency use in determining whether to design
training and development programs in-house or obtain these services
from a contractor or other external source?
* Does the agency use the most appropriate mix of centralized and
decentralized approaches for its training and development programs?
Implementation;
* What steps do agency leaders take to communicate the importance of
training and developing employees, and their expectations for training
and development programs to achieve results?
* How does the agency select employees to participate in training and
development efforts?
Evaluation;
* To what extent does the agency systematically plan for and evaluate
the effectiveness of its training and development efforts?
* How does the agency incorporate evaluation feedback into the
planning, design, and implementation of its training and development
efforts?
Source: GAO.
[End of table]
In addition, GAO training experts reviewed VBA materials, including
training curricula, lesson plans, and course evaluation forms, to
determine if these materials are consistent with accepted training
practices.
Assessment of VBA's Performance Management System for Claims
Processors:
In assessing the performance management system for VSRs and RVSRs, we
relied primarily on a set of accepted practices of effective public
sector performance management systems that has been compiled by
GAO.[Footnote 20] To identify these accepted practices, GAO reviewed
its prior reports on performance management that drew on the
experiences of public sector organizations both in the United States
and abroad. For the purpose of this review, we focused on the six
accepted practices most relevant for VBA's claims-processing workforce
(see table 5).
Table 5: Selected Accepted Practices for Effective Performance
Management Systems:
Practice: Aligning individual performance expectations with
organizational goals;
Description: Explicitly aligning individuals' daily activities with
broader results helps individuals see the connection between their work
and organizational goals and encourages individuals to focus on their
roles and responsibilities to help achieve those broader goals.
Practice: Connecting performance expectations to crosscutting goals;
Description: Fostering collaboration, interaction, and teamwork across
organizational boundaries to achieve results strengthens accountability
for these results.
Practice: Providing and routinely using performance information to
track organizational priorities;
Description: Providing objective performance information to both
managers and staff to show progress in achieving organizational results
and other priorities helps them manage during the year, identify
performance gaps, and pinpoint improvement opportunities.
Practice: Using competencies to provide a fuller assessment of
performance;
Description: Using competencies, which define the skills and supporting
behaviors that individuals need to effectively contribute to
organizational results, can provide a fuller picture of an individual's
performance.
Practice: Making meaningful distinctions in performance;
Description: Providing individuals with candid and constructive
feedback helps them maximize their contribution, and providing
management with objective and fact-based information that clearly
differentiates between different levels of performance enables it to
reward top performers and deal with poor performers.
Practice: Involving employees and stakeholders to gain ownership of
performance management systems;
Description: Actively involving employees and stakeholders in
developing the performance management system and providing ongoing
training on the system helps increase their understanding and ownership
of the organizational goals and objectives.
Source: GAO.
[End of table]
[End of section]
Appendix II: Fiscal Year 2007 Core Technical Training Requirements for
VSRs and RVSRs:
Position: Decision Review Officers (DRO); GS13/Rating Veterans Service
Representatives (RVSR); GS7-12; (Seasoned); Required: 80 hours; Any DRO
or RVSR who conducts a training session will also be given credit for
those training hours as part of their training requirement;
Course title or topic:
Effective Dates;
Diabetes Mellitus;
Leishmaniasis;
Original Compensation Ratings;
Original Pension Ratings;
Original DIC Ratings;
Rating re-opened claims;
Claims for Increase;
New and Material Evidence;
Re-opened DIC ratings;
Routine Future Exams;
3.105(e) reductions;
Paragraph 28/29/30 ratings;
Due Process Provisions;
Clear and unmistakable errors (3.105(a));
Ancillary Benefits;
Accrued Ratings;
Musculoskeletal issues;
Eye-Vision Issues;
Infectious Diseases;
Ear-Hearing;
Respiratory Disorders;
Cardiovascular Issues;
Digestive Issues;
Genitourinary System;
Gynecology;
Hemic/Lymphatic;
Endocrine (other than DM);
Neurological;
Mental Disorder (other than PTSD);
PTSD;
Special Monthly Compensation (SMC);
The Appeals Process;
Responsibilities of a DRO;
Hearings;
Informal Conferences;
Resolution of Claims;
Certifying a case to BVA;
Processing Remands;
Preparing a Statement of the Case (SOC);
Preparing a Supplemental Statement of the Case (SSOC);
Role of the Rating Specialist;
Benefit of the Doubt;
Weighing Evidence;
60 Hours of the required 80 Hours will be selected from the suggested
topics above. The remaining 20 hours will be selected at the Stations
discretion based upon their own individual quality review; (Training
provided from the above topics can be focused on a particular aspect of
the topic; i.e. Cold Injuries and Rating Hypertension from
Cardiovascular issues could be separate classes);
Participation in Agency Advancement Programs (i.e., LEAD, LVA) does not
substitute for Required training requirements;
Training source: C&P Training Website [hyperlink,
http://cptraining.vba.va.gov/C&P_Training/RVSR/RVSR_Tng_Curr.htm]; Fast
Letters' Training Letters; Court Decisions; TPSS (Can be used as
refresher training for seasoned employees by module); EPSS; Manuals;
Regulations; Additional Issue Specific Lesson Plans are under
development; (Lesson plans can be taken from the Centralized Training
Curriculum found on the C&P Intranet Training Site. If used as provided
they do not require C&P review and approval); These plans can and often
should be modified to focus in on a particular narrow issue of training
need. Modified lesson plans are to be submitted to C&P Service for
review and approval at least 30 days prior to delivery of training; Any
Challenge-oriented original lesson plan developed by Station personnel
is to be submitted to C&P Service for review and approval at least 30
days prior to delivery of training.) C&P Service Broadcasts that may be
provided during the course of the FY may be substituted in place of any
training scheduled on an hour by hour basis.
Position: Veteran Service Representative (VSR); GS 7-12 (Seasoned);
Required: 80 hours; Any Super Senior VSR, Senior VSR, or VSR, who
conducts a training session will also be given credit for those
training hours including preparation time as part of their training
requirement;
Course title or topic:
Reference Materials: Manual Training & WARMS;
C&P Website;
Claims Folder Maintenance;
Records Management;
POA/Service Orgs.;
Original Compensation Claims;
Re-opened Compensation Claims;
VA Form 21-526;
Establishing Veteran Status;
Claims Recognition;
Duty to Assist;
Requesting VA Exams;
Issue Specific Claims Development;
Asbestos Claims Development;
Herbicide Claims Development;
POW Claims Development;
Radiation Claims Development;
PTSD Claims Development;
Undiagnosed Illness Claims Development;
Dependency Issues; Contested Claims;
Deemed Valid and Common Law Marriages;
Continuous Cohabitation;
Pension;
SHARE;
COVERS;
MAP D;
MAP A;
Administrative Decisions;
Character of Discharge;
Line of Duty-Willful Misconduct;
Matching Programs;
Workload Management;
DEA Training;
Intro to Ratings;
Paragraph 29 & 30 Ratings;
Ratings & BDN;
BDN 301 Interface;
PCGL Award Letters;
Dependents and the BDN;
Compensation Offsets;
Drill Pay Waivers;
Pension Awards Processing & BDN;
Hospital Reductions;
Burial Benefits;
Death Pension;
Accrued Benefits;
Accrued Awards & the BDN;
Apportionments;
Special Monthly Pension;
Helpless Child;
Incompetency/Fiduciary Arrangements;
Claims Processing;
Auto Allowance and Adaptive Equipment;
Special Adapted Housing;
Special Home Adaptation Grants;
Incarcerated Veterans;
Processing Write Outs;
FOIA/Privacy Act;
Telephone & Interview Techniques;
Telephone Development;
IRIS;
Introduction to VACOLS;
Education Benefits;
Insurance Benefits;
National Cemetery;
VR&E Benefits;
Loan Guaranty Benefits;
General Benefits - FAQs;
Suicidal Caller Guidance;
Non-Receipt of BDN Payments;
Mail Handling; Income & Net Worth Determinations;
Bootcamp test and review of VSR;
Readiness Guide (2 HRS Required);
Reference Material Training and Navigation (1 HR Required);
Appeals and Ancillary Benefits;
Ready to Rate Development;
Customer Service;
FNOD Info and PMC Process;
Intro to Appeals Process;
DRO Selection Letter;
Income Adjustment Materials;
Income Adjustments;
60 Hours of the required 80 Hours will be selected from the suggested
topics above. The remaining 20 hours will be selected at the Stations
discretion based upon their own individual quality review;
Training source: C&P Training Website [hyperlink,
http://cptraining.vba.va.gov/C&P_Training/VSR/VSR_Curriculum.htm] or,
[hyperlink,
http://cptraining.vba.va.gov/C&P_Training/VSR/VSR_Curriculum.htm#pctt];
Fast Letters;
Training Letters;
Court Decisions;
TPSS (Can be used as refresher training for seasoned employees by
module);
EPSS;
Manuals;
Regulations;
Federal Benefits for Veterans and Dependents;
Business Line Internet Sites;
Conference Calls (VACO/C&P Service);
Star Reporter;
Additional Issue Specific Lesson Plans are under development;
(Lesson plans can be taken from the Centralized Training Curriculum
found on the C&P Intranet Training Site. If used as provided they do
not require C&P review and approval);
These plans can and often should be modified to focus in on a
particular narrow issue of training need. Modified lesson plans are to
be submitted to C&P Service for review and approval at least 30 days
prior to delivery of training;
Any Challenge-oriented original lesson plan developed by Station
personnel is to be submitted to C&P Service for review and approval at
least 30 days prior to delivery of training);
C&P Service Broadcasts that may be provided during the course of the FY
may be substituted in place of any training scheduled on an hour by
hour basis.
Position: Veterans Services Representative (VSR); GS 7-12 (New);
Required: Entire Curriculum; (Follow C&P Prescribed Curriculum for new
VSRs, as posted on intranet);
Course title or topic:
Curriculum is posted on C&P Training Intranet Site;
Claims Processing Prerequisites:
Human Resources and Orientation;
Computer Security and LAN;
Procedures;
Core Values;
Core Competencies and Your Job;
Voice of the Veteran video;
VA in Motion video;
VSR Handbook;
VA Terminology;
SHARE (BDN & CEST);
COVERS;
PIES;
Return with Honor Video;
MAPD;
AMIE/CAPRI;
Medical TPSS (Medical Terminology); Reader Focused Writing Tools;
Pre-Determination Team Training;
Overview of VA Mission; Reference Materials:
Manual Training & WARMS;
C&P Website;
Claims Folder Maintenance; Records Management;
POA/Service Organizations; Compensation; Original Compensation Claims;
Non-Original Compensation Claims;
VA Form 21-526, App. For Compensation or Pension;
Establishing Veteran Status;
Claims Recognition;
Duty to Assist;
Selecting the Correct Worksheet for VA Exams;
Issue Specific Claim Development;
Asbestos Claim Development;
Herbicide Claim Development;
POW Claim Development;
Radiation Claim Development;
PTSD Claim Development;
Undiagnosed Illness Claim Development;
Dependency;
Contested Claims;
Deemed Valid and Common-law Marriage;
Continuous Cohabitation;
Pension;
Intro. To Disability Pension;
Overview of SHARE (SSA);
Administrative Decision Process;
Character of Discharge;
Line of Duty - Willful Misconduct Claims Development;
Workload Management Utilizing WIPP;
DEA Training (req. added 4/06);
Post-Determination Team Training:
Intro to Ratings;
Paragraph 29 & 30 Ratings;
Ratings & the BDN;
BDN 301 Interface Video;
PCGL Award Letters;
PCGL;
Dependents & the BDN;
Compensation Offsets;
Drill Pay Waivers;
Star Reporter;
Pension Awards Processing & the BDN;
Hospital Reductions;
Burial Benefits;
Disallowance Processing;
DIC Benefits;
Death Pension;
Accrued Benefits;
Accrued Awards & the BDN;
Apportionment;
Special Monthly Pension;
Helpless Child;
Incompetency/Fiduciary Arrangements;
Claims Processing;
Automobile Allowance and Adaptive Equipment;
Specially Adapted Housing and Special Home Adaptation Grants;
Incarceration;
Processing Computer Write Outs;
DEA Training (req. added 4/06);
Public Contact Team Training;
FOIA/Privacy Act;
Communication Skills;
Telephone Development;
Inquiry Routing and Information System (IRIS);
Intro to VACOLS;
Other VBA Business Lines;
Customer Service Insurance Education (2 hrs);
Triage Team Training: FNOD Information & PMC Processing;
Appeals Team Training: Intro to Appeals Process VACOLS;
Pension Maintenance Centers;
Income Adjustment Materials: Income Adjustments;
Training source: [hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#cpp].
[hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#pred].
[hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#postd].
[hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#pctt];
VSR Core Curriculum Video;
VSR Curriculum;
VSR Core Curriculum Fast Ltr. 04-12;
VSR Core Curriculum Threshold Videos Video; [hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#ttt];
[hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#att];
[hyperlink,
http://cptraining.vba.va.gov/C&P_Training/vsr/VSR_Curriculum.htm#iam].
Source: VBA.
[End of table]
[End of section]
Appendix III: Excerpts from Fiscal Year 2007 Training Plans for Four
Regional Offices:
Each training plan we reviewed contained the same informational
categories, some of which were what courses were offered by the
regional office, whether or not the course was conducted, and how many
employees completed the training. Although the fiscal year 2007
training plans we reviewed include data on whether and when the course
was actually completed, the initial training plans submitted at the
beginning of the fiscal year of course do not have this information.
The lists provided below include the first 25 courses listed on each
plan alphabetically, a small sample of the courses that the regional
offices reported they completed for the fiscal year.
Table 6: Excerpt from Atlanta Regional Office Training Plan:
Course name: Accrued Benefits;
Number of employees completed: 15;
Total hours of training completed: 150.
Course name: Accrued Ratings (2 sessions conducted);
Number of employees completed: 47;
Total hours of training completed: 80.
Course name: Administrative Decisions;
Number of employees completed: 15;
Total hours of training completed: 60.
Course name: Ancillary Benefits;
Number of employees completed: 14;
Total hours of training completed: 14.
Course name: Appeals and Ancillary Benefits (2 sessions conducted);
Number of employees completed: 26;
Total hours of training completed: 41.
Course name: Apportionments (2 sessions conducted);
Number of employees completed: 29;
Total hours of training completed: 194.
Course name: Asbestos Claims Development;
Number of employees completed: 9;
Total hours of training completed: 9.
Course name: Auto Allowance/Special Adapted Housing/Special Home
Adaptation Grant;
Number of employees completed: 15;
Total hours of training completed: 30.
Course name: Benefits Delivery Network 301 Interface (2 sessions
conducted);
Number of employees completed: 48;
Total hours of training completed: 48.
Course name: Beneficiary Identification Records Locator Subsystem
Update;
Number of employees completed: 17;
Total hours of training completed: 17.
Course name: Blast Injuries (2 sessions conducted);
Number of employees completed: 20;
Total hours of training completed: 20.
Course name: Burial Benefits (2 sessions conducted);
Number of employees completed: 36;
Total hours of training completed: 100.
Course name: Board of Veterans Appeals Examinations;
Number of employees completed: 46;
Total hours of training completed: 69.
Course name: Compensation & Pension Website (2 sessions conducted);
Number of employees completed: 108;
Total hours of training completed: 270.
Course name: Change of Address/Power of Attorney Processing/No Record
Mail;
Number of employees completed: 17;
Total hours of training completed: 34.
Course name: Cardiovascular Issues;
Number of employees completed: 38;
Total hours of training completed: 76.
Course name: Certifying a Case to Board of Veterans Appeals;
Number of employees completed: 12;
Total hours of training completed: 12.
Course name: Character of Discharge;
Number of employees completed: 78;
Total hours of training completed: 78.
Course name: Claims Folder Maintenance (2 sessions conducted);
Number of employees completed: 17;
Total hours of training completed: 28.
Course name: Claims for Direct Service Connection/Aggravation/
Presumptive Service Connection;
Number of employees completed: 34;
Total hours of training completed: 34.
Course name: Claims for Increase;
Number of employees completed: 29;
Total hours of training completed: 58.
Course name: Claims Processing;
Number of employees completed: 139;
Total hours of training completed: 69.5.
Course name: Claims Recognition;
Number of employees completed: 84;
Total hours of training completed: 336.
Course name: Compensation Offsets (3 sessions conducted);
Number of employees completed: 167;
Total hours of training completed: 352.5.
Course name: Computer Security and LAN Procedures;
Number of employees completed: 6;
Total hours of training completed: 6.
Source: VBA.
Note: Atlanta's training plan reported the regional office conducted a
total of 133 courses for fiscal year 2007.
[End of table]
Table 7: Excerpt from Baltimore Regional Office Training Plan:
Course name: Accrued Benefits;
Number of employees completed: 5;
Total hours of training completed: 10.
Course name: Automated Medical Information Exchange/Compensation and
Pension Record Interchange;
Number of employees completed: 6;
Total hours of training completed: 48.
Course name: Appeals and Ancillary Benefits;
Number of employees completed: 3;
Total hours of training completed: 3.
Course name: Asbestos Claims Development;
Number of employees completed: 3;
Total hours of training completed: 3.
Course name: Access Standardized Performance Elements Nationwide;
Number of employees completed: 2;
Total hours of training completed: 2.
Course name: Auto Allowance and Adaptive Equipment (2 sessions
conducted);
Number of employees completed: 16;
Total hours of training completed: 8.
Course name: Benefits Delivery at Discharge Development;
Number of employees completed: 14;
Total hours of training completed: 21.
Course name: Benefits Delivery Network 301 Interface (2 sessions
conducted);
Number of employees completed: 5;
Total hours of training completed: 7.
Course name: Benefit of the Doubt;
Number of employees completed: 3;
Total hours of training completed: 12.
Course name: Burial Benefits (2 sessions conducted);
Number of employees completed: 7;
Total hours of training completed: 14.
Course name: Compensation & Pension Website (3 sessions conducted);
Number of employees completed: 15;
Total hours of training completed: 36.5.
Course name: Certifying a Case to Board of Veterans Appeals;
Number of employees completed: 3;
Total hours of training completed: 12.
Course name: Character of Discharge;
Number of employees completed: 15;
Total hours of training completed: 7.5.
Course name: Claims Folder Maintenance;
Number of employees completed: 7;
Total hours of training completed: 14.
Course name: Claims Recognition;
Number of employees completed: 5;
Total hours of training completed: 20.
Course name: Communication-Nonverbal Cues;
Number of employees completed: 3;
Total hours of training completed: 1.5.
Course name: Computer Security and LAN Procedures;
Number of employees completed: 6;
Total hours of training completed: 12.
Course name: Conducting a Field Exam;
Number of employees completed: 3;
Total hours of training completed: 1.5.
Course name: Continuous Cohabitation (2 sessions conducted);
Number of employees completed: 20;
Total hours of training completed: 20.
Course name: Core Values;
Number of employees completed: 5;
Total hours of training completed: 5.
Course name: Control of Veterans Records System (3 sessions conducted);
Number of employees completed: 10;
Total hours of training completed: 12.5.
Course name: Customer Service (5 sessions conducted);
Number of employees completed: 40;
Total hours of training completed: 416.
Course name: Dealing with Difficult Payee Situations;
Number of employees completed: 3;
Total hours of training completed: 3.
Course name: Deemed Valid and Common Law Marriages (2 sessions
conducted);
Number of employees completed: 20;
Total hours of training completed: 12.5.
Course name: Dependency Issues (3 sessions conducted);
Number of employees completed: 22;
Total hours of training completed: 26.5.
Source: VBA.
Note: Baltimore's training plan reported the regional office conducted
a total of 191 courses for fiscal year 2007.
[End of table]
Table 8: Excerpt from Milwaukee Regional Office Training Plan:
Course name: 8824e;
Number of employees completed: 1;
Total hours of training completed: 1.
Course name: Administrative Decisions;
Number of employees completed: 14;
Total hours of training completed: 91.
Course name: Advanced Data Manipulation in Excel (VA Learning Online);
Number of employees completed: 1;
Total hours of training completed: 4.
Course name: All - Litigation Hold Memo;
Number of employees completed: 130;
Total hours of training completed: 32.5.
Course name: All-Encryption Training;
Number of employees completed: 1;
Total hours of training completed: 0.5.
Course name: Ancillary Benefits;
Number of employees completed: 21;
Total hours of training completed: 42.
Course name: Auto Allowance and Adaptive Equipment;
Number of employees completed: 28;
Total hours of training completed: 28.
Course name: Blast Injuries (Video);
Number of employees completed: 33;
Total hours of training completed: 33.
Course name: Board of Veterans Appeals review;
Number of employees completed: 7;
Total hours of training completed: 14.
Course name: Compensation & Pension Website;
Number of employees completed: 41;
Total hours of training completed: 102.5.
Course name: Claims Assistant - Burials;
Number of employees completed: 4;
Total hours of training completed: 4.
Course name: Claims Assistant/Program Support Clerk - Power of
Attorney;
Number of employees completed: 24;
Total hours of training completed: 24.
Course name: Claims Assistant/Program Support Clerk - Share and Cest;
Number of employees completed: 21;
Total hours of training completed: 178.5.
Course name: Claims Assistant/Program Support Clerk - Veterans Appeals
Control and Locator System;
Number of employees completed: 25;
Total hours of training completed: 25.
Course name: Cardiovascular Issues;
Number of employees completed: 30;
Total hours of training completed: 180.
Course name: Challenge 07-02 Centralized Training;
Number of employees completed: 6;
Total hours of training completed: 720.
Course name: Challenge 07-02 Post Centralized Training;
Number of employees completed: 6;
Total hours of training completed: 1440.
Course name: Challenge 07-02 Pre-Req.;
Number of employees completed: 6;
Total hours of training completed: 720.
Course name: Claims Folder Maintenance;
Number of employees completed: 41;
Total hours of training completed: 82.
Course name: Claims Recognition;
Number of employees completed: 26;
Total hours of training completed: 26.
Course name: Character of Discharge Determinations, Line of Duty
Determinations, and Administrative Decisions;
Number of employees completed: 11;
Total hours of training completed: 24.75.
Course name: Compensation Offsets;
Number of employees completed: 27;
Total hours of training completed: 94.5.
Course name: Core Values;
Number of employees completed: 2;
Total hours of training completed: 3.5.
Course name: Control of Veterans Records System (2 sessions conducted);
Number of employees completed: 2;
Total hours of training completed: 3.
Course name: Compensation and Pension Examination Project;
Number of employees completed: 1;
Total hours of training completed: 18.
Source: VBA.
Note: Milwaukee's training plan reported the regional office conducted
a total of 323 courses for fiscal year 2007.
[End of table]
Table 9: Excerpt from Portland Regional Office Training Plan:
Course name: 020 Development;
Number of employees completed: 3;
Total hours of training completed: 16.5.
Course name: 2007 Veterans Service Center Management Workshop;
Number of employees completed: 1;
Total hours of training completed: 26.
Course name: 3.105(e) Reductions;
Number of employees completed: 15;
Total hours of training completed: 15.
Course name: 38 CFR 3.14 & Pyramiding;
Number of employees completed: 2;
Total hours of training completed: 0.5.
Course name: 5-Tier Performance Evaluations;
Number of employees completed: 7;
Total hours of training completed: 5.25.
Course name: 8824 Preparation;
Number of employees completed: 1;
Total hours of training completed: 5.
Course name: Absence & Leave Circular Training;
Number of employees completed: 13;
Total hours of training completed: 13.
Course name: Account Analysis;
Number of employees completed: 3;
Total hours of training completed: 6.
Course name: Account Audits;
Number of employees completed: 3;
Total hours of training completed: 6.
Course name: Accrued Awards & the Benefits Delivery Network;
Number of employees completed: 2;
Total hours of training completed: 2.
Course name: Accrued Ratings;
Number of employees completed: 16;
Total hours of training completed: 4.
Course name: Add Dependents/Verifying Service;
Number of employees completed: 18;
Total hours of training completed: 9.
Course name: Admin Decisions/Rebuilt/Special Monthly Compensation;
Number of employees completed: 4;
Total hours of training completed: 18.
Course name: Administrative Decisions;
Number of employees completed: 5;
Total hours of training completed: 2.5.
Course name: Agent Orange development;
Number of employees completed: 4;
Total hours of training completed: 4.
Course name: Amputation Rule;
Number of employees completed: 2;
Total hours of training completed: 0.5.
Course name: Ancillary Benefits;
Number of employees completed: 28;
Total hours of training completed: 28.
Course name: Appeal Procedures--Refresher;
Number of employees completed: 3;
Total hours of training completed: 5.25.
Course name: Appeals;
Number of employees completed: 33;
Total hours of training completed: 33.
Course name: Appeals and Ancillary Benefits (3 sessions conducted);
Number of employees completed: 34;
Total hours of training completed: 13.
Course name: Appeals--Training and Performance Support System modules;
Number of employees completed: 1;
Total hours of training completed: 16.
Course name: Application/eligibility;
Number of employees completed: 1;
Total hours of training completed: 3.5.
Course name: Apportionments (2 sessions conducted); Number of employees
completed: 4; Total hours of training completed: 14.5.
Course name: Asbestos Claims Development;
Number of employees completed: 23;
Total hours of training completed: 23.
Course name: Access Standardized Performance Elements Nationwide;
Number of employees completed: 6;
Total hours of training completed: 6.
Source: VBA.
Note: Portland's training plan reported the regional office conducted a
total of 509 courses for fiscal year 2007.
[End of table]
[End of section]
Appendix IV: Comments from the Department of Veterans Affairs:
The Secretary Of Veterans Affairs:
Washington:
May 9, 2008:
Mr. Daniel Bertoni:
Director:
Education, Workforce, and Income Security Issues:
U.S. Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Mr. Bertoni:
The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Office's (GAO) draft report, Veterans' Benefits:
Increased Focus on Evaluation and Accountability Would Enhance Training
and Performance Management for Claims Processors (GAO-08-561) and
agrees with GAO's conclusions and concurs with GAO's recommendations.
Robust employee training and effective performance management are
critical to VA's goal of improving overall timeliness and accuracy of
veterans claims processing. VA will closely monitor and evaluate the
success of our efforts to enhance claims processor performance.
The enclosure addresses each of GAO's recommendations in detail and
provides technical comments for clarity and accuracy. VA appreciates
the opportunity to comment on your draft report.
Sincerely yours,
Signed by: James B. Peake, M.D.
Enclosure:
Department Of Veterans Affairs (VA) Comments To Government
Accountability Office (GAO) Draft Report:
Veterans' Benefits: Increased Focus on Evaluation and Accountability
Would Enhance Training and Performance Management for Claims Processors
(GAO-08-561):
GAO recommends that the Secretary of Veterans Affairs should direct VBA
to:
* Collect and review feedback from staff on the training conducted at
the regional offices to determine:
- if the 80-hour annual training requirement is appropriate for all
VSRs and RVSRs.
Concur. The Veterans Benefits Administration (VBA) has an active
program for training evaluation driven by the Administration's
priorities. Recent evaluations in support of improving veterans service
representative (VSR) and rating veterans service representative (RVSR)
training have included evaluation of VBA's Training and Performance
Support System (TPSS) (2006-2007 and 2007-2008) and Challenge training
for VSRs and RVSRs (2007-2008). The 80-hour training requirement for
VSRs and RVSRs was established in FY 2006 to improve the knowledge and
skills of claims processors with the ultimate goal of improving
accuracy and consistency. The 80-hour training requirement has been
evaluated annually by the Central Office staff and maintained at its
current level to improve the organization's ability to serve veterans.
VBA will consult with its regional offices and evaluate the 80-hour
training requirement for new and seasoned VSRs and RVSRs prior to
issuance of the training plan call letter for FY 2009.
- the extent to which regional offices provide training that is
relevant to VSRs' and RVSRs' work, given varying levels of staff
experiences.
Concur. With the introduction of more than 3,100 new VSRs and RVSRs
into regional offices during FY 2007 and FY 2008, VBA's priority in
training has been focused on building the knowledge and skill level of
new employees through consistent curriculum and delivery methods. VBA
will continue its initiatives to provide relevant field training that
will hinge on increasing use of the VA Learning Management System (VA
LMS) to identify training achievements and gaps at the individual
level. In 2006, VBA established a long-term initiative for advanced
planning of annual training within regional offices. In its initial
phase (2006-2008), regional offices developed and submitted annual
training plans that were reviewed by Central Office officials for
thoroughness and relevance. Training plans listed the classes that each
regional office would conduct for each position (e.g., VSR/RVSR),
including the projected date and designated instructors selected from
each regional office's more experienced personnel. The initiative's
second phase began with the fielding of VA LMS this year, which will
allow for recording and tracking of training on an individual employee
level. While many regional offices keep records on individual
employees' training, VA LMS will become the VBA-wide method of
collecting and reporting this information. In FY 2009, regional offices
will establish quarterly curricula for VSRs and RVSRs in VA LMS.
Regional offices have the option of prescribing different curricula for
varying experience levels. Central Office staff will review the
curricula for thoroughness and relevance at the administration level,
but direct supervisors in the field will determine relevance at the
individual level, given varying levels of staff experiences. With VA
LMS, supervisors will have easy access to the training records of their
subordinates, and, therefore, be able to determine the training needs
of each employee to tailor the training curriculum when appropriate.
- whether regional offices find the TPSS module a useful learning tool
and, if not, what adjustments are needed to make it more useful.
Concur. This action has been completed. VBA will continue to collect
and evaluate field opinions of TPSS during evaluations, as it has
during evaluations of TPSS (2006-2007 and 2007-2008) and Challenge
training for VSRs and RVSRs (2007-2008). While GAO's opportunities to
collect feedback were confined to interview of a limited number of
employees from four regional offices, VBA's three evaluations examined
data collected from 37 regional office visits, 470 interviews, and
2,718 survey responses. The 2006-2007 TPSS evaluation found that TPSS
provided value to VBA, and identified areas for improved implementation
within regional offices. Actions have been assigned for the findings of
the TPSS evaluation of 2006-2007, and will be assigned based upon the
findings of the other two evaluations when the reports are finalized.
* Use information from its new learning management system to hold
individual VSRs and RVSRs accountable for completing whatever annual
training requirement it determines is appropriate.
Concur. The Department has selected a learning management system that
places oversight responsibility on supervisors for ensuring training
requirements are met. To support the use of VA LMS in the field, VBA
has established full-time or part-time training managers in all
regional offices to support the directors' training information needs,
and is providing classes that enable use of VA LMS to record individual
training assignments and completions. The establishment of job-position
curricula will provide visibility on requirements and accomplishments
to supervisors at all levels, and to Central Office staff. VBA is
working closely with the Department's VA LMS program staff to develop
standardized reports to provide training progress reports to regional
office and Central Office directors to increase visibility on training
achievements and deficiencies.
* GAO recommends the Secretary of Veterans Affairs should also examine
the distribution of claims processing staff across overall performance
categories to determine if its performance appraisal system clearly
differentiates between overall performance levels, and if necessary
adjust its system to ensure that it makes clear distinctions.
Concur: VBA will examine how the ratings of claims processing staff are
distributed over the performance categories to determine if its
performance appraisal system clearly differentiates between overall
performance levels. If necessary, VBA will develop recommendations for
adjustment of VA's performance appraisal system.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Daniel Bertoni (202) 512-7215 bertonid@gao.gov.
Staff Acknowledgments:
In addition to the contact named above, Clarita Mrena, Assistant
Director; Lorin Obler, Analyst-in-Charge; Carolyn S. Blocker; and David
Forgosh made major contributions to this report; Margaret Braley, Peter
Del Toro, Chris Dionis, Janice Latimer, and Carol Willett provided
guidance; Walter Vance assisted with study design; Charles Willson
helped draft the report; and Roger Thomas provided legal advice.
[End of section]
Related GAO Products:
Veterans' Benefits: Improved Management Would Enhance VA's Pension
Program. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-112].
Washington, D.C.: February 14, 2008.
Veterans' Disability Benefits: Claims Processing Challenges Persist,
while VA Continues to Take Steps to Address Them. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-08-473T]. Washington, D.C.:
February 14, 2008.
Disabled Veterans' Employment: Additional Planning, Monitoring, and
Data Collection Efforts Would Improve Assistance. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-07-1020]. Washington, D.C.:
September 12, 2007.
Veterans' Benefits: Improvements Needed in the Reporting and Use of
Data on the Accuracy of Disability Claims Decisions. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-03-1045]. Washington, D.C.:
September 30, 2003.
Human Capital: A Guide for Assessing Strategic Training and Development
Efforts in the Federal Government. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-03-893G]. Washington, D.C.: July 2003.
Results-Oriented Cultures: Creating a Clear Linkage between Individual
Performance and Organizational Success. [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-03-488]. Washington D.C.: March
14, 2003.
Major Management Challenges and Program Risks: Department of Veterans
Affairs. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-110].
Washington, D.C.: January 1, 2003.
Veterans' Benefits: Claims Processing Timeliness Performance Measures
Could Be Improved. [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-03-
282]. Washington, D.C.: December 19, 2002.
Veterans' Benefits: Quality Assurance for Disability Claims and Appeals
Processing Can Be Further Improved. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-02-806]. Washington, D.C.: August 16, 2002.
Veterans' Benefits: Training for Claims Processors Needs Evaluation.
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-601]. Washington,
D.C.: May 31, 2001.
Veterans Benefits Claims: Further Improvements Needed in Claims-
Processing Accuracy. [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/HEHS-99-35]. Washington, D.C.: March 1, 1999.
[End of section]
Footnotes:
[1] These practices are laid out primarily in two GAO reports: Human
Capital: A Guide for Assessing Strategic Training and Development
Efforts in the Federal Government, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-04-546G] (Washington, D.C.: March 2004) and Results
Oriented Cultures: Creating a Clear Linkage between Individual
Performance and Organizational Success, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-03-488] (Washington, D.C.: Mar.
14, 2003).
[2] 38 U.S.C. § 5312(a).
[3] While new claims processors are on probation, 100 percent of the
claims work they perform is quality reviewed by a supervisor. After
their probationary period, only a small sample of their claims is
quality reviewed.
[4] Typically, RVSRs are promoted VSRs, although in some instances, VA
hires RVSRs from outside of VA who have medical or legal experience.
[5] Staff who teach classes other than centralized training are not
required to take the week-long Instructor Development Course, although
they may do so if openings exist. They can also take an 8-hour
condensed course for regional instructors.
[6] In 2001, we reported that VBA had spent or obligated about $18.6
million of the estimated total TPSS program cost of $32 million. See
GAO, Veterans' Benefits: Training for Claims Processors Needs
Evaluation, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-01-601]
(Washington, D.C.: May 31, 2001).
[7] VBA defines an experienced VSR or RVSR as one who has been in that
position for 1 year or more.
[8] To determine if VSRs and RVSRs in a regional office are generally
meeting their annual training requirement, the aggregate number of
training hours completed in a given year by all staff in that office is
divided by the number of staff in that office.
[9] According to VBA officials, the board is made up of a mix of
regional office and central office staff from different VBA business
lines including Employee Development and Training, Human Resources, the
Compensation and Pension Service, and the Insurance Service.
[10] GAO, Veterans' Benefits: Improved Management Would Enhance VA's
Pension Program, [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-
112] (Washington, D.C.: Feb. 14, 2008).
[11] In 2001, GAO reported that VBA's TPSS may not fully achieve its
objectives of providing standardized training to new employees,
reducing the training period required for new employees, or improving
claims-processing accuracy and consistency. In the report, we
recommended actions the agency should consider in providing timely
standardized training and providing indicators of the impact of TPSS on
accuracy and consistency. In its technical comments on this report, VA
indicated it accomplished the first recommendation. See [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-01-601].
[12] The central office has set a minimum performance level for each
element that defines the fully successful level of performance.
Regional offices may set higher fully successful levels for their
staff, and three of the offices we visited had set a higher level for
at least one element. Regional offices also have discretion to set the
level for exceptional performance in each element for their staff.
[13] See GAO, Human Capital: Preliminary Observations on the
Administration's Draft Proposed "Working for America Act," [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-06-142T] (Washington, D.C.: Oct.
5, 2005).
[14] In three of the four offices we visited, staff members placed in
the outstanding and excellent categories receive bonuses, and in one of
these offices some staff in the fully successful category also receive
bonuses.
[15] Officials in the other two offices we visited reported no problems
with the performance appraisal formula. Officials in one of these
offices told us the current five-level system provides more flexibility
than the previous pass/fail system.
[16] We asked VA for fiscal year 2007 performance appraisal data for
VSRs and RVSRs nationally to determine whether the distribution of
staff across overall performance categories is similar at the national
level. While VA indicated that it collects performance appraisal data
for regional office staff, the agency was unable to provide us with
appraisal data specifically for VSRs and RVSRs, as these positions are
part of a broader job series.
[17] To determine each office's accuracy performance in fiscal year
2007, we used data obtained from VBA's Systematic Technical Accuracy
Review (STAR) system. In an earlier GAO report, Veterans' Benefits:
Further Changes in VBA's Field Office Structure Could Help Improve
Disability Claims Processing, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-06-149] (Washington, D.C.: Dec. 9, 2005), we identified
problems that affected the use of the STAR data to make distinctions in
accuracy among regional offices. However, for the purposes of site
selection for our current review, we judged the STAR data to be
sufficiently reliable. We made this determination based on a
sensitivity analysis we did on earlier year data that considered
sampled cases that were not sent in for STAR review. After this
analysis we found that even with the existing limitations in the STAR
data, Milwaukee and Baltimore had higher accuracy scores and Atlanta
and Portland had lower accuracy scores. Even though the sensitivity
analysis was done on earlier year data, the ranking of the four offices
was similar in fiscal year 2007, showing that the offices we deemed to
have higher accuracy scores in an earlier year still had higher
accuracy scores in fiscal year 2007 and the same remained true for the
offices with lower accuracy scores.
[18] One question we asked the regional offices was whether each course
on their fiscal year 2007 training plan addressed a core technical
training topic. For three of the offices, the data we received did not
cover all training hours provided during the fiscal year, but each
office provided data on at least 99 percent of its training hours.
[19] GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-04-546G] (Washington, D.C.: March
2004).
[20] GAO, Results-Oriented Cultures: Creating a Clear Linkage between
Individual Performance and Organizational Success, [hyperlink,
http://www.gao.gov/cgi-bin/getrpt?GAO-03-488] (Washington, D.C.: Mar.
14, 2003).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office:
441 G Street NW, Room LM:
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
