VA Health Care
VA Needs to Improve Accuracy of Reported Wait Times for Blind Rehabilitation Services Gao ID: GAO-04-949 July 22, 2004The Department of Veterans Affairs (VA) provides rehabilitation services to legally blind veterans. These services are intended to help them acquire the skills necessary to become more independent. Almost all of VA's rehabilitation services for legally blind veterans are provided at Blind Rehabilitation Centers (BRC), an inpatient program. VA reported that the average length of time a veteran waited to be admitted to a BRC increased from 168 to 210 days from fiscal years 1999 through 2003. GAO was asked to examine the accuracy of veterans' wait times for admission to BRCs. GAO's objective was to determine whether the average wait times for veterans seeking admission to BRCs reported by VA were accurate. GAO reviewed VA policies and procedures for determining the average length of time veterans wait to be admitted to a BRC. GAO also visited 5 of VA's 10 BRCs to evaluate the reliability of the data used to calculate wait times.
GAO found that the average length of time VA reported that veterans wait for admission to BRCs was inaccurate. Some data used to calculate wait times were incomplete or incorrect. For example, at one BRC GAO found that one or more of the data elements used to calculate the wait times--the date the BRC received the application, the earliest admission date offered to the veteran, and the date the veteran was admitted to the BRC--were missing from 31 percent of the records and incorrect in 13 percent of the records. GAO also found missing or inaccurate data at two other BRCs. In addition, GAO found that BRCs used different procedures for their calculations, which also contributed to the inaccurate average wait times. For example, two BRCs correctly ended the wait times calculations on the earliest admission date offered to the veteran, while the other three BRCs ended the wait times calculations on the date the veteran was admitted to the BRC. To enable VA to accurately assess wait times, it is essential for VA to develop more comprehensive instructions to calculate average wait times and for BRCs to adhere to them.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-949, VA Health Care: VA Needs to Improve Accuracy of Reported Wait Times for Blind Rehabilitation Services
This is the accessible text file for GAO report number GAO-04-949
entitled 'VA Health Care: VA Needs to Improve Accuracy of Reported Wait
Times for Blind Rehabilitation Services' which was released on July 22,
2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Requesters:
United States Government Accountability Office:
GAO:
July 2004:
VA Health Care:
VA Needs to Improve Accuracy of Reported Wait Times for Blind
Rehabilitation Services:
GAO-04-949:
GAO Highlights:
Highlights of GAO-04-949, a report to congressional requesters.
Why GAO Did This Study:
The Department of Veterans Affairs (VA) provides rehabilitation
services to legally blind veterans. These services are intended to help
them acquire the skills necessary to become more independent. Almost
all of VA‘s rehabilitation services for legally blind veterans are
provided at Blind Rehabilitation Centers (BRC), an inpatient program.
VA reported that the average length of time a veteran waited to be
admitted to a BRC increased from 168 to 210 days from fiscal years 1999
through 2003. GAO was asked to examine the accuracy of veterans‘ wait
times for admission to BRCs. GAO‘s objective was to determine whether
the average wait times for veterans seeking admission to BRCs reported
by VA were accurate. GAO reviewed VA policies and procedures for
determining the average length of time veterans wait to be admitted to
a BRC. GAO also visited 5 of VA‘s 10 BRCs to evaluate the reliability
of the data used to calculate wait times.
What GAO Found:
GAO found that the average length of time VA reported that veterans
wait for admission to BRCs was inaccurate. Some data used to calculate
wait times were incomplete or incorrect. For example, at one BRC GAO
found that one or more of the data elements used to calculate the wait
times”the date the BRC received the application, the earliest admission
date offered to the veteran, and the date the veteran was admitted to
the BRC”were missing from 31 percent of the records and incorrect in 13
percent of the records. GAO also found missing or inaccurate data at
two other BRCs. In addition, GAO found that BRCs used different
procedures for their calculations, which also contributed to the
inaccurate average wait times. For example, two BRCs correctly ended
the wait times calculations on the earliest admission date offered to
the veteran, while the other three BRCs ended the wait times
calculations on the date the veteran was admitted to the BRC. To enable
VA to accurately assess wait times, it is essential for VA to develop
more comprehensive instructions to calculate average wait times and for
BRCs to adhere to them.
What GAO Recommends:
GAO recommends that the Secretary of Veterans Affairs direct the Under
Secretary for Health to instruct (1) the program office to develop more
specific instructions for calculating wait times and (2) the BRCs to
adhere to these instructions by developing procedures to compile
complete and accurate information. VA concurred with GAO‘s
recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-04-949.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Cynthia A. Bascetta at
(202) 512-7101.
Contents:
Letter:
Results in Brief:
Background:
VA's Reported Wait Times for Accessing BRCs Were Inaccurate:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Scope and Methodology:
Appendix II: Comments from the Department of Veterans Affairs:
Table:
Table 1: Location of VA's Blind Rehabilitation Centers and the Number
of Authorized Beds:
Abbreviations:
BRC: Blind Rehabilitation Center VADepartment of Veterans Affairs
VIST: Visual Impairment Service Team:
United States Government Accountability Office:
Washington, DC 20548:
July 22, 2004:
The Honorable Bob Graham:
Ranking Minority Member:
Committee on Veterans' Affairs:
United States Senate:
The Honorable Rob Simmons:
Chairman:
Subcommittee on Health:
Committee on Veterans' Affairs:
House of Representatives:
The Department of Veterans Affairs (VA) provides rehabilitation
training services for legally blind[Footnote 1] veterans that are
intended to enable them to acquire the skills necessary to develop
personal independence, such as using magnification devices for reading.
Almost all of VA's rehabilitation services for legally blind veterans
are provided at Blind Rehabilitation Centers (BRC), a residential
inpatient program. VA reported that the average length of time a
veteran waited to be admitted to a BRC increased from 168 to 210 days
from fiscal years 1999 through 2003. In 2003, VA reported that
approximately 2,100 legally blind veterans received care in BRCs, with
average wait times ranging from 77 to 343 days.
Congress has been concerned for more than 10 years about the length of
time veterans wait to receive training at BRCs. In 1993 and again in
1995, Congress noted that veterans were unable to access services at
BRCs in a timely manner and encouraged VA to provide more services. You
expressed concern about the accuracy of the length of time VA reports
that veterans wait for admission to BRCs. Our objective was to
determine whether the average wait times for veterans seeking admission
to BRCs reported by VA were accurate.
To assess the accuracy of VA's reported wait times, we visited 5 of
VA's 10 BRCs that we selected based on differences in geographical
location and the number of beds available at the BRC. We met with BRC
officials and other VA employees who manage the health care provided to
legally blind veterans. We reviewed the procedures they followed to
calculate veterans' wait times and analyzed documents[Footnote 2] to
evaluate the reliability of the data used to calculate wait times at 3
of the BRCs. We were unable to assess the data at the other 2 BRCs
because application packages were not available. After veterans were
discharged, 1 BRC required that the application packages be destroyed
and the other BRC returned the packages to the medical facility that
referred the veteran. We also met with officials from VA's Blind
Rehabilitation Service Program Office (program office), which has
responsibility for blindness-related services, to review its policies
and procedures regarding wait times calculations. Our review was
conducted from September 2003 through July 2004 in accordance with
generally accepted government auditing standards. (See app. I for
additional information on our scope and methodology.)
Results in Brief:
The average length of time VA reported that veterans wait for admission
to BRCs was inaccurate. Some data used to calculate wait times--the
date the BRC received the application, the earliest admission date
offered to the veteran, and the date of admission--were incomplete or
incorrect. In addition, we found that BRCs used different procedures
for their calculations, which also contributed to the inaccurate
average wait times. We are making two recommendations to improve the
accuracy of VA's reported BRC wait times. VA agreed with our findings
and recommendations and indicated that it will provide a detailed
action plan.
Background:
In 2003, VA estimated that about 157,000 veterans were legally blind,
about 44,000 of them were enrolled in its health care system, and about
2,100 received rehabilitation training in BRCs. The blind
rehabilitation program is designed to improve the quality of life for
veterans who are legally blind. VA's Visual Impairment Service Team
(VIST) coordinators are responsible for managing legally blind
veterans' access to rehabilitation services, including reaching a
decision with the veteran about appropriate training and other
services. VIST coordinators prepare the veteran's application to a BRC
and ensure that the veteran has the medical examinations required for
admission, such as hearing examinations and low vision testing. VA's 10
BRCs provide training to legally blind veterans on an inpatient
residential basis. Table 1 lists the location of the BRCs and the
number of beds authorized for blind rehabilitation services.
Table 1: Location of VA's Blind Rehabilitation Centers and the Number
of Authorized Beds:
Location: American Lake, Washington[B];
Authorized beds[A]: 15.
Location: Augusta, Georgia[B];
Authorized beds[A]: 15.
Location: Birmingham, Alabama;
Authorized beds[A]: 32.
Location: Hines, Illinois[B];
Authorized beds[A]: 34.
Location: Palo Alto, California;
Authorized beds[A]: 32.
Location: San Juan, Puerto Rico;
Authorized beds[A]: 12.
Location: Tucson, Arizona[B];
Authorized beds[A]: 34.
Location: Waco, Texas;
Authorized beds[A]: 15.
Location: West Haven, Connecticut;
Authorized beds[A]: 34.
Location: West Palm Beach, Florida[B];
Authorized beds[A]: 18.
Location: Total;
Authorized beds[A]: 241.
Source: VA.
[A] As of May 2004.
[B] BRCs evaluated by GAO.
[End of table]
The training BRCs offer is comprehensive and individualized with the
goal of helping legally blind veterans meet their personal goals and
achieve a realistic level of independence. They offer a basic
rehabilitation program and computer training. The basic program trains
veterans in such areas as orientation and mobility (e.g., moving around
the home or using a cane to travel through different environments) or
visual skills (e.g., using closed circuit television or magnification
devices to read or write checks). Computer training teaches veterans
how to operate a computer; search the Internet; and send, receive, and
read e-mail messages.
Each month, BRCs are required to submit a report to the program office
that includes their calculation of the average number of days veterans
waited to be admitted to a BRC for training during the past 6 months.
VA defines an applicant's wait time as the number of days that elapse
from the date the BRC receives the application to the earliest
admission date offered to the veteran.
VA's Reported Wait Times for Accessing BRCs Were Inaccurate:
Wait times reported by the program office were not accurate for two
reasons. First, we found that some data VA used to calculate wait times
were missing and some that it used were incorrect. Second, VA did not
use consistent procedures to calculate wait times.
Data Used to Report Wait Times Were Missing or Inaccurate:
We determined that data elements BRCs used for calculating wait times
for both basic and computer training--the date the BRC received the
application, the earliest admission date offered to the veteran, and
the date the veteran was admitted to the BRC--were missing or
inaccurate. For example, during our review of data at one BRC, we found
that one or more of the data elements used to calculate the wait times
were missing from 31 percent of the records and incorrect in 13 percent
of the records. We also noted missing or incorrect data elements during
our review of records at two other BRCs. Specifically, at one BRC we
reviewed 30 records and found missing or inaccurate data in 24 of them.
At the other BRC, we found inaccurate data in 8 of 16 records. At the
remaining two BRCs, we could not validate the completeness or accuracy
of records used to calculate wait times because application packages
were not available. After veterans completed their training, one BRC
required that the application packages be destroyed and the other BRC
returned the packages to the medical facility that referred the
veteran. In addition, we found weaknesses in the way the BRCs ensured
complete and accurate data. For example, none of the BRCs stated they
validated data entry. Also, none of them checked their databases for
completeness (Is the data element filled in or has it been left blank?)
and reasonableness (Is the date the BRC received the application
earlier than the first admission date offered to the veteran?).
Further, one of them did not check for format accuracy (If the data
element is a date, is it formatted as a date, such as 5/24/2004?).
Procedures Used to Calculate Wait Times Were Inconsistent:
We found that the five BRCs we visited followed different procedures
for determining the average time a veteran must wait for admission to a
BRC. According to VA instructions, the BRC is to average the number of
days that elapsed from the date the application was received to the
earliest admission date offered to the veteran for veterans admitted
during the past 6 months. However, the instructions do not stipulate
whether the application package must be complete before putting the
veteran on the wait list or the length of time BRCs allow VIST
coordinators to submit the missing information. This lack of
specificity resulted in BRCs using different procedures to calculate
their wait times.
Although all five BRCs began their calculations when they received the
veteran's application--even if it was not complete--they allowed VIST
coordinators different lengths of time to provide the missing
information. According to the BRC officials at these locations, the
BRCs accepted incomplete packages as long as the basic information
needed to process the application was present, including documents to
prove eligibility, a recent physical examination, and an eye
examination proving legal blindness. For example, one BRC official
stated that the results of hearing examinations were not essential to
approve applications but were needed before veterans were admitted for
training. VIST coordinators who submitted incomplete applications could
get their veterans on the wait list earlier than VIST coordinators who
waited to submit complete applications. BRCs allowed different lengths
of time for VIST coordinators to provide missing information before
removing veterans from the wait list. For example, one BRC removed
veterans with incomplete applications from the wait list after 30 days,
while another waited at least 6 months.
Not all BRCs used the same procedures to stop the wait times
calculation. The program office instructs BRCs to stop the wait times
calculation on the earliest admission date offered to the veteran. Two
BRCs correctly ended the wait times calculation on that date, while the
other three ended the wait times on the date the veteran was actually
admitted, even if the veteran had refused earlier admission dates. For
example, one veteran was offered an admission date of October 27th. The
veteran declined that date and two subsequently offered dates and was
finally admitted on June 7th of the following year. Because the
veteran's first offer date was October 27th, according to the program
office instructions the wait times should have stopped on that date.
Instead, the BRC included the additional time between October 27th and
June 7th in its wait times calculation. In this example, the BRC
calculation overstated the veteran's wait times by over 7 months.
Conclusions:
While VA reports BRC wait times for veterans, the information does not
accurately reflect the time veterans wait for admission. Data used for
calculations are often incomplete or inaccurate. In addition, BRCs
followed different procedures for calculating wait times because VA's
instructions do not specify whether application packages must be
complete before putting the veteran on the wait list or the length of
time BRCs allow VIST coordinators to submit the missing information.
For VA to accurately assess wait times, it is essential for VA to
develop a consistent process for determining average wait times and for
BRCs to adhere to it.
Recommendations for Executive Action:
To improve VA's ability to effectively manage care for legally blind
veterans, we recommend that the Secretary of Veterans Affairs direct
the Under Secretary for Health to instruct (1) the program office to
develop more specific instructions for calculating wait times and (2)
the BRCs to adhere to these instructions by developing procedures to
compile complete and accurate information on the length of time
veterans wait for admission to BRCs.
Agency Comments:
In commenting on the draft of this report, VA agreed with our findings
and conclusions. VA stated that the report accurately conveyed the
variability and complexity of reporting wait times for admission to
BRCs. VA concurred with our recommendations and indicated that it will
provide a detailed action plan. VA's written comments are reprinted in
appendix II.
We are sending copies of this report to the Secretary of Veterans
Affairs and other interested parties. We also will make copies
available to others upon request. In addition, the report will be
available at no charge at the GAO Web site at http://www.gao.gov. If
you or your staff have any questions about this report, please call me
at (202) 512-7101 or Michael T. Blair, Jr. at (404) 679-1944. Cherie
Starck, Cynthia Forbes, and Janet Overton were key contributors to this
report.
Sincerely,
Signed by:
Cynthia A. Bascetta
Director, Health Care--Veterans' Health and Benefits Issues:
[End of section]
Appendix I: Scope and Methodology:
To determine whether VA accurately reports average wait times for
veterans seeking admission to Blind Rehabilitation Centers (BRC), we
examined the policies and procedures VA used to calculate the length of
time veterans wait for admission. We met with and obtained
documentation from officials at the Blind Rehabilitation Service
Program Office and from the five BRCs we visited. We selected the five
BRCs located in Tucson, Arizona; West Palm Beach, Florida; Augusta,
Georgia; Hines, Illinois; and American Lake, Washington, based on
differences in geographical location and the number of beds available
at the BRC. We met with BRC officials and veterans' case managers. We
reviewed documentation on the procedures BRC staff followed to
calculate veterans' wait times to assess the quality of the data the
five BRCs maintained. We also reviewed veterans' paper application
packages for BRC admission, printouts from the BRC's electronic
database used to manage the wait list, and veteran's computerized
medical records.
At the first BRC we visited, we compared the date the application was
received at the BRC in the electronic and paper files for a
judgmentally selected sample of 16 applicants. We identified data
elements that were inaccurate in either the paper or electronic records
in 8 of the 16 records. At the second BRC, we chose a random sample of
100 veterans, stratified by fiscal year, from the 578 that were
admitted in fiscal years 2000 through 2003. A BRC official then
provided printed copies of the data from the electronic file used to
calculate the wait times and the folders containing the paper
application packages. We reviewed the selected files to compare the
data elements used for calculating wait times--the date the BRC
received the application, the earliest admission date offered to the
veteran, and the date the veteran was admitted to the BRC--from the two
sources. We determined that one or more of these data elements were
missing from 31 percent of the records with a margin of error of plus
or minus 8 percent. Additionally, we found that one or more of these
data elements were inaccurate in 13 percent of the records with a
margin of error of plus or minus 6 percent.[Footnote 3]
For the remaining three BRCs, we attempted to verify that the problems
we found at the first two BRCs were also found at the other facilities.
At the third BRC we randomly sampled 10 admissions each from fiscal
years 2001 through 2003.[Footnote 4] A BRC official provided printed
copies of the data from the electronic file used to calculate the wait
times and the folders containing the paper application packages. We
reviewed the two sources to compare the data elements used for
calculating wait times. To determine if similar data problems also
existed at this BRC, we compared the percentage of inaccurate or
missing data elements to a tolerable level of 10 percent--that is, not
less than 10 percent of the records at this BRC had one or more
inaccurate or missing data elements. We found that 24 of the 30 records
had inaccurate or missing data elements. We confirmed that the percent
of inaccurate and incomplete data elements was not less than 10 percent
by performing a one-sided significance test at the 95-percent
confidence level. We were unable to assess the data at the other two
BRCs because application packages were not available. After veterans
completed their training, one BRC required that the application
packages be destroyed and the other BRC returned the packages to the
medical facility that referred the veteran. Our results from these five
BRCs cannot be generalized to other facilities.
Appendix II: Comments from the Department of Veterans Affairs:
THE SECRETARY OF VETERANS AFFAIRS
WASHINGTON
July 9, 2004:
Ms. Cynthia A. Bascetta
Director:
Health Care Team:
U. S. General Accounting Office
441 G Street, NW
Washington, DC 20548:
Dear Ms. Bascetta:
The Department of Veterans Affairs (VA) has reviewed the General
Accounting Office's (GAO) draft report, VA HEALTH CARE: VA Needs to
Improve Accuracy of Reported Wait Times for Blind Rehabilitation
Services (GAO-04-949) and agrees with your conclusions. The report
accurately conveys the variability and complexity of reporting wait
times for admission to the Veterans Health Administration's (VHA) Blind
Rehabilitation Centers based on data from 3 of its 10 centers. GAO
addresses the need for a systems approach to data management leading to
less variability and more consistency in reporting wait times.
Compliance from all the Blind Rehabilitation Centers is an important
component to the systems change and will provide consistency in VA's
data collection and reporting. VA concurs with GAO's recommendations.
The Blind Rehabilitation Program Office recognized the need for
consistency and standardization and is developing an electronic system
for reporting wait times. Implementation is part of the Information
Technology (IT) update of the Blind Rehabilitation IT package Version
5.0. VA anticipates completing implementation in the fourth quarter of
Fiscal Year 2005. The program office, in conjunction with the Office of
the Deputy Under Secretary for Health for Operations and Management,
will develop compliance reporting requirements for Blind Rehabilitation
Centers. VA believes this will ensure accountability for accurate data
calculation through medical center and network directors. In the
interim, VHA will clarify the current policy so that the BRCS will
report wait times data more accurately and consistently.
The Department will provide a detailed action plan in its response to
GAO's final report. VA appreciates the opportunity to comment on your
draft report.
Sincerely yours,
Signed by:
Anthony J. Principi:
[End of section]
FOOTNOTES
[1] VA defines legal blindness as when the patient's best-corrected
central visual acuity with ordinary eyeglasses or contacts is 20/200 or
less in the better eye (as measured by the Snellen Visual Acuity Chart)
or when the field of useful vision is 20 degrees or less in the better
eye. People who are legally blind can only at best read the big "E" on
the eye chart or see as if they are looking through a paper towel tube,
according to VA.
[2] These documents included veterans' applications for BRC admission,
printouts from the BRC's electronic database used to manage the wait
list, and veterans' computerized medical records.
[3] The categories of records with missing data elements and those with
inaccurate data elements are not mutually exclusive.
[4] We had planned to sample from fiscal years 2000 through 2003 but
records from fiscal year 2000 had been destroyed.
GAO's Mission:
The General Accounting Office, the audit, evaluation and investigative
arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site (www.gao.gov) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office 441 G Street NW, Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000
TDD: (202) 512-2537
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, Managing Director,
NelliganJ@gao.gov
(202) 512-4800
U.S. General Accounting Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
