Medicare
Accuracy of Responses from the 1-800-MEDICARE Help Line Should Be Improved
Gao ID: GAO-05-130 December 8, 2004
In March 1999, the Centers for Medicare & Medicaid Services (CMS) implemented a telephone help line--1-800-MEDICARE--to provide information about program eligibility, enrollment, and benefits. The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (MMA) directed GAO to examine several issues related to this 24-hour help line and the customer service representatives (CSRs) who staff it. In this report, GAO evaluated (1) the accuracy of the information the help line provides, (2) the training given to CSRs, and (3) CMS's efforts to monitor the accuracy of information provided through the help line.
The 1-800-MEDICARE help line provided accurate answers to 61 percent of the 420 calls we made and inaccurate answers to 29 percent. We were not able to obtain any answers for the remaining 10 percent of our calls at the time we placed them. Most of these calls were not answered because they were transferred to other contractors responsible for processing Medicare claims that were not open for business at the time we called or these calls were inadvertently disconnected. To facilitate accurate responses, the 1-800-MEDICARE help line provides CSRs with written answers--called "scripts"--that CSRs use during a call. When CSRs provided inaccurate information, it was largely because they did not seem to access and effectively use a script that answered our questions. CMS and its contractor do not routinely pretest the scripts to ensure that they are understandable to CSRs or potential callers. The training for CSRs meets CMS's requirements, but it is not sufficient to ensure that CSRs are able to answer questions accurately on the help line. Before handling calls, CSRs must complete about 2 weeks of classroom training; accurately answer two simulated calls consecutively out of six; and score at least 90 percent on a written exam. In addition, all CSRs receive ongoing training. However, the results from our calls indicate that the testing and simulated call answering did not sufficiently measure whether CSRs were prepared to answer questions accurately. CMS delegates most accuracy monitoring to one of its contractors and reviews the results. The bulk of the monitoring focuses on how accurately individual CSRs answer questions. However, this monitoring does not systematically track questions answered inaccurately by CSRs as a group, which could help target training and script improvement. Through two smaller studies that measured how accurately specific questions were answered, CMS was able to identify areas to improve scripts and training.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director:
Team:
Phone:
GAO-05-130, Medicare: Accuracy of Responses from the 1-800-MEDICARE Help Line Should Be Improved
This is the accessible text file for GAO report number GAO-05-130
entitled 'Medicare: Accuracy of Responses from the 1-800-MEDICARE Help
Line Should Be Improved' which was released on December 08, 2004.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
December 2004:
Medicare:
Accuracy of Responses from the 1-800-MEDICARE Help Line Should Be
Improved:
GAO-05-130:
GAO Highlights:
Highlights of GAO-05-130, a report to congressional committees
Why GAO Did This Study:
In March 1999, the Centers for Medicare & Medicaid Services (CMS)
implemented a telephone help line”1-800-MEDICARE”to provide information
about program eligibility, enrollment, and benefits. The Medicare
Prescription Drug, Improvement, and Modernization Act of 2003 (MMA)
directed GAO to examine several issues related to this 24-hour help
line and the customer service representatives (CSRs) who staff it. In
this report, GAO evaluated (1) the accuracy of the information the help
line provides, (2) the training given to CSRs, and (3) CMS‘s efforts to
monitor the accuracy of information provided through the help line.
What GAO Found:
The 1-800-MEDICARE help line provided accurate answers to 61 percent of
the 420 calls we made and inaccurate answers to 29 percent. We were not
able to obtain any answers for the remaining 10 percent of our calls at
the time we placed them. Most of these calls were not answered because
they were transferred to other contractors responsible for processing
Medicare claims that were not open for business at the time we called
or these calls were inadvertently disconnected. To facilitate accurate
responses, the 1-800-MEDICARE help line provides CSRs with written
answers”called ’scripts“”that CSRs use during a call. When CSRs
provided inaccurate information, it was largely because they did not
seem to access and effectively use a script that answered our
questions. CMS and its contractor do not routinely pretest the scripts
to ensure that they are understandable to CSRs or potential callers.
Percentage of Calls with Accurate, Inaccurate, and No Answers:
[See PDF for image]
Note: Based on 420 calls placed in July 2004.
[End of figure]
The training for CSRs meets CMS‘s requirements, but it is not
sufficient to ensure that CSRs are able to answer questions accurately
on the help line. Before handling calls, CSRs must complete about 2
weeks of classroom training; accurately answer two simulated calls
consecutively out of six; and score at least 90 percent on a written
exam. In addition, all CSRs receive ongoing training. However, the
results from our calls indicate that the testing and simulated call
answering did not sufficiently measure whether CSRs were prepared to
answer questions accurately.
CMS delegates most accuracy monitoring to one of its contractors and
reviews the results. The bulk of the monitoring focuses on how
accurately individual CSRs answer questions. However, this monitoring
does not systematically track questions answered inaccurately by CSRs
as a group, which could help target training and script improvement.
Through two smaller studies that measured how accurately specific
questions were answered, CMS was able to identify areas to improve
scripts and training.
What GAO Recommends:
To improve the accuracy of the information the help line provides, GAO
recommends that CMS
(1) revise procedures so that calls are not transferred to other
contractors that are closed,
(2) assess current scripts and pretest new and revised scripts to
ensure that they are understandable, (3) provide more testing of CSRs‘
ability to accurately answer questions and use the results to target
training efforts as needed, and (4) monitor the accuracy rate for each
frequently asked question and use the results to modify scripts or
provide training, if necessary. CMS agreed with the recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-05-130.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Leslie G. Aronovitz at
(312) 220-7600.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
1-800-MEDICARE Provided Accurate Information for Less Than Two-thirds
of the Calls We Placed:
CSR Training Met CMS's Requirements, But Was Not Sufficient to Ensure
Accurate Responses:
Monitoring Generally Focused on Performance of Individual CSRs, Rather
than Accuracy Rates for Specific Questions:
Conclusions:
Recommendations:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix IIPrescription Drug Discount Card Questions and Information
Used to Develop Accuracy Criteria:
Appendix III: $600 Credit Question and Information Used to Develop
Accuracy Criteria:
Appendix IV: Medigap Question and Information Used to Develop Accuracy
Criteria:
Appendix V: Power Wheelchair Question and Information Used to Develop
Accuracy Criteria:
Appendix VI: Medicare Part B Question and Information Used to Develop
Accuracy Criteria:
Appendix VII: Eye Glasses/Exam Question and Information Used to Develop
Accuracy Criteria:
Appendix VIII: Comments from the Department of Health & Human Services:
Appendix IX: GAO Contact and Staff Acknowledgments:
GAO Contact:
Acknowledgments:
Tables:
Table 1: Questions and Criteria Used to Determine Answer Accuracy:
Table 2: Summary of Unanswered Calls by Question:
Figure:
Figure 1: Percentage of Calls with Accurate, Inaccurate, and No
Answers, by Question:
Abbreviations:
CDC: Centers for Disease Control and Prevention:
CMS: Centers for Medicare & Medicaid Services:
CSR: customer service representative:
IRS: Internal Revenue Service:
MMA :Medicare Prescription Drug, Improvement, and Modernization Act of
2003:
United States Government Accountability Office:
Washington, DC 20548:
December 8, 2004:
The Honorable Charles E. Grassley:
Chairman:
The Honorable Max Baucus:
Ranking Minority Member:
Committee on Finance:
United States Senate:
The Honorable Joe Barton:
Chairman:
The Honorable John D. Dingell:
Ranking Minority Member:
Committee on Energy and Commerce:
House of Representatives:
The Honorable William M. Thomas:
Chairman:
The Honorable Charles B. Rangel:
Ranking Minority Member:
Committee on Ways and Means:
House of Representatives:
Medicare is a federal program that helps pay for a variety of health
care services and items for approximately 41 million elderly and
disabled beneficiaries. One of the responsibilities of the Centers for
Medicare & Medicaid Services (CMS), the federal agency that administers
Medicare, is to provide beneficiaries and other members of the public
with clear, accurate, and timely information about the program. To help
do so, in March 1999, CMS implemented a nationwide toll-free telephone
help line--1-800-MEDICARE--which Medicare beneficiaries, their
families, and other members of the public can call to ask questions
about program eligibility, enrollment, and benefits. By 2001, the help
line had customer service representatives (CSR) answering calls 24
hours a day, 7 days a week. For information about coverage and payment
for medical services and items, beneficiaries and others have also been
able to call the companies under contract with CMS to process and pay
Medicare claims.[Footnote 1]
In 2004, the 1-800-MEDICARE help line significantly expanded its
operations in order to handle an increased number of calls. This
increase occurred after the enactment of the Medicare Prescription
Drug, Improvement, and Modernization Act of 2003 (MMA) in December
2003, which established a prescription drug discount card program for
Medicare beneficiaries and required that information on this new
program be made available through the 1-800-MEDICARE help line. During
the 6 months following the enactment of MMA, the 1-800-MEDICARE help
line handled over 9 million calls, more than triple the number handled
in the previous 6 months. Many of these callers asked about the
prescription drug benefit that will be available beginning in 2006, the
prescription drug discount cards available in the interim, and the $600
credit for prescription drugs purchased by low-income beneficiaries
with the cards. In response to the increased call volume, in the first
half of 2004, CMS added over 800 CSRs--more than doubling the number of
staff who had previously been available to respond to 1-800-MEDICARE
help line inquiries. In June 2004, CMS also increased the number of
contractors managing the 1-800-MEDICARE help line from one to two.
MMA also required that GAO examine the accuracy and consistency of
answers provided through the 1-800-MEDICARE help line and the training
and education given to its CSRs.[Footnote 2] To address this
requirement, as discussed with the congressional committees with
jurisdiction over Medicare, we evaluated (1) the accuracy of
information that 1-800-MEDICARE provides, (2) the training given to 1-
800-MEDICARE CSRs, and (3) CMS's efforts to monitor the accuracy of
information provided through the 1-800-MEDICARE help line.
To evaluate the accuracy of information provided by 1-800-MEDICARE, we
made 420 calls to the 1-800-MEDICARE help line during July 2004 and
posed 1 of 6 questions about the Medicare program during each
call.[Footnote 3] For each question, we randomly placed calls at
different times of the day and days of the week to match the typical
pattern of calls reported by the 1-800-MEDICARE help line in April
2004. To develop our 6 questions, we initially formulated 20 questions
from the 100 topics most frequently addressed by the 1-800-MEDICARE
help line's CSRs in April 2004. For each of our questions we developed
criteria that we used to define an accurate response, using information
on the Medicare Web site's frequently asked questions section.[Footnote
4] CMS confirmed the responses we provided, and provided us with the
written answers that the CSRs would be expected to use to respond to
each question. From the list of 20 questions, we chose 6 questions,
asked each one a total of 70 times, and evaluated the accuracy of the
responses using information provided by CMS. If the CSR provided some
information on the topic, but did not provide sufficient and complete
information to meet our criteria, we considered the answer to be
inaccurate. To evaluate the training provided to CSRs, we interviewed
officials at the primary contractor[Footnote 5] and CMS staff, reviewed
the instructional materials used in CSR training, and observed a
training session in June 2004. In addition, we reviewed previous GAO
reports on the training provided by help lines, including the training
provided to the CSRs answering calls on the Internal Revenue Service's
(IRS) help line.[Footnote 6] To evaluate CMS's efforts to monitor the
accuracy of information provided through the help line, we interviewed
CMS officials and officials from the primary 1-800-MEDICARE contractor.
In addition, we reviewed related documents and information about call
centers in other industries. We did not verify the reliability of CMS's
monitoring data. Appendix I includes a more detailed discussion of our
scope and methodology. Our work was conducted from May 2004 through
December 2004 in accordance with generally accepted government auditing
standards.
Results in Brief:
In response to our 420 calls to the 1-800-MEDICARE help line, CSRs
accurately answered the questions posed in 61 percent of the calls we
placed. The accuracy rate varied significantly among the six questions
we posed. Twenty-nine percent of the calls were answered inaccurately.
We were not able to obtain any answers to our questions for the
remaining 10 percent of our calls at the time we placed them. This
generally occurred when our morning, evening, and weekend calls were
transferred to claims administration contractors that were not open for
business at the time we called or when our calls were inadvertently
disconnected. To facilitate accurate and consistent responses, the 1-
800-MEDICARE help line provides CSRs with information in the form of
written answers--known as "scripts"--that CSRs can access by typing in
key words on their computers during a call. The scripts are designed to
address the help line's frequently asked questions or provide links to
additional information. However, we found that CSRs provided inaccurate
information largely because they did not always understand enough about
the Medicare program to access a script that answered the question or
could not clearly explain the material in the scripts that they were
using. For example, one question about income eligibility for the $600
prescription drug credit was answered inaccurately in 55 out of 70
calls, generally because the CSRs did not seek other needed information
from a second script to correctly answer the question. In addition,
CSRs sometimes did not understand the language used in the scripts or
other written material available to them. For example, in answering a
question on power wheelchair payment, one CSR confused "trunk
strength"--which is upper body strength--with car trunk space and
incorrectly explained that Medicare would only cover a power wheelchair
if a beneficiary had adequate space to put it in the trunk of his car.
As this example shows, when CSRs do not understand the scripts, they
provide inaccurate answers. Nevertheless, CMS and its contractor do not
routinely pretest the scripts to ensure that they are understandable to
the CSRs or potential callers.
The training for CSRs meets CMS's requirements, but it is not
sufficient to ensure that CSRs are able to answer questions accurately
on the 1-800-MEDICARE help line. As required by CMS, all CSRs receive
training. The primary contractor, which is responsible for training all
newly hired CSRs, provides them with a 2-week session of classroom
instruction on accessing and using scripts, customer service etiquette,
and general information about the Medicare program. As part of the
training, CMS requires newly hired CSRs to score 90 percent or higher
on a written exam before they are allowed to answer questions on the
help line. CMS requires CSRs to pass an exercise answering six
simulated calls. As required by CMS, both contractors also provide
their CSRs with periodic refresher training on Medicare program changes
and other issues. Although the 1-800-MEDICARE contractors meet CMS's
training requirements by providing instruction and testing, the number
of inaccurate responses we received on our calls indicates that not all
CSRs had the necessary knowledge and skills to answer our questions
accurately. Testing how accurately CSRs answer frequently asked
questions using scripts provides a better measure of their ability to
handle calls. However, in their testing, CMS and its contractors do not
emphasize CSRs' ability to answer questions accurately using scripts.
For example, to pass the simulated call handling exercise, new CSRs
have to demonstrate their ability to use scripts to answer calls
accurately on only two consecutive calls out of six possible attempts.
Similarly, on the written exam that newly hired CSRs must pass before
they work on the help line, 24 of the exam's 52 questions ask CSRs to
identify scripts with information to answer specific questions while
the remaining 28 questions target other skills needed by help line
staff. Further, while all CSRs receive continuing training, they are
not required to demonstrate that they have effectively mastered new
material in answering calls.
CMS delegates most accuracy monitoring to its primary contractor and
reviews the results, but this monitoring is not designed to identify
and systematically track responses to questions that CSRs as a group
commonly answer incorrectly. The primary contractor listens to the
conversations CSRs have with callers, observes the scripts they select
on their computers to answer questions, and then scores the CSRs'
performance in several categories, including customer service etiquette
and accuracy. Calls can be evaluated as acceptable if CSRs provide some
correct information, even if they do not provide enough information to
accurately answer the question. As required by CMS, the primary
contractor evaluates four calls per month for each CSR--which is a
standard amount for the help line industry--and reports the results to
CMS. Although the primary contractor's monitoring includes an
assessment of CSRs' accuracy, the monitoring does not categorize and
identify accuracy rates by question, which could help CMS target
training and other improvement efforts. In contrast to most of the 1-
800-MEDICARE help line monitoring efforts, on two occasions CMS
contracted for studies that examined accuracy rates by question. Like
our methodology, these studies used specific criteria to evaluate an
answer's completeness and accuracy. The findings from these studies
were similar to ours--accuracy rates varied by question and were
sometimes lower than the accuracy rates found through the primary
contractor's monitoring. For example, one study found that CSRs
accurately answered between 39 and 69 percent of the questions asked
about Medicare prescription drug discount card and benefit; in the same
month the subcontractor found an overall accuracy rate for all
questions to be about 94 percent. CMS used information from these
smaller studies that focused on the accuracy rates for specific
questions to improve the scripts and training for CSRs.
To improve the accuracy of responses provided to the 1-800-MEDICARE
help line callers, we are making several recommendations. Specifically,
we are recommending that CMS revise procedures so that calls are not
transferred to claims administration contractors that are closed for
business at that time; assess current scripts and pretest new and
revised scripts before approving them to ensure that they are
understandable to CSRs and potential callers; provide more testing of
CSRs' ability to use scripts to accurately answer the most frequently
asked questions, and use the results to target additional training
efforts as needed; and monitor the accuracy rate of each frequently
asked question and use the results to modify scripts or provide
training, if necessary.
In its comments on a draft of this report, CMS agreed with our
recommendations and provided additional detail on the challenges it
faced in administering 1-800-MEDICARE after the massive call volume
increase that followed MMA. CMS also described the steps that it was
considering, or had begun, to address our recommendations. CMS
expressed concern that we considered answers from 1-800-MEDICARE CSRs
to be inaccurate if they were incomplete. However, we believe that our
accuracy criteria included the minimum detail needed to avoid
misleading callers.
Background:
CMS administers the 1-800-MEDICARE help line to answer beneficiaries'
questions about Medicare eligibility, enrollment, and benefits. The
help line currently operates 24 hours a day, 7 days a week, and has
eight call center locations that are run by two contractors. As of
October 2004, the primary contractor managed 1,332 of the 2,137 CSRs
and operated seven of the eight 1-800-MEDICARE call centers. In
addition, the primary contractor is responsible for other activities,
such as distributing program material requested by callers, training
all new CSRs before they handle calls on the 1-800-MEDICARE help line,
and researching answers to more complex questions some callers may
have. Prior to 2004, one contractor managed the 1-800-MEDICARE help
line. In June 2004, in response to increasing call volume,[Footnote 7]
CMS hired a second contractor, which in October 2004 managed 805 CSRs
and operated one of the eight 1-800-MEDICARE call centers.
A call placed to 1-800-MEDICARE is answered initially by an interactive
voice response system. This is an automated system that, depending on
the caller's responses to the system's automated prompts, routes a call
to a CSR or to other information sources. These other information
sources can include the other help lines maintained by Medicare's
claims administration contractors or recorded information.
All CSRs must have a high school diploma or its equivalent, but they
are not required to be knowledgeable about the Medicare program at the
time they are hired. To help provide clear, accurate, and timely
answers to callers' questions, CMS expects the CSRs to use written
scripts, which contain information about the program.[Footnote 8] CSRs
listen to a caller's question and then type in related keywords to
generate a list of suggested scripts that could be used to answer the
question. The CSRs select the script they consider best suited to
answer the question and read either excerpts or the entire script. CSRs
can also consult other information sources. For example, CSRs can use
Web-based tools available on the Medicare Web site[Footnote 9] to help
beneficiaries select a prescription drug discount plan, nursing home,
or home health agency.
Because the types of questions frequently posed to 1-800-MEDICARE
change in response to program or other policy changes, new scripts may
need to be created or existing ones updated. Either CMS or the primary
contractor may decide to develop a new script or update an existing one
for clarification or in response to program changes. CMS officials
approve scripts that are developed by the primary contractor, and check
them for accuracy and completeness.
1-800-MEDICARE Provided Accurate Information for Less Than Two-thirds
of the Calls We Placed:
The 1-800-MEDICARE help line provided accurate answers to 61 percent of
the 420 calls we made. The accuracy rate varied significantly among the
six questions we posed. Overall, 29 percent of our calls were answered
inaccurately. In general, CSRs erred because they did not understand
enough about the Medicare program to access the script with information
to answer the question or clearly explain the material in it. In
addition, for 10 percent of the calls we placed, we were unable to get
a response to our question at the time we contacted the 1-800-MEDICARE
help line, mainly due to problems when CSRs transferred calls.
Accuracy Rate Varied Significantly by Question:
In response to our calls to the 1-800-MEDICARE help line, CSRs answered
our questions accurately for 256 out of 420 calls, a rate of 61
percent. The criteria we developed to determine the information that
constituted an accurate answer for each question are shown in table 1.
(A more detailed version of the questions and information to answer
them are in apps. II through VII.) The criteria were based on answers
we developed from information on the Medicare Web site and were
confirmed by CMS, which provided us with the scripts that contained
information to answer the questions.[Footnote 10] We considered all
calls we placed to the 1-800-MEDICARE help line to be part of our test
of its accuracy, even if the call was transferred to a claims
administration contractor to provide the answer.
Table 1: Questions and Criteria Used to Determine Answer Accuracy:
Medicare prescription drug discount card questions:
Question 1: Drug card;
Question: Medicare prescription drug discount card questions: What drug
card can my father-in-law get that will cover all of his drugs at a
particular pharmacy and have his drugs cost the least amount?
Criteria: An accurate response would include at least one prescription
drug discount card recommendation with the lowest total prices for the
drugs the beneficiary used.
Question 2: $600 credit;
Question: Medicare prescription drug discount card questions: Can my
mother qualify for the $600 credit with three specified sources and
amounts of income?
Criteria: An accurate response would indicate that a beneficiary with
the amount of income from the specified sources could receive the $600
credit.
Question 3: Medigap[A];
Question: Medicare prescription drug discount card questions: Can my
grandmother get a Medicare-approved prescription drug discount card if
she has Medicare and a Medigap policy?
Criteria: An accurate response would indicate that a beneficiary could
have a Medigap policy and still receive a Medicare-approved
prescription drug discount card.
Other Medicare coverage or eligibility questions:
Question 4: Power wheelchair;
Question: Medicare prescription drug discount card questions: Will
Medicare pay for a power wheelchair for my father?
Criteria: An accurate response would indicate that (1) a physician must
prescribe the power wheelchair or determine it to be medically
necessary, and (2) a power wheelchair requires a copayment on the part
of the Medicare beneficiary.
Question 5: Medicare part B[B] enrollment;
Question: Medicare prescription drug discount card questions: Should my
husband sign up for part B if I am still working and we have health
insurance coverage from my employer?
Criteria: An accurate response would indicate that a beneficiary could
wait to enroll and then sign up for Medicare part B during a special
enrollment period.
Question 6: Eye exam and glasses;
Question: Medicare prescription drug discount card questions: Will
Medicare pay for an eye exam and a new pair of eyeglasses if my
mother's prescription has changed?
Criteria: An accurate response would indicate that Medicare does not
pay for routine eye exams and eyeglasses.
Source: GAO analysis.
Notes: The analysis is based on a report of the most frequently
accessed 1-800-MEDICARE scripts in May 2004 provided by CMS, scripts
related to these questions provided by CMS on July 1, 2004, and
information from Medicare's Web site on frequently asked questions and
their answers, http://medicare.custhelp.com/cgi-bin/medicare.cfg/php/
enduser/std_alp.php, accessed on May 21, 2004.
[A] Medigap is a privately purchased health insurance policy that
supplements Medicare by paying for some of the health care costs that
are not covered by the program.
[B] Eligible individuals can choose to enroll in part B and pay a
monthly premium. Medicare part B services include physician and
outpatient hospital services, diagnostic tests, mental health services,
outpatient physical and occupational therapy, ambulance services, some
home health services, durable medical equipment, prosthetics,
orthotics, and medical supplies.
[End of table]
The percentage of calls CSRs answered accurately varied by question, as
shown in figure 1. For example, CSRs accurately answered 81 percent of
the calls asking whether a beneficiary could receive a prescription
drug discount card if they had a Medigap policy. The answer to the
Medigap question was clearly described in a script, which allowed CSRs
to respond with the highest accuracy rate for all of our questions.
Similarly, for question 1--choosing a prescription drug discount card-
-CSRs answered accurately 76 percent of the time. By July 2004, when we
placed our calls, a large number of CSRs had been recently hired and
trained specifically to answer this question, using a script and a Web-
based tool.[Footnote 11] In contrast, for question 2 calls about the
$600 prescription drug credit, CSRs answered inaccurately 79 percent of
the time. CSRs scored poorly on this question primarily because they
based their answers on the beneficiary's total income without
considering that some specific types of income should not be included
in the calculation of eligibility for the credit. CSRs would have had
to access two scripts to correctly answer the question, because the
more general script on the topic did not contain all of the needed
information. Question 5, which addressed Medicare part B enrollment,
also had a relatively high inaccuracy rate--41 percent. We were not
able to obtain an answer to some of our questions at the time that we
called, most commonly when CSRs or the interactive voice response
system transferred calls concerning questions 4 and 6 to other help
lines.
Figure 1: Percentage of Calls with Accurate, Inaccurate, and No
Answers, by Question:
[See PDF for image]
Note: Numbers may not add to 100 percent due to rounding.
[End of figure]
Inaccurate Responses Were Largely Due to Ineffective Use of Scripts:
CSRs responded inaccurately to our questions largely because they did
not seem to know enough to effectively use the scripts. According to a
CMS official and the primary contractor's staff, CSRs are expected to
use scripts to guide their discussion with callers; they are not
supposed to rely solely on acquired knowledge of Medicare to answer
questions. We found, however, that in responding to our questions CSRs
usually had one of four problems using scripts. The CSRs (1) did not
seem to access a script, even when one was available; (2) did not seem
to access a script with the right information to answer the question;
(3) did not obtain enough information from the script; or (4)
misunderstood some of the words in the scripts.
We found instances when CSRs did not seem to access scripts when
responding to calls. For example, when responding to our calls
concerning the prescription drug discount card question, 2 CSRs
indicated that they were not able to inform the caller about which card
had the lowest drug prices--even though 53 other CSRs successfully used
a script and a Web-based tool to answer this question. One other CSR
referred our caller to AARP for an answer, rather than respond with the
appropriate script and Web-based tool.[Footnote 12] These 3 CSRs did
not seem to know how to correctly answer this question, which was
addressed by one of the most commonly accessed script for the first
half of the year. During 20 of the calls to answer our question on
whether a spouse should enroll in Medicare part B if he had current
employment-based health insurance, CSRs told our callers that enrolling
in Medicare was a personal decision and they could not answer the
question, which we classified as an inaccurate answer. They did not
seem to recognize that they could access a script that contained
information designed to answer that question.
CSRs sometimes seemed to be accessing the wrong script to answer our
question. For example, in answering the question on whether a
beneficiary could receive a prescription drug discount card if she had
a Medigap policy, one CSR incorrectly stated that the caller needed to
complete a survey before receiving an answer. There is a script
available that provides the answer to the Medigap question, but the
script does not mention a survey. This CSR seemed to be using a
different script about the prescription drug discount card, which has
the right information to answer our question on the best prescription
drug discount card to choose.
In some cases, CSRs did not obtain enough information from the scripts
they were using to accurately answer the question we asked. For
example, to answer our question concerning whether a beneficiary could
qualify for the $600 credit toward prescription drug purchases, the CSR
should consider the source, as well as the amount of the beneficiary's
income. Some sources of income are not counted in determining a
beneficiary's eligibility for the $600 credit. According to CMS, to
answer this question accurately the CSR would have to access two
different scripts. The first script provides general information about
the $600 credit and refers CSRs to the second script, which lists the
sources of income that are not included in the eligibility calculation.
However, the CSRs who answered this question incorrectly in 55 calls--
or 79 percent of the time--focused on the total amount of income and
did not seem to seem to consider the sources of the income or to access
and use information from the second script. In 14 of the calls--or 20
percent of the time--CSRs were able to answer this question correctly,
because they did consider the sources and amounts of income that we
indicated the beneficiary had.
Finally, CSRs sometimes misinterpreted or did not understand the words
they were reading from the scripts or other written materials. For
example, to answer our Medigap question, a CSR incorrectly told the
caller that the beneficiary would automatically receive a prescription
drug discount card if enrolled in a Medigap plan. The CSR may have been
confusing Medigap policies with Medicare managed care plans, because
both are discussed in the script that answered this question. In
another example, for our question related to power wheelchair coverage,
a CSR misread the requirement that a beneficiary must have adequate
trunk--or upper body--strength. The CSR mistakenly informed us that a
Medicare beneficiary needs to have adequate "trunk space" in order to
qualify for a power wheelchair. When we asked for clarification, the
CSR stated that Medicare requires a qualifying beneficiary to have
adequate trunk space in his or her car to hold a power wheelchair.
Similarly, during one of our calls about eye exam and glasses payment,
the CSR informed us that an eye exam would be covered and then stated,
"the only part of the exam that is not covered is 'refraction,' but I
don't know exactly what that is." Because the CSR did not understand
that a typical eye exam would be considered a refraction, she gave the
caller the incorrect impression that Medicare would pay for a routine
eye exam.
CMS and contractor staff acknowledged that scripts for the 1-800-
MEDICARE help line are not routinely pretested to ensure that both the
CSR and the caller can understand the script before it is used to
answer callers' questions.[Footnote 13] On occasion, the 1-800-MEDICARE
contractor has obtained CSRs' feedback on draft scripts before they are
used on the 1-800-MEDICARE help line to ensure that scripts can be
easily read and understood. But this is not done as a routine step
before new and revised scripts are used in handling calls. In addition,
even if the CSRs consider the script understandable, it may still be
confusing to Medicare beneficiaries. We found that pretesting to ensure
that written material is understandable to its intended audience is a
standard practice used to develop effective communications materials.
For example, prior to issuing the first Medicare & You
handbook[Footnote 14] nationwide to beneficiaries, CMS conducted
consumer testing of its publication to evaluate its effectiveness as a
communication tool. CMS has revised subsequent editions of the handbook
to make it easier to read and use, based on feedback from
beneficiaries. Moreover, other HHS agencies, such as the Centers for
Disease Control and Prevention[Footnote 15] and the Substance Abuse and
Mental Health Services Administration,[Footnote 16] have developed
guidance on steps for ensuring that written material is understandable
for intended readers and pretesting the materials before use.[Footnote
17]
Ten Percent of Calls Were Not Answered, Often When Calls Were
Transferred to Other Contractors' Help Lines:
We did not receive answers to our questions for 10 percent (42) of the
420 calls we placed at the time we originally called. Several reasons
accounted for this, as table 2 shows. For half (21) of the unanswered
calls, the CSRs or the interactive voice response system transferred
the calls placed during morning, evening, and weekend hours to claims
administration contractors that were not open for business at the time
of our call. Although the 1-800-MEDICARE help line is open 24 hours a
day, 7 days a week, these other help lines are not. The transferred
calls pertained to our questions concerning Medicare coverage about
power wheelchairs and eye exams and glasses. The 1-800-MEDICARE CSRs or
the interactive voice response system transfer such questions to the
claims administration contractors' help lines because these contractors
generally have greater knowledge about Medicare coverage
issues.[Footnote 18] Once our calls were transferred to closed help
lines, we generally heard recordings that stated the contractors'
regular business hours and suggested calling back at that time.
However, for 7 of those 21 calls, the contractors' recorded messages
did not provide a telephone number to use to call back during the
stated business hours.[Footnote 19]
Table 2: Summary of Unanswered Calls by Question:
Medicare prescription drug discount card questions:
Question 1: Drug card;
Call transferred to closed help line: 0;
Call disconnected: 3;
Web-based tool inoperative: 4;
Call routed to a wrong telephone number: 0;
Total unanswered calls: 7.
Question 2: $600 credit;
Call transferred to closed help line: 0;
Call disconnected: 1;
Web-based tool inoperative: 0;
Call routed to a wrong telephone number: 0;
Total unanswered calls: 1.
Question 3: Medigap;
Call transferred to closed help line: 0;
Call disconnected: 2;
Web-based tool inoperative: 0;
Call routed to a wrong telephone number: 0;
Total unanswered calls: 2.
Other Medicare coverage or eligibility questions:
Question 4: Power wheelchair;
Call transferred to closed help line: 10;
Call disconnected: 5;
Web-based tool inoperative: 0;
Call routed to a wrong telephone number: 0;
Total unanswered calls: 15.
Question 5: Medicare part B enrollment;
Call transferred to closed help line: 0;
Call disconnected: 0;
Web-based tool inoperative: 0;
Call routed to a wrong telephone number: 0;
Total unanswered calls: 0.
Question 6: Eye exam/glasses;
Call transferred to closed help line: 11;
Call disconnected: 5;
Web-based tool inoperative: 0;
Call routed to a wrong telephone number: 1;
Total unanswered calls: 17.
Total number of responses by category;
Call transferred to closed help line: 21;
Call disconnected: 16;
Web-based tool inoperative: 4;
Call routed to a wrong telephone number: 1;
Total unanswered calls: 42.
Source: GAO analysis.
[End of table]
The second most common reason we did not receive answers to our calls
was that our calls were disconnected. Sixteen of the 42 unanswered
calls were disconnected. For example, calls were disconnected before we
were able to obtain a response to our questions. In one instance, the
call was placed on hold for 30 minutes and then was disconnected. Four
calls made on the same day did not receive a response because computer
maintenance prevented the CSRs from accessing the Web-based tool they
needed to use to answer our question about the prescription drug
discount card. Finally, one other call was unanswered because the call
was routed to a wrong telephone number.
CSR Training Met CMS's Requirements, But Was Not Sufficient to Ensure
Accurate Responses:
As required by CMS, both newly hired and experienced CSRs receive
training to help them answer questions posed on the 1-800-MEDICARE help
line. The training for newly hired CSRs includes instruction on
accessing and using scripts, customer service etiquette, and
information about the Medicare program. As part of the training, CMS
requires newly hired CSRs to score 90 percent or higher on a written
exam before they handle calls on the help line. All CSRs also receive
continuing training, and take written quizzes on the new material.
Although the 1-800-MEDICARE contractors met CMS's training requirements
by providing instruction and testing, the testing does not fully
measure CSRs' ability to accurately answer real questions from callers.
The primary contractor develops and conducts the training new CSRs
receive. Most of the training consists of 2 weeks of classroom
instruction. In general, the instruction introduces CSRs to scripts and
provides general information about the Medicare program. For example,
in a training session we observed for newly hired CSRs in June 2004,
the instructors helped the CSRs prepare for the types of inquiries that
might be expected from callers on the 1-800-MEDICARE help line. The
instructors posed different questions to the class, and each CSR
attempted to identify and access a script with the right information to
answer the instructor's question.[Footnote 20] One CSR would be
selected to read the script that they chose, and participants discussed
whether they thought this was the script with the best information to
answer the question. After completing their initial instruction, CMS
requires the new CSRs to score at least 90 percent on a written exam
and successfully complete a call handling simulation exercise before
they answer calls on the help line without supervision. To successfully
complete the call handling simulation exercise, CSRs must accurately
answer two consecutive simulated help line calls out of six possible
attempts. In addition, new CSRs generally spend about 4 hours listening
to calls answered by an experienced CSR.
In addition to the initial training for newly hired CSRs, CMS requires
all CSRs to receive continuing training.[Footnote 21] Continuing
training is delivered through three methods: refresher classes, online
broadcast announcements, and small group meetings. Weekly refresher
classes provide a means of instructing CSRs about Medicare program
changes. Following each refresher training class, CSRs complete a short
quiz to show that they understand the new information. While there is
no minimum score that CSRs must achieve on the short quiz, CMS staff
informed us that help line supervisors review each quiz to ensure that
any questions that posed problems for CSRs would be addressed with
further training. To provide CSRs with information quickly, the primary
contractor sends online broadcast announcements to each CSR's computer
workstation. These online announcements usually contain information
that may affect CSRs' responses to help line questions, such as news
about a change in a specific script. Lastly, small group meetings of
about 12 CSRs and their supervisor are held for 30 minutes each week so
that CSRs can discuss topics that can help them improve their call
handling skills.
After gaining experience in answering calls, some CSRs receive 4
additional days of special training and are promoted to a senior
position. These CSRs receive classroom training on using Web-based
computer programs that can assist Medicare beneficiaries in selecting a
managed care plan, a nursing home, or other Medicare-related services.
Like other CSRs, they must score 90 percent or higher on a written
exam, and successfully complete a simulated call handling exercise
before they can handle calls using the Web-based computer programs.
Currently, about 200 senior CSRs answer calls on the 1-800-MEDICARE
help line.
Although all CSRs receive training and are tested as required, the
responses we received indicate that not all CSRs had the necessary
knowledge and skills to answer our questions accurately. In our
opinion, testing how effectively CSRs use scripts to answer frequently
asked questions provides the best measure of their preparation to do
so. While 24 of the exam's 52 questions ask CSRs to identify scripts
that could be used to answer specific inquiries, the remaining 28
questions target other skills.[Footnote 22] In addition to the written
test, new CSRs must appropriately answer questions posed in two
consecutive simulated calls before they staff the help line.[Footnote
23] This simulated call handling and some of the written exam questions
are the only measures of the CSRs' ability to accurately answer calls
using scripts. In combination, the test and the two simulated calls do
not appear to be a sufficient measure of new CSRs' ability to
accurately answer the most frequently asked questions, given our
findings on the accuracy of their responses. Further, while all CSRs
receive continuing training, they are not required to demonstrate that
they have effectively mastered the new material in handling calls.
Developing a more targeted assessment of where CSRs need to augment
their skills helped focus another help line's training efforts and
allowed it to meet its accuracy goals. In 2001, we assessed the
telephone help line maintained by the IRS to answer taxpayers'
questions and found that it had not met the agency's goals for
accurately answering general questions about tax law and specific
questions about individuals' tax returns in 2001.[Footnote 24] In
response, the IRS analyzed the specific types of inquiries within the
area of tax law and individual returns that were answered inaccurately
and identified the knowledge and skills its CSRs needed to answer
questions more accurately. The IRS also identified the CSRs most in
need of training to improve accuracy in those knowledge and skill areas
and provided additional training to them before call volume increased
for the 2002 tax season. By the end of the training period, these CSRs
were required to be certified by their managers as capable of providing
correct responses to taxpayer questions. The IRS also assigned
responsibility for selected tax law topics to individual call center
managers, making them accountable for ensuring that CSRs were trained
and could accurately address inquiries on these topics. After these
initiatives were complete, we found that the help line had improved its
accuracy enough to meet its 2001 goals. In the span of 1 year, the
accuracy rate on answering tax law questions increased from 79 to 85
percent and the accuracy rate for answering questions about
individuals' tax returns increased from 88 to 91 percent.[Footnote 25]
Monitoring Generally Focused on Performance of Individual CSRs, Rather
than Accuracy Rates for Specific Questions:
CMS monitors the 1-800-MEDICARE help line mostly by requiring its
primary contractor to evaluate four individual conversations that each
CSR has with callers each month. Based on these conversations, the
primary contractor evaluates the performance of individual CSRs in
several categories, including accuracy, and reports the overall results
to CMS. CMS also occasionally directly monitors a small number of
individual CSRs' calls. However, the contractor's and CMS's monitoring
does not systematically track the accuracy rates for commonly asked
questions. As a result, the monitoring does not assess how accurately
CSRs as a group answer particular questions, which could help CMS
target additional training efforts. Two smaller evaluation efforts did
focus on specific questions answered inaccurately, and these targeted
monitoring efforts provided information that CMS used to improve CSR
training and the scripts used on the help line.
CMS's Primary Contractor Monitors Performance of Individual CSRs:
At the time of our review, CMS had delegated most of the responsibility
for monitoring the accuracy of the 1-800-MEDICARE help line to the
primary contractor, while maintaining oversight by reviewing the
primary contractor's results. To monitor 1-800-MEDICARE, the primary
contractor focuses on the performance of individual CSRs, evaluating
four calls per month for each person. The primary contractor evaluates
either live conversations--known as blind monitoring--or recorded
conversations on the help line, while viewing displays of the CSRs'
computer activity during calls. Viewing the CSRs' computer activity
allows the primary contractor's staff to observe the scripts or other
materials that CSRs access to answer callers' questions. After
monitoring a call, the primary contractor's supervisory staff uses a
checklist to evaluate the CSR's response to the caller. Help line
supervisors share the results with each CSR to help improve
performance. The primary contractor provides monthly reports to CMS on
the results of its monitoring.[Footnote 26] The primary contractor has
a subcontractor, which is responsible for conducting some independent
call monitoring, as well as reviewing the results of some of the
primary contractor's call monitoring. In addition to the four calls per
CSR per month that CMS requires the primary contractor to monitor, the
subcontractor monitors up to one additional call per month per CSR. The
subcontractor reports its monitoring results monthly to CMS and the
primary contractor.
In addition to meeting CMS requirements, the amount of call monitoring
per CSR approximates industry standards. A survey of 735 North American
call centers that represent help lines in various industries, including
telecommunications, financial services, and health care, found a wide
variance in the number of calls monitored per month. The most commonly
reported monthly monitoring frequencies were 4 to 5 calls per CSR or 10
or more calls per CSR.[Footnote 27]
The evaluation checklist used by CMS's contractor for monitoring calls
indicates that a CSR's performance should include certain components--
such as using an appropriate greeting, showing respect to the caller,
actively listening to the caller, responding accurately to the
question, providing a complete response, using appropriate tone and
speed, offering to provide additional information if necessary, and
ending the call politely. The primary contractor's staff uses the
checklist to evaluate both the customer service skills and knowledge
skills demonstrated during a call and classify these into one of four
categories--"unacceptable," "needs improvement," "achieves
expectations," and "exceeds expectations."
CMS requires the primary contractor to reach a quality rating of
"achieves expectations" or higher for at least 90 percent of the total
number of CSR calls evaluated each month. The primary contractor
evaluates a call as either "accurate" or "inaccurate," and because
accuracy is weighted more heavily than other components, a CSR cannot
provide inaccurate information on a call and still have the call scored
as "achieves expectations." However, in contrast to our methodology for
this report, a CSR can provide incomplete information--information that
is correct but does not fully answer the question--and still have the
call scored as "achieves expectations." In addition, the evaluation
checklist does not indicate the specific criteria used to determine a
call's accuracy.
Although CMS's main role in monitoring the help line is to review the
efforts of the primary contractor, the agency also conducts some
monitoring of CSRs on its own. Like the primary contractor, CMS
occasionally uses blind monitoring to evaluate the performance of
individual CSRs--listening to real-time calls and watching the scripts
and other materials the CSRs use. CMS does not conduct blind monitoring
routinely and document the results, and therefore, CMS staff could not
provide us any information on the extent of this monitoring.[Footnote
28] According to a CMS official, the agency conducts blind monitoring
on a "limited and as-needed" basis.
CMS Has Tracked Accuracy Rates by Question on Occasion:
Although CMS's primary 1-800-MEDICARE contractor monitors the accuracy
of individual CSRs, CMS does not use the regular monthly monitoring to
identify trends in inaccurate responses by question. Specifically, the
primary contractor does not routinely classify or categorize CSRs'
answers by specific question to identify the questions that
collectively were answered less accurately. While routine information
about a question's accuracy rate could be used to target improvement
efforts, CMS has only taken this approach twice in recent years. Both
of these efforts were small compared to the primary contractor's
monitoring. The larger effort to monitor accuracy by question lasted 29
months and involved 300 calls a month, whereas the primary contractor
evaluated about 7,350 calls in July 2004.[Footnote 29]
CMS contracted for a study to evaluate the 1-800-MEDICARE help line's
accuracy in answering specific questions, but did not receive results
quickly enough to immediately address problems. Beginning in January
2002 and until May 2004, the CMS contractor hired to assess the
"Medicare & You" program[Footnote 30] placed about 300 calls per month
to the 1-800-MEDICARE help line. These callers used a set of
hypothetical scenarios to assess how specific questions were answered.
This study also established criteria specifying the information an
accurate answer should provide[Footnote 31] and made a distinction
between fully responsive answers--in other words, complete and accurate
answers--or partially responsive answers--not complete but providing
some accurate information.[Footnote 32] For the first 19 months
studied, the average percentage of calls that received fully responsive
answers ranged from under 40 percent to over 90 percent, depending on
the question and the period of time studied.
The study helped CMS identify questions that the CSRs were answering
less accurately. However, CMS staff told us that the agency received
the reports 4 to 5 months after monitoring occurred, which did not
allow CMS to immediately address any identified problems. Nevertheless,
CMS staff indicated to us that the results of these evaluations were
used to identify areas where CSR refresher training was needed. Due to
funding constraints, this project was terminated in May 2004. CMS told
us it planned to resume a similar project in November 2004 around the
time when the next cycle for the "Medicare & You" program contract
begins.
In another study, CMS measured 1-800-MEDICARE's accuracy by question
rather than individual CSR and found that certain questions were
answered more accurately than others. In April 2004, a private
consultant group that was under contract with CMS placed 49 calls to 1-
800-MEDICARE to determine whether CSRs were relaying accurate
information about Medicare's new prescription drug discount card and
benefit.[Footnote 33] The study established specific criteria on the
information that CSRs should include for an answer to be
accurate.[Footnote 34] Evaluating accuracy by question, the study found
that CSRs accurately answered between 39 percent and 69 percent of the
questions asked about the new Medicare prescription drug discount card
and benefit. In contrast, in that same month the subcontractor--using
its evaluation checklist--determined that the overall accuracy rate for
the calls it measured on all topics to be about 94 percent.
As a result of this study, CMS improved its scripts and related
training. The private consultant's report indicated that CSRs were
having difficulty distinguishing between the Medicare prescription drug
coverage benefit that will be in effect in 2006 with the Medicare
prescription drug discount card that is currently available. CMS
responded by clarifying the scripts used to answer these questions and
improving the related materials used to train CSRs. For example, CMS
worked with the contractor to rename the titles of the different
scripts to include the term "benefit" or "card" as a method of
differentiating between them. When the subcontractor noted about 3
months later that a few CSRs were continuing to confuse the
prescription drug discount card and prescription drug benefit, CMS
further clarified the scripts and its primary contractor conducted
additional refresher training to attempt to correct the problem.
Conclusions:
Each year, millions of Medicare beneficiaries, their family members,
and other callers rely on the 1-800-MEDICARE help line for information.
Providing them with accurate answers is critical to keeping them
informed about Medicare's benefits. However, we found that 6 out of 10
calls were answered accurately, 3 out of 10 calls were answered
inaccurately, and we were not able to get a response for 1 out of 10
calls.
To answer inquiries accurately, CSRs have to be able to effectively
access and use scripts. Given the lack of prior Medicare knowledge
among CSRs, the 1-800-MEDICARE help line's script-based approach is a
reasonable means to facilitate accurate and consistent responses to
caller's questions. However, this approach makes CSRs--and thus the
help line they support--dependent on the clarity and accuracy of the
scripts available. Pretesting scripts might have identified ones that
were difficult to understand by either the CSR or by potential callers,
but this is not routinely done. Further, the training that CSRs receive
on using scripts is also essential to their ability to answer questions
accurately. However, the written exam that newly hired CSRs must pass
and the continuing training quizzes do not measure the ability to use
information in scripts to provide accurate and complete answers on the
help line.
Monitoring the help line could identify areas where CSRs' knowledge and
skills are lacking. Although CMS ensures that the amount of the
contractor's monitoring per CSR falls within industry standards, the
bulk of the monitoring methods are not designed to systematically
assess how accurately CSRs as a group answer particular questions.
Evaluating how accurately particular questions are addressed is an
important step to improving scripts and CSR training for those topics.
Finally, 1-800-MEDICARE is advertised as providing information 24 hours
a day, 7 days a week, but we could not always obtain answers to our
questions when we called. When we called with questions about Medicare
payment for power wheelchairs and coverage of eye exams and glasses,
the help line frequently transferred our calls to claims administration
contractors that were closed at the time. For a third of these
transferred calls, we were not given a call-back number. This practice
of transferring calls to claims administration contractors that are
closed, in effect, reduces the benefit of a 24-hour help line to a
business-hour help line for many beneficiaries.
Recommendations:
In order to improve the accuracy of responses made on the 1-800-
MEDICARE help line and callers' ability to have their questions
answered, we recommend that the CMS Administrator take four actions:
* Assess the current scripts for the most commonly asked questions to
ensure that they are understandable to CSRs and potential callers and
if not, revise them as needed and pretest new and revised scripts to
ensure that CSRs can effectively use them to accurately answer callers'
questions.
* Enhance testing of CSRs' skills in accurately answering the most
commonly asked questions using scripts and, if needed, provide
additional training to improve the accuracy and completeness of their
responses.
* Supplement current monitoring efforts to include a systematic review
of the accuracy of information provided by the CSRs as a group for the
most frequently asked questions and use the results to modify scripts
or provide more training, as needed.
* Revise routing procedures and technology to ensure that calls are not
transferred or referred to claims administration contractors during
nonbusiness hours.
Agency Comments and Our Evaluation:
In its written comments on a draft of this report, which are reprinted
in appendix VIII, CMS agreed with our recommendations and stated that
it had begun several efforts to address them. CMS also provided more
detail on the challenges it faced in administering 1-800-MEDICARE due
to the massive increase in call volume that occurred after the passage
of the MMA.
CMS agreed with our recommendation to assess current scripts and
pretest new and revised scripts to ensure that they are understandable.
In its comments, CMS stated that the written information used to
develop 1-800-MEDICARE scripts often comes from Medicare publications
that have been consumer tested as part of the publication preparation
process. Language that has undergone some consumer testing is often
incorporated into the scripts to improve clarity. While this step may
be helpful, we believe that pretesting scripts verbally is also
important, as consumer testing of material intended for written
publication may not be adequate to determine whether the scripts are
understandable to CSRs and the public. CMS also stated that it is
considering implementing an editorial board to review scripts, which we
believe would be another positive step to help assure the scripts'
clarity.
CMS agreed with our recommendation to enhance testing of CSRs' ability
to accurately answer questions and provide additional training, as
needed. CMS indicated that it was reassessing its testing requirements
to determine better ways to ensure that CSRs are prepared to handle
calls, once they are certified. The agency stated that it planned to
benchmark its efforts against industry standards to determine more
effective approaches. In its comments, CMS expressed concern that we
had characterized customer service skills as less meaningful than
knowledge skills. While we believe both are important, in keeping with
our congressional mandate, this report focused on the accuracy of
information provided by 1-800-MEDICARE and did not address the quality
of its CSRs' customer service skills.
CMS agreed with our recommendation to supplement its current monitoring
efforts by including a systematic review of the accuracy of information
provided by CSRs as a group for the most frequently asked questions and
using the results to modify scripts and provide more training, as
needed. CMS indicated that it believed it had done a good job
developing a quality assurance program that focuses on the most
important requirements for both accuracy and customer service skills
needed to answer calls from the elderly population. We agree that CMS
has focused on quality assurance for 1-800-MEDICARE. Our recommendation
did not deal with changing, but with enhancing, its quality assurance
efforts. To address our recommendation, CMS indicated that it would
implement a plan to develop trend information on the results of its
quality assurance activities and would focus on improving the accuracy
of responses to frequently asked questions.
CMS agreed with our recommendation to revise procedures so that calls
are not transferred to claims administration contractors during their
nonbusiness hours. The agency indicated that although the 1-800-
MEDICARE CSRs are available 24 hours a day, 7 days a week, they do not
have access to claim-specific information. Therefore, the 1-800-
MEDICARE CSRs would have to direct callers asking questions about
specific claims to contact the claims administration contractors during
their normal business hours. Our report focused on questions for which
the CSRs had scripted responses and did not need to access claim-
specific information. Nevertheless, for 7 of the 21 calls that were
routed to claims administration contractors that were closed at the
time we called, the contractors' recorded messages did not provide a
telephone number to call back during stated business hours. In its
comments, CMS indicated that it had implemented additional routing
plans to address this problem and is expanding access to claims data
that will help reduce this problem in the future. CMS also raised
concerns that we did not release the detailed audit documentation on
our test calls while our work was still in progress. GAO's policy does
not allow us to provide audit documentation to an agency while work is
ongoing. At the time of CMS's request, we described our policy and
offered to verbally provide more detail on telephone disconnections,
but CMS did not follow up with us to obtain this information. After our
report is published, we will address this request.
Finally, CMS expressed concern that we did not describe the criteria we
used to evaluate the accuracy of responses to our six questions and
stated that incomplete answers should not be considered inaccurate
responses. As noted in the draft report, table 1 lists the criteria
that we established for each of our six questions. We developed these
criteria so that we could objectively evaluate responses received from
CSRs. For four of these questions (numbers 1, 2, 3, and 6), there was
only one element in the correct response, so an incomplete response was
not possible.[Footnote 35] For the remaining two questions, we
considered the answer accurate if it included two elements. We believe
that by not including both elements for each of these questions,
callers would be left with a false impression, rather than with an
accurate answer. For example, in evaluating the response to the
question of whether Medicare would pay for a power wheelchair, we
thought it was important for the caller to know that (1) the wheelchair
needed to be prescribed by a physician and (2) the beneficiary would be
responsible for a copayment. Because the copayment for a power
wheelchair is at least $1,000, we believe that it would be misleading
not to mention either a copayment or cost sharing when a caller asks
whether Medicare pays for this item. Likewise, needing to have a
physician prescribe the power wheelchair is a Medicare requirement, and
we did not think a response could be accurate without mentioning it.
For the question on Medicare part B enrollment, we thought that it was
important to know that a beneficiary could wait to enroll but, once
other health insurance coverage ended, had a limited time period to
enroll in part B without incurring higher premiums. Without knowing
both elements of this answer, beneficiaries would not have enough
information to guide their decision on part B enrollment and,
therefore, the answer provided would be misleading.
We are sending copies of this report to the Administrator of CMS,
appropriate congressional committees, and other interested parties. We
will also make copies available to others upon request. This report is
also available at no charge on GAO's Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please call
me on (312) 220-7600. An additional GAO contact and other staff who
made contributions to this report are listed in appendix IX.
Signed by:
Leslie G. Aronovitz:
Director, Health Care--Program Administration and Integrity Issues:
[End of section]
Appendix I: Scope and Methodology:
To determine the accuracy of information provided, we placed a total of
420 calls to the 1-800-MEDICARE help line. We made 420 calls in order
to have a sample that was large enough to determine if differences in
accuracy were significant. We selected six questions about Medicare--
three related to the Medicare prescription drug discount card and three
related to Medicare coverage or eligibility for benefits.[Footnote 36]
We asked each of the selected questions a total of 70 times. We
randomly placed calls at different times of the day and different days
of the week between July 8 and July 30, 2004, to match the daily and
hourly pattern of calls reported by 1-800-MEDICARE in April 2004.
To select the 6 questions, we initially chose 20 questions that related
to the 100 topics most frequently addressed by the 1-800-MEDICARE help
line's CSRs in May 2004 and developed criteria for an accurate response
from information on the Medicare Web site's frequently asked questions
section.[Footnote 37] We then presented the 20 questions and answers to
Centers for Medicare & Medicaid Services (CMS) officials, who provided
us with a script number and text for each question. CMS officials did
not object to using any of the 6 questions that we ultimately chose, or
suggest that the answers that we had provided for these questions were
incorrect. We informed both CMS and one of the 1-800-MEDICARE
contractors that we would be placing these calls. However, we did not
tell them which 6 of the 20 questions we selected, or the specific
dates and times when we would be placing our calls.
Before placing our calls, we created a scenario with fictional names
and zip codes for each of the six questions to make them sound more
realistic. (Appendices II to VII contain the scenarios that we used.)
We made pretest calls for each question before we finalized its
wording. During our actual calls, the CSRs were not aware that their
responses would be included in a research study. We recorded the length
of, and routing process for, each call. We evaluated the accuracy of
the responses by CSRs to the 420 calls we placed by whether key
information was provided.
The results from our 420 calls are limited only to those calls and are
not generalizable to the population of calls routinely made to call
centers by beneficiaries or other callers. Although the six questions
we posed were among topics most often accessed by CSRs, they do not
encompass all of the questions callers might ask. In addition, we did
not verify the reliability of CMS's monitoring data.
To examine the training provided to 1-800-MEDICARE CSRs, we interviewed
officials representing CMS and the 1-800-MEDICARE contractor
responsible for training CSRs. We reviewed the primary contractor's
training requirements and the instructional materials that are used to
educate new CSRs. We also observed a training session for new CSRs at a
1-800-MEDICARE call center. In addition, we reviewed previous GAO
reports on the operations of other help lines, including the training
provided to the CSRs answering calls on the Internal Revenue Service's
help line.
To evaluate CMS's role in overseeing the accuracy of information
provided through the 1-800-MEDICARE help line, we interviewed officials
from CMS and one of the two 1-800-MEDICARE contractors about their
monitoring and oversight activities. For our first objective, we
focused on the accuracy of information provided by the CSR regardless
of the contractor who managed their work. For our other two objectives,
we relied on information provided by one of the two contractors--which
we refer to as the primary contractor. We also identified CMS
requirements for call center operations and reviewed contractor reports
to identify the types of problems encountered through the help line. We
performed our work from May 2004 through December 2004 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix II: Prescription Drug Discount Card Questions and Information
Used to Develop Accuracy Criteria:
For question 1, which is about choosing a prescription drug discount
card, we used scenarios with different combinations of prescription
drugs and one of four different locations, in order to ensure our
anonymity. If a CSR named one of the two prescription drug discount
cards with the lowest cost for the combination of prescription drugs in
the scenario posed, we considered it to be a correct response to our
question. To ensure that we obtained the correct answer for each
question, we periodically checked the prescription drug prices using
the prescription drug tool on the Medicare.gov site. This is the same
tool CSRs used to answer our questions. The answers shown in this
appendix were accurate as of July 15, 2004.
Question 1a posed to CSRs:
My father-in-law lives in Wayne, Pennsylvania, and wants to continue to
shop at Yorke Apothecary (located at 110 S. Wayne Ave., Wayne,
Pennsylvania, 19087). What drug card can he get that will cover all of
his drugs at Yorke Apothecary, and cost the least amount?
He takes the following drugs:
Hydrochlorothiazide; 25MG; 30 TABS.
Lipitor; 20MG; 30 TABS.
Warfarin sodium; 5MG; 30 TABS.
[End of table]
Other information to provide to the CSR if asked:
* He is single.
* He lives in Wayne, Pennsylvania, in Delaware County. His zip code is
19087.
* He currently has fee-for-service Medicare with no other drug
benefits.
* He does not use an American Indian Health pharmacy.
* He does not live in a long-term care facility.
* He has $20,000 in annual income and is not interested in any drug
assistance programs, including the $600 credit.
* His sources of income are a pension and Social Security, but the
amount from each is unknown.
* He has some bank accounts, but their value is unknown.
* The amount he currently pays for drugs is unknown.
* Default answer for other questions: "I don't know."
Two prescription drug discount cards listed on Medicare.gov with the
lowest prices for the combination of drugs in our scenario:
myPharmaCare Prescription Drug Discount Card;
1-800-601-3002;
Monthly drug costs: $116.69;
Annual enrollment fee: $25.00.
U Share Prescription Drug Discount Card;
1-800-707-3914;
Monthly drug costs: $115.59;
Annual enrollment fee: $19.95.
Question 1b posed to CSRs:
My father lives in Homewood, Illinois, and wants to continue to shop at
the K-Mart Pharmacy in Homewood, Illinois (located at 17550 Halsted
Rd., Homewood, Illinois, 60430). What drug card can he get that will
cover all of his drugs at the K-Mart Pharmacy, and cost the least
amount?
He takes the following drugs:
Aricept; 10 MG; 30 TABS;
Celebrex; 100 MG; 30 TABS.
[End of table]
Other information to provide to the CSR if asked:
* He is single.
* He lives in Homewood, Illinois, in Cook County. His zip code is
60430.
* He currently has fee-for-service Medicare with no other drug
benefits.
* He does not use an American Indian Health pharmacy.
* He does not live in a long-term care facility.
* He has $20,000 in annual income and is not interested in any drug
assistance programs, including the $600 credit.
* His sources of income are a pension and Social Security, but the
amount from each is unknown.
* He has some bank accounts, but their value is unknown.
* The amount he currently pays for drugs is unknown.
* Default answer for other questions: "I don't know."
Two prescription drug discount cards listed on Medicare.gov with the
lowest prices for the combination of drugs in our scenario:
U Share Prescription Drug Discount Card;
1-800-707-3914;
Monthly drug costs: $174.91;
Annual enrollment fee: $19.95:
Any of several prescription drug discount cards available with this
combination of drugs priced at $182.80.[Footnote 38]
Question 1c posed to CSRs:
My father lives in Cincinnati, Ohio, and wants to continue to shop at
the CVS Pharmacy (located at 3195 Linwood Ave., Cincinnati, Ohio). What
drug card can he get that will cover all of his drugs at the CVS
Pharmacy, and cost the least amount?
He takes the following drugs:
Lipitor; 10 MG; 30 TABS.
Ambien; 5 MG; 30 TABS.
Lipitor: Vioxx; 25 MG; 30 TABS.
[End of table]
Other information to provide to the CSR if asked:
* He is single.
* He lives in Cincinnati, Ohio, in Hamilton County. His zip code is
45226.
* He currently has fee-for-service Medicare with no other drug
benefits.
* He does not use an American Indian Health pharmacy.
* He does not live in a long-term care facility.
* He has $20,000 in annual income and is not interested in any drug
assistance programs, including the $600 credit.
* His sources of income are a pension and Social Security, but the
amount from each is unknown.
* He has some bank accounts, but their value is unknown.
* The amount he currently pays for drugs is unknown.
* Default answer for other questions: "I don't know."
Two prescription drug discount cards listed on Medicare.gov with the
lowest prices for the combination of drugs in our scenario:
myPharmaCare Prescription Drug Discount Card;
1-800-601-3002;
Monthly drug costs: $202.79;
Annual enrollment fee: $25.00:
Anthem Prescription Drug Discount Card;
1-800-730-2804;
Monthly drug costs: $209.87;
Annual enrollment fee: $14.95:
Question 1d posed to CSRs:
My father lives in Brooklyn, New York, and wants to continue to shop at
the Neergaard Pharmacy (located at 454 Fifth Avenue, in Brooklyn, New
York). What drug card can he get that will cover all of his drugs at
the Neergaard Pharmacy, and cost the least amount?
He takes the following drugs:
Cardura: 2 MG; 30 TABS.
Hydrochlorothiazide; 25 MG; 30 TABS.
Lisinopril; 5 MG; 30 TABS.
[End of table]
Other information to provide to the CSR if asked:
* He is single.
* He lives in Brooklyn, New York, in Kings County. His zip code is
11215.
* He currently has fee-for-service Medicare with no other drug
benefits.
* He does not use an American Indian Health pharmacy.
* He does not live in a long-term care facility.
* He has $20,000 in annual income and is not interested in any drug
assistance programs, including the $600 credit.
* His sources of income are a pension and Social Security, but the
amount from each is unknown.
* He has some bank accounts, but their value is unknown.
* The amount he currently pays for drugs is unknown.
* Default answer for other questions: "I don't know."
Two prescription drug discount cards listed on Medicare.gov with the
lowest prices for the combination of drugs in our scenario:
EnvisionRx Plus Prescription Drug Discount Card;
1-866-250-2005;
Monthly drug costs: $46.33;
Annual enrollment fee: $30.00:
Any of several prescription drug discount cards available with this
combination of drugs priced at $50.45.[Footnote 39]
[End of section]
Appendix III: $600 Credit Question and Information Used to Develop
Accuracy Criteria:
Question 2 posed to CSRs:
I've heard about the $600 credit that can help pay for prescriptions
and wanted to know if my mother was eligible for it. Could she qualify
for the credit? I know she has three sources of income.
* She has about $765 per month from Social Security.
* She also gets $250 each month in rental income from the apartment in
the downstairs part of her house. She has a tenant that pays rent to
her.
* She's also getting a payout from my father's life insurance policy of
$70 each month.
Other information to provide to the CSR if asked:
* She is single and lives alone.
* She only has fee-for-service Medicare as health insurance.
* She owns her house.
* She lives in Miami, Florida, 33129.
* Default answer for other questions: "I don't know."
Information from Medicare.gov that GAO used to develop accuracy
criteria:
If your annual gross income is currently no more than $12,569 ($1,048
per month) as a single person or no more than $16,862 ($1,406 per
month) for a married couple, you might qualify for a $600 credit to
help pay for your prescription drugs and Medicare may pay your annual
enrollment fee. If you and your spouse both qualify for the credit, the
credit will be put on each of your cards.
You cannot qualify for the credit if you already have outpatient
prescription drug coverage from Medicaid, TRICARE for Life,[Footnote
40] Federal Employees Health Benefit Plan, or other health coverage
that includes outpatient prescription drugs such as an employer group
health plan. Note: If your other coverage is provided through a
Medicare Advantage plan or a Medigap plan, you may still qualify.
The following sources of income should be included when calculating
your gross income for your $600 credit enrollment form:
* Employee compensation (salary, wages, tips, bonuses, awards, etc.)
* Unemployment compensation:
* Pensions and annuities:
* Social Security benefits (including Social Security Equivalent
portion of Railroad Retirement):
* Railroad Retirement benefits:
* Veterans Affairs benefits:
* Military and government disability pensions - armed forces, Public
Health Service, National Oceanic and Atmospheric Administration,
Foreign Service (based on date pension began, combat-related pension,
etc.)
* Individual Retirement Account distributions:
* Interest (savings accounts, checking accounts, etc.)
* Ordinary dividends (stocks, bonds, etc.)
* Refunds, credits, or offsets of state and local income taxes:
* Alimony received:
* Business income:
* Capital gains:
* Farm income:
* Rental real estate, royalties, partnerships, trusts, etc.
* Other gains (sale or exchange of business property):
* Other income (lottery winnings, awards, prizes, raffles, etc.)
The following sources of income should not be included when calculating
your income for $600 credit enrollment form:
* Inheritances and gifts (taxed to estate or giver if not under limits
for exemption):
* Interest on state and local government obligations (e.g., bonds):
* Workers compensation payments:
* Federal Employees Compensation Act payments:
* Supplemental Security Income benefits:
* Income from national senior service corps programs:
* Public welfare and other public assistance benefits:
* Proceeds from sale of a home:
* Lump sum life insurance benefits paid upon death of insured:
* Life insurance benefits paid in installments:
* Accelerated life insurance death benefit payments (e.g., viatical
settlements, terminal illness, chronic illness):
* Medical Savings Accounts withdrawals for medical expenses:
* Payments from long-term care insurance policies (subject to
limitation):
* Accident or health insurance policy benefits:
* Accident compensatory damages:
* Child support payments received:
* Most foster care provider payments received:
* Disaster Relief grants:
* Disability payments as the result of a terrorist attack:
[End of section]
Appendix IV: Medigap Question and Information Used to Develop Accuracy
Criteria:
Question 3 posed to CSRs:
I'm calling with a question about my grandmother. She is 69 and she has
Medicare, and she also has a Medigap policy. Could you please tell me
if she can still get a Medicare-approved drug discount card?
Other information to provide to the CSR if asked:
* She is single and lives alone.
* She lives in Miami, Florida. I don't know the zip code off-hand.
* She is not in a long-term care facility.
* She is enrolled only in Medicare fee-for-service. She doesn't have a
Medicare managed care plan.
* She is not enrolled in Medicaid.
* I don't think she's interested in the $600 credit right now; I was
just wondering if she could get the prescription drug discount card.
* Default answer for other questions: "I don't know."
Information from Medicare.gov that GAO used to develop accuracy
criteria:
Having a Medigap policy does not preclude a Medicare beneficiary from
being eligible for a Medicare prescription drug discount card.
[End of section]
Appendix V: Power Wheelchair Question and Information Used to Develop
Accuracy Criteria:
For question 4 about power wheelchairs, we provided the CSRs with one
of four different city and state combinations, as shown below. The four
city and state combinations were randomly assigned to different power
wheelchair calls. We did this to ensure that if our call was
transferred to one of the four claims administrator contractors that
administer Medicare's durable medical equipment claims--including
power wheelchair claims--we were not biasing our results toward any
particular claims administrator.
Question 4 posed to CSRs:
My father is having trouble getting around. He has a hard time walking
and doesn't have much upper body strength. Could you please tell me if
Medicare will pay for a power wheelchair for him?
Other information to provide to the CSR if asked:
* He is enrolled in Medicare, both parts A and B.
* He lives in [select one, based on random assignment]:
* Philadelphia, Pennsylvania. His zip code is 19105.
* Detroit, Michigan. His zip code is 48209.
* Pensacola, Florida. His zip code is 32516.
* Scottsdale, Arizona. His zip code is 85262.
* His doctor has suggested he get a power wheelchair to improve his
mobility.
* He doesn't have enough strength to use a manual wheelchair.
* He lives alone and is not married.
* Default answer for other questions: "I don't know."
Information from Medicare.gov that GAO used to develop accuracy
criteria:
Power wheelchairs and/or scooters are covered if they are medically
necessary based on Medicare's criteria for coverage. In order for
Medicare to cover a power wheelchair/scooter, the beneficiary's doctor
must provide a prescription or certificate of medical
necessity[Footnote 41] that states that he needs it because of his
medical condition.
If your father qualifies for coverage, Medicare will pay 80 percent of
the Medicare-allowed amount.[Footnote 42],[Footnote 43]
[End of section]
Appendix VI: Medicare Part B Question and Information Used to Develop
Accuracy Criteria:
Question 5 posed to CSRs:
Should my husband sign up for part B if I am still working and we have
health insurance coverage from my employer?
Other information to provide to the CSR if asked:
* My husband is about to turn 65 next January.
* If asked whether working for a large or small employer: I work for
the federal government.
* I have full medical coverage, including dental and vision.
* My husband is fully covered under my insurance plan.
* Neither of us is disabled.
* The city/zip code information that corresponds with the location of
the caller.
* Default answer for other questions: "I don't know."
Information from Medicare.gov that GAO used to develop accuracy
criteria:
Your husband might want to wait to sign up for part B, because he would
have to pay the monthly part B premium and the benefits may be of
limited value as long as the group health plan [spouse's insurance] is
the primary payer. You could save on monthly premiums by waiting to
sign up.
If your husband doesn't sign up for part B when first eligible because
he has group health coverage through an employer, he can sign up for
Medicare part B during a special enrollment period. This can be anytime
he is still covered by the employer's group health plan or during the 8
months following the month when either the coverage or the employment
ends--whichever is first.
Most people who sign up for Medicare part B benefits during a special
enrollment period do not pay higher premiums.
[End of section]
Appendix VII: Eye Glasses/Exam Question and Information Used to Develop
Accuracy Criteria:
Question 6 posed to CSRs:
My mother is 66 and is enrolled in Medicare. She has been complaining
lately that she is having trouble reading the paper and thinks she may
need new eyeglasses. Will Medicare pay for an eye exam and a new pair
of eyeglasses if her prescription has changed?
Other information to provide to the CSR if asked:
* The city/zip code information that corresponds with the location of
the caller.
* She is not married.
* She is enrolled in Medicare fee-for-service only.
* I do not know the name of the county she lives in.
* Default answer for other questions: "I don't know."
Information from Medicare.gov that GAO used to develop accuracy
criteria:
Medicare does not pay for routine eye exams, eyeglasses, or contact
lenses. The beneficiary must pay 100 percent of these services.
[End of section]
Appendix VIII: Comments from the Department of Health & Human Services:
DEPARTMENT OF HEALTH & HUMAN SERVICES:
Centers for Medicare & Medicaid Services:
Administrator:
Washington, DC 20201:
NOV 30 2004:
TO: Leslie G. Aronovitz:
Director, Health Care -Program Administration and Integrity Issues:
Government Accountability Office:
FROM: Mark B. McClellan, M.D., Ph.D.:
Administrator:
SUBJECT: Government Accountability Office (GAO) Draft Report: MEDICARE:
Accuracy of Response from the 1-800-MEDICARE Help Line Should Be
Improved, (GAO-05-130):
The Centers for Medicare & Medicaid Services (CMS) thanks GAO for it's
analysis and recommendations to help improve 1-800-MEDICARE. We concur
with the recommendations and have taken steps to implement further
improvements. We appreciate the work GAO has done and hope that we will
be able to benefit from more of their specific work products in
developing our improvement strategies. In addition to the
recommendations, we are asking for additional information from GAO (see
Attachment A). We hope GAO will provide us with the information we have
discussed with GAO regarding the findings on disconnected calls, so we
can determine if these are, as we suspect, the result of telephone
network problems.
While we concur fully with the goals specified by GAO, we also want to
stress some basic points related to their work and findings. The 1-800-
MEDICARE has experienced astounding growth since it was first launched
at the end of 1998. In 2000, the first full calendar year of nationwide
operations, the call center received 3.6 million calls asking about
Medicare benefits, and health plan information and coverage, handled by
approximately 300 customer service representatives (CSRs). In four
short years, CMS has expanded the service to be available 24 hours a
day, 7 days a week, added 3,000 CSRs, provided additional training on
such important topics as choosing a nursing home, long-tern care, and
choosing a drug discount card. We were faced with an unprecedented
volume of calls about a new part of the Medicare program that required
new training efforts and many new CSRs, and we believe we responded as
well as we reasonably could given the unique and demanding
circumstances. During the month of May 2004 alone, 1-800-MEDICARE
received 3.8 million calls as a result of the overwhelming interest in
the Prescription Drug Discount Card. The Prescription Drug Discount
Card and the subsequent drug benefit, which takes effect January 2006,
are historic improvements to the Medicare program.
We are continually making improvements to the services provided through
1-800-MEDICARE and were already implementing steps to make enhancements
prior to the GAO study. The GAO findings complement the work we are
doing to reduce the number of agent transfers and ensure that callers
reach the CSR who can help them with their Medicare question. While 1-
800-MEDICARE is open 24 hours a day, 7 days a week, the claims
contractors are not, and we have addressed the problem of calls being
routed to offices that are closed. In addition, we are currently
reassessing various elements of our testing requirements and training
protocol.
We are concerned about GAO's criteria for designating answers as
"inaccurate," particularly because they consider an "incomplete" answer
to be inaccurate. We train our CSRs to be attentive to the caller's
needs and tailor the response to the caller's specific inquiry, as well
as, the caller's interest and understanding level.
We have structured our responses to make these key points, and we have
provided further details regarding the GAO report below. It is
important to provide some additional general background information
that we believe is relevant to the study. This supplementary
information provides valuable information that will help the readers of
the report understand some of the broader circumstances that
contributed to the findings. We then respond specifically to each of
the four findings in the order they have been presented by the GAO. For
ease of review, our response to each finding will address four distinct
subject areas:
1. Our concurrence or non-concurrence with the finding;
2. Additional information that we believe is pertinent in understanding
why the specific finding was identified;
3. Questions and/or concerns we have with the methodology used in the
study including terminology issues, the need for additional
information, etc.; and:
4. The steps we are taking to implement the recommendations made
including timeframes where appropriate.
Background Information:
Based on our analysis of past call volumes and planned programmatic
impacts to 1-800-MEDICARE, we anticipated and were staffing for an
annual call volume of approximately seven million calls in Fiscal Year
2004. The Medicare Prescription Drug, Improvement, and Modernization
Act of 2003 (MMA) and the news surrounding it caused the call volume to
increase exponentially. For the month of May 2004 alone, 1-800-MEDICARE
received approximately 3.8 million calls. In this single 30-day period,
our call volume exceeded 50 percent of the call volume estimated for an
entire year and it was not even during our regularly busy Fall ad
campaign. We were not prepared to handle the extreme increase in calls
received as a result of the legislation. To further illustrate the
sheer explosion in call volume we experienced. we were notified in
early May by MCI, our telecommunications provider, that they had to
suspend delivery of calls because of concerns that the 1-800-MEDICARE
calls traffic would crash their entire voice network. To meet the
unprecedented high call volume, CMS rapidly worked to increase 1-800-
MEDICARE capacity (both staffing and network infrastructure) and
improve the level of service provided to callers.
In response to this urgent and compelling need, we prepared a scope of
work, negotiated a contract, hired, trained, and installed the
necessary infrastructure to immediately add 1,000 additional CSRs to
the I-800-MEDICARE team. This was all accomplished in approximately 15
business days. In addition, we aggressively worked with the existing 1-
800-MEDICARE contractor (Pearson GS) to recruit, hire, train, and place
on the telephones additional CSRs and network routing improvements to
ensure we were doing everything possible to meet the needs of our
callers. To enable us to have more CSRs on the telephones faster, CMS
implemented an expedited training program that focused on the use of
the Prescription Drug Assistance Program (PDAP) tool, and responded to
the top drug card questions with specific CSR groups. Callers with
questions that went 'beyond the scope of these basic topics were
transferred to a more experienced CSR for further assistance. This
effort immediately raised our CSR workforce by 30 percent to 3,000
CSRs, which is now our baseline staffing level for ongoing operations.
The purpose of this background section is to set the context of the
call center environment in which GAO was tasked to conduct a study. It
is not our intention to justify less than top quality service with
circumstances that were outside of our control. It is our goal to help
readers understand that we were doing everything possible to react to
unprecedented call volume and some of the decisions we were making in
the areas of staffing; training, content review, and network
enhancements were made while considering many factors, the first of
which has always been how we can get the best possible service to our
callers under very difficult circumstances.
GAO Recommendation:
Revise routing procedures and technology to ensure that calls are not
transferred or referred to claims administration contractors during
non-business hours.
CMS Response:
We concur with the recommendation.
Relevant Information:
Enactment of the MMA in December 2003, resulted in the immediate
legislative requirement to consolidate each of the 97 individual
contractor telephone numbers into a single number (1-800-MEDICARE). It
was always our goal as part of the Virtual Call Center Strategy (VCS)
to rebuild the beneficiary telephone network to allow for a single
entry point that callers would use to contact Medicare. The legislation
that required the single 1-800 number was effective upon enactment. fhe
initial planning estimates for consolidating to a single 1-800 number
indicated that it would take between 12 to 18 months to consolidate the
network. To respond to the legislative requirements, we rebuilt the
entire network and routing plans so that a single point of entry could
be routed to one of 97 different lines of business. We built and
implemented the first Medicare speech automated system to help callers
more easily access the desired service. We developed an extremely
complex solution behind the scenes ensuring that each caller would,
after answering a few simple questions, be routed to the contractor
best suited to answer the caller's specific question. Due to the
compressed timeframes, we were not able to implement all aspects of the
original plan. As a result, some calls were routed to contractor
locations after their normal hours of operation. In this situation, a
message was played indicating that the contractor was closed for the
day.
Current Approach:
We have since implemented additional routing plans that ensure callers
are not transferred to a site that is closed, but it is important to
note that while the base 1-800-MEDICARE CSRs are available 24 hours a
day, 7 days a week, the claims processing contractor's CSRs are only
available during normal business hours. With this in mind, it should be
understood that while we have amended the routing plans to ensure that
callers are sent to a general 1-800-MEDICARE CSR in situations where
the claims processing contractor is closed, those CSRs do not have
access to the claim detail that exists in the contractor's systems. The
result is that while we will be answering calls with a "live" person in
all instances, there will be times when we are not able to provide the
information a caller needs, depending on the hour of the day. We are
expanding the availability of access to claims data that will help us
reduce this problem in the future.
Concerns:
It was never our goal to implement the specific routing solution that
was in place during the study as a long-term application. We understand
the impact this process has on customer service, and we would never be
satisfied with that level of service on a permanent basis. However, we
found it necessary to make short-term adjustments to the plans in order
to comply with legislative requirements. Given the scope of work
associated with consolidating the vast network of phone numbers and
associated applications into a single phone number, speech routing
application, and the timeframes associated with the legislative
requirement, it was not technically possible to develop and implement a
solution that would address all of the requirements to complete the
work within the required mandate timeframe. Since implementation, we
have improved the functionality of the application and the specific
accomplishments we have made are outlined below under "Action Plan."
More importantly, it was not our intent to implement a system where
claims processing contractors would be available 24 hours a day, 7 days
a week concurrent, with the implementation of a single I-800 number
network. We believe that the calls identified as receiving "No Answers"
in the report would more appropriately be referred to as "Calls routed
to closed locations."
The report indicates that there were some portions of this subset of
calls that were "inadvertently disconnected." We find this portion of
the report particularly troublesome, and it is one of the reasons why
we immediately contacted GAO to request more detailed information on
this issue. It is disheartening for CMS to learn that GAO is not
willing to immediately release detailed information to CMS that would
aid in our troubleshooting in what appears to be a network problem
during an ongoing audit. We believe you understand from this and past
audits that telephone networks are vastly complex. There are literally
thousands of places where a problem with a single line of code or
hardware can result in some sort of service degradation. It appears,
based on the statements made in the report, that some portions of the
calls are being affected by a network issue that we have not yet
identified. As we explained to the GAO representative immediately upon
receiving the draft report, specific call data from the audit would
save extensive CMS staff resources and valuable time locating the
problem independent of having your data. More importantly, it would
allow CMS to respond more quickly with a solution, thus, reducing the
number of callers who will now continue to have some percentage of
calls "inadvertently disconnected." It is our sincere request that you
provide us with the crucial information upon final completion of your
report.
Action Plan:
We have deployed an Advanced Routing Database in the speech application
that reviews the hours of operations for each of the 47 claims call
centers that are now connected to 1-800-MEDICARE. The database checks
to make sure that the call center is open before calls are routed to
one of the claims call centers. If the call center is closed for any
reason, the call will be routed to one of the general Medicare CSRs,
and the CSR will try to assist the caller with general information or
instruct the caller to call back during business hours when the claims
system is available.
We have developed new information tools for the general Medicare CSRs
to use which makes it easy for them to find the correct Medicare claims
center for their caller and its correct hours of operation, prior to
transferring the call.
We have implemented weekly business and technical process meetings with
each of the call centers (both general and claims) to improve process
and technologies. As a part of these meetings, the need for all CSR
refresher training is identified and scheduled. We have built failsafe
software into the transfer technology that will not allow the CSR to
drop the caller into closed or invalid call center's queues. If the
queue cannot accept the call when the CSR transfers it, the network
forces the call back to the transferring CSR. This software helps to
prevent CSRs from "dropping" calls and keeps the caller from getting
lost in the transfer process.
GAO Recommendation:
Assess the current scripts for the most commonly asked questions to
ensure that they are understandable to CSRs and potential callers and
if not, revise them as needed and pretest new and revised scripts to
ensure that CSRs can effectively use them to accurately answer callers'
questions.
CMS Response:
We concur with the recommendation.
Relevant Information:
We are always interested in improving the way we develop and implement
scripts in the Desktop application. However, it is inaccurate, as the
report implies, to say that there is no testing of the scripts used by
1-800-MEDICARE CSRs. We based the majority of our scripts on content
that has been consumer tested in the development of Medicare
publications. In addition, we hold regular focus groups to identify
ways to explain concepts to beneficiaries in our static publications as
well as our communication messages. We often incorporate that language
into scripts; thus, there has been some degree of beneficiary testing.
It is also true that 1-800-MEDICARE is the front-line of our customer
service process and, as such, it is the very first place people go when
"hot topics" need to be addressed. Scripts are often developed to
address urgent or emergent issues, and there are situations where
sufficient time to fully pre-test/focus tests does not exist.
Concerns:
No concerns identified.
Action Plan:
We will continue our focused feedback sessions with CSRs that center on
pre-testing scripts. CSRs participate in content workgroups to develop
the MMA scripts and job aids.
We continue to hold weekly meetings with the call centers to discuss
how to make the scripts more user-friendly, complete, understandable,
and comprehensive.
Additionally, we have a desktop tool that allows CSRs to make
recommended changes to scripts. This feedback is usually the result of
caller feedback indicating the script is confusing, not helpful, etc.
CSR feedback is compiled weekly and the content is shared with CMS
staff. call center management, the learning and development team, as
well as other 1-800-MEDICARE functional areas. CSR feedback is reviewed
as part of the weekly meetings to determine call trends, the need for
additional scripts or script modifications, identify training
opportunities, and assist with overall continuous improvement at 1-800-
MEDICARE.
Based on feedback received, we are considering the implementation of an
editorial board that would review words and phases to make
recommendations such as changing "trunk strength" to "upper body
strength."
GAO Recommendation:
Enhance testing of CSR's skills in accurately answering the most
commonly asked questions using scripts and, if needed, provide
additional training to improve the accuracy and completeness of their
responses.
CMS Response:
We concur with the recommendation presented.
Relevant Information:
Addressed in Concerns section.
Concerns:
We believe the report reflects an incorrect understanding of the
training requirements. The specific contractual requirement is for a
CSR to "Achieve Expectations on or above two practice calls in a row."
The contractor addresses this by using a series of six simulated calls,
and the class trainers sit with the individual CSRs to determine, on a
ease-by-case basis, their success at handling the simulated calls. If
the CSR does not do well on the skill assessments on the simulated
calls, whether they pass or fail, the trainers provide the CSR with
additional coaching and mentoring. The intent is to provide the
necessary training to ensure that the CSR is ready for real-time
operations. In many of the cases where a CSR "fails" at a practice call
scenario, it may be due to the CSR being nervous, and the extra
tutoring helps to alleviate failure. This is not a situation where the
trainers just run through the six simulated calls until the CSR gets
two in a row correct, but is intended to provide the CSR with any
additional training they need in areas of weakness.
In the case of customer service, we disagree with the recommendation
that the questions targeted for the skills other than knowledge are not
as meaningful. It is very important that the CSRs can handle themselves
in all areas of customer service and even if they know how to get to
the right script and provide the correct answer, they will not have
credibility if they do not sound and act professionally.
Action Plan:
We are reassessing the testing requirements currently in place to
determine ways to better ensure that the CSRs are prepared to handle
calls at the point that they are certified. We are adjusting the
training protocol to spread testing of the CSR's ability to handle
calls over the length of the training period rather than just at the
conclusion and to focus on categories of calls, especially on the most
frequently asked questions. This approach allows more time for the
trainers to coach and mentor the CSRs prior to the next phase of their
job experience, which includes side-by-side work with an experienced
CSR We will also benchmark against industry standards to determine more
effective approaches, ensuring that the CSRs are prepared to respond to
calls upon completion of the classroom training.
GAO Recommendation:
Supplement current monitoring efforts to include a systematic review of
the accuracy of information provided by the CSRs as a group for the
most frequently asked questions and use the results to modify scripts
or provide more training, as needed.
CMS Response:
We concur with the recommendation to pursue trend analysis. However, we
cannot concur with the comment on the observations made in this
section, because we do not have access to the data GAO used to arrive
at their findings. (Noted below under "Concerns"):
Relevant Information:
We agree that more analysis could be conducted to better identify
global deficiencies across our CSR base. At the same time, we believe
that we have done a very good job developing a quality assurance
program that focuses on the important requirements for both accuracy
and customer service skills needed to answer calls from an elderly
population.
We require our contractor to monitor each CSR's call performance on a
monthly basis using a national quality scorecard. The accuracy rate for
the months July through October 2004 is as follows: July 89 percent;
August 91 percent; September 90 percent; October 92 percent. Our
national quality assurance scorecard measures the entire customer
service experience across three broad categories-accuracy,
completeness, and call action-and a subset of criteria within each of
these categories. For example, accuracy is assessed on the basis of
factors such as whether the CSR understood the question or whether they
used the correct script(s).
In order to have an independent assessment of performance we conduct
"mystery shopping" through a specialized quality evaluation contractor.
A response is classified as "fully responsive" only if all key points
are conveyed to the caller. For example, for the mystery shopping on
prescription drugs, the calls scored 85 percent or above as "fully
responsive."
In addition, we require our contractor to conduct 1,000 beneficiary
satisfaction surveys each month to determine customer satisfaction with
the services provided. For the month of July, 92 percent were satisfied
with the service they received. This is consistent with other months.
We also believe that we have done a very good job implementing
technology to help support this effort and keep costs under control. We
have not gotten to the point of regularly taking a more global review
of the data we collect or looking at it from a higher perspective;
(i.e., trends).
Concerns:
In the report, GAO identified 29 percent of the calls made to 1-800-
MEDICARE resulted in "inaccurate answers." The methodology used in
identifying an answer as inaccurate was "if the CSR provided
information on the topic but did not provide sufficient and complete
information to meet our criteria, we considered the answer to be
inaccurate." Consequently, it is difficult for:
IS to understand specifically what the specific criteria for a correct
answer. As mentioned in he report, CMS provided GAO with a broad list
of questions and various scripts that may have been used in answering
the questions. It is important to note, it is not the policy of CMS to
require all related scripts to be read to a caller, nor is it CMS
policy to require the entire text of any single script to be read. CSRs
are instructed to read only those portions of the scripts that pertain
to the callers' specific inquiry. Furthermore, there are judgment
decisions the CSRs must make with regard to how much information the
caller really needs and their level of interest and understanding. In
addition, CMS requires the CSRs to remain objective and to not provide
opinions to the callers. Based on the information provided in the draft
report it is not possible for CMS to identify what portion of the
responses deemed "inaccurate" by GAO. We hope to receive call details
and specific measurement criteria from the list of calls made to
further investigate this issue. Additionally, we question the GAO's use
of the word "inaccurate" and "incomplete" answers to classify a
grouping of responses. We do not consider an "incomplete" answer to be
an "inaccurate" answer. Again, we thank you for the opportunity to
review and comment on this report.
Action Plan:
CMS has deployed trained staff to monitor CSR calls themselves and
provide recommendations on changes and improvements for content. We
have implemented ongoing analysis of trending reports from all
monitoring sources to ensure that issues are identified and changes
made as a result of the findings of these groups.
We appreciate the feedback provided and we will implement a plan that
ensures that results of our Quality Assurance (QA) activities are
trended and that we have a process in place that takes the results of
that trending to the next step. Each month we will focus efforts on a
specific Frequently Asked Questions activity based on QA findings from
our other measures. This will enable us to better ensure that the CSRs
handling these questions, which make up a large majority of the issues
our callers want to discuss, can do so more accurately and effectively.
Attachment:
ATTACHMENT A:
The CMS is requesting additional information from the call sample that
was conducted. This information will be extremely helpful to us in our
efforts to troubleshoot and correct network problems and it will also
support our efforts to improve the consistency and accuracy of the
responses we provide to callers:
For each of the calls in the Study we are requesting the following
information:
Date of call Time of call Number that the call was placed from:
The phone number of the switch (in the event calls were not made from
individual's homes) The name that the tester used on the call:
The specific question asked How each call was scored The answer that
was considered to be correct The answer that was received:
In addition, it would be very helpful if the following information
could be provided separately for the subset of calls that were
identified as "Inadvertently disconnected" in the report:
Date of call Time of call Number that the call was placed from:
The phone number of the switch (in the event calls were not made from
individual's homes):
[End of section]
Appendix IX: GAO Contact and Staff Acknowledgments:
GAO Contact:
Sheila K. Avruch, (202) 512-7277:
Acknowledgments:
Shaunessye D. Curry, Joy L. Kraybill, Krister P. Friday, Sari B.
Shuman, Mary W. Reich, Ramsey L. Asaly, Alexis Chaudron, Perry G.
Parsons, and Leslie Spangler made key contributions to this report.
FOOTNOTES
[1] These companies, known as Medicare claims administration
contractors, maintain other toll-free help lines that are open during
normal business hours. In June 2004, CMS began a process to discontinue
listing these separate help line numbers and have 1-800-MEDICARE serve
as the only number that callers would use to access information about
Medicare eligibility, enrollment, benefits, coverage, and payment. As
of September 30, 2004, this process was complete.
[2] Pub. L. No. 108-173, § 923(d)(2), 117 Stat. 2066, 2395.
[3] We made 420 calls in order to have a sample that was large enough
to determine if differences in accuracy were significant.
[4] http://medicare.custhelp.com/cgi-bin/medicare.cfg/php/enduser/
std_alp.php.
[5] For our first objective, we focused on the accuracy of information
provided by the CSRs, regardless of the contractor that managed their
work. For our next two objectives, we relied on information provided by
one of the two contractors--which we refer to as the primary
contractor--because this contractor provided the training for all new
CSRs and managed CSRs who answered 80 percent of the calls on the help
line in July 2004. CMS does not use the term "primary contractor,"
because both companies contract directly with CMS. We are using the
term primary contractor to distinguish the one that had a larger role
during the time that we were conducting our calls.
[6] GAO, IRS's 2002 Tax Filing Season: Returns and Refunds Processed
Smoothly; Quality of Assistance Improved, GAO-03-314 (Washington, D.C.:
Dec. 20, 2002); GAO, IRS Telephone Assistance: Limited Progress and
Missed Opportunities to Analyze Performance in the 2001 Filing Season,
GAO-02-212 (Washington, D.C.: Dec. 7, 2001).
[7] In May 2004, 1-800-MEDICARE received over 3 and a half million
calls, which was more calls received than in any prior month.
Throughout the summer of 2004, 1-800-MEDICARE received between 1
million and 2 million calls each month.
[8] There are over 670 scripts for 1-800-MEDICARE in use.
[9] www.Medicare.gov.
[10] For some topics, such as the prescription drug discount card,
there are several scripts that deal with the topic, but generally one
script provides the best information to accurately answer a specific
question, according to CMS. For each of our proposed questions, CMS
provided the name and number of the script that it considered as having
the best information for CSRs to use in formulating a response.
[11] This tool is also available on Medicare's Web site, at http://
www.medicare.gov/AssistancePrograms/home.asp.
[12] AARP is an association for individuals 50 years of age and older.
It provides information, such as tips for healthy living and retirement
planning; advocacy on issues affecting the elderly; and services, such
as supplemental health insurance.
[13] In its comments on a draft of this report, CMS informed us that
the written information used to develop 1-800-MEDICARE scripts often
comes from Medicare publications that have been consumer tested as part
of the publication preparation process. Language that has undergone
some consumer testing is often incorporated into the scripts to improve
clarity.
[14] CMS distributes a handbook called Medicare & You to all Medicare
beneficiary households. The Medicare & You handbook describes the
Medicare program and its benefits.
[15] The Centers for Disease Control and Prevention is an agency of the
U.S. Department of Health and Human Services that provides federal
leadership to develop and apply disease prevention and control,
environmental health, and health promotion and education activities
designed to improve the health of people of the United States.
[16] The Substance Abuse and Mental Health Services Administration is
an agency of the U.S. Department of Health and Human Services
established to focus attention, programs, and funding on improving the
lives of people with, or at risk for, mental and substance abuse
disorders.
[17] U.S. Department of Health and Human Services, Centers for Disease
Control and Prevention, Scientific and Technical Information: Simply
Put (Atlanta, Ga.: 1999) (at http://www.cdc.gov/publications.htm,
downloaded Sept. 21, 2004), and U.S. Department of Health and Human
Services, Substance Abuse and Mental Health Services Agency, Technical
Assistance Bulletin: Pretesting is Essential; You Can Choose From
Various Methods. (1994, http://www.health.org/_usercontrols/
printpage.aspx?from downloaded Sept. 21, 2004).
[18] Each of our six questions was posed during evening and weekend
hours, as well as during normal business hours, using the same random
pattern of calling during certain hours of each day over the course of
3 weeks. However, because questions 4 and 6 were often transferred or
referred more often to other contractors' help lines, these were the
questions that were not answered when the contractors were closed for
business. In contrast, the 1-800-MEDICARE help line did not routinely
transfer questions 1, 2, 3, and 5 to other contractors.
[19] For 14 transferred calls, our callers were provided with a
telephone number and were able to complete the calls later. For 10 of
these 14 calls completed later (71 percent), claims administration
contractors' CSRs answered our questions accurately.
[20] There can be more than one script on a particular topic, but often
one of the scripts is the best one to answer a specific question about
that topic.
[21] While the primary contractor provided all training for newly hired
CSRs, both the primary and secondary contractors provided continuing
training for their CSRs.
[22] The test measures several types of information and skills that
CSRs need. It consists of 52 questions: 14 questions on choosing a
script that can be used to answer a question; 10 matching questions to
choose definitions of Medicare terms; 10 true or false questions where
CSRs were instructed to use a script to answer a question about the
Medicare program; 6 multiple choice questions on call handling
etiquette; and 12 matching questions to identify the organizations to
which CSRs could refer certain questions--for example, the Social
Security Administration for questions about enrolling in Medicare. The
guide for scoring each question on scripts indicates several scripts
that could be named as possible responses, including the script that
would best address the question posed. If the CSR chooses any of the
scripts that are listed as possible responses, the answer is scored
correctly, even if the CSR does not choose the script with the best
information to answer the question.
[23] The call handling exam for new CSRs consists of up to six
questions. In order to successfully pass, the new CSR must adequately
handle two of these calls consecutively.
[24] GAO-02-212.
[25] GAO-03-314.
[26] The primary contractor also surveys callers to assess their
satisfaction with the information they received from 1-800-MEDICARE.
These customer satisfaction survey results are also reported to CMS in
standard reports.
[27] Incoming Calls Management Institute, Call Center Monitoring Study
II Final Report (Annapolis, Md.: 2002).
[28] This monitoring occurs in addition to the four calls per CSR per
month that are monitored by the primary contractor, and the calls
monitored by the subcontractor.
[29] In addition, the subcontractor monitored about 1,400 calls in July
2004.
[30] The National Medicare Education Program publishes the Medicare &
You handbook and maintains other activities to communicate with
Medicare beneficiaries. The 1-800-MEDICARE help line is one of the
activities in the program.
[31] For example, for one question related to Medicare's coverage of a
piece of durable medical equipment, the study defined a fully
responsive answer as one that informed the caller that a prescription
would be needed for the item, specified that the beneficiary needed a
supplier who accepts Medicare, provided an overview of the Medicare
billing process, shared information on how to contact the contractors
that process durable medical equipment claims, and indicated that
supplemental insurance might pay part of the costs.
[32] If CSRs provided partially correct but incomplete information that
did not fully address our criteria, we scored the calls as
"inaccurate."
[33] This research was part of a CMS effort to determine whether CSRs
were prepared to handle questions about program changes resulting from
the MMA.
[34] The April 2004 test calls used a scorecard that included up to
nine different elements that measured accuracy, depending on the
question the caller used. For example, the call could be scored on
whether the CSR accurately informed the caller which Medicare
beneficiaries would be eligible for a prescription drug discount card,
how a beneficiary could apply for a prescription drug discount card,
and when a beneficiary could start to use their prescription drug
discount card.
[35] For question 1, the name of either one of the two prescription
drug plans with the lowest cost for the drugs was the correct answer.
For questions 2 and 3, the correct answer was "yes," and for question
6, the correct answer was "no."
[36] Except for a script used when transferring calls, each of the top
15 scripts accessed in May 2004 by 1-800-MEDICARE CSRs was designed to
answer various questions about the prescription drug discount card
program.
[37] http://medicare.custhelp.com/cgi-bin/medicare.cfg/php/enduser/
std_alp.php.
[38] If a CSR mentioned any prescription drug discount card that cost
$182.80 per month, we considered the response to be accurate.
[39] If a CSR mentioned any prescription drug discount card that cost
$50.45 per month, we considered the response to be accurate.
[40] TRICARE for Life provides secondary military health coverage
available for Medicare-eligible uniformed services beneficiaries,
their eligible family members, and survivors enrolled in Medicare part
B.
[41] A certificate of medical necessity is a standardized form that
physicians complete to indicate that an item is needed for medical
reasons.
[42] The Medicare.gov Web site notes that Medicare will pay 80 percent
after the $100.00 part B deductible is met, but we did not require CSRs
to mention the deductible for our accuracy criteria.
[43] According to the HHS Office of the Inspector General, the median
Medicare reimbursement for a purchased power wheelchair was $5,297 in
2003.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548: