Medicare
Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved Gao ID: GAO-06-654 May 3, 2006On January 1, 2006, Medicare began providing coverage for outpatient prescription drugs through its new Part D benefit. Beneficiaries who enroll in Part D may choose a drug plan from those offered by private plan sponsors under contract to the Centers for Medicare & Medicaid Services (CMS), which administers the Part D benefit. Beneficiaries have until May 15, 2006, to enroll in the Part D benefit and select a plan without the risk of penalties. GAO was asked to review the quality of CMS's communications on the Part D benefit. GAO examined 70 CMS publications to select 6 documents for review and contracted with the American Institutes for Research to evaluate the clarity of these texts; made 500 calls to the 1-800-MEDICARE help line; and contracted with the Nielsen Norman Group to evaluate the usability of the Medicare Web site.
The information given in the six sample documents that GAO reviewed describing the Part D benefit was largely complete and accurate, although this information lacked clarity. The documents were unclear in two ways. First, although about 40 percent of seniors read at or below the fifth-grade level, the reading levels of these documents ranged from seventh grade to postcollege. Second, on average, the six documents did not comply with about half of 60 common guidelines for good communication. For example, the documents used too much technical jargon and often did not define difficult terms, such as formulary. Moreover, 16 beneficiaries and advisers that GAO tested reported frustration with the documents' lack of clarity and had difficulty completing the tasks assigned to them. Although the documents lacked clarity, they informed readers of enrollment steps and factors affecting coverage decisions and were consistent with laws, regulations, and agency guidance. Customer service representatives (CSR) responded to the 500 calls GAO placed to CMS's 1-800-MEDICARE help line accurately and completely about two-thirds of the time. Of the remainder, 18 percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, and about 3 percent received incomplete responses. In addition, about 5 percent of GAO's calls were not answered, primarily because of disconnections. Accuracy and completeness rates of CSRs' responses varied significantly across the five questions GAO asked. For example, while CSRs provided accurate and complete responses to calls about beneficiaries' eligibility for extra help 90 percent of the time, the accuracy rate for calls concerning the drug plan that would cost the least for a specified beneficiary was 41 percent. For this question, the CSRs responded inappropriately for 35 percent of the calls by explaining that they could not identify the least costly plan without the beneficiary's personal information--even though CSRs had the information needed to answer the question. The time GAO callers waited to speak with CSRs also varied, ranging from no wait time to over 55 minutes. For 75 percent of the calls--374 of the 500--the wait was less than 5 minutes. The Part D benefit portion of the Medicare Web site can be difficult to use. GAO's test of the site's overall usability--the ease of finding needed information and performing various tasks--resulted in scores of 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate that using the site can be difficult. For example, the prescription drug plan finder was complicated to use and some of its key functions, such as "continue" and "choose a drug plan," were often not visible on the page without scrolling down.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-06-654, Medicare: Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved
This is the accessible text file for GAO report number GAO-06-654
entitled 'Medicare: Communications to Beneficiaries on the Prescription
Drug Benefit Could Be Improved' which was released on May 3, 2006.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Requesters:
May 2006:
Medicare:
Communications to Beneficiaries on the Prescription Drug Benefit Could
Be Improved:
GAO-06-654:
GAO Highlights:
Highlights of GAO-06-654, a report to congressional requesters.
Why GAO Did This Study:
On January 1, 2006, Medicare began providing coverage for outpatient
prescription drugs through its new Part D benefit. Beneficiaries who
enroll in Part D may choose a drug plan from those offered by private
plan sponsors under contract to the Centers for Medicare & Medicaid
Services (CMS), which administers the Part D benefit. Beneficiaries
have until May 15, 2006, to enroll in the Part D benefit and select a
plan without the risk of penalties.
GAO was asked to review the quality of CMS‘s communications on the Part
D benefit. GAO examined 70 CMS publications to select 6 documents for
review and contracted with the American Institutes for Research to
evaluate the clarity of these texts; made 500 calls to the 1-800-
MEDICARE help line; and contracted with the Nielsen Norman Group to
evaluate the usability of the Medicare Web site.
What GAO Found:
The information given in the six sample documents that GAO reviewed
describing the Part D benefit was largely complete and accurate,
although this information lacked clarity. The documents were unclear in
two ways. First, although about 40 percent of seniors read at or below
the fifth-grade level, the reading levels of these documents ranged
from seventh grade to postcollege. Second, on average, the six
documents did not comply with about half of 60 common guidelines for
good communication. For example, the documents used too much technical
jargon and often did not define difficult terms, such as formulary.
Moreover, 16 beneficiaries and advisers that GAO tested reported
frustration with the documents‘ lack of clarity and had difficulty
completing the tasks assigned to them. Although the documents lacked
clarity, they informed readers of enrollment steps and factors
affecting coverage decisions and were consistent with laws,
regulations, and agency guidance.
Customer service representatives (CSR) responded to the 500 calls GAO
placed to CMS‘s 1-800-MEDICARE help line accurately and completely
about two-thirds of the time. Of the remainder, 18 percent of the calls
received inaccurate responses, 8 percent of the responses were
inappropriate given the question asked, and about 3 percent received
incomplete responses. In addition, about 5 percent of GAO‘s calls were
not answered, primarily because of disconnections. Accuracy and
completeness rates of CSRs‘ responses varied significantly across the
five questions GAO asked. For example, while CSRs provided accurate and
complete responses to calls about beneficiaries‘ eligibility for extra
help 90 percent of the time, the accuracy rate for calls concerning the
drug plan that would cost the least for a specified beneficiary was 41
percent. For this question, the CSRs responded inappropriately for 35
percent of the calls by explaining that they could not identify the
least costly plan without the beneficiary‘s personal information”even
though CSRs had the information needed to answer the question. The time
GAO callers waited to speak with CSRs also varied, ranging from no wait
time to over 55 minutes. For 75 percent of the calls”374 of the 500”the
wait was less than 5 minutes.
The Part D benefit portion of the Medicare Web site can be difficult to
use. GAO‘s test of the site‘s overall usability”the ease of finding
needed information and performing various tasks”resulted in scores of
47 percent for seniors and 53 percent for younger adults, out of a
possible 100 percent. While there is no widely accepted benchmark for
usability, these scores indicate that using the site can be difficult.
For example, the prescription drug plan finder was complicated to use
and some of its key functions, such as ’continue“ and ’choose a drug
plan,“ were often not visible on the page without scrolling down.
What GAO Recommends:
GAO is recommending that the CMS Administrator enhance the quality of
its communications by taking actions to improve written materials, its
1-800-MEDICARE help line, and the Medicare Web site. CMS said that
GAO‘s findings did not present a complete and accurate picture of its
activities. However, CMS said that it supports the goals of GAO‘s
recommendations and is already taking steps to implement them.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-654].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Leslie G. Aronovitz at
aronovitzl@gao.gov or (312) 220-7600.
[End of section]
Contents:
Letter:
Results in Brief:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Briefing on Medicare Part D:
Appendix II: Objectives, Scope, and Methodology:
Appendix III: Comments from the Centers for Medicare & Medicaid
Services:
Appendix IV: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Sample of Six Selected Documents:
Table 2: Questions and Criteria Used to Evaluate Accuracy:
Abbreviations:
AIR: American Institutes for Research:
CMS: Centers for Medicare & Medicaid Services:
CSR: customer service representative:
MMA: Medicare Prescription Drug, Improvement, and Modernization Act of
2003:
NN/g: Nielsen Norman Group:
SHIP: State Health Insurance Assistance Program:
SMOG: Simplified Measure of Gobbledygook:
United States Government Accountability Office:
Washington, DC 20548:
May 3, 2006:
Congressional Requesters:
In the most significant change to the Medicare program since its
inception, the Medicare Prescription Drug, Improvement, and
Modernization Act of 2003 (MMA)[Footnote 1] established an outpatient
prescription drug benefit in Medicare, known as the Part D benefit.
Coverage for this new benefit began on January 1, 2006. Until this
time, Medicare, the federal program that finances health care benefits
for about 42 million elderly and disabled beneficiaries, had not
generally provided coverage for outpatient prescription drugs.
Beneficiaries may choose a Part D plan[Footnote 2] from multiple plans
offered by private sponsors[Footnote 3] under contract to the Centers
for Medicare & Medicaid Services (CMS),[Footnote 4] the agency that is
responsible for administering the Medicare program, including the Part
D benefit. These plans differ in the drugs they cover and the
pharmacies they use. In addition, the costs to the enrollee for the
monthly premium, the annual deductible, and co-payments for covered
drugs vary by plan. As of April 20, 2006, more than 30 million of
Medicare's 42 million beneficiaries were enrolled in a Part D plan or
had other outpatient prescription drug coverage. Beneficiaries have
until May 15, 2006, to select a plan without the risk of penalties in
the form of higher premiums.
Given the newness and complexity of the Part D benefit, it is critical
that beneficiaries and their advisers, including members of their
families, understand the available options so that beneficiaries can
make informed decisions on whether to enroll in Part D, and if so,
which drug plan to choose. Beneficiaries need to compare drug plans in
light of their anticipated prescription drug needs and existing
arrangements for paying for these drugs. In addition to comparing costs
and drug coverage, beneficiaries need to consider whether the plans
they are comparing have contracted with a local or mail-order pharmacy
that will provide a convenient means of filling their prescriptions.
As part of its responsibilities, CMS has undertaken outreach and
education efforts to provide beneficiaries and their advisers with the
information they need about Part D through various media, including
written documents, the 1-800-MEDICARE help line,[Footnote 5] and the
Medicare Web site.[Footnote 6] As of December 2005, CMS has produced
more than 70 written documents to explain Part D to beneficiaries.
Medicare & You--the beneficiary handbook--is the most widely available
of these documents and was sent directly to beneficiaries in October
2005. Other CMS documents are targeted to specific groups of
beneficiaries, such as dual-eligible beneficiaries[Footnote 7] and
beneficiaries with Medicare Advantage or Medigap policies.[Footnote 8]
Since March 1999, CMS has administered its nationwide 1-800-MEDICARE
help line to answer beneficiaries' questions about the Medicare
program. As of December 2005, about 7,500 customer service
representatives (CSR) were handling calls on the help line, which
operates 24 hours a day, 7 days a week, and is run by two CMS
contractors. Calls are answered by an automated system and are routed
to a CSR for specific questions, including those about Part D. CMS
provides CSRs with detailed scripts to use in answering the questions.
CSRs type in related keywords to generate a list of suggested scripts
for a given question, select the script they consider best suited to
the question, and read excerpts or the entire script. Call center
contractors write the scripts, and CMS checks them for accuracy and
completeness. CSRs can also consult other information sources, such as
the Medicare Web site. CMS does not allow CSRs to offer individualized
guidance to callers, including advice in choosing a drug plan. CMS's
Medicare Web site provides information about all aspects of the
Medicare program. The Web site contains basic information about the
Part D benefit; suggests factors for beneficiaries to consider when
choosing a plan; describes situations common to beneficiaries with
guidance on next steps to take in deciding whether to enroll and what
plan to choose; lists frequently asked questions; and allows users to
view, print, or order publications. In addition, the site contains
information on cost, coverage, and convenience of individual plans.
There is also a tool that allows beneficiaries to enroll directly in
the plan they have chosen.
CMS has also arranged for State Health Insurance Assistance Programs
(SHIP) to provide Part D information on request to Medicare
beneficiaries and their advisers. Currently, CMS provides grants to the
54 SHIPs--one in each state, the District of Columbia, the Virgin
Islands, Puerto Rico, and Guam. State SHIPs provide subgrants to over
1,300 local organizations to assist SHIPs in their efforts. In total,
SHIPs rely on over 12,000 trained counselors, most of whom are
volunteers, to provide free counseling and assistance via telephone and
face-to-face sessions, public education presentations and programs, and
media activities.
Widespread confusion among beneficiaries about the costs and coverage
under the new benefit has been reported by the media and others. For
example, according to an October 2005 survey by a research
organization, some beneficiaries are unaware of the penalties for late
enrollment and others did not realize that beneficiaries had to sign up
for the benefit.[Footnote 9] In light of your interest in ensuring that
Medicare beneficiaries receive the information they need to make
informed decisions, you asked us to examine the quality of the
information being provided on the Part D benefit. In this report, we
examined:
* the extent to which CMS's written documents describe the Part D
benefit in a clear, complete, and accurate manner;
* the effectiveness of CMS's 1-800-MEDICARE help line in providing
accurate, complete, and prompt responses to callers inquiring about the
Part D benefit;
* whether CMS's Medicare Web site presents information on the Part D
benefit in a usable manner; and:
* how CMS has used SHIPs to respond to the needs of Medicare
beneficiaries for information on the Part D benefit.
We briefed your staff regarding the results of our review on April 19,
2006. Appendix I contains information we provided during our briefing
to your staff.
To evaluate CMS's written documents describing the Part D benefit, we
examined 70 relevant CMS publications and selected a sample of six
documents for in-depth review. These documents represent a variety of
document types, content, and target audiences and include Section 6 of
the Medicare & You beneficiary handbook, which discusses Part D. To
assess the clarity of the sample documents, we contracted with the
American Institutes for Research (AIR), a firm with experience in
evaluating written documents. AIR evaluated the texts by using three
standard readability tests;[Footnote 10] 60 commonly recognized good
communications practices; and user testing with 11 Medicare
beneficiaries and 5 advisers to beneficiaries, all of whom were asked
to perform 18 specified tasks related to enrollment, coverage, costs,
penalty, and informational resources and provide feedback about their
experiences. To evaluate completeness, we reviewed the sample documents
to determine if they included sufficient information for the
beneficiaries to identify (1) their next steps in deciding whether to
enroll and what plan to choose and (2) important factors, such as
penalty provisions, that could affect their coverage decisions. To
evaluate accuracy, we reviewed the sample documents for consistency
with MMA, regulations, and CMS guidance.
To assess the accuracy, completeness, and promptness of the help line
responses, we made 500 calls to 1-800-MEDICARE, posing one of five
questions about Part D in each call so that each question was asked 100
times. To develop the questions, we considered topics listed on the
Medicare Web site and topics addressed in scripts frequently accessed
by CSRs. To develop our criteria for evaluating the accuracy and
completeness of CSRs' responses, we used three resources: (1) the
prescription drug finder tool on the Medicare Web site, (2) the 1-800-
MEDICARE scripts, and (3) input from CMS officials. We also recorded
the length of each call, including wait times, and the time it took to
be connected to a CSR.
To assess whether the Medicare Web site presents information on the
Part D benefit in a usable manner, we contracted with the Nielsen
Norman Group (NN/g), a firm with expertise in Web design. NN/g
conducted three evaluations: (1) it calculated an overall usability
score for the site--considering factors such as site navigation,
customer support, and presentation of online forms--to reflect the ease
of finding necessary information and performing various tasks; (2) it
determined the usability of 137 detailed aspects of the Web site,
including aspects of Web design, online tools, and writing style; and
(3) it tested the ability of seven participants (five beneficiaries and
two advisers to beneficiaries) to complete a total of 34 user tests to
determine the ease of performing a variety of Web-related tasks, such
as browsing the site and determining how to join a plan. We also
reviewed the results of CMS's analysis of its Web site's compliance
with requirements that federal government Web sites be accessible to
people with disabilities.
Finally, to examine how CMS has used SHIPs to meet the information
needs of beneficiaries regarding Part D, we obtained information about
SHIPs, their funding, changes made in response to the new benefit, and
the impact of Part D on the demand for SHIP services. In addition, we
interviewed CMS officials who monitor SHIP activities as well as SHIP
coordinators in the five states with the largest populations of
Medicare beneficiaries--California, Florida, New York, Pennsylvania,
and Texas.
We performed our work from November 2005 through May 2006 in accordance
with generally accepted government auditing standards. For more
information on our methodology, see appendix II.
Results in Brief:
The sample of CMS's written documents we reviewed describing the Part D
benefit to Medicare beneficiaries and their advisers were largely
complete and accurate, but the information these documents presented
lacked clarity. The documents were unclear in two ways. First, about 40
percent of seniors read at or below the fifth-grade level, but the
reading levels of the documents ranged from seventh grade to
postcollege. As a result, documents at these levels are not completely
clear and understandable for many seniors. Second, on average, the six
documents did not comply with about half of the 60 commonly recognized
guidelines for good communications. For example, although the documents
included concise and descriptive headings, they used too much technical
jargon and often did not define difficult terms, such as
formulary.[Footnote 11] The 11 beneficiaries and 5 advisers we tested
reported frustration with the documents' lack of clarity as they
encountered difficulties in understanding and attempting to complete 18
specified tasks. For example, none of these beneficiaries and only 2 of
the advisers were able to complete the task of computing their
projected total out-of-pocket costs for a plan that provided Part D's
standard coverage. Only 1 of the 18 tasks was completed by all
beneficiaries and advisers. Even those who were able to complete a
given task expressed confusion and frustration as they worked to
comprehend the relevant text. Although the sample documents lacked
clarity, the information presented in them was generally complete. The
documents informed readers of next steps in determining whether to
enroll and what plan to choose, and of important factors that could
affect their coverage decisions. The information in the sample
documents was also generally accurate when evaluated for consistency
with MMA, implementing regulations, and agency guidance.
Responses to the 500 calls we placed to CMS's 1-800-MEDICARE help line
regarding the Part D benefit were frequently accurate and complete.
However, we nonetheless received a substantial number of responses that
were inaccurate, incomplete, or inappropriate and that sometimes
involved an extensive wait before we could speak to a CSR. CSRs
answered 67 percent of the calls accurately and completely. Of the
remainder, 18 percent of the calls received inaccurate responses, 8
percent of the responses were inappropriate given the question asked,
and about 3 percent received incomplete responses. In addition, about 5
percent of our calls were not answered, primarily because of
disconnections.[Footnote 12] Accuracy and completeness rates of CSRs'
responses varied significantly for the five questions we asked. For
example, for the question on whether a beneficiary qualifies for extra
help, CSRs provided an accurate and complete response 90 percent of the
time. However, for a question concerning which drug plan would cost the
least for a beneficiary with certain specified prescription drug needs,
the accuracy rate was 41 percent. CSRs inappropriately responded 35
percent of the time that this question could not be answered without
personal identifying information--such as the beneficiary's Medicare
number or date of birth--even though the CSRs could have answered our
question using CMS's Web-based prescription drug plan finder tool. The
amount of time we waited to speak with a CSR also varied, ranging from
no wait time to over 55 minutes. For 75 percent of the calls--374 of
the 500--we waited less than 5 minutes. For the remainder of the calls,
62 were answered in less than 15 minutes, 39 calls were answered in
from 15 minutes to less than 25 minutes, and 25 led to a wait of 25
minutes or more.
We found that the Part D benefit portion of the Medicare Web site can
be difficult to use. In our evaluation of overall usability--the ease
of finding needed information and performing various tasks--we found
usability scores of 47 percent for seniors and 53 percent for younger
adults, out of a possible 100 percent. While there is no widely
accepted benchmark for usability, these scores indicate that using the
site can be difficult. For example, tools such as the drug plan finder
were complicated to use, and forms that collect information online from
users were difficult to correct if the user made an error. In our
evaluation of the usability of 137 detailed aspects of the Part D
portion of the site, including features of Web design and online tools,
we found that 70 percent of these aspects could be expected to cause
users confusion. For example, key functions of the prescription drug
plan finder tool, such as the "continue" and "choose a drug plan"
buttons, were often not visible on the page without scrolling down. In
our evaluation of the ability of seven participants to collectively
complete 34 user tests, we found that on average, participants were
able to proceed slightly more than halfway through each test. In
addition, CMS evaluated whether its Web site complied with pertinent
federal requirements regarding accessibility for people with
disabilities in March 2006. Although CMS has established features to
make information on its Web site accessible to disabled users, it found
that two requirements were not met, making it difficult for the
visually impaired to use. A CMS official told us that the agency made
the appropriate corrections on April 20, 2006. Because of time
constraints, we did not verify that these corrections were made.
CMS relies on SHIPs to play a significant role in providing counseling
and education on the Part D benefit to Medicare beneficiaries. CMS
increased SHIP funding from $12 million for the 2003 SHIP grant
year[Footnote 13] to $31.7 million for the 2005 grant year. CMS kept
funding relatively high for the 2006 grant year--$30 million--to ensure
that SHIPs continued to play an important role in educating
beneficiaries about Part D. The number of beneficiaries served by SHIPs
has also increased. During the 2004 SHIP grant year, SHIPs served
approximately 2.52 million people. During the first 9 months of the
2005 SHIP grant year--when CMS was gearing up its outreach and
education on Part D--SHIPs served approximately 3.3 million
individuals, an increase of nearly 770,000 from the prior full grant
year. CMS attributes the increase in demand for SHIP services--as
reflected in increases in the number of calls, face-to-face assistance,
and referrals from the 1-800-MEDICARE help line--to beneficiaries' need
for assistance on Part D. The average number of calls per month
referred from the help line to SHIPs, for example, increased from about
16,000 referrals for May through September 2005 to an average of about
43,000 for October and November 2005, about the time Part D enrollment
began. According to CMS officials, this increased demand can be
attributed to callers seeking advice on choosing a drug plan. Unlike
CSRs on the help line, SHIP counselors can offer individualized
guidance to callers on enrollment and plan selection. SHIP coordinators
in the five states we contacted confirmed that there was a substantial
increase in the demand for their services because of the new Part D
benefit. For example, the California SHIP served over 120,000 people in
January 2006, compared to about 35,000 served in all of 2005.
Conclusions:
Within the past 6 months, millions of Medicare beneficiaries have been
making important decisions about their prescription drug coverage and
have needed access to information about the new Part D benefit to make
appropriate choices. CMS faced a tremendous challenge in responding to
this need and, within short time frames, developed a range of outreach
and educational materials to inform beneficiaries and their advisers
about Part D. To disseminate these materials, CMS largely added
information to existing resources, including written documents, such as
Medicare & You; the 1-800-MEDICARE help line; the Medicare Web site;
and support for SHIPs. However, CMS has not ensured that its
communications to beneficiaries and their advisers are provided in a
manner that is consistently clear, complete, accurate, and usable. Six
months have passed since these materials were first made available to
beneficiaries, and their limitations could result in confusion among
those seeking to make coverage decisions. Although the initial
enrollment period for Part D will end on May 15, 2006, CMS will
continue to play a pivotal role in providing beneficiaries with
information about the drug benefit during the year and in subsequent
enrollment periods. CMS has an opportunity to enhance its
communications on the Part D benefit. This would allow beneficiaries
and their advisers to be better prepared when deciding whether to
enroll in the benefit, and if enrolling, which drug plan to choose.
Recommendations for Executive Action:
In order to improve the Part D benefit education and outreach materials
that CMS provides to Medicare beneficiaries, we are recommending that
the CMS Administrator take the following four actions:
* Ensure that CMS's written documents describe the Part D benefit in a
manner that is consistent with commonly recognized communications
guidelines and that is responsive to the intended audience's needs.
* Determine why CSRs frequently do not search for available drug plans
if the caller does not provide personal identifying information.
* Monitor the accuracy and completeness of CSRs' responses to callers'
inquiries and identify tools targeted to improve their performance in
responding to questions concerning the Part D benefit, such as
additional scripts and training.
* Improve the usability of the Part D portion of the Medicare Web site
by refining Web-based tools, providing workable site navigation
features and links, and making Web-based forms easier to use and
correct.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from CMS (see
app. III). CMS said that it did not believe our findings presented a
complete and accurate picture of its Part D communications activities.
CMS discussed several concerns regarding our findings on its written
documents and the 1-800-MEDICARE help line. However, CMS did not
disagree with our findings regarding the Medicare Web site or the role
of SHIPs. CMS also said that it supports the goals of our
recommendations and is already taking steps to implement them, such as
continually enhancing and refining its Web-based tools.
CMS discussed concerns regarding the completeness and accuracy of our
findings in terms of activities we did not examine, as well as those we
did. CMS stated that our findings were not complete because our report
did not examine all of the agency's efforts to educate Medicare
beneficiaries and specifically mentioned that we did not examine the
broad array of communication tools it has made available, including the
development of its network of grassroots partners throughout the
country. We recognize that CMS has taken advantage of many vehicles to
communicate with beneficiaries and their advisers. However, we focused
our work on the four specific mechanisms that we believed would have
the greatest impact on beneficiaries--written materials, the 1-800-
MEDICARE help line, the Medicare Web site, and the SHIPs. In addition,
CMS stated that our report is based on information from January and
February 2006, and that it has undertaken a number of activities since
then to address the problems we identified. Although we appreciate
CMS's efforts to improve its Part D communications to beneficiaries on
an ongoing basis, we believe it is unlikely that the problems we
identified in this report could have been corrected yet given their
nature and scope.
CMS raised two concerns with our examination of a sample of written
materials. First, it criticized our use of readability tests to assess
the clarity of the six sample documents we reviewed. For example, CMS
said that common multisyllabic words would inappropriately inflate the
reading level. However, we found that reading levels remained high
after adjusting for 26 multisyllabic words a Medicare beneficiary would
encounter, such as Social Security Administration. CMS also pointed out
that some experts find such assessments to be misleading. Because we
recognize that there is some controversy surrounding the use of reading
levels, we included two additional assessments to supplement this
readability analysis--the assessment of design and organization of the
sample documents based on 60 commonly recognized communications
guidelines and an examination of the usability of six sample documents,
involving 11 beneficiaries and 5 advisers.
Second, CMS expressed concern about our examination of the usability of
the six sample documents. The participating beneficiaries and advisers
were called on to perform 18 specified tasks, after reading the
selected materials, including a section of the Medicare & You handbook.
CMS suggested that the task asking beneficiaries and advisers to
calculate their out-of-pocket drug costs was inappropriate because
there are many other tools that can be used to more effectively compare
costs. We do not disagree with CMS that there are a number of ways
beneficiaries may complete this calculation; however, we nonetheless
believe that it is important that beneficiaries be able to complete
this task on the basis of reading Medicare & You, which, as CMS points
out, is widely disseminated to beneficiaries, reaching all beneficiary
households each year. In addition, CMS noted that it was not able to
examine our detailed methodology regarding the clarity of written
materials--including assessments performed by one of our contractors
concerning readability and document design and organization. We plan to
share this information with CMS, once our report has become public.
Finally, CMS took issue with one aspect of our evaluation of the 1-800-
MEDICARE help line. Specifically, CMS said the 41 percent accuracy rate
associated with one of the five questions we asked was misleading,
because, according to CMS, we failed to analyze 35 of the 100
responses. However, we disagree. This question addressed which drug
plan would cost the least for a beneficiary with certain specified
prescription drug needs. We analyzed these 35 responses to this
question and found the responses to be inappropriate. The CSRs would
not provide us with the information we were seeking because we did not
supply personal identifying information, such as the beneficiary's
Medicare number or date of birth. We considered such responses
inappropriate because the CSRs could have answered this question
without personal identifying information by using CMS's Web-based
prescription drug plan finder tool. Although CMS said that it has
emphasized to CSRs, through training and broadcast messages, that it is
permissible to provide the information we requested without requiring
information that would personally identify a beneficiary, in these 35
instances, the CSR simply told us that our question could not be
answered. CMS also said that the bulk of these inappropriate responses
were related to our request that the CSR use only brand-name drugs.
This is incorrect--none of these 35 responses were considered incorrect
or inappropriate because of a request that the CSR use only brand-name
drugs--as that was not part of our question.
As arranged with your offices, unless you publicly announce the
contents of this report earlier, we plan no further distribution of it
until 30 days after its date. At that time, we will send copies of this
report to the Secretary of Health and Human Services, the Administrator
of the Centers for Medicare & Medicaid Services, and other interested
parties. We will also make copies available to others on request. In
addition, the report will be available at no charge on the GAO Web site
at [Hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (312) 220-7600 or aronovitzl@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix IV.
Signed By:
Leslie G. Aronovitz:
Director:
Health Care:
List of Requesters:
The Honorable John D. Dingell:
Ranking Minority Member:
Committee on Energy and Commerce:
House of Representatives:
The Honorable Henry A. Waxman:
Ranking Minority Member:
Committee on Government Reform:
House of Representatives:
The Honorable Charles B. Rangel:
Ranking Minority Member Committee on Ways and Means:
House of Representatives:
The Honorable Sherrod Brown:
Ranking Minority Member:
Subcommittee on Health Committee on Energy and Commerce:
House of Representatives:
The Honorable Pete Stark:
Ranking Minority Member:
Subcommittee on Health Committee on Ways and Means:
House of Representatives:
[End of section]
Appendix I: Briefing on Medicare Part D:
Medicare Part D: CMS Communications to Beneficiaries Could Be Improved:
Briefing for Congressional Staff Updated:
Contents:
* Purpose and Objectives:
* Objective 1: Written Documents:
* Objective 2: 1-800-MEDICARE Help Line:
* Objective 3: Medicare Web Site:
* Objective 4: State Health Insurance Assistance Programs:
Purpose and Objectives:
* The Medicare Prescription Drug, Improvement, and Modernization Act of
2003 (MMA) established the new Part D outpatient prescription drug
benefit.
* The Centers for Medicare & Medicaid Services (CMS) is responsible for
overseeing this new benefit. CMS has taken steps to inform
beneficiaries and their advisers about Part D using written documents,
a toll-free help line, and an Internet Web site. CMS also gives State
Health Insurance Assistance Programs (SHIP) funds to provide
information about the Medicare program, including Part D.
* We assessed information that CMS provides to Medicare beneficiaries
to educate them about Part D. Specifically, we assessed:
1. The extent to which CMS's written documents describe the Part D
benefit in a clear, complete, and accurate manner.
2. The effectiveness of CMS's 1-800-MEDICARE help line in providing
accurate, complete, and prompt responses to callers inquiring about the
Part D benefit.
3. Whether the Medicare Web site presents information on the Part D
benefit in a usable manner.
4. How CMS has used SH I Ps to respond to the needs of Medicare
beneficiaries for information on the Part D benefit.
Objective 1: Written Documents Methodology:
* We performed in-depth review of a sample of six CMS documents
describing the Part D benefit. The sample was selected to represent a
variety of document types, content, and target audiences.
* We contracted with the American Institutes for Research (AIR) to
assess the clarity of sample documents.
Sample documents: Medicare & You (Section 6: Medicare Prescription Drug
Coverage);
Target audience: All beneficiaries.
Sample documents: Things to Think about When You Compare Plans;
Target audience: All beneficiaries.
Sample documents: Frequently Asked Questions about: Retiree
Prescription Drug Coverage & the New Medicare Prescription Drug
Coverage;
Target audience: Beneficiaries with employer or union coverage.
Sample documents: The Auto-Enrollment Notice;
Target audience: Dual-eligible beneficiaries[a].
Sample documents: Quick Facts about Medicare's New Coverage for
Prescription Drugs for People with a Medicare Health Plan with
Prescription Drug Coverage;
Target audience: Beneficiaries with Medicare Advantage [B].
Source: GAO.
Sample documents: Do You Have a Medigap Policy[c] with Prescription
Drug Coverage?;
Target audience: Beneficiaries with Medigap.
[a] Dual-eligible beneficiaries are Medicare beneficiaries who receive
full Medicaid benefits for services not covered by Medicare.
[b] Medicare Advantage replaces the Medicare+Choice managed care
program and expands the availability of private health plan options to
Medicare beneficiaries.
[c] Medigap policies provide supplemental health coverage sold by
private insurers to help pay for Medicare cost-sharing requirements, as
well as for some services not provided by Medicare.
[End of table]
* To determine the clarity of the sample of Part D written documents
describing the Part D benefit, AIR:
- evaluated text by sentence length and the number of syllables using
three standard readability tests Flesch-Kincaid, SMOG, and Fry:
- assessed the design and organization of the documents based on 60
commonly recognized written communications guidelines, including those
to aid senior readers; and:
* tested the usability of sample documents with 16 participants-1 1
Medicare beneficiaries, including 1 disabled beneficiary who was under
65, and 5 advisers to beneficiaries.
- Everyone was asked to perform 18 specified tasks related to
enrollment, coverage, costs, penalty, and informational resources. They
were also asked to provide feedback about their experiences.
- Although the size of the group was small, research shows that as few
as 5 individuals can provide meaningful insights into common problems.
* To evaluate completeness, we reviewed the sample documents to
determine if they included sufficient information to identify (1) next
steps in determining whether to enroll and what plan to choose and (2)
important factors, such as penalty provisions, that could affect
coverage decisions.
* To evaluate accuracy, we reviewed the sample documents for
consistency with laws, regulations, and CMS guidance.
Objective 1: Written Documents-Documents Lack Clarity:
* Readability assessment: Sample documents explaining the Part D
benefit are written at a reading level that is difficult for many
seniors.
- Reading levels for the sample documents were challenging for at least
the 40 percent of seniors, who read at or below the 5th grade level.
* Reading level estimates for the sample texts[Footnote 1] ranged from
7th grade to postcollege level.
* Reading levels remain challenging for at least 40 percent of seniors
even after adjusting for 26 multisyllabic words, such as Medicare,
Medicare Advantage, and Social Security Administration. After the
adjustment, the estimated reading level ranged from 8th to 12th grade.
* Document design and organization assessment: The sample documents
demonstrated adherence to about half of the 60 commonly recognized
written communications guidelines, on average.
- Desirable features: The documents;
* were written with a respectful and polite tone,
* were free of cliches and slang,
* contained useful contact information,
* included concise and descriptive headings, and:
* generally followed graphic and formatting guidelines.
- Undesirable features: The documents;
* used too much technical jargon,
* often did not define difficult terms,
* included sentences and some paragraphs that were too long, and:
* did not use sufficient summaries to assist the reader in identifying
key points.
* Usability assessment: Beneficiaries and advisers to beneficiaries
were frustrated by the documents' lack of clarity and often could not
complete the 18 assigned tasks.
- One of the 18 assigned tasks was completed by all beneficiaries and
advisers.
- Eleven of the 18 assigned tasks were completed by at least half of
the beneficiaries and advisers.
- Four of the 18 assigned tasks were completed by 2 or fewer of the 11
beneficiaries.
- Nine of the 18 assigned tasks, were completed by 2 or fewer of the 5
advisers.
* Some of the tasks that proved difficult included:
- computing projected total out-of-pocket costs for a plan that
provided Part D's standard coverage (successfully completed by none of
the 11 beneficiaries and 2 of the 5 advisers),
- evaluating whether it was possible to enroll in Medicare Part D and
keep drug coverage from a retiree health plan (successfully completed
by 2 beneficiaries and 2 advisers), and:
- determining the course of action for dual-eligibles who are
automatically enrolled in a plan that does not cover all drugs used
(successfully completed by 4 beneficiaries and 1 adviser).
- Participants described documents as too wordy, confusing, and hard to
follow.
- Participants struggled with technical terms, such as "classes of
commonly prescribed drugs" and "formulary," which is a list of drugs
covered by a plan.
- Even when most participants were able to complete the tasks, they
expressed confusion and frustration.
Objective 1: Written Documents-Documents are Generally Complete:
* Our analysis showed that the sample documents were generally complete
and informed readers of next steps in determining whether to enroll and
what plan to choose as well as important factors that could affect
their coverage decisions. For example:
- All documents reviewed provided sources of assistance and relevant
contact information, which could aid in identifying next steps for
coverage decisions.
- All documents reviewed provided the dates of the start of initial
program enrollment and coverage.
* However, our analysis also identified a few exceptions where the
documents did not mention some important issues. For example:
- Medicare & You noted that drug plan information may change, but made
no mention of possible changes on the pages beneficiaries would use to
compare coverage and select a plan. Such information is needed because
drug plans can change their covered drugs and prices.
- The documents did not provide sufficient information about the
cumulative effect of the penalty for missing the initial enrollment
deadline.
* Our analysis showed that the sample documents were generally accurate
and that the text was consistent with MMA, implementing regulations,
and agency guidance.
* However, we noted a few misleading statements in Medicare & You. For
example:
- The document implied that if a beneficiary's doctor applied for an
exception it would be granted, whereas exceptions to the formulary are
granted at each plan sponsor's discretion.
- The document outlined the minimum requirements for standard coverage
by Part D plans. However, it did not indicate that few plans offer this
exact coverage and that beneficiaries should be prepared to compare
plans with varying premiums, co-payments, and covered drugs to choose
plans that best suit them.
Objective 2: 1-800-MEDICARE Help Line Methodology:
* We placed 500 calls to 1-800-MEDICARE, posing one of five questions
in each call, so that each question was asked 100 times. To develop the
questions, we considered topics listed on the Medicare Web site and
obtained help line reports that listed the scripts that customer
service representatives (CSR) frequently accessed to respond to
callers' questions.
* Calls were randomly placed at different times of the day and on
different days of the week from January 17 to February 7, 2006, to
match the daily and hourly pattern of calls reported by 1-800-MEDICARE
in October 2005.
* To evaluate the accuracy and completeness of CSRs' responses to our
five questions, we used three resources:
- the prescription drug finder tool on the Medicare Web site,
- the 1-800-MEDICARE scripts prepared by CMS and contractors for CSRs
to use in responding to callers' questions, and:
- input from CMS officials on the criteria we used to evaluate
responses.
* To evaluate the promptness of the help line in answering calls, we
recorded the length of time it took to connect to a CSR for each call.
* CSRs' responses were scored in one of five categories based on
specific criteria we developed:
- Accurate and Complete - responses met our defined criteria;
- Inappropriate - responses reflected the need for personal beneficiary
information, which was not actually required to answer the question;
- Inaccurate - responses did not meet our defined criteria;
- Incomplete - responses partially met our defined criteria;
- Unanswered - calls did not receive responses from CSRs.
Table:
Question: 1. What drug plan can a beneficiary get that will cover all
of his/her [specified] drugs at a [specified] pharmacy; have a mail-
order option; and cost the least amount annually with [or without] a
deductible?;
Criteria: An accurate and complete response would identify the
prescription drug plan that has the lowest estimated annual cost for
the drugs the beneficiary uses.
Question: 2. Can a beneficiary who is in a nursing home and not on
Medicaid sign up for a prescription drug plan?;
Criteria: An accurate and complete response would indicate that such a
beneficiary can choose whether to enroll in a Medicare prescription
drug plan.
Question: 3. Can a beneficiary enroll in the Medicare prescription drug
program and keep his/her current Medigap policy?;
Criteria: An accurate and complete response would inform the caller
that enrolling for the prescription drug benefit would depend on
whether the beneficiary's Medigap plan was creditable- that is, whether
the coverage it provided was at least as good as Medicare's standard
prescription drug coverage-or noncreditable. The CSR response would
also mention that the beneficiary's Medigap plan should have sent
him/her information that outlines options.
Question: 4. What options does a beneficiary, who has retiree health
insurance with prescription drug coverage that is not as good as the
Medicare prescription drug coverage, have as it relates to the Medicare
benefit?;
Criteria: An accurate and complete response would indicate that a
beneficiary has two options: (1) keep current health plan and join the
prescription drug plan later with a penalty; or (2) drop current
coverage and join a Medicare drug plan.
Question: 5. How do I know if a beneficiary qualifies for extra help?;
Criteria: An accurate and complete response would refer the beneficiary
to the Social Security Administration.
Source: GAO.
[End of table]
Objective 2: 1-800-MEDICARE Responses Often Accurate and Complete, but
Some Not:
* We found that the 1-800-MEDICARE help line provided accurate and
complete answers to 334 of our 500 calls, a rate of about 67 percent.
In addition, it provided accurate but incomplete answers for about 3
percent of our calls.
Figure: Distribution of Unanswered Calls and Accurate and Complete,
Inaccurate, Incomplete, and Inappropriate Responses[a]:
[See PDF for image]
Source: GAO.
[a] Percentages exceed 100 because of rounding.
[End of figure]
Objective 2: 1-800-MEDICARE-Variation in Results for Individual
Questions:
* The accuracy and completeness of responses to our five questions
varied significantly, from 41 percent to 90 percent.
- Q1 - 41 percent
- Q2 - 79 percent
- Q3 - 66 percent
- Q4 - 58 percent
- Q5 - 90 percent
- Average for all questions-67 percent:
Figure: Distribution of Unanswered Calls and Accurate and Complete,
Inaccurate, Incomplete, and Inappropriate Responses by Question:
[See PDF for image]
Source. GAO.
[End of figure]
* CSRs answered some questions better than others. For example:
- CSRs accurately and completely answered question 5 (whether a
beneficiary qualifies for extra help), which had a specific script, 90
percent of the time.
- CSRs accurately and completely answered question 2 (whether a
beneficiary in a nursing home, who was not on Medicaid, could sign up
for the drug benefit) 79 percent of the time even though there was no
specific script for the question.
- CSRs' responses for question 3 (whether a beneficiary with a Medigap
policy could enroll in the drug benefit) were accurate and complete 66
percent of the time. Many of the responses were inaccurate because they
did not provide adequate information about creditable and noncreditable
coverage.
- The accuracy and completeness rate for question 4 (about retiree
health insurance) was 58 percent. Many of the responses were inaccurate
because the CSRs did not follow the available script or provide
sufficient information about the implications of the beneficiary's
decision.
- CSRs' responses to question 1 (which requires CSRs to use the
prescription drug plan finder Web tool) were accurate and complete less
than 50 percent of the time. The rate is largely caused by CSRs'
inappropriate responses35 out of 100 times that they were unable to
answer the question without personal identifying information, such as
the beneficiary's Medicare number or date of birth.
* We did not obtain answers for 23 of the calls we placed because of
unintentional disconnections, intentional disconnections, or an
inoperative Web tool.
- Unintentional disconnections occurred when the system inadvertently
disconnected the call (19 calls).
- Intentional disconnections were programmed by the telephone company
when wait times were projected to exceed 20 minutes (3 calls).
- The prescription drug plan finder Web tool used by CSRs was not
operative at the time of our call (1 call).
Objective 2: 1-800-MEDICARE-Variation in Wait Times:
* The amount of time we had to wait to speak with a CSR varied
significantly, ranging from no wait to more than 55 minutes.
- About 75 percent of calls were connected in less than 5 minutes.
- For calls where we waited more than 5 minutes to speak to a CSR, the
wait time ranged from 5 minutes to over 55 minutes.
* Sixty-two calls were on hold from 5 to 14 minutes, 59 seconds;
* Thirty-nine calls were on hold from 15 to 24 minutes, 59 seconds;
* Twenty-five calls were on hold 25 minutes or more.
- For both intentional and unintentional disconnections, we often
waited more than 5 minutes before the disconnection occurred. In one
case, we were placed on hold for 54 minutes before being disconnected.
Objective 3: Medicare Web Site Methodology:
* To evaluate the usability of the Part D benefit portion of the
Medicare Web site, we contracted with Nielsen Norman Group (NN/g), a
firm with expertise in Web design.
* In addition, we reviewed CMS's efforts to comply with section 508 of
the Rehabilitation Act of 1973, as amended (29 U.S.C. § 794d).
- Section 508 requires that all federal Web sites be designed to make
information and services fully available to individuals with
disabilities.
- Our review included an examination of CMS's March 2006 report
assessing the compliance of its Medicare Web site with this federal
requirement and discussions with CMS officials.
* NN/g performed the following three separate evaluations:
- Evaluation one: NN/g calculated an overall score of the site's
usability, to reflect the ease of finding necessary information and
performing various tasks. For this calculation, NN/g considered various
factors, such as site navigation, customer support, and presentation of
online forms.
- Evaluation two: NN/g evaluated in detail the usability of 137
detailed aspects of the Part D benefit portion of the Web site. Topics
included:
* Web design (e.g., home page, navigation, search function, graphics,
and organization);
* tools (e.g., plan finder);
* writing style (e.g., tone, content, legibility, and readability);
* accessibility (e.g., availability of site version for the blind);
and;
* languages (e.g., links for users who have difficulty reading English).
- Evaluation three: NN/g conducted a total of 34 user tests to
determine the ease of performing a variety of Web-related tasks, such
as browsing the site, making a change in address, finding plan
information under certain scenarios, comparing Medigap and Part D drug
coverage, and determining how to join a plan.
* NN/g asked five Medicare beneficiaries who were not disabled and two
advisers to beneficiaries to perform one or more user tests each using
the Web site.
* At the end of the user tests, the seven participants were asked to
provide feedback about their experiences.
Objective 3: Medicare Web Site Difficult to Use:
* Based on NN/g evaluations, we concluded that the Part D benefit
portion of the Medicare Web site can be challenging to use.
- For evaluation one, the calculated usability scores indicate a need
for improvement. The usability score was 47 percent for seniors and 53
percent for younger adults. While there is no widely accepted benchmark
for usability, these scores indicate that using the site can be
difficult. For example, tools such as the drug plan finder were
complicated to use, and forms that collect information online from
users were difficult to correct if the user made an error.
- Evaluation two showed that the Part D benefit portion of the Web site
was difficult to use. About 70 percent of the 137 detailed aspects of
the site were presented in a manner that could be expected to cause a
medium or high level of confusion. For example,
* important functions in the plan finder tool the "continue" and
"choose a drug plan" buttons are often not visible on the page;
* plan finder tool defaults to generic drugs, complicating users'
search for drug plans covering brand-name drugs;
* information to assist navigation was often not helpful for example,
text labels associated with links were not always functioning; and:
* the writing style presented some challenges for example, material was
written at the 11th grade level.
- For evaluation three, the 34 user tests showed that the site was a
challenge for the seven participants to use. For example:
* For 12 of the 34 tests, participants' initial reactions were that
they would not be able to complete the tests and wanted to quit trying.
* On average, participants were able to proceed slightly more than
halfway through each of the 34 tests.
* When asked for feedback on their experience with using the site, the
seven participants, on average, indicated high frustration levels and
low satisfaction.
* To comply with section 508 of the Rehabilitation Act, CMS has
established features to make information on its Medicare Web site
accessible to disabled users. For example, CMS provides a "screen
reader" version of the site for the visually impaired. This technology
translates text and data into spoken words.
* CMS's March 2006 review of its site's compliance with section 508
showed that two requirements were not met:
- The plan finder did not provide alternative text for all images that
is, there was no text for the screen reader to read. Therefore, images
could not be translated into spoken words for the visually impaired.
- The plan finder did not allow screen readers to recognize form fields
and translate forms into spoken words. As a result, visually impaired
users would not have been able to complete Web-based forms.
* A CMS official told us that the agency made the necessary corrections
on April 20, 2006, but we did not verify that these corrections were
made.
Objective 4: SHIP Methodology:
* We interviewed CMS officials and reviewed documentation they provided
about SHIPs' role in educating beneficiaries about the Part D benefit.
* We contacted the SHIP coordinators in California, Florida, New York,
Texas, and Pennsylvania the five states with the most Medicare
beneficiaries. Together, these states accounted for about 35 percent of
the country's total Medicare population in 2004.
Objective 4: SHIPs' Responses to Beneficiaries' Needs Concerning Part D:
* According to CMS, it relies on SHIPs to play a significant role in
beneficiary counseling and education on the Part D benefit.
* In anticipation of the increased demand for SHIP services regarding
the Part D benefit, CMS increased SHIP funding in recent years. Funding
for the 2003 SHIP grant year[Footnote 2] was $12 million, and it
reached $31.7 million for the 2005 grant year. CMS kept funding
relatively high for the 2006 grant year $30 million to ensure that
SHIPs continue to play an important role in educating beneficiaries
about the Part D benefit.
* During the 2004 SHIP grant year, SHIPs served approximately 2.52
million people. According to preliminary data for the first 9 months of
the 2005 SHIP grant year when CMS was gearing up its outreach and
education on Part D SHIPs served approximately 3.3 million individuals,
an increase of nearly 770,000 from the prior full grant year. CMS
attributes this increase in demand for services to beneficiaries' need
for assistance on the Part D benefit.
* The average number of calls referred from the 1-800-MEDICARE help
line to SHIPs has increased significantly.
- The monthly average of number of calls referred to SHIPs increased
from 16,000 referrals for May through September 2005 to approximately
43,000 for October and November 2005, the months around the time when
enrollment in the Part D benefit began.
- According to CMS officials, this increased demand was influenced by
callers seeking advice on choosing a drug plan. Unlike CSRs on the help
line, SHIP counselors can offer individualized guidance to callers.
* Specifically, the five SHIPs we contacted experienced a large
increase in demand for their services because of the Part D benefit.
- California served over 120,000 people in January 2006, compared to
about 35,000 served in all of 2005.
- Florida, mostly during November and December of 2005, held at least
six "phone bank" events where SHIP counselors were available to take
calls on the Part D benefit during live newscasts. Florida plans to
hold two additional phone banks as the May 15 enrollment deadline
approaches.
- New York reported nearly doubling its formal training sessions for
SHIP counselors in 2005, to prepare them for the demand for services
related to the Part D benefit.
- Texas counseled 45,719 clients and conducted 523 outreach events from
November 15, 2005the official start of the enrollment period to March
22, 2006.
- Pennsylvania held over 3,000 enrollment events, which were attended
by more than 130,000 people, from May 2005 to February 28, 2006.
* The SHIP officials in four of the five states we contacted indicated
that the demand for their services related to the Part D benefit has
declined since the benefit began in January 2006. However, each SHIP
contacted expects a surge in demand as the May 15 enrollment deadline
approaches.
* Since December 2005, CMS has been conducting biweekly meetings with
its regional offices, which interact directly with SHIP offices, to
gauge SHIPs' ability to meet the demands of beneficiaries.
Agency Comments:
We discussed our findings with CMS officials on March 13, 2006.
9 CMS officials did not comment on our findings.
Footnotes:
[1] Estimates have a likely margin of error of [plus or minus] two
grades.
[2] A SHIP grant year begins on April 1 of the year the funds become
available.
[End of section]
Appendix II: Objectives, Scope, and Methodology:
In this report, we assessed (1) the extent to which the Centers for
Medicare & Medicaid Services' (CMS) written documents describe the
Medicare Part D prescription drug benefit in a clear, complete, and
accurate manner; (2) the effectiveness of CMS's 1-800-MEDICARE help
line in providing accurate, complete, and prompt responses to callers
inquiring about the Part D benefit; (3) whether CMS's Medicare Web site
presents information on the Part D benefit in a usable manner; and (4)
how CMS has used State Health Insurance Assistance Programs (SHIP) to
respond to the needs of Medicare beneficiaries for information on the
Part D benefit. To obtain information on CMS's efforts to educate
beneficiaries about Part D, we interviewed agency officials responsible
for Part D written documents, the 1-800-MEDICARE help line, the
Medicare Web site, and SHIPs. Following our briefing of congressional
staff on April 19, 2006, the briefing slides were updated to reflect
CMS's reported correction to the Medicare Web site to comply with
section 508 of the Rehabilitation Act of 1973.[Footnote 14] We
determined that the data used were sufficiently reliable for the
purposes of this report.
Written Documents:
To assess the clarity, completeness, and accuracy of written documents,
we compiled a list of all available CMS-issued Part D benefit
publications intended to inform beneficiaries and their advisers and
selected a sample of 6 from the 70 CMS documents available, as of
December 7, 2005, for in-depth review, as shown in table 1. The sample
Part D documents were chosen to represent a variety of publication
types, such as frequently asked questions and fact sheets available to
beneficiaries about the Part D drug benefit. We selected documents that
targeted all beneficiaries or those with unique drug coverage concerns,
such as dual-eligibles and beneficiaries with Medigap.[Footnote 15]
Table 1: Sample of Six Selected Documents:
Document: Medicare & You, Section 6: Medicare Prescription Drug
Coverage;
Target audience: All beneficiaries.
Document: Things to Think about When You Compare Plans;
Target audience: All beneficiaries.
Document: Frequently Asked Questions about: Retiree Prescription Drug
Coverage & the New Medicare Prescription Drug Coverage;
Target audience: Beneficiaries with employer or union coverage.
Document: Introduction to the Auto-Enrollment Notice;
Target audience: Dual-eligible beneficiaries[A].
Document: Quick Facts about Medicare's New Coverage for Prescription
Drugs for People with a Medicare Health Plan with Prescription Drug
Coverage;
Target audience: Beneficiaries with Medicare Advantage[B].
Document: Do You Have a Medigap Policy with Prescription Drug
Coverage?;
Target audience: Beneficiaries with Medigap.
Source: GAO.
[A] Dual-eligible beneficiaries are Medicare beneficiaries who receive
full Medicaid benefits for services not covered by Medicare.
[B] Medicare Advantage replaced the Medicare+ Choice managed care
program and expanded the availability of private health plan options to
Medicare beneficiaries.
[End of table]
To evaluate clarity, we contracted with the American Institutes for
Research (AIR)--a firm with experience in evaluating written material.
AIR evaluated the texts of the six sample documents using three
methodologies:
1. three standard readability tests;[Footnote 16]
2. 60 commonly recognized written communications guidelines, including
practices to aid senior readers; and:
3. user testing with 11 Medicare beneficiaries and 5 advisers to
beneficiaries, who performed 18 specified tasks related to enrollment,
coverage, cost, penalty, and information resources and provided
feedback about their experiences.
We reviewed the sample documents for completeness to determine whether
they contained sufficient information to allow the beneficiaries to
identify (1) their next steps in determining whether to enroll and what
plan to choose and (2) important factors, such as penalty provisions,
that could affect their coverage decisions. To identify those important
factors associated with the Part D benefit, we reviewed relevant laws,
regulations, and 1-800-MEDICARE scripts prepared for customer service
representatives (CSR) to read to callers and obtained information from
advocacy groups. To evaluate the accuracy of information, we reviewed
the sample materials for compliance with laws, regulations, and CMS
guidance.
The 1-800-MEDICARE Help Line:
To determine the accuracy and completeness of information provided
regarding the Part D benefit, we placed a total of 500 calls to the 1-
800-MEDICARE help line. We posed one of five questions about Part D in
each call, so that each question was asked 100 times. Each question was
pretested before we finalized its wording. We randomly placed calls at
different times of the day and different days of the week from January
17 to February 7, 2006. Our calling times were chosen to match the
daily and hourly pattern of calls reported by 1-800-MEDICARE in October
2005. We informed CMS officials that we would be placing calls;
however, we did not tell them the questions we would ask or the
specific dates and times that we would be placing our calls.
To select the five questions, we considered topics identified in the
Medicare Web site's frequently asked questions. In addition, we
considered topics most frequently addressed by 1-800-MEDICARE CSRs
based on help line reports. To evaluate the accuracy of CSRs' responses
to our five questions, we used three resources: (1) the prescription
drug plan finder tool on the Medicare Web site, (2) 1-800-MEDICARE
scripts, and (3) input obtained from CMS officials on the criteria we
used for evaluating CSR responses. Table 2 lists the questions we asked
and the criteria we used to evaluate the accuracy of responses.
Table 2: Questions and Criteria Used to Evaluate Accuracy:
Question: 1. What drug plan can a beneficiary get that will cover all
of his/her [specified] drugs at a [specified] pharmacy; have a mail-
order option; and cost the least amount annually with [or without] a
deductible?;
Criteria: An accurate and complete response would identify the
prescription drug plan that has the lowest estimated annual cost for
the drugs the beneficiary uses.
Question: 2. Can a beneficiary who is in a nursing home and not on
Medicaid sign up for a prescription drug plan?;
Criteria: An accurate and complete response would indicate that a
beneficiary can choose whether to enroll in a Medicare prescription
drug plan.
Question: 3. Can a beneficiary enroll in the Medicare prescription drug
program and keep his/her current Medigap policy?;
Criteria: An accurate and complete response would inform the caller
that enrolling for the prescription drug benefit would depend on
whether the beneficiary's Medigap plan was creditable--that is, whether
the coverage it provided was at least as good as Medicare's standard
prescription drug coverage-or noncreditable. The CSR response would
also mention that the beneficiary's Medigap plan should have sent
him/her information that outlined options.
Question: 4. What options does a beneficiary, who has retiree health
insurance with prescription drug coverage that is not as good as the
Medicare prescription drug coverage, have as it relates to the Medicare
benefit?;
Criteria: An accurate and complete response would indicate that a
beneficiary has two options: (1) keep current health plan and join the
prescription drug plan later with a penalty or (2) drop current
coverage and join a Medicare drug plan.
Question: 5. How do I know if a beneficiary qualifies for extra help?;
Criteria: An accurate and complete response would refer the beneficiary
to the Social Security Administration.
Source: GAO.
[End of table]
When placing our calls, we identified ourselves as a beneficiary's
relative, but did not provide CSRs with specific identifying
information, such as a Medicare beneficiary number or date of birth.
During our calls, CSRs were not aware that their responses would be
included in a research study. We recorded the length of each call,
including wait times, and the time it took before being connected to a
CSR. We evaluated the accuracy and completeness of the responses by
CSRs to the 500 calls by determining whether key information was
provided.
The results from our 500 calls are limited to those calls and are not
generalizable to the universe of calls made to the help line. The
questions we asked were limited to matters concerning the Part D
benefit and do not encompass all of the questions callers might ask.
Medicare Web Site:
We contracted with the Nielsen Norman Group (NN/g)--a firm with
expertise in Web design--to assess the usability of the Part D
information available on the Medicare Web site. This study consisted of
three separate evaluations. First, NN/g compared the site's compliance
with established usability guidelines to determine a usability score to
reflect the ease of finding necessary information and performing
various tasks. Specifically, to determine the usability scores, NN/g
evaluated various aspects of the Web site using industry-recognized
"good" Web design practices, as indicated by the contractor, and the
collective body of knowledge from NN/g internal reports and experts, or
NN/g usability guidelines.[Footnote 17]
Second, NN/g determined the degree of difficulty associated with 137
detailed aspects of Web site design for the Part D portion of the site.
The 137 aspects fall into the following general categories:
* overall Web design (e.g., home page, navigation, search function,
graphics, and overall organization);
* tools (e.g., plan finder);
* writing style (e.g., content, tone, legibility, and readability);
* accessibility (e.g., availability of a version of the Web site for
the blind); and:
* languages (e.g., availability of languages other than English).
NN/g determined the difficulty level in using each of the 137 aspects.
NN/g noted aspects that had good design and would not be expected to
cause confusion. For those aspects with a design that would be expected
to cause confusion, NN/g ranked the associated difficulty level as
high, medium, or low.[Footnote 18]
Third, NN/g performed a qualitative evaluation on January 20 and 23,
2006, to test the ability of five Medicare beneficiaries and two
beneficiary advisers to perform specified tasks related to Medicare
beneficiaries using the Web site and to obtain feedback about
participants' experiences. While the results are not statistically
valid, these users provided important insights into the usability of
the Medicare Web site. Participants were asked to "think out loud" as
they worked through their tasks, while an NN/g facilitator observed
their behavior and took notes. NN/g gave each task a score. At the end
of their sessions, NN/g asked participants for input regarding their
confidence in the answers they obtained from the Web site, and their
overall satisfaction and frustration levels associated with using the
site.
Finally, we obtained the results of CMS's March 2006 review of its Web
site's compliance with section 508 of the Rehabilitation Act of 1973,
as amended. This law requires federal agencies to make the information
on their Web sites accessible to people with disabilities. We also
discussed the results of this review with agency officials and followed
up with them to determine the status of CMS's corrective actions.
State Health Insurance Assistance Programs:
To determine the role of SHIPs in helping Medicare beneficiaries
understand Part D, we interviewed CMS officials who monitor SHIPs'
activities. We also reviewed information that we obtained from CMS
officials and other sources on the program, its funding, changes made
in response to the introduction of Part D, and the impact of Part D on
the demand for SHIP services. In addition, we interviewed SHIP
officials in California, Florida, New York, Texas, and Pennsylvania--
the five states with the largest Medicare populations--to obtain
information on the experience of their SHIPs with Part D.
We conducted our work from November 2005 through May 2006 in accordance
with generally accepted government auditing standards.
[End of section]
Appendix III: Comments from the Centers for Medicare & Medicaid
Services:
Department Of Health & Human Services:
Centers for Medicare & Medicaid Services:
200 Independence Avenue SW:
Washington, DC 20201:
TO: Leslie G. Aronovitz:
Director:
Health Care:
FROM: Mark B. McClellan, M.D., PhD:
Administrator:
SUBJECT: Government Accountability Office's (GAO) Draft report,
"Medicare: Communications to the Beneficiaries on the Prescription Drug
Benefit Could Be Improved" (GAO-06-654):
The Centers for Medicare & Medicaid Services has reviewed the findings
in the GAO report entitled MEDICARE: Communications to Beneficiaries on
the Prescription Drug Benefit Could Be Improved (GAO-06-654) regarding
CMS communications on the Part D benefit. Having clear and effective
communication about Medicare's new prescription drug coverage is one of
the Agency's critical priorities. We have worked very hard to ensure
that Medicare beneficiaries have the information they need to make
decisions about enrolling in a drug plan that works for them. We are
pleased that the millions of beneficiaries who have enrolled in Part D
are experiencing very high rates of satisfaction with their coverage.
Each week, tens of thousands of beneficiaries are enrolling in Part D,
which gives them real savings and protections for the future.
While we greatly appreciate the feedback from your report and have
already worked to implement your recommendations, we do not believe
that your findings present a complete or accurate picture of the Part D
communication activities. We understand that the report is based on
studies of particular aspects of some of our communications tools at
one point in time three months ago, in January and early February 2006.
In addition to the many "continuous improvement" activities we have
undertaken to address startup issues in the drug benefit since that
time, there are much more extensive internal and external evaluations
of our communications activities completed before, during, and after
that time which have different conclusions, as we note below. These
evaluations have used well-established methods which have been clearly
documented and reviewed; in contrast, you have not yet responded to our
requests for information on the methods you have applied. Additionally,
your report does not address the unique breadth and depth of CMS
activities to educate and to reach out to people with Medicare and the
community that supports them in their health care decisions. From the
outset, it was clear that no single source of information would be
adequate or preferred by all of our beneficiaries. Consequently, we
have expanded the range of tools available and vastly expanded our
local partnerships to help beneficiaries use them, partnering with more
than ten thousand diverse public and private organizations around the
country in this effort.
Importantly, the report does not look at this broad array of
communication tools to help Medicare beneficiaries consider their drug
plan options. For example, the report dismisses all of the tools used
by our customer service representatives and our website for
beneficiaries that provide personalized identifying information to
enable us to provide them with personally customized service. The vast
majority of our callers provide such personal identification, yet these
tools were not evaluated. In fact, the report misleadingly states that
we provided the right information on a lower share of cases because
some customer service representatives sought to get this personal
information to serve the beneficiary more quickly and effectively.
Where GAO did actually get information on drug costs, as thousands of
callers get every day, customer service representatives provided
accurate information at a much higher rate. As another example, GAO
evaluated whether beneficiaries could calculate their out-of-pocket
drug costs in the standard Medicare benefit by hand, using only the
Medicare and You handbook, but very few beneficiaries have opted to use
the handbook in this way because: (1) there are far better tools
available for quickly and automatically calculating drug costs on the
web, on the phone, and through our partner organizations, and (2) over
90 percent of our beneficiaries are choosing plans with benefits other
than the standard plan, because they prefer features like zero
deductibles, flat copays, and coverage in the "donut hole."
Beneficiaries are overwhelmingly using other tools to make effective
cost comparisons.
In fact, the drug plan finder element of the website has received 164.6
million page views between November 15, 2005 and April 26, 2006. The
Frequently Asked Questions (FAQ) section of www.medicare.gov has been
accessed more than one million times since January l, 2006. CMS has
also responded to more than 19,000 emails received through the FAQ
section, with 93% of them being resolved satisfactorily in the first
response.
Finally, there is no attention in the report at all to major aspects of
our communications activities and expenditures, such as the expansion
of our community based education and outreach efforts through an
extensive network of grassroots partners across the country. This
significant emphasis on reaching people where they live, work, play and
pray is a key component of our success in reaching millions of people
with Medicare and those who work on their behalf. No mention is made of
the specialized campaigns targeting African American, Hispanic,
American Indians, Asian American and Pacific Islander and in low income
communities. These campaigns utilize new partnerships, employ materials
in other languages and specialized paid media campaigns. These targeted
campaigns within the broader campaign allow us to reach all segments of
the Medicare population, including those who might benefit from the low
income subsidy and those with language and other cultural barriers to
accessing information.
We believe that there have been a number of key elements to our
successful education campaign. First, we recognized early on that we
would need to supplement our proven traditional communications tools,
including the Medicare & You Handbook, the 1-800-MEDICARE line, and the
State Health Insurance Assistance Programs (SHIPs) with additional
advanced technology and grassroots resources, as well as use earned and
paid media opportunities. Second, we determined that the provision of
personalized assistance and one-on-one counseling was the key
ingredient to success. This necessitated our building a grassroots
network of traditional and non-traditional partners who were willing to
be trained to provide the one-on-one counseling. We strongly believe
this is important for beneficiaries to make confident decisions about
their Part D plan. We knew we would have to develop a grassroots
capacity and local networks to supplement the CMS regional structure to
provide the necessary education and enrollment assistance at the
community level. This would involve reaching out, not just to our
traditional partners such as the SHIPs, but to all the groups and
organizations that have contact with our beneficiaries on a daily basis
"where they work, where they play, and where they pray."
We appreciate any and all ideas for improving our communications
efforts, and we take very seriously the four tasks that GAO recommends
to improve CMS' education efforts. We support the goal of these tasks
and have already taken many steps to meet them.
Ensure that CMS's written documents describe the Part D benefit in a
manner that is consistent with commonly recognized communications
guidelines and that is responsive to the intended audience needs. - CMS
employs a wide variety of consumer research techniques, simple language
best practices, and independent evaluations in both English and in
Spanish documents to ensure the readability and usefulness of our
educational materials including those describing Part D. These tests
have demonstrated that CMS written documents follow best practice
guidelines for written communications with the intended audiences.
These techniques and practices are summarized in Attachment A.Because
of the importance of this topic, we are always interested in improving
our written products. We look forward to an opportunity to review what
GAO used in its review and will compare them to the evaluation methods
we are already using, as soon as GAO is willing to provide the
methodological details.
Determine why CSRs frequently do not search for available drug plans if
the caller does not provide personal identifying information. -As
discussed with the GAO reviewers, CMS has instructed CSRs, in cases
where that information is unavailable, to perform a search that
provides general information on the plan options available to the
beneficiary. Our web tools have always been set up to support such
"unauthenticated" searches as well.
1-800 MEDICARE CSRs do have the ability to conduct a general search for
callers who do not have their Medicare number. If the person provides
personal information, the authenticated search, other information that
may influence their decision is pulled into the search, e.g., low
income subsidy status or coverage through a retiree drug subsidy.
Because this path provides more robust and specific results, CMS has
encouraged CSRs to stress the importance of an authenticated
Prescription Drug Plan Finder search to callers. The importance of
authenticated searches is stressed in the CSR training materials and
scripts. We have placed warnings throughout the training materials
about the downside of proceeding without the personalized information
and CSRs do suggest that the person call back when they have it.
Even so, we know that there are occasions in which someone may not want
to provide this information, or another caller may be inquiring on
behalf of a beneficiary and not have the information, or a reporter or
analyst may be calling for information. It has been emphasized to CSRs
that non-authenticated general information is to be shared if the
caller is unable to provide specific information that would enable a
more detailed search. An example of relevant CSR instructions follows.
"If a caller indicates they are calling for someone else and just wants
general information on plans available in their area, you do not need
to personalize the search if the caller does not want to. You can
provide general plan information and send a personalized booklet if
requested." CMS has a comprehensive quality review process on calls and
we will continue to monitor calls to ensure that CSRs are pursuing the
general search when appropriate.
At the same time, we believe that GAO presents this finding in a way
that is incorrect and misleading. We believe that the 41 % accuracy
rate unfairly portrayed how accurately CMS answers questions on drug
plan options without beneficiary personal identification information,
when the GAO failed to analyze 35 out of the 41 responses. In
actuality, when the responses are analyzed, correct answers are
actually being provided a majority of the time. Further, the bulk of
the responses characterized as "inaccurate" were related to the test
caller's request that the CSR use only brand name drugs (i.e., no
generic drug substitution). This request is highly unusual in our call
experience as generic versions of a drug are identical in their
clinical effects. However, we have subsequently modified the web tool
used by our CSRs to make it easier to override the generic drug
substitution logic in the tool.
Monitor the accuracy and completeness of CSRs responses to callers'
inquiries and identify tools targeted to improve their performance in
responding to questions concerning the Part D benefit, such as
additional scripts and training.-We have worked hard to ensure
beneficiaries have access to accurate and clear information when they
call 1-800-Medicare. Our ongoing monitoring program, which evaluates a
random sample of hundreds of actual calls received each month, has
found that calls to 1-800-MEDICARE in 2006 have been answered
accurately 93 percent of the time. The high accuracy rate is reflected
in high rates of overall satisfaction from 1-800-MEDICARE callers,
which averaged 84 to 85 percent in February and March.
Improve the usability of the Part D portion of the Medicare website by
refining web-based tools, providing workable navigation features and
links, and making web-based forms easier to use and correct. - CMS is
continually enhancing and refining their web-based tools to provide
Medicare beneficiaries and their caregivers the information needed to
compare, choose and enroll in a prescription drug plan that best meet
their needs. We summarize some of our recent enhancements below. Online
enrollment has been highly successful, as evidenced by the 3 million
beneficiaries who have enrolled in the prescription drug plans using
CMS' web-based drug plan finder. Our partner organizations have used
the web tools to assist millions more with their enrollment-related
needs. The high level of online enrollment and use by partners
indicates that many people have found that this resource is useful and
effective for undertaking the most important step of enrolling in a
drug plan, and we are pleased that thousands more are using it every
day.
We cannot emphasize enough CMS' commitment to continuously improve the
communications with beneficiaries and other constituents. We want our
websites to continue to be recognized as benchmarks for excellence.
Attachment A outlines improvements that we have made to the website
since the GAO review and we believe demonstrate our continued
commitment to excellence.
All of our communications methods, in conjunction with our far-reaching
grassroots efforts, have helped provide the important information about
Part D needed by beneficiaries, providers and partners to ensure the
Medicare drug program is a success. In fact, the vast majority of
beneficiaries are using their coverage to save money and get protection
for the future: actual premiums and drug costs are much lower than had
been expected because of strong competition, and because beneficiaries
are using the enrollment tools to choose plans that save them more
(over 73 percent of beneficiaries are enrolling in plans stand-alone
prescription drug plans with premiums below the average); the drug
plans are successfully filling over three million prescriptions a day;
and each week hundreds of thousands of beneficiaries are enrolling in
the new program.
Tab A attached provides additional details about our communications
materials and approaches. Also attached are technical comments for your
consideration in Tab B. We will use the findings of the GAO report
going forward as we continue our commitment to ensure that Medicare
beneficiaries have the information they need to make informed health
care decisions.
Tab A:
Detailed Information About Part D Education And Outreach:
Over the past two years, we have dedicated significant resources to the
development and implementation of an extensive education and outreach
campaign surrounding Medicare prescription drug coverage, including a
variety of beneficiary publications and materials, the 1-800-MEDICARE
helpline, the Medicare Prescription Drug Plan Finder web tool on
www.medicare.gov, personalized assistance via the State Health
Insurance Assistance Program (SHIP) counseling program, and local
enrollment events. All of these initiatives are rooted in a foundation
of continuous quality improvement that involves identifying the
information that needs to be conveyed, using consumer research to
determine the most effective messages and vehicles, preparing materials
accordingly, and measuring material effectiveness. This thorough,
comprehensive and careful process ensures that all of our educational
materials are as accurate, clear and informative as possible.
Handbook and other written materials:
CMS has produced and disseminated an unprecedented number of written
communication products on Medicare prescription drug coverage. These
materials meet their intended goal of quickly and easily providing
action-oriented information on a variety of topics related to Part D.
Written materials exist in the form of booklets, brochures, fact sheets
and letters. Some key communication products are available in Braille
and audiotape, and many have been translated into alternate languages
to increase accessibility to information.
Medicare & You Handbook:
* The Handbook is an important information source for all Medicare
beneficiaries on the Medicare program and their medical and drug
coverage. Each year, all beneficiary households receive a copy and we
know from our consumer research that beneficiaries keep it to use as a
reference source. Our customer surveys of beneficiaries who read the
Medicare & You 2006 Handbook, conducted in January-February 2006,
showed that 72 percent were "very" or "somewhat satisfied" with the
Handbook.
* For 2006, we updated the Medicare & You Handbook to reflect
information on the new Medicare prescription drug coverage by including
a summary of the new coverage and information on how it can help
Medicare beneficiaries in different situations. In addition, we
reorganized the Handbook to help Medicare beneficiaries decide whether
and how to choose among alternative plans. For example, a prominently
highlighted box on the inside cover of the Handbook serves to remind
beneficiaries that they need to make a choice about prescription drug
coverage for 2006. Beneficiaries are directed to the specific Handbook
section that provides more details on how to select a prescription drug
plan.
* The Medicare & You Handbook has been designed to assist beneficiaries
in deciding how to choose a plan based on cost, coverage, convenience
and peace of mind both now and in the future. In addition to general
information, the Handbook includes information for beneficiaries based
upon their current prescription drug coverage status.
* CMS uses a series of steps before, during, and after printing the
Medicare & You Handbook to ensure accuracy. Some steps may be combined
or omitted as appropriate for other targeted publications and deadlines
for publication.
* Before printing the Handbook, CMS conducts multiple rounds of
internal review by program staff experts in components throughout CMS.
CMS also subjects the Handbook to expert review by external
organizations. CMS solicits comments from an extensive list of advocacy
groups, academic partners, industry trade organizations, Congressional
staff, and other interested stakeholders. CMS writers/editors do the
final proofing. Finally, the CMS Office of External Affairs/Graphics
reviews the Handbook. CMS provides a final desktop publishing
troubleshooting check to ensure that materials include only the files
(such as logos, photos, and fonts) that CMS has legal rights to use.
* During the printing process, CMS reviews printer "blueline" copies.
CMS reviews first proofs from the printer to ensure the publication
layout is accurate. CMS has an opportunity to correct printer errors
(generally something that was altered in the transfer from electronic
file to print plate) or make author's alterations (errors previously
missed) before printing begins. Specially trained CMS and/or GPO staff
go on-site to the print contractor to conduct quality assurance
inspections of the publication, checking for errors as the Handbook is
being printed.
* After printing, CMS carefully monitors and investigates reports of
errors in publications, including tracking related feedback from
representatives at 1-800-MEDICARE. CMS corrects publications, as
needed, and issues updated electronic files and/or errata sheets to
accompany printed publications.
* CMS is very concerned about the readability of our publications. We
have to balance the often competing goals of explaining technical
information about Medicare coverage in clear and simple language while
ensuring its accuracy. We go to great lengths to explain terms that
beneficiaries need to understand to address readability concerns. For
example, all publications include phone numbers and web sites, in case
people need more information. CMS has found that this contact
information is nearly universally identified and understood by
beneficiaries.
* GAO noted readability test score findings as evidence that our
written documents lacked clarity. CMS doesn't routinely perform
readability tests like the Fry, SMOG, FOG or Flesch-Kincaid on
completed publications. Our writers may use these tests as tools during
the drafting process to provide a rough estimate of the readability
level and identify elements such as passive sentences, which can be
readily improved. These kinds of tests rely largely on counting
syllables per word, words per sentence, and sentences per paragraph to
determine a "grade level" readability score which we do not find to be
a useful parameter in gauging "readability" of Medicare materials
because there are terms that may be unfamiliar to the Medicare
population. As such, we go to great lengths to explain concepts that
may be readily understood. For example, "Medicare," "deductible,"
"formulary" and "prescription" are all multi-syllabic words that would
inflate scores in these types of reading tests. However, they are terms
for which there are few or no simpler substitutes. People with Medicare
(and in health insurance generally), commonly recognize most of these
terms. Where they don't, as with "formulary," we use them with careful
explanation in context, which also inflates the readability test scores
by adding words to the sentence. Such tests would not account for this
phenomenon and it is not usually accounted for by omitting certain
words in the scoring process given how many terms for which we provide
detailed explanations.
* These readability test scores are somewhat misleading and incomplete
as a measure of the ease or difficulty of materials.
- Plain language and literary experts like Roger Shuy and the
Georgetown University Round Table on Language and Linguistics, the
Social Security Administration, the Maine AHEC Health Literacy Center,
the Delegates Assembly of the International Reading Association, and
the U.S. Securities and Exchange Commission state that individual's
tested literacy level and their ability to read and understand
materials written at the corresponding grade level rarely match.
- Test scores don't take into account other criteria that improve
clarity of message, like navigational cues and graphic elements.
- It's challenging to account for multi-syllabic terms like "Medicare"
or "prescription" that are widely-understood and/or for which there are
no simpler alternatives.
- When appropriate, our publications provide a glossary to help
beneficiaries understand words that may be new to them. The Medicare &
You handbook contains such a glossary, as do our other large booklets.
However, glossaries would mitigate the goals of brief fact sheets and
letters, and therefore, for these types of materials, every effort is
made to define difficult terms in context, which can inflate standard
readability test scores.
* As an additional measure of clarity, GAO states they used 60
"commonly recognized guidelines" to evaluate our publications. It is
difficult to sufficiently comment on the findings without knowing these
60 criteria, beyond the handful of examples in the report. However, it
is important to note that to the best of our knowledge, these
guidelines were compiled from multiple sources for the purposes of this
evaluation and are not commonly recognized as a set. We look forward to
the opportunity to review these guidelines and their relationship to
our publications in the future, to assess where improvements might be
made.
* To evaluate and improve the usability of Medicare publications, CMS
hires contractors to conduct research with beneficiaries, caregivers,
and other people who help beneficiaries. CMS uses focus groups to help
us understand what information is important to beneficiaries. We also
conduct cognitive interviews to test how well beneficiaries understand
the content in our draft publications. Our drafts are revised based on
the feedback that we receive.
* Consumer testing for the Handbook dates back to 1998. Over the years,
we have qualitatively tested the Handbook with over 1000 aged and
disabled beneficiaries, caregivers, and Medicare counselors. Each year,
the basic testing is conducted in two rounds to allow for iterative
improvements. Lessons learned from year to year are applied to each new
version of the book.
* Multiple methods are used to test the book. The most heavily relied
on method is cognitive interviews where participants are given tasks
"cold," that is without prior preparation. We've also relied on triads
and focus groups which allow participants to generate ideas on how to
improve the book.
* We also conduct "diary groups" where beneficiaries are asked to make
comments on the book as they read through it at home and are then
brought in for focus groups. Tested content developed for particular
publications is also used in other publications as-appropriate. This
overlap ensures consistency across CMS publications.
* Information collected from beneficiaries earlier this year indicated
that 61 percent of respondents said the Medicare & You Handbook was
"very easy" or "somewhat easy" to understand.
* CMS elicited feedback from more than 300 beneficiaries on Part D
materials. The Medicare & You handbook language was tested by a testing
contractor, BearingPoint, with over 150 beneficiaries. This testing
helped us simplify our language and explain concepts more clearly.
* GAO used similar testing methods on a smaller scale to evaluate the
clarity of our written materials. We are interested in reviewing the
details of the 18 tasks that were used the interviews conducted with
beneficiaries and beneficiary advisors, and understanding which tasks
correlated to which tested products. GAO's report provides no details
on the tasks that respondents completed successfully, and describes
only three tasks that were difficult. These three indicate that the
purpose and expectations of these publications may have been
overlooked. The primary goal of our written communications in this
phase was awareness - to make beneficiaries aware of the new coverage,
aware that they needed to take some action, and aware of the resources
available to help them make decisions. None of these publications were
intended to independently lead a reader through such complex activities
as computing projected out-of-pocket costs. Other feedback on our
publications shows they are successful in meeting their intended goals.
* The National Association of Government Communicators critiqued the
Medicare & You 2005 Handbook for the 2004 Blue Pencil Competition. The
handbook received positive feedback in the judges' ratings. The judges
rated the handbook in categories such as writing, editing, purpose,
design, printing, cost effectiveness, and dissemination.
- The judges strongly agreed that the writing was clear, concise, and
appropriate for its intended audience.
- One judge wrote, "Given the complexity of this subject, the writing
is extremely clear and easy to understand. Technical terms are well
explained, and needed information is easy to locate."
- In the area of design, another judge commented that, "Choice of font,
typeface, and size; leading; and margins made the book attractive,
while ensuring accessibility for users (especially seniors). Use of
blue headings and other design elements contributed to ease of use, as
well."
- In the category of purpose, the judges strongly agreed that the
purpose of the handbook is clear and that the handbook gets its message
across with well-supported topics. As an overall final comment, a judge
wrote, "This entry is very well suited to its purpose and audience."
* CMS began preparations for the 2007 Medicare & You Handbook in late
December 2005. To date, staff and leadership have held input meetings
with key advocates and stakeholders, tested early draft revisions with
beneficiaries, established a firm project plan, and instituted
additional quality assurance and proofing processes. The Handbook is
currently on schedule for its required mailing in the fall of this
year, with a comprehensive external review process ending this week and
extensive consumer testing scheduled in mid-May.
1-800-MEDICARE:
It is a top priority at CMS to ensure that beneficiaries have timely
access to accurate information and receive satisfactory service when
contacting 1-800-MEDICARE.
* Between 2004 and the beginning of the open enrollment period, CMS
conducted numerous activities to prepare for the prescription drug
benefit, including the development of a comprehensive training
curriculum on the prescription drug benefit and the Plan Finder tool
for Customer Service Representatives (CSRs). Since November 15, 2005,
CMS has made continuous updates to scripts and reference materials for
CSRs to ensure they are able to communicate accurate information to
beneficiaries and people calling on behalf of beneficiaries.
* CMS's quality monitoring program has found that in 2006, calls to 1-
800-MEDICARE have been accurate 93 percent of the time. This quality
monitoring program is conducted by contractors who run the call
centers. CMS monitors at least 4 calls per month for each of our
thousands of CSRs to identify improvement and training opportunities.
* These are not just mystery shopping calls, which are limited to
topics chosen by researchers, but actual calls which are representative
of the information Medicare beneficiaries want to know. To ensure
reliability and accuracy, all monitors score a sample of calls on a
weekly basis and meet to review their approaches. The data is analyzed
constantly and is used to take immediate corrective action. This work
is overseen by a team within CMS dedicated to the quality of the 1-800-
MEDICARE call centers.
* Examples of topics receiving the highest volume of inquiries at our
call centers include:
- How to enroll in a plan to obtain prescription drug coverage
- Complaints about drug coverage:
- How to apply for the limited-income subsidy:
* Since the beginning of the new prescription drug benefit, CMS has
taken many steps to help beneficiaries get the information they need to
select a drug plan. For example, CMS acquired additional infrastructure
including telephone lines and workstations at call center sites.
* CMS increased the number of customer service representatives (CSRs)
from 3,000 in June 2004 to as many as 7,800 to handle beneficiary calls
with minimal wait times.
* On average, from November 15, 2005 to April 12, 2006, callers have
experienced wait times of less than 2 minutes, with longer waits
sometimes occurring during peak call periods. Call volume to 1-800-
MEDICARE peaked around 400,000 calls per day in mid-November when
enrollment began, and again in early to mid-January. Currently, call
volume reaches 200,000 calls per day on the highest volume day and
levels out around 150,000 per day during the remainder of the week.
Call volumes have continued to increase slightly since then.
* CMS recognizes that not all beneficiaries are able to use, or have
access to, the internet, which is the platform for the useful Medicare
Prescription Drug Plan Finder tool. As part of our outreach and
communication efforts, CMS trained additional staff exclusively on the
use of the Medicare Prescription Drug Plan Finder tool so that they
could be dedicated to answering calls only about the prescription drug
benefit and available plan options.
* We expanded responsibilities and provided additional training for
some CSRs and advanced training for others. We required CSRs to take
written exams and test calls for certification before allowing them to
take live calls. All CSRs have one week of classroom training followed
by two or three additional days of practice calls, simulation, quality
monitoring, and follow-up coaching to ensure peak performance. Finally,
we monitored newly-trained CSRs and those who would benefit from
additional coaching at a higher level.
* This year, CMS implemented a 1-800 MEDICARE caller satisfaction
survey conducted by Pacific Consulting Group, an independent
contractor. This survey provides 1) satisfaction tracking over time and
2) an early warning system that can point to potential service
problems. Improvements can then be implemented relatively quickly to
enhance caller satisfaction. These CMS customer satisfaction surveys
indicate that the bulk of callers who interact with our CSRs, 87
percent are satisfied with their experience. They are particularly
pleased with how courteous and patient the CSRs are (rated at 97
percent). These responses came not only from people with Medicare, but
also friends or relatives calling on their behalf, who made up 34
percent of callers during March 2006.
* Currently, 500 surveys are conducted each week with 400 callers who
spoke with CSRs and 100 callers who used the Interactive Voice Response
System.
The data below depict results from weekly calls for those callers that
spoke to a CSR. The results show the percentage of respondents in the
weeks January 16th, February 27th and March 6th that strongly or
somewhat agree with the statements listed below.
Table:
Survey Metric(% agree-strongly or somewhat to the following
statements): CSR was helpful;
Week of January 16th: 84%;
Week of February 27th: 89%;
Week of March 6th: 88%.
Survey Metric(% agree-strongly or somewhat to the following
statements): CSR understood issue or concern;
Week of January 16th: 83%;
Week of February 27th: 86%;
Week of March 6th: 88%.
Survey Metric(% agree-strongly or somewhat to the following
statements): CSR explained things to me in a way that I could
understand;
Week of January 16th: 83%;
Week of February 27th: 86%;
Week of March 6th: 84%.
Survey Metric(% agree-strongly or somewhat to the following
statements): I received all of the information I needed;
Week of January 16th: 67%;
Week of February 27th: 72%;
Week of March 6th: 73%.
Survey Metric(% agree-strongly or somewhat to the following
statements): The CSR was knowledgeable;
Week of January 16th: 81%;
Week of February 27th: 86%;
Week of March 6th: 85%.
Survey Metric(% agree-strongly or somewhat to the following
statements): I received information specific to my issue;
Week of January 16th: 75%;
Week of February 27th: 80%;
Week of March 6th: 80%.
Survey Metric(% agree-strongly or somewhat to the following
statements): Overall I am satisfied;
Week of January 16th: 79%;
Week of February 27th: 84%;
Week of March 6th: 85%.
[End of table]
* Pharmacists are a key partner in the implementation of the Medicare
prescription drug benefit. To ensure that pharmacists have access to
the information they need to assist beneficiaries at the pharmacy
counter, CMS developed a dedicated pharmacist 1-866 telephone line.
Incoming calls through the dedicated pharmacist line are routed to the
head of the queue at the 1-800 MEDICARE number, wait times are
substantially lower than the overall average for beneficiaries and
other individuals calling the 1-800-MEDICARE line. This helps to
relieve any burden on pharmacists, and also ensures that pharmacists
are able to assist beneficiaries immediately at the pharmacy counter.
* CMS is well-prepared to handle increased call-volume that may occur
before the May 15tH enrollment deadline. We have increased the number
of CSRs from 3,000 in June of 2004 to 6,000 CSRs for May enrollments.
We have also acquired additional infrastructure including telephone
lines and workstations at call center sites. We have refined our CSR
scripts by reducing redundant information, indexing scripts for quick
access, and including probing questions to help the CSRs better
identify callers' concerns.
* Despite our efforts, some beneficiaries will wait until the deadline
is near, but our top priority is to encourage people to enroll now and
avoid the rush.
Medicare.gov:
* To ensure that the new Plan Finder tool was well-designed and easily
used by beneficiaries and other individuals, CMS worked with a
professional website development contractor, CGI Federal and a
subcontractor, Navigation Arts.
* As the Medicare Prescription Drug Plan Finder was being designed, CMS
engaged in multiple rounds of consumer testing to ensure its usefulness
and simplicity. CMS conducted three rounds of in-depth interviews with
Medicare beneficiaries to obtain feedback as drafts of the tool were
developed throughout 2005. Final interviews that focused on messages
tailored specifically for beneficiaries based on their insurance
information were conducted in September 2005. CMS conducts ongoing
consumer research to continue to improve understandability and
usability.
* CMS also conducts thorough and ongoing analyses of possible outliers
in data, including the Medicare Prescription Drug Plan Finder plan
pricing data, pharmacy network, mismatched formulary identifiers (NDC
codes), and other missing formulary data. If problems are found with a
plan's data, information on the plan will be suppressed from the
website until CMS works with the plan to correct its information and
properly display it.
* We are proud to say that CMS has received a number of awards for its
website from independent organizations. These awards include the
"eHealthcare Leadership Award" at the Ninth Annual Internet Conference,
the "2005 Pioneer Award" at the E-Gov Institute and Federal Computer
Week, and the "Independent Technology Supporting Service to Our
Country" award at the Eighth Annual Technology Gala to benefit Juvenile
Diabetes.
* We believe that the website has been extremely successful in
providing beneficiaries, their caregivers and CMS partners with clear,
accurate and timely information to help them enroll in drug plans. In
fact, CSRs at 1-800 MEDICARE have access to the Plan Finder to help
beneficiaries find the information they need about choosing a plan,
enrolling in a plan, or other issues related to accessing their
prescription drug coverage. The Plan Finder also has been a critical
tool for SHIPS and other partners, such as the ABC Coalition and
Medicare Today, to use when conducting outreach to beneficiaries.
* Results from a web-based customer satisfaction survey conducted by
MSInteractive, a subsidiary of Market Strategies that specializes in
web-site satisfaction research, were very positive. This research,
conducted in December 2005, focused only on the prescription drug plan
finder tool.
* The survey indicated that content, interactivity, and navigability
have the greatest impact on satisfaction. During development of the
tool, CMS contracted with a web design firm to leverage their expertise
on these impacts. CMS continues to focus on these areas in future
enhancements and updates.
* The site's "appearance" and "privacy" scored highly, but had no
impact on overall satisfaction.
- 66 percent of those who enrolled were either "somewhat" or "strongly
satisfied" with the tool.
- 80 percent of those who enrolled would recommend the tool to a friend.
- 70 percent of users agreed with this statement, "I know more about
the Medicare Prescription Drug Plans now that I've used this site."
- Regular internet users had higher ratings of the site.
* In January and February 2006, Abt conducted a telephone survey of a
random sample of beneficiaries and found that:
- 14 percent of respondents used the www.medicare.gov website to get
information about Medicare;
- 60 percent said it was "very easy" or "somewhat easy" to understand
the information from www.medicare.gov;
- Beneficiaries who rated their satisfaction with the information
received from medicare.gov as "very/somewhat" satisfied outnumbered the
"dissatisfied" beneficiaries 71 percent to 19 percent. Seven percent of
beneficiaries were neither "satisfied nor dissatisfied."
* Overall, the drug plan finder element of the website has received
164.6 million page views between November 15, 2005 and April 26, 2006.
We do not have a way to differentiate whether those hits were from
beneficiaries or their caregivers.
* To date, 3 million beneficiaries have enrolled in prescription drug
plans using the Plan Finder. That indicates that at least that many
people were satisfied enough with the information they received to
undertake the most important step of enrolling in a drug plan.
* The Frequently Asked Questions (FAQ) section of www.medicare.gov has
been accessed more than one million times since January 1, 2006. CMS
has also responded to more than 19,000 emails received through the FAQ
section, with 93% of them being resolved satisfactorily in the first
response.
State Health Insurance Assistance Programs (SHIPS):
* While the SHIPs play a significant role in beneficiary counseling and
education on Part D, CMS has also created a national grassroots network
of more than 24,000 partners and 140 coalitions that rely on
traditional tools to help them provide personalized counseling to
Medicare beneficiaries every day.
* The network CMS built is diverse and committed, with members from
every sector, including advocacy groups, government agencies, service
clubs, faith-based organizations, benefits counselors, trained
volunteers and healthcare professionals such as doctors and pharmacists.
* This extensive, grassroots-level partnership is truly unprecedented
for the Medicare program. It's reaching out to people with Medicare all
over the country... "where they live, work, play, and pray." This
approach has helped personalize Medicare in every corner of the country.
*Preliminary data from the State Health Insurance Assistance Programs
(SHIPs) shows that individual in-person and telephone contacts,
presentations and meetings reached a total of 4.5 million clients,
compared to 2.5 million in the previous grant period.
Other Selected Activities:
* The Mobile Office Tour has traveled 500,000 miles since last fall and
approximately half of the territory covered and events have been in
rural areas, in an attempt to reach out to a variety of beneficiaries
and partners at the local level. We knew we would have to develop a
grassroots capacity and local networks to supplement the CMS regional
structure to provide the necessary education and enrollment assistance
at the community level. This would involve reaching out, not just to
our traditional partners such as the SHIPS, but to all the groups and
organizations that have contact with our beneficiaries on a daily basis
"where they work, where they play, and where they pray." We needed to
involve individuals and institutions: family members and friends;
current and former employers; churches and synagogues; financial
advisors and community centers, to name but a few.
* CMS is reaching out directly to beneficiaries through an extensive
paid and earned media campaign focusing on press and radio, both of
which are highly localized in informing beneficiaries of special events
in their neighborhoods.
* As we approach May 15, many members of the Cabinet whose agencies
have helped build awareness of the prescription drug benefit through
their own programs have joined efforts with CMS, including the United
States Department of Agriculture, Department of Commerce, Department of
Labor and Housing and Urban Development.
* To minimize a possible last minute rush to enroll, CMS is making a
monumental effort to enroll beneficiaries well before the May 15tH
deadline. In the past month, there have been 1,000 events per week
across the country to provide beneficiaries with personalized help so
they understand the prescription drug coverage options available to
them and they can enroll in a plan. In our enrollment efforts, we are
targeting beneficiaries who may qualify for the low-income subsidy and
beneficiaries who live in rural areas. Our enrollment events are fully
coordinated with the Social Security Administration (SSA) to assist
beneficiaries in applying for extra help, as well as to help them
enroll in a plan.
[End of section]
Appendix IV: GAO Contact and Staff Acknowledgments:
GAO Contact:
Leslie G. Aronovitz (312) 220-7600 or aronovitzl@gao.gov:
Acknowledgments:
In addition to the contact named above, Susan T. Anthony and Geraldine
Redican-Bigott, Assistant Directors; Ramsey L. Asaly; Enchelle Bolden;
Laura Brogan; Shaunessye D. Curry; Chir-Jen Huang; M. Peter Juang; Ba
Lin; Michaela M. Monaghan; Roseanne Price; Pauline Seretakis; Margaret
J. Weber; and Craig H. Winslow made contributions to this report.
[End of section]
FOOTNOTES
[1] Pub. L. No. 108-173, § 101, 117 Stat. 2066, 2071-2152 (to be
codified at 42 U.S.C. §§ 1395w-101--1395w-152). MMA redesignated the
previous part D of title XVIII of the Social Security Act as part E and
inserted a new part D after part C.
[2] For Part D standard coverage, Medicare pays on average 75 percent
of prescription drug costs up to $2,250, after a $250 deductible.
Beneficiaries then pay their next $2,850 in drug costs. If their drug
costs exceed this amount, Medicare will pay about 95 percent of their
additional costs for the rest of the calendar year.
[3] Drug plan sponsors include insurance companies and other private
organizations.
[4] CMS is an agency in the Department of Health and Human Services.
[5] In December 2004, we reported on the information being provided to
beneficiaries through the Medicare help line on eligibility,
enrollment, and benefits. (See GAO, Medicare: Accuracy of Responses
from the 1-800-MEDICARE Help Line Should Be Improved, GAO-05-130
(Washington, D.C.: Dec. 8, 2004).)
[6] The Medicare Web site is www.medicare.gov.
[7] Dual-eligible beneficiaries are Medicare beneficiaries who are also
eligible for Medicaid--the federal-state health program for low-income
individuals--and receive full Medicaid benefits for services not
covered by Medicare.
[8] Medicare Advantage replaced the Medicare+Choice managed care
program and expanded the availability of private health plan options to
Medicare beneficiaries. Medigap policies provide supplemental health
coverage sold by private insurers to help pay for Medicare cost-sharing
requirements, as well as for some services not provided by Medicare.
[9] The Henry J. Kaiser Family Foundation, The Medicare Drug Benefit:
Beneficiaries Perspectives Just Before Implementation, [Hyperlink,
http://kff.org/]. kaiserpolls/med111005nr.cfm (downloaded Apr. 26,
2006).
[10] The three tests were the Flesch-Kincaid Grade Level, the SMOG
(Simplified Measure of Gobbledygook) Reading Grade Level, and the Fry
Readability Estimate. These tests use such measures as sentence length
and the number of syllables in a selection of text to arrive at a
reading level, which is expressed in terms of school grade level.
[11] A formulary is a list of prescription drugs covered by a health
plan.
[12] The percentages related to the responses we received to our 500
calls exceed 100 percent because of rounding.
[13] A SHIP grant year begins on April 1 of the year the funds become
available.
[14] 29 U.S.C. § 794d (2000).
[15] Medigap policies provide supplemental health coverage sold by
private insurers to help pay for Medicare cost-sharing requirements, as
well as for some services not provided by Medicare.
[16] The three tests were the Flesch-Kincaid Grade Level, the SMOG
(Simplified Measure of Gobbledygook) Reading Grade Level, and the Fry
Readability Estimate. The tests use such measures as sentence length
and the number of syllables in a selection of text to arrive at a
reading level, which is expressed in terms of school grade level.
[17] These guidelines are presented in an NN/g report called Web
Usability for Senior Citizens: 46 Design Guidelines Based on Usability
Studies with People Age 65 and Older. For this study, NN/g conducted
usability tests of 17 Web sites with 44 seniors. Based on the test
findings, NN/g developed 46 design guidelines that would make Web sites
more attractive to seniors.
[18] In addition, NN/g indicated cases where an aspect was not
functioning correctly from a Web site development standpoint by giving
it a "bug" mark.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
