Medicare

Quality of CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved Gao ID: GAO-06-715T May 4, 2006

Today's hearing focuses on Medicare Part D, the program's new outpatient prescription drug benefit. On January 1, 2006, Medicare began providing this benefit, and beneficiaries have until May 15, 2006, to enroll without the risk of penalties. The Centers for Medicare & Medicaid Services (CMS), which administers the Part D benefit, has undertaken outreach and education efforts to inform beneficiaries and their advisers. GAO was asked to discuss how CMS can better ensure that Medicare beneficiaries are informed about the Part D benefit. This testimony is based on Medicare: CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved, GAO-06-654 (May 3, 2006).

Information given in the six sample documents that GAO reviewed describing the Part D benefit was largely complete and accurate, although this information lacked clarity. First, about 40 percent of seniors read at or below the fifth-grade level, but the reading levels of these documents ranged from seventh grade to postcollege. Second, on average, the six documents we reviewed did not comply with about half of 60 common guidelines for good communication. For example, the documents used too much technical jargon and often did not define difficult terms. Moreover, 16 beneficiaries and advisers that GAO tested reported frustration with the documents' lack of clarity and had difficulty completing the tasks assigned to them. Customer service representatives (CSRs) answered about two-thirds of the 500 calls GAO placed to CMS's 1-800-MEDICARE help line accurately and completely. Of the remainder, 18 percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, and about 3 percent received incomplete responses. In addition, about 5 percent of GAO's calls were not answered, primarily because of disconnections. The accuracy and completeness of CSRs' responses varied significantly across the five questions. For example, while CSRs provided accurate and complete responses to calls about beneficiaries' eligibility for financial assistance 90 percent of the time, the accuracy rate for calls concerning the drug plan that would cost the least for a beneficiary with specified prescription drug needs was 41 percent. For this question, the CSRs responded inappropriately for 35 percent of the calls by explaining that they could not identify the least costly plan without the beneficiary's personal information--even though CSRs had the information needed to answer the question. The time GAO callers waited to speak with CSRs also varied, ranging from no wait time to over 55 minutes. For 75 percent of the calls--374 of the 500--the wait was less than 5 minutes. The Part D benefit portion of the Medicare Web site can be difficult to use. GAO's test of the site's overall usability--the ease of finding needed information and performing various tasks--resulted in scores of 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate that using the site can be difficult. For example, the prescription drug plan finder was complicated to use and some of its key functions, such as "continue" and "choose a drug plan," were often not visible on the page without scrolling down.

GAO-06-715T, Medicare: Quality of CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved This is the accessible text file for GAO report number GAO-06-715T entitled 'Medicare: Quality of CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved' which was released on May 5, 2006. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Testimony: Before the Subcommittee on Health, Committee on Ways and Means, House of Representatives: Medicare: Quality of CMS Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved: Statement of Leslie G. Aronovitz: Director: Health Care: GAO-06-715T: GAO Highlights: Highlights of GAO-06-715T, a testimony before the Subcommittee on Health, Committee on Ways and Means, U.S. House of Representatives. Why GAO Did This Study: Today‘s hearing focuses on Medicare Part D, the program‘s new outpatient prescription drug benefit. On January 1, 2006, Medicare began providing this benefit, and beneficiaries have until May 15, 2006, to enroll without the risk of penalties. The Centers for Medicare & Medicaid Services (CMS), which administers the Part D benefit, has undertaken outreach and education efforts to inform beneficiaries and their advisers. GAO was asked to discuss how CMS can better ensure that Medicare beneficiaries are informed about the Part D benefit. This testimony is based on Medicare: Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved, GAO-06-654 (May 3, 2006). What GAO Found: Information given in the six sample documents that GAO reviewed describing the Part D benefit was largely complete and accurate, although this information lacked clarity. First, about 40 percent of seniors read at or below the fifth-grade level, but the reading levels of these documents ranged from seventh grade to postcollege. Second, on average, the six documents we reviewed did not comply with about half of 60 common guidelines for good communication. For example, the documents used too much technical jargon and often did not define difficult terms. Moreover, 16 beneficiaries and advisers that GAO tested reported frustration with the documents‘ lack of clarity and had difficulty completing the tasks assigned to them. Customer service representatives (CSRs) answered about two-thirds of the 500 calls GAO placed to CMS‘s 1-800-MEDICARE help line accurately and completely. Of the remainder, 18 percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, and about 3 percent received incomplete responses. In addition, about 5 percent of GAO‘s calls were not answered, primarily because of disconnections. The accuracy and completeness of CSRs‘ responses varied significantly across the five questions. For example, while CSRs provided accurate and complete responses to calls about beneficiaries‘ eligibility for financial assistance 90 percent of the time, the accuracy rate for calls concerning the drug plan that would cost the least for a beneficiary with specified prescription drug needs was 41 percent. For this question, the CSRs responded inappropriately for 35 percent of the calls by explaining that they could not identify the least costly plan without the beneficiary‘s personal information”even though CSRs had the information needed to answer the question. The time GAO callers waited to speak with CSRs also varied, ranging from no wait time to over 55 minutes. For 75 percent of the calls”374 of the 500”the wait was less than 5 minutes. The Part D benefit portion of the Medicare Web site can be difficult to use. GAO‘s test of the site‘s overall usability”the ease of finding needed information and performing various tasks”resulted in scores of 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate that using the site can be difficult. For example, the prescription drug plan finder was complicated to use and some of its key functions, such as ’continue“ and ’choose a drug plan,“ were often not visible on the page without scrolling down. What GAO Recommends: In its May 2006 report, GAO recommended that the CMS Administrator enhance the quality of its communications on the Part D benefit, including clarifying written materials, monitoring the accuracy and completeness of help line responses, and improving the usability of the Medicare Web site. CMS said that GAO‘s findings did not present a complete and accurate picture of its activities. However, CMS said that it supports the goals of GAO‘s recommendations and is already taking steps to implement them. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-715T]. To view the full product, including the scope and methodology, click on the link above. For more information, contact Leslie G. Aronovitz at aronovitzl@gao.gov or (312) 220-7600. [End of section] Mr. Chairman and Members of the Subcommittee: I am pleased to be here today as you discuss the Medicare outpatient prescription drug benefit, known as the Part D benefit, which was established by the Medicare Prescription Drug, Improvement, and Modernization Act of 2003[Footnote 1] with coverage beginning on January 1, 2006. Until this time, Medicare, the program that finances health care benefits for about 42 million elderly and disabled beneficiaries, had not generally provided coverage for outpatient prescription drugs. Beneficiaries who opt to enroll in Part D may choose a drug plan from those offered by private plan sponsors under contract to the Centers for Medicare & Medicaid Services (CMS), which administers the Part D benefit. These plans differ in the drugs covered, pharmacies used, and enrollee costs. As of April 20, 2006, more than 30 million of Medicare's 42 million beneficiaries were enrolled in a Part D plan or had other outpatient prescription drug coverage. Beneficiaries have until the end of the current enrollment period, May 15, 2006, to enroll in the Part D benefit and select a plan without the risk of penalties in the form of higher premiums. Given the newness and complexity of the Part D benefit, it is critical that beneficiaries and their advisers, including members of their families, understand the options available to them. Understanding these options enables beneficiaries to make informed decisions on whether to enroll in the Part D benefit, and if they decide to enroll, which drug plan to choose. As part of its responsibilities, CMS has undertaken efforts to provide beneficiaries and their advisers with the information they need about the Part D benefit through various media, including written documents, the 1-800-MEDICARE help line,[Footnote 2] and the Medicare Web site.[Footnote 3] CMS's education efforts are important because widespread confusion has been reported among beneficiaries about the costs and coverage under the new benefit. You and others have expressed interest in ensuring that Medicare beneficiaries receive the information they need to make informed decisions. My remarks today will focus on (1) the extent to which CMS's written documents describe the Part D benefit in a clear, complete, and accurate manner; (2) the effectiveness of CMS's 1-800-MEDICARE help line in providing accurate, complete, and prompt responses to callers inquiring about the Part D benefit; and (3) whether CMS's Medicare Web site presents information on the Part D benefit in a usable manner. My testimony will summarize findings of a report we released yesterday that examines CMS's Medicare Part D benefit communications to beneficiaries in more detail and includes recommendations to the CMS Administrator for improving the quality of the agency's Part D benefit education and outreach materials.[Footnote 4] To address these issues, we interviewed CMS officials responsible for written documents about the Part D benefit, the 1-800-MEDICARE help line, and the Medicare Web site. To assess the clarity, completeness, and accuracy of written documents, we performed an in-depth review of a sample of six CMS documents describing the Part D benefit, including contracting with the American Institutes for Research (AIR), a firm with experience in evaluating written documents, to assess their clarity. (See app. I for a list of written documents reviewed.) We also placed 500 calls to 1-800-MEDICARE, posing one of five questions related to the Part D benefit in each call so that each question was asked 100 times. We evaluated the accuracy and completeness of customer service representatives' (CSR) responses to these questions. (See app. II for the questions and criteria we used to evaluate the accuracy and completeness of CSR responses to calls we made to 1-800 MEDICARE.) To assess the usability of the Part D benefit information available on the Medicare Web site, we contracted with the Nielsen Norman Group (NN/g), an expert on Web design. We conducted our work from November 2005 through May 2006, in accordance with generally accepted government auditing standards. In summary, CMS successfully developed a large volume of information about the new Part D benefit and made it available to beneficiaries through a variety of sources, despite the challenge of developing this information within a short time frame. However, the quality of CMS's communications to beneficiaries and their advisers about the Part D benefit could be improved. For example, although the six CMS written documents we reviewed were largely accurate and complete, they often lacked clarity. Specifically, while about 40 percent of seniors read at or below the fifth-grade level, the reading levels of the documents ranged from seventh grade to postcollege. Moreover, the six documents used too much technical jargon and often did not define difficult terms. Similarly, although 67 percent of the responses to the 500 calls we placed to CMS's 1-800-MEDICARE help line regarding the Part D benefit were accurate and complete, we nonetheless received a substantial number of responses that were not. Eighteen percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, about 3 percent received incomplete responses, and about 5 percent of our calls were not answered, primarily due to disconnections.[Footnote 5] In addition, our review of the Part D benefit portion of the Medicare Web site showed that this site can be difficult to use. In usability tests that examined the ease of finding needed information and performing various tasks, we found that, for overall usability, the Web site scored 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate that using the site can be difficult. Therefore, in the report we issued yesterday, we made specific recommendations to the CMS Administrator to enhance the quality of the agency's communications on the Part D benefit, including clarifying written materials, monitoring the accuracy and completeness of CSR's responses to callers' inquiries, and improving the usability of the Part D benefit portion of the Medicare Web site. In its comments on a draft of our report (see app. III), CMS said that it supports the goals of our recommendations and is already taking steps to implement them. However, CMS said that our findings did not present a complete and accurate picture of its Part D benefit communications activities. We believe that our report provides an accurate examination of the CMS communications mechanisms that have the greatest impact on beneficiaries (see app. IV). Background: CMS has undertaken steps to educate beneficiaries about the Part D benefit using written documents, a toll-free help line, and the Medicare Web site. To explain the Part D benefit to beneficiaries, CMS had produced more than 70 written documents as of December 2005. Medicare & You--the beneficiary handbook--is the most widely available and was sent directly to beneficiaries in October 2005. Other written documents were targeted to specific groups of beneficiaries, such as dual-eligible beneficiaries[Footnote 6] and beneficiaries with Medicare Advantage or Medigap policies.[Footnote 7] Beneficiaries can obtain answers to questions about the Part D benefit by calling the 1-800-MEDICARE help line. This help line, which is administered by CMS, was established in March 1999, to answer beneficiaries' questions about the Medicare program. As of December 2005, about 7,500 CSRs were handling calls on the help line, which operates 24 hours a day, 7 days a week, and is run by two CMS contractors. CMS provides CSRs with detailed scripts to use in answering the questions. Call center contractors write the scripts, and CMS checks them for accuracy and completeness. In addition, CMS's Medicare Web site provides information about various aspects of the Medicare program. The Web site contains basic information about the Part D benefit, suggests factors for beneficiaries to consider when choosing plans and provides guidance on enrollment and plan selection. It also lists frequently asked questions, and allows users to view, print, or order publications. In addition, the site contains information on cost and coverage of individual plans. There is also a tool that allows beneficiaries to enroll directly in the plan they have chosen. Clarity of CMS Written Documents Could Be Improved: Although the six sample documents we reviewed informed readers of enrollment steps and factors affecting coverage, they lacked clarity in two ways. First, about 40 percent of seniors read at or below the fifth- grade level, but the reading levels of the documents ranged from seventh grade to postcollege. As a result, these documents are challenging for many seniors. Even after adjusting the text for 26 multisyllabic words, such as Medicare, Medicare Advantage, and Social Security Administration, the estimated reading level ranged from seventh to twelfth grade, a reading level that would remain challenging for at least 40 percent of seniors. Second, on average, the six documents we reviewed did not comply with about half of the 60 commonly recognized guidelines for good communications. For example, although the documents included concise and descriptive headings, they used too much technical jargon and often did not define difficult terms such as formulary.[Footnote 8] The 11 beneficiaries and 5 advisers we tested reported frustration with the documents' lack of clarity as they encountered difficulties in understanding and attempting to complete 18 specified tasks. For example, none of these beneficiaries and only 2 of the advisers were able to complete the task of computing their projected total out-of- pocket costs for a plan that provided Part D standard coverage. Only one of 18 specified tasks was completed by all beneficiaries and advisers. Even those who were able to complete a given task expressed confusion as they worked to comprehend the relevant text. Help Line Responses Frequently Complete and Accurate, but Varied By Question: Of the 500 calls we placed to CMS's 1-800-MEDICARE help line regarding the Part D benefit, CSRs answered about 67 percent of the calls accurately and completely. Of the remainder, 18 percent of the calls received inaccurate responses, 8 percent of the responses were inappropriate given the question asked, and about 3 percent received incomplete responses. In addition, about 5 percent of our calls were not answered, primarily because of disconnections.[Footnote 9] The accuracy and completeness of CSR responses varied significantly across our five questions. (See fig. 1.) For example, while CSRs provided accurate and complete responses to calls about beneficiaries' eligibility for financial assistance 90 percent of the time, the accuracy rate for calls concerning the drug plan that would cost the least for a beneficiary with specified prescription drug needs was 41 percent. CSRs inappropriately responded 35 percent of the time that this question could not be answered without personal identifying information--such as the beneficiary's Medicare number or date of birth--even though the CSRs could have answered our question using CMS's Web-based prescription drug plan finder tool. CSRs' failure to read the correct script also contributed to inaccurate responses. The time GAO callers waited to speak with CSRs also varied, ranging from no wait time to over 55 minutes. For 75 percent of the calls--374 of the 500--the wait was less than 5 minutes. Figure 1: Variation in CSRs' Responses for Individual Questions: [See PDF for image] Source: GAO. [End of figure] Part D Benefit Portion of Medicare Web Site Can Be Challenging to Use: We found that the Part D benefit portion of the Medicare Web site can be difficult to use. In our evaluation of overall usability--the ease of finding needed information and performing various tasks--we found usability scores of 47 percent for seniors and 53 percent for younger adults, out of a possible 100 percent. While there is no widely accepted benchmark for usability, these scores indicate difficulties in using the site. For example, tools such as the drug plan finder were complicated to use, and forms that collect information on-line from users were difficult to correct if the user made an error. We also evaluated the usability of 137 detailed aspects of the Part D benefit portion of the site, including features of Web design and on- line tools, and found that 70 percent of these aspects could be expected to cause users confusion. For example, key functions of the prescription drug plan finder tool, such as the "continue" and "choose a drug plan" buttons, were often not visible on the page without scrolling down. In addition, the drug plan finder tool defaults--or is automatically reset--to generic drugs, which may complicate users' search for drug plans covering brand name drugs. The material in this portion of the Web site is written at the 11th grade level, which can also present challenges to some users. Finally, in our evaluation of the ability of seven participants to collectively complete 34 user tests, we found that on average, participants were only able to proceed slightly more than half way though each test. When asked about their experiences with using the Web site, the seven participants, on average, indicated high levels of frustration and low levels of satisfaction. Concluding Observations: Within the past 6 months, millions of Medicare beneficiaries have been making important decisions about their prescription drug coverage and have needed access to information about the new Part D benefit to make appropriate choices. CMS faced a tremendous challenge in responding to this need and, within short time frames, developed a range of outreach and educational materials to inform beneficiaries and their advisers about the Part D benefit. To disseminate these materials, CMS largely added information to existing resources, including written documents, such as Medicare & You; the 1-800-MEDICARE help line; and the Medicare Web site. However, CMS has not ensured that its communications to beneficiaries and their advisers are provided in a manner that is consistently clear, complete, accurate, and usable. Although the initial enrollment period for the Part D benefit will end on May 15, 2006, CMS will continue to play a pivotal role in providing beneficiaries with information about the drug benefit in the future. The recommendations we have made would help CMS to ensure that beneficiaries and their advisers are prepared when deciding whether to enroll in the benefit, and if enrolling, which drug plan to choose. Mr. Chairman, this concludes my prepared remarks. I would be happy to respond to any questions that you or other Members of the subcommittee may have at this time. Contact and Acknowledgments: For further information regarding this statement, please contact Leslie G. Aronovitz at (312) 220-7600. Contact points for our Offices of Congressional Relations and Public Affairs may be found in the last page of this statement. Susan T. Anthony and Geraldine Redican-Bigott, Assistant Directors; Shaunessye D. Curry; Helen T. Desaulniers; Margaret J. Weber; and Craig H. Winslow made key contributions to this statement. [End of section] Appendix I: Sample of CMS Written Documents Reviewed: To assess the clarity, completeness, and accuracy of written documents, we compiled a list of all available CMS-issued Part D benefit publications intended to inform beneficiaries and their advisers and selected a sample of 6 from the 70 CMS documents available, as of December 7, 2005, for in-depth review, as shown in Table 1. The sample documents were chosen to represent a variety of publication types, such as frequently asked questions and fact sheets available to beneficiaries about the Part D benefit. We selected documents that targeted all beneficiaries or those with unique drug coverage concerns, such as dual-eligibles and beneficiaries with Medigap plans. Table 1: Sample of Six Selected Documents: Document: Medicare & You, Section 6: Medicare Prescription Drug Coverage; Target audience: All beneficiaries. Document: Things to Think about When You Compare Plans; Target audience: All beneficiaries. Document: Frequently Asked Questions about: Retiree Prescription Drug Coverage & the New Medicare Prescription Drug Coverage; Target audience: Beneficiaries with employer or union coverage. Document: Introduction to the Auto-Enrollment Notice; Target audience: Dual-eligible beneficiaries[A]. Document: Quick Facts about Medicare's New Coverage for Prescription Drugs for People with a Medicare Health Plan with Prescription Drug Coverage; Target audience: Beneficiaries with Medicare Advantage[B]. Document: Do You Have a Medigap Policy with Prescription Drug Coverage?; Target audience: Beneficiaries with Medigap[C]. Source: GAO. [A] Dual-eligible beneficiaries are Medicare beneficiaries who receive full Medicaid benefits for services not covered by Medicare. [B] Medicare Advantage replaced the Medicare + Choice managed care program and expanded the availability of private health plan options to Medicare beneficiaries. [C] Medigap policies provide supplemental health coverage sold by private insurers to help pay for Medicare cost-sharing requirements, as well as for some services not provided by Medicare. [End of table] [End of section] Appendix II: Questions and Criteria Used to Evaluate Accuracy and Completeness of CSR's Help Line Responses: To determine the accuracy and completeness of information provided regarding the Part D benefit, we placed a total of 500 calls to the 1- 800-MEDICARE help line. We posed one of five questions about the Part D benefit in each call, so that each question was asked 100 times. Table 2 summarizes the questions we asked and the criteria we used to evaluate the accuracy of responses. Table 2: Questions and Criteria Used to Evaluate Accuracy and Completeness: Question GAO Asked MEDICARE help line CSRs: 1. What drug plan can a beneficiary get that will cover all of his/her [specified] drugs at a [specified] pharmacy, have a mail-order option; and cost the least amount annually with [or without] a deductible?; Criteria GAO used to evaluate accuracy and completeness of CSR Responses: An accurate and complete response would identify the prescription drug plan that has the lowest estimated annual cost for the drugs the beneficiary uses. Question GAO Asked MEDICARE help line CSRs: 2. Can a beneficiary who is in a nursing home and not on Medicaid sign up for a prescription drug plan?; Criteria GAO used to evaluate accuracy and completeness of CSR Responses: An accurate and complete response would indicate that a beneficiary can choose whether to enroll in a Medicare prescription drug plan. Question GAO Asked MEDICARE help line CSRs: 3. Can a beneficiary enroll in the Medicare prescription drug program and keep his/her current Medigap policy?; Criteria GAO used to evaluate accuracy and completeness of CSR Responses: An accurate and complete response would inform the caller that enrolling for the prescription drug benefit would depend on whether the beneficiary's Medigap plan was creditable- -that is, whether the coverage it provided was at least as good as Medicare's standard prescription drug coverage--or noncreditable. The CSR response would also mention that the beneficiary's Medigap plan should have sent him/her information that outlined options. Question GAO Asked MEDICARE help line CSRs: 4. What options does a beneficiary, who has retiree health insurance with prescription drug coverage that is not as good as the Medicare prescription drug coverage, have as it relates to the Medicare benefit?; Criteria GAO used to evaluate accuracy and completeness of CSR Responses: An accurate and complete response would indicate that a beneficiary has two options: (1) keep current health plan and join the prescription drug plan later with a penalty, or (2) drop current coverage and join a Medicare drug plan. Question GAO Asked MEDICARE help line CSRs: 5. How do I know if a beneficiary qualifies for financial assistance?; Criteria GAO used to evaluate accuracy and completeness of CSR Responses: An accurate and complete response would refer the beneficiary to the Social Security Administration. Source: GAO. [End of table] [End of section] Appendix III: Comments from the Centers for Medicare & Medicaid's Services: Department Of Health & Human Services: Centers for Medicare & Medicaid Services: 200 Independence Avenue SW: Washington, DC 20201: TO: Leslie G. Aronovitz: Director: Health Care: FROM: Mark B. McClellan, M.D., PhD: Administrator: SUBJECT: Government Accountability Office's (GAO) Draft report, "Medicare: Communications to the Beneficiaries on the Prescription Drug Benefit Could Be Improved" (GAO-06-654): The Centers for Medicare & Medicaid Services has reviewed the findings in the GAO report entitled MEDICARE: Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved (GAO-06-654) regarding CMS communications on the Part D benefit. Having clear and effective communication about Medicare's new prescription drug coverage is one of the Agency's critical priorities. We have worked very hard to ensure that Medicare beneficiaries have the information they need to make decisions about enrolling in a drug plan that works for them. We are pleased that the millions of beneficiaries who have enrolled in Part D are experiencing very high rates of satisfaction with their coverage. Each week, tens of thousands of beneficiaries are enrolling in Part D, which gives them real savings and protections for the future. While we greatly appreciate the feedback from your report and have already worked to implement your recommendations, we do not believe that your findings present a complete or accurate picture of the Part D communication activities. We understand that the report is based on studies of particular aspects of some of our communications tools at one point in time three months ago, in January and early February 2006. In addition to the many "continuous improvement" activities we have undertaken to address startup issues in the drug benefit since that time, there are much more extensive internal and external evaluations of our communications activities completed before, during, and after that time which have different conclusions, as we note below. These evaluations have used well-established methods which have been clearly documented and reviewed; in contrast, you have not yet responded to our requests for information on the methods you have applied. Additionally, your report does not address the unique breadth and depth of CMS activities to educate and to reach out to people with Medicare and the community that supports them in their health care decisions. From the outset, it was clear that no single source of information would be adequate or preferred by all of our beneficiaries. Consequently, we have expanded the range of tools available and vastly expanded our local partnerships to help beneficiaries use them, partnering with more than ten thousand diverse public and private organizations around the country in this effort. Importantly, the report does not look at this broad array of communication tools to help Medicare beneficiaries consider their drug plan options. For example, the report dismisses all of the tools used by our customer service representatives and our website for beneficiaries that provide personalized identifying information to enable us to provide them with personally customized service. The vast majority of our callers provide such personal identification, yet these tools were not evaluated. In fact, the report misleadingly states that we provided the right information on a lower share of cases because some customer service representatives sought to get this personal information to serve the beneficiary more quickly and effectively. Where GAO did actually get information on drug costs, as thousands of callers get every day, customer service representatives provided accurate information at a much higher rate. As another example, GAO evaluated whether beneficiaries could calculate their out-of-pocket drug costs in the standard Medicare benefit by hand, using only the Medicare and You handbook, but very few beneficiaries have opted to use the handbook in this way because: (1) there are far better tools available for quickly and automatically calculating drug costs on the web, on the phone, and through our partner organizations, and (2) over 90 percent of our beneficiaries are choosing plans with benefits other than the standard plan, because they prefer features like zero deductibles, flat copays, and coverage in the "donut hole." Beneficiaries are overwhelmingly using other tools to make effective cost comparisons. In fact, the drug plan finder element of the website has received 164.6 million page views between November 15, 2005 and April 26, 2006. The Frequently Asked Questions (FAQ) section of www.medicare.gov has been accessed more than one million times since January l, 2006. CMS has also responded to more than 19,000 emails received through the FAQ section, with 93% of them being resolved satisfactorily in the first response. Finally, there is no attention in the report at all to major aspects of our communications activities and expenditures, such as the expansion of our community based education and outreach efforts through an extensive network of grassroots partners across the country. This significant emphasis on reaching people where they live, work, play and pray is a key component of our success in reaching millions of people with Medicare and those who work on their behalf. No mention is made of the specialized campaigns targeting African American, Hispanic, American Indians, Asian American and Pacific Islander and in low income communities. These campaigns utilize new partnerships, employ materials in other languages and specialized paid media campaigns. These targeted campaigns within the broader campaign allow us to reach all segments of the Medicare population, including those who might benefit from the low income subsidy and those with language and other cultural barriers to accessing information. We believe that there have been a number of key elements to our successful education campaign. First, we recognized early on that we would need to supplement our proven traditional communications tools, including the Medicare & You Handbook, the 1-800-MEDICARE line, and the State Health Insurance Assistance Programs (SHIPs) with additional advanced technology and grassroots resources, as well as use earned and paid media opportunities. Second, we determined that the provision of personalized assistance and one-on-one counseling was the key ingredient to success. This necessitated our building a grassroots network of traditional and non-traditional partners who were willing to be trained to provide the one-on-one counseling. We strongly believe this is important for beneficiaries to make confident decisions about their Part D plan. We knew we would have to develop a grassroots capacity and local networks to supplement the CMS regional structure to provide the necessary education and enrollment assistance at the community level. This would involve reaching out, not just to our traditional partners such as the SHIPs, but to all the groups and organizations that have contact with our beneficiaries on a daily basis "where they work, where they play, and where they pray." We appreciate any and all ideas for improving our communications efforts, and we take very seriously the four tasks that GAO recommends to improve CMS' education efforts. We support the goal of these tasks and have already taken many steps to meet them. Ensure that CMS's written documents describe the Part D benefit in a manner that is consistent with commonly recognized communications guidelines and that is responsive to the intended audience needs. - CMS employs a wide variety of consumer research techniques, simple language best practices, and independent evaluations in both English and in Spanish documents to ensure the readability and usefulness of our educational materials including those describing Part D. These tests have demonstrated that CMS written documents follow best practice guidelines for written communications with the intended audiences. These techniques and practices are summarized in Attachment A.Because of the importance of this topic, we are always interested in improving our written products. We look forward to an opportunity to review what GAO used in its review and will compare them to the evaluation methods we are already using, as soon as GAO is willing to provide the methodological details. Determine why CSRs frequently do not search for available drug plans if the caller does not provide personal identifying information. -As discussed with the GAO reviewers, CMS has instructed CSRs, in cases where that information is unavailable, to perform a search that provides general information on the plan options available to the beneficiary. Our web tools have always been set up to support such "unauthenticated" searches as well. 1-800 MEDICARE CSRs do have the ability to conduct a general search for callers who do not have their Medicare number. If the person provides personal information, the authenticated search, other information that may influence their decision is pulled into the search, e.g., low income subsidy status or coverage through a retiree drug subsidy. Because this path provides more robust and specific results, CMS has encouraged CSRs to stress the importance of an authenticated Prescription Drug Plan Finder search to callers. The importance of authenticated searches is stressed in the CSR training materials and scripts. We have placed warnings throughout the training materials about the downside of proceeding without the personalized information and CSRs do suggest that the person call back when they have it. Even so, we know that there are occasions in which someone may not want to provide this information, or another caller may be inquiring on behalf of a beneficiary and not have the information, or a reporter or analyst may be calling for information. It has been emphasized to CSRs that non-authenticated general information is to be shared if the caller is unable to provide specific information that would enable a more detailed search. An example of relevant CSR instructions follows. "If a caller indicates they are calling for someone else and just wants general information on plans available in their area, you do not need to personalize the search if the caller does not want to. You can provide general plan information and send a personalized booklet if requested." CMS has a comprehensive quality review process on calls and we will continue to monitor calls to ensure that CSRs are pursuing the general search when appropriate. At the same time, we believe that GAO presents this finding in a way that is incorrect and misleading. We believe that the 41 % accuracy rate unfairly portrayed how accurately CMS answers questions on drug plan options without beneficiary personal identification information, when the GAO failed to analyze 35 out of the 41 responses. In actuality, when the responses are analyzed, correct answers are actually being provided a majority of the time. Further, the bulk of the responses characterized as "inaccurate" were related to the test caller's request that the CSR use only brand name drugs (i.e., no generic drug substitution). This request is highly unusual in our call experience as generic versions of a drug are identical in their clinical effects. However, we have subsequently modified the web tool used by our CSRs to make it easier to override the generic drug substitution logic in the tool. Monitor the accuracy and completeness of CSRs responses to callers' inquiries and identify tools targeted to improve their performance in responding to questions concerning the Part D benefit, such as additional scripts and training.-We have worked hard to ensure beneficiaries have access to accurate and clear information when they call 1-800-Medicare. Our ongoing monitoring program, which evaluates a random sample of hundreds of actual calls received each month, has found that calls to 1-800-MEDICARE in 2006 have been answered accurately 93 percent of the time. The high accuracy rate is reflected in high rates of overall satisfaction from 1-800-MEDICARE callers, which averaged 84 to 85 percent in February and March. Improve the usability of the Part D portion of the Medicare website by refining web-based tools, providing workable navigation features and links, and making web-based forms easier to use and correct. - CMS is continually enhancing and refining their web-based tools to provide Medicare beneficiaries and their caregivers the information needed to compare, choose and enroll in a prescription drug plan that best meet their needs. We summarize some of our recent enhancements below. Online enrollment has been highly successful, as evidenced by the 3 million beneficiaries who have enrolled in the prescription drug plans using CMS' web-based drug plan finder. Our partner organizations have used the web tools to assist millions more with their enrollment-related needs. The high level of online enrollment and use by partners indicates that many people have found that this resource is useful and effective for undertaking the most important step of enrolling in a drug plan, and we are pleased that thousands more are using it every day. We cannot emphasize enough CMS' commitment to continuously improve the communications with beneficiaries and other constituents. We want our websites to continue to be recognized as benchmarks for excellence. Attachment A outlines improvements that we have made to the website since the GAO review and we believe demonstrate our continued commitment to excellence. All of our communications methods, in conjunction with our far-reaching grassroots efforts, have helped provide the important information about Part D needed by beneficiaries, providers and partners to ensure the Medicare drug program is a success. In fact, the vast majority of beneficiaries are using their coverage to save money and get protection for the future: actual premiums and drug costs are much lower than had been expected because of strong competition, and because beneficiaries are using the enrollment tools to choose plans that save them more (over 73 percent of beneficiaries are enrolling in plans stand-alone prescription drug plans with premiums below the average); the drug plans are successfully filling over three million prescriptions a day; and each week hundreds of thousands of beneficiaries are enrolling in the new program. Tab A attached provides additional details about our communications materials and approaches. Also attached are technical comments for your consideration in Tab B. We will use the findings of the GAO report going forward as we continue our commitment to ensure that Medicare beneficiaries have the information they need to make informed health care decisions. Tab A: Detailed Information About Part D Education And Outreach: Over the past two years, we have dedicated significant resources to the development and implementation of an extensive education and outreach campaign surrounding Medicare prescription drug coverage, including a variety of beneficiary publications and materials, the 1-800-MEDICARE helpline, the Medicare Prescription Drug Plan Finder web tool on www.medicare.gov, personalized assistance via the State Health Insurance Assistance Program (SHIP) counseling program, and local enrollment events. All of these initiatives are rooted in a foundation of continuous quality improvement that involves identifying the information that needs to be conveyed, using consumer research to determine the most effective messages and vehicles, preparing materials accordingly, and measuring material effectiveness. This thorough, comprehensive and careful process ensures that all of our educational materials are as accurate, clear and informative as possible. Handbook and other written materials: CMS has produced and disseminated an unprecedented number of written communication products on Medicare prescription drug coverage. These materials meet their intended goal of quickly and easily providing action-oriented information on a variety of topics related to Part D. Written materials exist in the form of booklets, brochures, fact sheets and letters. Some key communication products are available in Braille and audiotape, and many have been translated into alternate languages to increase accessibility to information. Medicare & You Handbook: * The Handbook is an important information source for all Medicare beneficiaries on the Medicare program and their medical and drug coverage. Each year, all beneficiary households receive a copy and we know from our consumer research that beneficiaries keep it to use as a reference source. Our customer surveys of beneficiaries who read the Medicare & You 2006 Handbook, conducted in January-February 2006, showed that 72 percent were "very" or "somewhat satisfied" with the Handbook. * For 2006, we updated the Medicare & You Handbook to reflect information on the new Medicare prescription drug coverage by including a summary of the new coverage and information on how it can help Medicare beneficiaries in different situations. In addition, we reorganized the Handbook to help Medicare beneficiaries decide whether and how to choose among alternative plans. For example, a prominently highlighted box on the inside cover of the Handbook serves to remind beneficiaries that they need to make a choice about prescription drug coverage for 2006. Beneficiaries are directed to the specific Handbook section that provides more details on how to select a prescription drug plan. * The Medicare & You Handbook has been designed to assist beneficiaries in deciding how to choose a plan based on cost, coverage, convenience and peace of mind both now and in the future. In addition to general information, the Handbook includes information for beneficiaries based upon their current prescription drug coverage status. * CMS uses a series of steps before, during, and after printing the Medicare & You Handbook to ensure accuracy. Some steps may be combined or omitted as appropriate for other targeted publications and deadlines for publication. * Before printing the Handbook, CMS conducts multiple rounds of internal review by program staff experts in components throughout CMS. CMS also subjects the Handbook to expert review by external organizations. CMS solicits comments from an extensive list of advocacy groups, academic partners, industry trade organizations, Congressional staff, and other interested stakeholders. CMS writers/editors do the final proofing. Finally, the CMS Office of External Affairs/Graphics reviews the Handbook. CMS provides a final desktop publishing troubleshooting check to ensure that materials include only the files (such as logos, photos, and fonts) that CMS has legal rights to use. * During the printing process, CMS reviews printer "blueline" copies. CMS reviews first proofs from the printer to ensure the publication layout is accurate. CMS has an opportunity to correct printer errors (generally something that was altered in the transfer from electronic file to print plate) or make author's alterations (errors previously missed) before printing begins. Specially trained CMS and/or GPO staff go on-site to the print contractor to conduct quality assurance inspections of the publication, checking for errors as the Handbook is being printed. * After printing, CMS carefully monitors and investigates reports of errors in publications, including tracking related feedback from representatives at 1-800-MEDICARE. CMS corrects publications, as needed, and issues updated electronic files and/or errata sheets to accompany printed publications. * CMS is very concerned about the readability of our publications. We have to balance the often competing goals of explaining technical information about Medicare coverage in clear and simple language while ensuring its accuracy. We go to great lengths to explain terms that beneficiaries need to understand to address readability concerns. For example, all publications include phone numbers and web sites, in case people need more information. CMS has found that this contact information is nearly universally identified and understood by beneficiaries. * GAO noted readability test score findings as evidence that our written documents lacked clarity. CMS doesn't routinely perform readability tests like the Fry, SMOG, FOG or Flesch-Kincaid on completed publications. Our writers may use these tests as tools during the drafting process to provide a rough estimate of the readability level and identify elements such as passive sentences, which can be readily improved. These kinds of tests rely largely on counting syllables per word, words per sentence, and sentences per paragraph to determine a "grade level" readability score which we do not find to be a useful parameter in gauging "readability" of Medicare materials because there are terms that may be unfamiliar to the Medicare population. As such, we go to great lengths to explain concepts that may be readily understood. For example, "Medicare," "deductible," "formulary" and "prescription" are all multi-syllabic words that would inflate scores in these types of reading tests. However, they are terms for which there are few or no simpler substitutes. People with Medicare (and in health insurance generally), commonly recognize most of these terms. Where they don't, as with "formulary," we use them with careful explanation in context, which also inflates the readability test scores by adding words to the sentence. Such tests would not account for this phenomenon and it is not usually accounted for by omitting certain words in the scoring process given how many terms for which we provide detailed explanations. * These readability test scores are somewhat misleading and incomplete as a measure of the ease or difficulty of materials. - Plain language and literary experts like Roger Shuy and the Georgetown University Round Table on Language and Linguistics, the Social Security Administration, the Maine AHEC Health Literacy Center, the Delegates Assembly of the International Reading Association, and the U.S. Securities and Exchange Commission state that individual's tested literacy level and their ability to read and understand materials written at the corresponding grade level rarely match. - Test scores don't take into account other criteria that improve clarity of message, like navigational cues and graphic elements. - It's challenging to account for multi-syllabic terms like "Medicare" or "prescription" that are widely-understood and/or for which there are no simpler alternatives. - When appropriate, our publications provide a glossary to help beneficiaries understand words that may be new to them. The Medicare & You handbook contains such a glossary, as do our other large booklets. However, glossaries would mitigate the goals of brief fact sheets and letters, and therefore, for these types of materials, every effort is made to define difficult terms in context, which can inflate standard readability test scores. * As an additional measure of clarity, GAO states they used 60 "commonly recognized guidelines" to evaluate our publications. It is difficult to sufficiently comment on the findings without knowing these 60 criteria, beyond the handful of examples in the report. However, it is important to note that to the best of our knowledge, these guidelines were compiled from multiple sources for the purposes of this evaluation and are not commonly recognized as a set. We look forward to the opportunity to review these guidelines and their relationship to our publications in the future, to assess where improvements might be made. * To evaluate and improve the usability of Medicare publications, CMS hires contractors to conduct research with beneficiaries, caregivers, and other people who help beneficiaries. CMS uses focus groups to help us understand what information is important to beneficiaries. We also conduct cognitive interviews to test how well beneficiaries understand the content in our draft publications. Our drafts are revised based on the feedback that we receive. * Consumer testing for the Handbook dates back to 1998. Over the years, we have qualitatively tested the Handbook with over 1000 aged and disabled beneficiaries, caregivers, and Medicare counselors. Each year, the basic testing is conducted in two rounds to allow for iterative improvements. Lessons learned from year to year are applied to each new version of the book. * Multiple methods are used to test the book. The most heavily relied on method is cognitive interviews where participants are given tasks "cold," that is without prior preparation. We've also relied on triads and focus groups which allow participants to generate ideas on how to improve the book. * We also conduct "diary groups" where beneficiaries are asked to make comments on the book as they read through it at home and are then brought in for focus groups. Tested content developed for particular publications is also used in other publications as-appropriate. This overlap ensures consistency across CMS publications. * Information collected from beneficiaries earlier this year indicated that 61 percent of respondents said the Medicare & You Handbook was "very easy" or "somewhat easy" to understand. * CMS elicited feedback from more than 300 beneficiaries on Part D materials. The Medicare & You handbook language was tested by a testing contractor, BearingPoint, with over 150 beneficiaries. This testing helped us simplify our language and explain concepts more clearly. * GAO used similar testing methods on a smaller scale to evaluate the clarity of our written materials. We are interested in reviewing the details of the 18 tasks that were used the interviews conducted with beneficiaries and beneficiary advisors, and understanding which tasks correlated to which tested products. GAO's report provides no details on the tasks that respondents completed successfully, and describes only three tasks that were difficult. These three indicate that the purpose and expectations of these publications may have been overlooked. The primary goal of our written communications in this phase was awareness - to make beneficiaries aware of the new coverage, aware that they needed to take some action, and aware of the resources available to help them make decisions. None of these publications were intended to independently lead a reader through such complex activities as computing projected out-of-pocket costs. Other feedback on our publications shows they are successful in meeting their intended goals. * The National Association of Government Communicators critiqued the Medicare & You 2005 Handbook for the 2004 Blue Pencil Competition. The handbook received positive feedback in the judges' ratings. The judges rated the handbook in categories such as writing, editing, purpose, design, printing, cost effectiveness, and dissemination. - The judges strongly agreed that the writing was clear, concise, and appropriate for its intended audience. - One judge wrote, "Given the complexity of this subject, the writing is extremely clear and easy to understand. Technical terms are well explained, and needed information is easy to locate." - In the area of design, another judge commented that, "Choice of font, typeface, and size; leading; and margins made the book attractive, while ensuring accessibility for users (especially seniors). Use of blue headings and other design elements contributed to ease of use, as well." - In the category of purpose, the judges strongly agreed that the purpose of the handbook is clear and that the handbook gets its message across with well-supported topics. As an overall final comment, a judge wrote, "This entry is very well suited to its purpose and audience." * CMS began preparations for the 2007 Medicare & You Handbook in late December 2005. To date, staff and leadership have held input meetings with key advocates and stakeholders, tested early draft revisions with beneficiaries, established a firm project plan, and instituted additional quality assurance and proofing processes. The Handbook is currently on schedule for its required mailing in the fall of this year, with a comprehensive external review process ending this week and extensive consumer testing scheduled in mid-May. 1-800-MEDICARE: It is a top priority at CMS to ensure that beneficiaries have timely access to accurate information and receive satisfactory service when contacting 1-800-MEDICARE. * Between 2004 and the beginning of the open enrollment period, CMS conducted numerous activities to prepare for the prescription drug benefit, including the development of a comprehensive training curriculum on the prescription drug benefit and the Plan Finder tool for Customer Service Representatives (CSRs). Since November 15, 2005, CMS has made continuous updates to scripts and reference materials for CSRs to ensure they are able to communicate accurate information to beneficiaries and people calling on behalf of beneficiaries. * CMS's quality monitoring program has found that in 2006, calls to 1- 800-MEDICARE have been accurate 93 percent of the time. This quality monitoring program is conducted by contractors who run the call centers. CMS monitors at least 4 calls per month for each of our thousands of CSRs to identify improvement and training opportunities. * These are not just mystery shopping calls, which are limited to topics chosen by researchers, but actual calls which are representative of the information Medicare beneficiaries want to know. To ensure reliability and accuracy, all monitors score a sample of calls on a weekly basis and meet to review their approaches. The data is analyzed constantly and is used to take immediate corrective action. This work is overseen by a team within CMS dedicated to the quality of the 1-800- MEDICARE call centers. * Examples of topics receiving the highest volume of inquiries at our call centers include: - How to enroll in a plan to obtain prescription drug coverage - Complaints about drug coverage: - How to apply for the limited-income subsidy: * Since the beginning of the new prescription drug benefit, CMS has taken many steps to help beneficiaries get the information they need to select a drug plan. For example, CMS acquired additional infrastructure including telephone lines and workstations at call center sites. * CMS increased the number of customer service representatives (CSRs) from 3,000 in June 2004 to as many as 7,800 to handle beneficiary calls with minimal wait times. * On average, from November 15, 2005 to April 12, 2006, callers have experienced wait times of less than 2 minutes, with longer waits sometimes occurring during peak call periods. Call volume to 1-800- MEDICARE peaked around 400,000 calls per day in mid-November when enrollment began, and again in early to mid-January. Currently, call volume reaches 200,000 calls per day on the highest volume day and levels out around 150,000 per day during the remainder of the week. Call volumes have continued to increase slightly since then. * CMS recognizes that not all beneficiaries are able to use, or have access to, the internet, which is the platform for the useful Medicare Prescription Drug Plan Finder tool. As part of our outreach and communication efforts, CMS trained additional staff exclusively on the use of the Medicare Prescription Drug Plan Finder tool so that they could be dedicated to answering calls only about the prescription drug benefit and available plan options. * We expanded responsibilities and provided additional training for some CSRs and advanced training for others. We required CSRs to take written exams and test calls for certification before allowing them to take live calls. All CSRs have one week of classroom training followed by two or three additional days of practice calls, simulation, quality monitoring, and follow-up coaching to ensure peak performance. Finally, we monitored newly-trained CSRs and those who would benefit from additional coaching at a higher level. * This year, CMS implemented a 1-800 MEDICARE caller satisfaction survey conducted by Pacific Consulting Group, an independent contractor. This survey provides 1) satisfaction tracking over time and 2) an early warning system that can point to potential service problems. Improvements can then be implemented relatively quickly to enhance caller satisfaction. These CMS customer satisfaction surveys indicate that the bulk of callers who interact with our CSRs, 87 percent are satisfied with their experience. They are particularly pleased with how courteous and patient the CSRs are (rated at 97 percent). These responses came not only from people with Medicare, but also friends or relatives calling on their behalf, who made up 34 percent of callers during March 2006. * Currently, 500 surveys are conducted each week with 400 callers who spoke with CSRs and 100 callers who used the Interactive Voice Response System. The data below depict results from weekly calls for those callers that spoke to a CSR. The results show the percentage of respondents in the weeks January 16th, February 27th and March 6th that strongly or somewhat agree with the statements listed below. Table: Survey Metric(% agree-strongly or somewhat to the following statements): CSR was helpful; Week of January 16th: 84%; Week of February 27th: 89%; Week of March 6th: 88%. Survey Metric(% agree-strongly or somewhat to the following statements): CSR understood issue or concern; Week of January 16th: 83%; Week of February 27th: 86%; Week of March 6th: 88%. Survey Metric(% agree-strongly or somewhat to the following statements): CSR explained things to me in a way that I could understand; Week of January 16th: 83%; Week of February 27th: 86%; Week of March 6th: 84%. Survey Metric(% agree-strongly or somewhat to the following statements): I received all of the information I needed; Week of January 16th: 67%; Week of February 27th: 72%; Week of March 6th: 73%. Survey Metric(% agree-strongly or somewhat to the following statements): The CSR was knowledgeable; Week of January 16th: 81%; Week of February 27th: 86%; Week of March 6th: 85%. Survey Metric(% agree-strongly or somewhat to the following statements): I received information specific to my issue; Week of January 16th: 75%; Week of February 27th: 80%; Week of March 6th: 80%. Survey Metric(% agree-strongly or somewhat to the following statements): Overall I am satisfied; Week of January 16th: 79%; Week of February 27th: 84%; Week of March 6th: 85%. [End of table] * Pharmacists are a key partner in the implementation of the Medicare prescription drug benefit. To ensure that pharmacists have access to the information they need to assist beneficiaries at the pharmacy counter, CMS developed a dedicated pharmacist 1-866 telephone line. Incoming calls through the dedicated pharmacist line are routed to the head of the queue at the 1-800 MEDICARE number, wait times are substantially lower than the overall average for beneficiaries and other individuals calling the 1-800-MEDICARE line. This helps to relieve any burden on pharmacists, and also ensures that pharmacists are able to assist beneficiaries immediately at the pharmacy counter. * CMS is well-prepared to handle increased call-volume that may occur before the May 15tH enrollment deadline. We have increased the number of CSRs from 3,000 in June of 2004 to 6,000 CSRs for May enrollments. We have also acquired additional infrastructure including telephone lines and workstations at call center sites. We have refined our CSR scripts by reducing redundant information, indexing scripts for quick access, and including probing questions to help the CSRs better identify callers' concerns. * Despite our efforts, some beneficiaries will wait until the deadline is near, but our top priority is to encourage people to enroll now and avoid the rush. Medicare.gov: * To ensure that the new Plan Finder tool was well-designed and easily used by beneficiaries and other individuals, CMS worked with a professional website development contractor, CGI Federal and a subcontractor, Navigation Arts. * As the Medicare Prescription Drug Plan Finder was being designed, CMS engaged in multiple rounds of consumer testing to ensure its usefulness and simplicity. CMS conducted three rounds of in-depth interviews with Medicare beneficiaries to obtain feedback as drafts of the tool were developed throughout 2005. Final interviews that focused on messages tailored specifically for beneficiaries based on their insurance information were conducted in September 2005. CMS conducts ongoing consumer research to continue to improve understandability and usability. * CMS also conducts thorough and ongoing analyses of possible outliers in data, including the Medicare Prescription Drug Plan Finder plan pricing data, pharmacy network, mismatched formulary identifiers (NDC codes), and other missing formulary data. If problems are found with a plan's data, information on the plan will be suppressed from the website until CMS works with the plan to correct its information and properly display it. * We are proud to say that CMS has received a number of awards for its website from independent organizations. These awards include the "eHealthcare Leadership Award" at the Ninth Annual Internet Conference, the "2005 Pioneer Award" at the E-Gov Institute and Federal Computer Week, and the "Independent Technology Supporting Service to Our Country" award at the Eighth Annual Technology Gala to benefit Juvenile Diabetes. * We believe that the website has been extremely successful in providing beneficiaries, their caregivers and CMS partners with clear, accurate and timely information to help them enroll in drug plans. In fact, CSRs at 1-800 MEDICARE have access to the Plan Finder to help beneficiaries find the information they need about choosing a plan, enrolling in a plan, or other issues related to accessing their prescription drug coverage. The Plan Finder also has been a critical tool for SHIPS and other partners, such as the ABC Coalition and Medicare Today, to use when conducting outreach to beneficiaries. * Results from a web-based customer satisfaction survey conducted by MSInteractive, a subsidiary of Market Strategies that specializes in web-site satisfaction research, were very positive. This research, conducted in December 2005, focused only on the prescription drug plan finder tool. * The survey indicated that content, interactivity, and navigability have the greatest impact on satisfaction. During development of the tool, CMS contracted with a web design firm to leverage their expertise on these impacts. CMS continues to focus on these areas in future enhancements and updates. * The site's "appearance" and "privacy" scored highly, but had no impact on overall satisfaction. - 66 percent of those who enrolled were either "somewhat" or "strongly satisfied" with the tool. - 80 percent of those who enrolled would recommend the tool to a friend. - 70 percent of users agreed with this statement, "I know more about the Medicare Prescription Drug Plans now that I've used this site." - Regular internet users had higher ratings of the site. * In January and February 2006, Abt conducted a telephone survey of a random sample of beneficiaries and found that: - 14 percent of respondents used the www.medicare.gov website to get information about Medicare; - 60 percent said it was "very easy" or "somewhat easy" to understand the information from www.medicare.gov; - Beneficiaries who rated their satisfaction with the information received from medicare.gov as "very/somewhat" satisfied outnumbered the "dissatisfied" beneficiaries 71 percent to 19 percent. Seven percent of beneficiaries were neither "satisfied nor dissatisfied." * Overall, the drug plan finder element of the website has received 164.6 million page views between November 15, 2005 and April 26, 2006. We do not have a way to differentiate whether those hits were from beneficiaries or their caregivers. * To date, 3 million beneficiaries have enrolled in prescription drug plans using the Plan Finder. That indicates that at least that many people were satisfied enough with the information they received to undertake the most important step of enrolling in a drug plan. * The Frequently Asked Questions (FAQ) section of www.medicare.gov has been accessed more than one million times since January 1, 2006. CMS has also responded to more than 19,000 emails received through the FAQ section, with 93% of them being resolved satisfactorily in the first response. State Health Insurance Assistance Programs (SHIPS): * While the SHIPs play a significant role in beneficiary counseling and education on Part D, CMS has also created a national grassroots network of more than 24,000 partners and 140 coalitions that rely on traditional tools to help them provide personalized counseling to Medicare beneficiaries every day. * The network CMS built is diverse and committed, with members from every sector, including advocacy groups, government agencies, service clubs, faith-based organizations, benefits counselors, trained volunteers and healthcare professionals such as doctors and pharmacists. * This extensive, grassroots-level partnership is truly unprecedented for the Medicare program. It's reaching out to people with Medicare all over the country... "where they live, work, play, and pray." This approach has helped personalize Medicare in every corner of the country. *Preliminary data from the State Health Insurance Assistance Programs (SHIPs) shows that individual in-person and telephone contacts, presentations and meetings reached a total of 4.5 million clients, compared to 2.5 million in the previous grant period. Other Selected Activities: * The Mobile Office Tour has traveled 500,000 miles since last fall and approximately half of the territory covered and events have been in rural areas, in an attempt to reach out to a variety of beneficiaries and partners at the local level. We knew we would have to develop a grassroots capacity and local networks to supplement the CMS regional structure to provide the necessary education and enrollment assistance at the community level. This would involve reaching out, not just to our traditional partners such as the SHIPS, but to all the groups and organizations that have contact with our beneficiaries on a daily basis "where they work, where they play, and where they pray." We needed to involve individuals and institutions: family members and friends; current and former employers; churches and synagogues; financial advisors and community centers, to name but a few. * CMS is reaching out directly to beneficiaries through an extensive paid and earned media campaign focusing on press and radio, both of which are highly localized in informing beneficiaries of special events in their neighborhoods. * As we approach May 15, many members of the Cabinet whose agencies have helped build awareness of the prescription drug benefit through their own programs have joined efforts with CMS, including the United States Department of Agriculture, Department of Commerce, Department of Labor and Housing and Urban Development. * To minimize a possible last minute rush to enroll, CMS is making a monumental effort to enroll beneficiaries well before the May 15tH deadline. In the past month, there have been 1,000 events per week across the country to provide beneficiaries with personalized help so they understand the prescription drug coverage options available to them and they can enroll in a plan. In our enrollment efforts, we are targeting beneficiaries who may qualify for the low-income subsidy and beneficiaries who live in rural areas. Our enrollment events are fully coordinated with the Social Security Administration (SSA) to assist beneficiaries in applying for extra help, as well as to help them enroll in a plan. [End of section] Appendix IV: Agency Comments and Our Evaluation: We received written comments on a draft of our report from CMS (see app. III). CMS said that it did not believe our findings presented a complete and accurate picture of its Part D communications activities. CMS discussed several concerns regarding our findings on its written documents and the 1-800-MEDICARE help line. However, CMS did not disagree with our findings regarding the Medicare Web site or the role of SHIPs. CMS also said that it supports the goals of our recommendations and is already taking steps to implement them, such as continually enhancing and refining its Web-based tools. CMS discussed concerns regarding the completeness and accuracy of our findings in terms of activities we did not examine, as well as those we did. CMS stated that our findings were not complete because our report did not examine all of the agency's efforts to educate Medicare beneficiaries and specifically mentioned that we did not examine the broad array of communication tools it has made available, including the development of its network of grassroots partners throughout the country. We recognize that CMS has taken advantage of many vehicles to communicate with beneficiaries and their advisers. However, we focused our work on the four specific mechanisms that we believed would have the greatest impact on beneficiaries--written materials, the 1-800- MEDICARE help line, the Medicare Web site, and the SHIPs. In addition, CMS stated that our report is based on information from January and February 2006, and that it has undertaken a number of activities since then to address the problems we identified. Although we appreciate CMS's efforts to improve its Part D communications to beneficiaries on an ongoing basis, we believe it is unlikely that the problems we identified in our report could have been corrected yet given their nature and scope. CMS raised two concerns with our examination of a sample of written materials. First, it criticized our use of readability tests to assess the clarity of the six sample documents we reviewed. For example, CMS said that common multisyllabic words would inappropriately inflate the reading level. However, we found that reading levels remained high after adjusting for 26 multisyllabic words a Medicare beneficiary would encounter, such as Social Security Administration. CMS also pointed out that some experts find such assessments to be misleading. Because we recognize that there is some controversy surrounding the use of reading levels, we included two additional assessments to supplement this readability analysis--the assessment of design and organization of the sample documents based on 60 commonly recognized communications guidelines and an examination of the usability of six sample documents, involving 11 beneficiaries and 5 advisers. Second, CMS expressed concern about our examination of the usability of the six sample documents. The participating beneficiaries and advisers were called on to perform 18 specified tasks, after reading the selected materials, including a section of the Medicare & You handbook. CMS suggested that the task asking beneficiaries and advisers to calculate their out-of-pocket drug costs was inappropriate because there are many other tools that can be used to more effectively compare costs. We do not disagree with CMS that there are a number of ways beneficiaries may complete this calculation; however, we nonetheless believe that it is important that beneficiaries be able to complete this task on the basis of reading Medicare & You, which, as CMS points out, is widely disseminated to beneficiaries, reaching all beneficiary households each year. In addition, CMS noted that it was not able to examine our detailed methodology regarding the clarity of written materials--including assessments performed by one of our contractors concerning readability and document design and organization. We plan to share this information with CMS. Finally, CMS took issue with one aspect of our evaluation of the 1-800- MEDICARE help line. Specifically, CMS said the 41 percent accuracy rate associated with one of the five questions we asked was misleading, because, according to CMS, we failed to analyze 35 of the 100 responses. However, we disagree. This question addressed which drug plan would cost the least for a beneficiary with certain specified prescription drug needs. We analyzed these 35 responses to this question and found the responses to be inappropriate. The CSRs would not provide us with the information we were seeking because we did not supply personal identifying information, such as the beneficiary's Medicare number or date of birth. We considered such responses inappropriate because the CSRs could have answered this question without personal identifying information by using CMS's Web-based prescription drug plan finder tool. Although CMS said that it has emphasized to CSRs, through training and broadcast messages, that it is permissible to provide the information we requested without requiring information that would personally identify a beneficiary, in these 35 instances, the CSR simply told us that our question could not be answered. CMS also said that the bulk of these inappropriate responses were related to our request that the CSR use only brand-name drugs. This is incorrect--none of these 35 responses were considered incorrect or inappropriate because of a request that the CSR use only brand-name drugs--as that was not part of our question. (290546): [End of section] FOOTNOTES: [1] Pub. L. No. 108-173, � 101, 117 Stat. 2066, 2071-2152 (to be codified at 42 U.S.C. �� 1395w-101--1395w-152). The MMA redesignated the previous part D of title XVIII of the Social Security Act as part E and inserted a new part D after part C. [2] In December 2004, we reported on the information being provided to beneficiaries through the Medicare help line on eligibility, enrollment, and benefits. See GAO, Medicare: Accuracy of Responses from the 1-800-MEDICARE Help Line Should Be Improved, GAO-05-130 (Washington, D.C.: Dec. 8, 2004). [3] The Medicare Web site is www.medicare.gov. [4] GAO, Medicare: Communications to Beneficiaries on the Prescription Drug Benefit Could Be Improved, GAO-06-654 (Washington, D.C.: May 3, 2006). [5] The percentages related to the responses we received to our 500 calls exceed 100 percent due to rounding. [6] Dual-eligible beneficiaries are Medicare beneficiaries who are also eligible for Medicaid--the federal-state health program for low-income individuals--and receive full Medicaid benefits for services not covered by Medicare. [7] Medicare Advantage replaced the Medicare+Choice managed care program and expanded the availability of private health plan options to Medicare beneficiaries. Medigap policies provide supplemental health coverage sold by private insurers to help pay for Medicare cost-sharing requirements, as well as for some services not provided by Medicare. [8] A formulary is a list of prescription drugs covered by a health plan. [9] The percentages related to the responses we received to our 500 calls exceed 100 percent because of rounding. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: