Clinical Labs

CMS and Survey Organization Oversight Is Not Sufficient to Ensure Lab Quality Gao ID: GAO-06-879T June 27, 2006

Today's hearing focuses on oversight of clinical labs. The Clinical Laboratory Improvement Amendments of 1988 (CLIA) strengthened quality requirements for labs that perform tests to diagnose or treat disease. About 36,000 labs that perform certain complex tests must be surveyed biennially by a state survey agency, a state CLIA-exempt program, or a private accrediting organization. CMS oversees implementation of CLIA requirements, which includes determining the CLIA equivalency of the inspection requirements used by exempt states and accrediting organizations. GAO was asked to discuss (1) the quality of lab testing and (2) the adequacy of CLIA oversight. To examine these issues, GAO analyzed data on lab performance and reviewed the procedures used by CMS and survey organizations to implement CLIA and oversee lab performance. This testimony is based on the GAO report, Clinical Lab Quality: CMS and Survey Organization Oversight Should Be Strengthened, GAO-06-416 (June 16, 2006).

In summary, insufficient data exist to identify the extent of serious quality problems at labs. When CMS implemented revised CLIA survey requirements in 2004, it modified historical state survey agency findings and, as a result, data prior to 2004 no longer reflect key survey requirements in effect at the time of those surveys. The limited data available suggest that state survey agency inspections do not identify all serious deficiencies. In addition, the lack of a straightforward method to link similar requirements across survey organizations makes it virtually impossible to assess lab quality in a standardized manner. Furthermore, CMS does not effectively use available data, such as the proportion of labs with serious deficiencies or proficiency testing results, to monitor lab quality. Proficiency testing is an objective measurement of a lab's ability to consistently produce accurate test results. GAO's analysis of proficiency testing data suggests that lab quality may not have improved at hospital labs in recent years. Oversight of clinical lab quality is not adequate to ensure that labs are meeting CLIA requirements. Weaknesses in five areas mask real and potential quality problems at labs. First, the balance struck between the CLIA program's educational and regulatory goals is sometimes inappropriately skewed toward education, which may result in understatement of survey findings. For example, even though the initial test failure rates were high, CMS instructed state survey agencies not to cite deficiencies during the first two years of required Pap smear proficiency testing to allow labs and their staff to become familiar with the program. Second, the manner in which one accrediting organization structures its survey teams raised concerns about appropriate levels of training and the appearance of a conflict of interest that could undermine the integrity of the survey process. Third, concerns about anonymity and lab workers' lack of familiarity with how to file a complaint suggests that some quality problems are not being reported. Fourth, based on the large number of labs with proposed sanctions from 1998 through 2004 that were never imposed--even for labs with the same serious deficiencies on consecutive surveys--it is unclear how effective CMS's enforcement process is at motivating labs to consistently comply with CLIA requirements. Finally, CMS is not meeting its requirement to determine in a timely manner the continued equivalency of accrediting organization and exempt-state program inspection requirements and processes, nor has the agency reviewed changes to accrediting organization and exempt-state program inspection requirements before implementation.

GAO-06-879T, Clinical Labs: CMS and Survey Organization Oversight Is Not Sufficient to Ensure Lab Quality This is the accessible text file for GAO report number GAO-06-879T entitled 'Clinical Labs: CMS and Survey Organization Oversight is Not Sufficient to Ensure Lab Quality' which was released on June 27, 2006. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Testimony: Before the Subcommittee on Criminal Justice, Drug Policy and Human Resources, Committee on Government Reform, House of Representatives: United States Government Accountability Office: GAO: For Release on Delivery Expected at 2:00 p.m. EDT: Tuesday, June 27, 2006: Clinical Labs: CMS and Survey Organization Oversight Is Not Sufficient to Ensure Lab Quality: Statement of Leslie G. Aronovitz: Director, Health Care: GAO-06-879T: GAO Highlights: Highlights of GAO-06-879T, a testimony before the Subcommittee on Criminal Justice, Drug Policy and Human Resources, Committee on Government Reform, House of Representatives Why GAO Did This Study: Today‘s hearing focuses on oversight of clinical labs. The Clinical Laboratory Improvement Amendments of 1988 (CLIA) strengthened quality requirements for labs that perform tests to diagnose or treat disease. About 36,000 labs that perform certain complex tests must be surveyed biennially by a state survey agency, a state CLIA-exempt program, or a private accrediting organization. CMS oversees implementation of CLIA requirements, which includes determining the CLIA equivalency of the inspection requirements used by exempt states and accrediting organizations. GAO was asked to discuss (1) the quality of lab testing and (2) the adequacy of CLIA oversight. To examine these issues, GAO analyzed data on lab performance and reviewed the procedures used by CMS and survey organizations to implement CLIA and oversee lab performance. This testimony is based on the GAO report, Clinical Lab Quality: CMS and Survey Organization Oversight Should Be Strengthened, GAO-06-416 (June 16, 2006). What GAO Found: In summary, insufficient data exist to identify the extent of serious quality problems at labs. When CMS implemented revised CLIA survey requirements in 2004, it modified historical state survey agency findings and, as a result, data prior to 2004 no longer reflect key survey requirements in effect at the time of those surveys. The limited data available suggest that state survey agency inspections do not identify all serious deficiencies. In addition, the lack of a straightforward method to link similar requirements across survey organizations makes it virtually impossible to assess lab quality in a standardized manner. Furthermore, CMS does not effectively use available data, such as the proportion of labs with serious deficiencies or proficiency testing results, to monitor lab quality. Proficiency testing is an objective measurement of a lab‘s ability to consistently produce accurate test results. GAO‘s analysis of proficiency testing data suggests that lab quality may not have improved at hospital labs in recent years. Oversight of clinical lab quality is not adequate to ensure that labs are meeting CLIA requirements. Weaknesses in five areas mask real and potential quality problems at labs. First, the balance struck between the CLIA program‘s educational and regulatory goals is sometimes inappropriately skewed toward education, which may result in understatement of survey findings. For example, even though the initial test failure rates were high, CMS instructed state survey agencies not to cite deficiencies during the first two years of required Pap smear proficiency testing, to allow labs and their staff to become familiar with the program. Second, the manner in which one accrediting organization structures its survey teams raised concerns about appropriate levels of training and the appearance of a conflict of interest that could undermine the integrity of the survey process. Third, concerns about anonymity and lab workers‘ lack of familiarity with how to file a complaint suggests that some quality problems are not being reported. Fourth, based on the large number of labs with proposed sanctions from 1998 through 2004 that were never imposed”even for labs with the same serious deficiencies on consecutive surveys”it is unclear how effective CMS‘s enforcement process is at motivating labs to consistently comply with CLIA requirements. Finally, CMS is not meeting its requirement to determine in a timely manner the continued equivalency of accrediting organization and exempt-state program inspection requirements and processes, nor has the agency reviewed changes to accrediting organization and exempt-state program inspection requirements before implementation. What GAO Recommends: In a report released today, GAO made numerous recommendations to the CMS Administrator that would strengthen program oversight. CMS noted that the report provided insights into areas where it can improve oversight and said that it would implement 11 of GAO‘s 13 recommendations. [Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-06-879T]. To view the full product, including the scope and methodology, click on the link above. For more information, contact Leslie G. Aronovitz at (312) 220-7600 or aronovitzl@gao.gov. [End of Section] Mr. Chairman and Members of the Subcommittee: I am pleased to be here today as you discuss oversight of the quality of testing performed by the nation's clinical laboratories. Clinical lab tests are one of the most frequently billed Medicare procedures and, according to the American Clinical Laboratory Association, affect an estimated 70 percent of medical decisions.[Footnote 1] Ensuring accurate and reliable lab test results is critical because erroneous results may lead to improper treatment, unnecessary mental and physical anguish for patients, and higher health care costs. Concerns about the quality of lab testing resulted in enactment of the Clinical Laboratory Improvement Amendments of 1988 (CLIA).[Footnote 2] In recent years, despite CLIA, lab quality problems in several states have raised questions about the adequacy of lab oversight. The Centers for Medicare & Medicaid Services (CMS) is responsible for overseeing compliance with clinical lab testing requirements. As of December 2005, there were approximately 193,000 labs nationwide ranging from very small physician office labs that conduct fewer than 2,000 tests annually, to hospital labs that conduct millions of tests each year. Most clinical labs regulated under CLIA must obtain a certificate from CMS, but only about 19 percent--those that conduct moderate-to high-complexity tests--undergo biennial inspections, which are also referred to as surveys. During the surveys, inspectors assess lab compliance with mandated personnel and testing standards. In addition, surveyed labs must participate in proficiency testing, a program that requires them to test samples with unknown characteristics that are then graded by an external party. Labs with serious deficiencies may be sanctioned. Labs may choose to be surveyed by (1) their state survey agency, under contract with CMS; (2) their state CLIA-exempt program for labs in New York and Washington; or (3) one of six private accrediting organizations. State survey agency inspections use CLIA requirements that are intended to help ensure valid and reliable lab tests; the two state CLIA-exempt programs and six accrediting organizations survey labs using their own requirements that CMS has determined to be at least equivalent to CLIA's. Each survey organization is also responsible for investigating complaints about lab quality.[Footnote 3] My remarks today will focus on (1) the quality of lab testing and (2) the effectiveness of CMS and survey organization oversight of the CLIA program. My testimony summarizes the findings of a report we released today that examines these issues in more detail and includes numerous recommendations to the CMS Administrator for improving the quality of laboratory testing through closer oversight of clinical labs and the administration of CLIA standards. To determine what is known about the quality of lab testing, we analyzed data on serious deficiencies identified during surveys by state survey agencies using CMS's On-Line Survey, Certification, and Reporting system (OSCAR). We requested comparable data on serious deficiencies from state CLIA-exempt programs and the three largest accrediting organizations--the College of American Pathologists (CAP), COLA, and the Joint Commission on Accreditation of Healthcare Organizations (JCAHO)--which together survey about 97 percent of accredited labs.[Footnote 4] We also analyzed proficiency testing data- -another indicator of a lab's ability to produce accurate test results. To evaluate the effectiveness of CLIA program oversight, we reviewed the processes used to ensure the quality of clinical lab testing and analyzed available data related to these issues. Based on our review and discussions with CMS and survey organization officials, we focused on several key issues: (1) the balance struck between the regulatory and educational goals of lab surveys, (2) the implications of CAP's use of volunteer surveyors from neighboring labs to conduct inspections, (3) how survey organizations facilitate the filing of complaints, (4) the use of sanctions to encourage compliance, (5) CMS's process for determining that the standards used by state CLIA-exempt programs and accrediting organizations are at least equivalent to those of CLIA, and (6) the results of validation reviews that are intended to assess the adequacy of inspections by survey organizations. In addition, we interviewed officials from CMS, three CMS regional offices,[Footnote 5] 10 state survey agencies,[Footnote 6] the New York and Washington CLIA- exempt programs, and the three largest accrediting organizations. We conducted our work from January 2005 through May 2006 in accordance with generally accepted government auditing standards. In summary, insufficient data exist to identify the extent of serious quality problems at labs. When CMS implemented revised CLIA survey requirements in 2004, it modified historical state survey agency findings stored in its OSCAR database and, as a result, data prior to 2004 no longer reflect key survey requirements in effect at the time of those surveys. The limited data available suggest that state survey agency inspections do not identify all serious deficiencies. In addition, the lack of a straightforward method to link similar requirements across survey organizations makes it virtually impossible to assess lab quality in a standardized manner, such as identifying the proportion of labs with condition-level deficiencies, which indicate serious or systemic quality problems. Furthermore, CMS does not effectively use available data--such as the proportion of labs with serious deficiencies or proficiency testing results--to monitor lab quality. Proficiency testing is the one available data source that can be used to uniformly compare lab quality across survey organizations. Although CMS noted that proficiency testing trend data show a decrease in failures for labs as a whole, we found that the data suggest that quality may not have improved at hospital labs in recent years. Despite the importance of, and the statutory requirement for, quarterly proficiency testing, CMS requires proficiency testing for almost all laboratory tests only three times a year. Regarding oversight of clinical lab quality, we found that it is inadequate to ensure that labs are meeting CLIA requirements. Weaknesses in six areas mask real and potential quality problems at labs. First, the balance struck between the CLIA program's educational and regulatory goals is sometimes inappropriately skewed toward education, which may result in understatement of survey findings. In one instance, CMS instructed state survey agencies not to cite deficiencies for Pap smear proficiency test results during the first two years of required testing, to allow labs and their staff to become familiar with the program. Second, the way one accrediting organization structures its volunteer survey teams raised concerns about appropriate levels of training and the appearance of a conflict of interest. Third, although few labs were the subject of a complaint each year from 2002 through 2004--significantly less than one complaint per lab per year-- concerns about anonymity and lab workers' lack of familiarity with how to file a complaint suggest that some quality problems may not be reported. Fourth, based on the large number of labs with proposed sanctions from 1998 through 2004 that were never imposed--even for labs with the same serious, condition-level deficiencies on consecutive surveys--it is unclear how effective CMS's enforcement process is at motivating labs to consistently comply with CLIA requirements. Fifth, CMS is not meeting its requirement to determine in a timely manner the continued equivalency of accrediting organization and exempt-state inspection requirements and processes. For example, New York's and COLA's reviews were about 4 years and 3 years past due, respectively, as of December 2005. Moreover, CMS allows the implementation of changes to accrediting organization and exempt-state program inspection requirements between periodic equivalency determinations before it reviews the proposed changes. CMS attributed these delays to having insufficient staff. Finally, validation reviews--one of CMS's most important oversight tools--do not provide an independent assessment of the extent to which surveys identify all serious deficiencies because many are performed simultaneously with such surveys. Accordingly, in the report we released today, we made specific recommendations to the CMS Administrator to standardize survey findings across survey organizations in order to make meaningful comparisons; strengthen survey, complaint, and enforcement processes; and improve oversight of the CLIA program. In its comments on a draft of our report, CMS endorsed our overall conclusion that quality assurance for the nation's clinical labs should be strengthened and said that it would take action in response to 11 of our 13 recommendations. CMS provided an alternative assessment of lab quality, and disagreed with our recommendations concerning the frequency of proficiency testing and the extent of simultaneous accrediting organization validation reviews. CMS also expressed concern about identifying and sanctioning labs with repeat condition level deficiencies. After considering CMS's comments, we believe that implementing our recommendations is necessary to improve oversight of labs and accrediting organizations. Background: A clinical lab is generally defined as a facility that examines specimens derived from humans for the purpose of disease diagnosis, prevention, and treatment, or health assessment of individuals. Labs conduct a wide range of tests that are categorized as waived tests or as moderate-or high-complexity tests. Approximately 81 percent of all labs (about 157,000) are not subject to routine biennial surveys because they perform (1) "waived" tests, which are generally simple tests that have an insignificant risk of erroneous results, such as those approved for home use[Footnote 7] or (2) tests performed during the course of a patient visit with a microscope on specimens that are not easily transportable. CLIA establishes more stringent requirements for the 19 percent (about 36,000) of labs performing moderate-or high- complexity testing, including the requirement for a survey and participation in routine proficiency testing. Surveys examine lab compliance with CLIA program requirements in several areas including: personnel qualifications, proficiency testing,[Footnote 8] quality control, quality assurance, and recordkeeping. Survey Organizations: In general, labs have a choice of who conducts their surveys--state survey agencies using CLIA inspection requirements or other survey organizations that use requirements CMS has determined to be at least equivalent to CLIA's. CMS contracts with state survey agencies in most states to inspect labs against CLIA requirements.[Footnote 9] CLIA established an approval process to allow states and private accrediting organizations to use their own requirements to survey labs. As noted earlier, New York and Washington operate CLIA-exempt programs and CMS has approved six private, nonprofit accrediting organizations to survey labs--the American Association of Blood Banks (AABB), the American Osteopathic Association (AOA), the American Society of Histocompatibility and Immunogenetics (ASHI), CAP, COLA, and JCAHO. The requirements of both state CLIA-exempt programs and accrediting organizations must be reviewed by CMS at least every 6 years to ensure CLIA equivalency, but may be more stringent than those of CLIA. Figure 1 lists the three types of survey organizations and indicates whether they survey labs under CLIA requirements, or use their own CLIA- equivalent requirements. It also shows the percentage of labs performing moderate-to high-complexity testing surveyed by each type of organization. In general, state survey agencies, COLA, and Washington's CLIA-exempt program survey physician office labs, while New York's CLIA- exempt program, CAP, and JCAHO survey hospital labs. Figure 1: Types of Survey Organizations, Requirements Used to Survey Labs, and Percentage of Labs Surveyed by Each Organization, as of December 2005: [See PDF for image] Source: GAO. [A] Washington is not included as it has only a CLIA-exempt program. [B] New York uses CLIA-equivalent requirements to inspect larger hospital labs under the state's CLIA-exempt program and CLIA requirements to inspect smaller labs, including physician office labs. Only the labs in the CLIA-exempt program are counted here. [C] Some labs are counted more than once because labs may be accredited by more than one organization. While some labs in New York may be accredited, they are still subject to biennial surveys by the state survey agency or the state CLIA-exempt program, because New York does not authorize accreditation as the basis for lab licensure. [End of Figure] Surveys and Complaint Investigations: Survey organizations (1) conduct surveys and complaint investigations and (2) monitor proficiency test results submitted by surveyed labs three times a year. Surveys are typically conducted by former or current lab workers, who assess lab compliance with CLIA or CLIA- equivalent requirements. Generally, surveyors verify that lab personnel are appropriately qualified to conduct testing, evaluate proficiency test records, check equipment and calibration to ensure that appropriate quality control measures are in place, and determine whether the lab has a quality assurance plan and uses it to, among other things, appropriately identify and resolve problems affecting testing quality. Surveys also include an educational component to assist labs in understanding how to comply with CLIA requirements. Lab survey requirements are classified as either "standard-" or "condition-" level. Deficiencies are also characterized as standard-or condition-level based on the requirement in which the deficiency occurs. Standard-level deficiencies denote problems that generally are not serious, while condition-level deficiencies are cited when the problems are serious or systemic in nature. When deficiencies are found during surveys or complaint investigations, labs are required to submit a plan of correction, detailing how and when they will address the deficiencies. Additionally, CMS can impose principal or alternative sanctions, or both. Principal sanctions include revocation of a CLIA certificate, cancellation of the right to receive Medicare payments, or limits on testing. Alternative sanctions, authorized by Congress to give CMS more flexibility to achieve lab compliance, are less severe and include civil money penalties or on-site monitoring. For condition- level deficiencies that do not involve an imminent and serious threat to patient health and a significant hazard to public health, labs have an opportunity to correct the deficiencies, which we refer to as a grace period, before the sanctions are imposed. If a lab is unable to correct a deficiency during this grace period, CMS determines whether to impose sanctions. CMS Oversight: CMS, including its 10 regional offices, oversees state and accrediting organization survey activities. CMS reviews and approves initial and subsequent applications from exempt-state programs and accrediting organizations to ensure CLIA equivalency. Validation reviews are one of CMS's primary oversight tools. Federal surveyors in CMS regional offices are responsible for conducting validation reviews of state survey agency and exempt-state program inspections, but state survey agency staff conduct the validation reviews of accrediting organization inspections.[Footnote 10] An objective of these reviews is to determine if all condition-level deficiencies were identified. These reviews are conducted within 60 days of a state's, or 90 days of an accrediting organization's, survey of a lab. Insufficient Data Exist to Identify Extent of Serious Lab Quality Problems: The extent of serious quality problems at labs is unclear because CMS has incomplete data on condition-level deficiencies identified by state survey agencies prior to 2004. Survey results for 2004 show substantial variability across states, which suggests that state survey agencies do not conduct surveys in a consistent manner. We also found that the lack of a straightforward linkage between CLIA requirements and the CLIA- equivalent requirements of some survey organizations makes it virtually impossible to assess lab quality in a standardized manner. CMS does not effectively use available data, such as the results of surveys and proficiency testing, to monitor and assess lab quality. Although CMS noted that proficiency testing trend data show a decrease in failures for labs as a whole, the data suggest that quality may not have improved at hospital labs for the period 1999 through 2003. Limited Quality Data for Labs Inspected by State Survey Agencies Suggest Survey Inconsistencies: CMS's OSCAR database contains limited data on the quality of labs inspected by state survey agencies and, as a result, it is not possible to analyze changes in the quality of lab testing over time. In January 2004, CMS implemented revised CLIA survey requirements and modified the existing OSCAR data--state survey agency findings--to reflect the changes. The revisions affected approximately two-thirds of the CLIA condition-level requirements.[Footnote 11] As a result of the data modifications, the findings for surveys conducted prior to 2004 no longer reflect all key condition-level requirements in effect at the time of those surveys. Based on the available 2004 OSCAR data (which represent about one half of all labs surveyed by state survey agencies), we found that 6.3 percent of labs had condition-level deficiencies. However, variability in the OSCAR data suggests that labs are not surveyed in a consistent manner. In 2004, the percentage of labs that were reported to have condition-level deficiencies varied considerably by state, ranging from none in 6 states to about 25 percent of labs in South Carolina. Based on interviews with CMS and 10 state survey agencies, it appears that at least some of this variability is due to differences in states' approaches to conducting their surveys as opposed to true differences in lab quality. For example, CMS told us that, because there is not a prescriptive checklist to guide the survey process, the reliance on state surveyor judgment results in variations in the citing of deficiencies. In fact, officials in several states said that there are circumstances under which condition-level deficiencies would not be cited, such as if the lab staff were new or if the lab had a good history of compliance. As a result, available data likely understate the extent of serious quality problems at labs. Quality of Labs Inspected by Survey Organizations Is Very Difficult to Measure in a Standardized Manner: Differences in the inspection requirements used by survey organizations make it virtually impossible to measure lab quality in a standardized manner. Because exempt-state programs and accrediting organizations do not classify inspection requirements and related deficiencies with the same criteria used by state survey agencies--as either standard-or condition-level--they cannot easily identify the proportion of surveyed labs with condition-level deficiencies.[Footnote 12] We asked exempt-state programs and accrediting organizations what percentage of their requirements, and any deficiencies cited for failure to meet those requirements, indicated serious problems that were equivalent to CLIA condition-level deficiencies. CAP and COLA crosswalked their recent survey findings to CLIA condition-level requirements. Although their analysis suggested that from about 56 to 68 percent of labs surveyed during 2004 had a deficiency in at least one condition-level requirement, they acknowledged that these proportions overstated the subset of labs with serious problems. JCAHO did not crosswalk its inspection requirements to those of CLIA because staff would have had to manually review each survey report to determine which deficiencies were equivalent to deficiencies in CLIA condition- level requirements.[Footnote 13] Despite the difficulty of identifying CLIA equivalent condition-level deficiencies, two of the three accrediting organizations we reviewed have systems to identify labs they survey that have serious quality problems. COLA estimated that about 9 percent of labs it surveyed in 2004 were subject to closer scrutiny because of the seriousness of the problems identified. According to JCAHO, about 5 percent of the labs it surveyed in 2004 were not in compliance with a significant number of requirements. The third accrediting organization, CAP, has criteria for identifying labs that warrant greater scrutiny, but CAP officials told us that identifying such labs had to be accomplished on a case by case basis, rather than through a database inquiry.[Footnote 14] CMS Use of Data for Monitoring Lab Quality Is Limited: CMS does not effectively use available data, such as the results of surveys and proficiency testing data, to monitor and assess lab quality. Although CMS tracks the most frequently cited deficiencies at labs in an effort to improve quality, it does not routinely track the proportion of labs, by state, in which state survey agencies identify condition-level deficiencies--those that denote serious or systemic problems. As noted earlier, variability in survey findings suggests inconsistencies in how surveys are conducted. CMS also does not require exempt-state programs and accrediting organizations to routinely submit data on serious deficiencies identified at the labs they inspect, unless the deficiencies pose immediate jeopardy to the public or an individual's health. We also found that CMS does not effectively use proficiency testing data to assess clinical lab quality. Proficiency testing is an important indicator of lab quality because it is an objective assessment of a lab's ability to produce accurate test results and is conducted more frequently than surveys--three times a year versus once every 2 years. In the absence of comparable survey data, proficiency testing results provide a uniform way to assess the quality of lab testing across survey organizations. Although CMS's analysis of proficiency testing data showed improvements over time, our analysis of proficiency testing data for 1999 through 2003 suggests that there has been an increase in proficiency testing failures for labs inspected by CAP and JCAHO, which generally inspect hospital labs, and a decrease in such failures for labs surveyed by state survey agencies and COLA, which tend to inspect physician office labs. Importantly, CMS's decision to require proficiency testing for almost all laboratory tests only three times a year is inconsistent with the statutory requirement. CLIA requires that proficiency testing be conducted "on a quarterly basis, except where the Secretary determines for technical and scientific reasons that a particular examination or procedure may be tested less frequently (but not less often than twice per year)."[Footnote 15] In CMS's 1992 rule implementing CLIA, the agency provided a rationale for reducing the frequency of proficiency testing, but did not provide a technical and scientific basis for reducing the frequency for particular procedures or tests.[Footnote 16] CMS told us that officials from CMS and the Centers for Disease Control and Prevention had together determined that the reduced frequency was based on technical and scientific grounds and supplied a brief, undated narrative which it attributed to the Centers for Disease Control and Prevention. However, the narrative focused on the relative costs and benefits of proficiency testing at various intervals and did not include an analysis of the technical and scientific considerations with regard to particular tests that presented a basis for reducing the frequency. CLIA Program Oversight Is Inadequate: Oversight by CMS and survey organizations is not adequate to ensure that labs meet CLIA requirements. For example, the goal of educating lab workers during surveys takes precedence over the identification and reporting of deficiencies, while the use of volunteer rather than staff surveyors by one accrediting organization raises questions about appropriate levels of training and the appearance of a conflict of interest. The significant increase in complaints since CAP took steps to help ensure that lab workers know how to file a complaint suggests that some quality problems at labs inspected by some survey organizations may not be reported. In addition, sanctions are not being used effectively as an enforcement tool to promote labs' compliance with CLIA requirements, as evidenced by the relatively few labs with repeat condition-level deficiencies on consecutive surveys from 1998 through 2004 that had sanctions imposed. Furthermore, CMS is not meeting its responsibility to determine that accrediting organization and exempt-state program requirements and processes continue to be at least equivalent to CLIA's. Finally, ongoing CMS validation reviews do not provide an independent assessment of the extent to which surveys identify all condition-level deficiencies--primarily due to their timing. Balance Between Educational and Regulatory Roles by CMS and Survey Organizations Appears to Be Inappropriate: The goal of educating lab workers sometimes takes precedence over, or precludes, the identification and reporting of deficiencies that affect the quality of lab testing.[Footnote 17] For example, surveyors from one state survey agency told us they do not cite condition-level deficiencies when lab workers are new but prefer to educate the new staff. As a result, data on the quality of lab testing and trends in quality over time may be misleading. CMS also appears to be inappropriately stressing education over regulation. For instance, in its 2005 implementation of proficiency testing for lab technicians who interpret Pap smears, a test for cervical cancer, CMS instructed state surveyors to refrain from citing deficiencies at labs whose staff fail the tests in 2005 or 2006. According to CMS, this educational focus allows labs and their staff to become familiar with the proficiency testing program; however, it is important to note that there was about a 13-year time lag between the 1992 regulations that implemented CLIA and the 2005 implementation of Pap smear proficiency testing.[Footnote 18] In addition, CMS noted that it was concerned about some of the high initial Pap smear proficiency testing failure rates. An inappropriate balance between the educational and regulatory roles is also evident in some accrediting organization practices. For instance, for COLA, the process of educating labs begins even prior to a survey, when labs are encouraged to complete a self-assessment to identify COLA requirements with which they are not in compliance. A CAP surveyor we interviewed with over 30 years of lab experience estimated that the majority of pathologists--individuals who generally serve as CAP survey team leaders--view surveys as educational, rather than as assessments of compliance with lab requirements. Use of Volunteer Surveyors by CAP Raises Concerns: The use of volunteer inspectors by CAP raises concerns about appropriate levels of training and the appearance of a conflict of interest. Although state survey agencies, exempt-state programs, COLA, and JCAHO employ dedicated staff surveyors, CAP relies primarily on volunteer teams consisting of lab workers from other CAP-inspected labs to conduct surveys.[Footnote 19] In contrast to the mandatory training and continuing education programs in place for the staff surveyors of other survey organizations, training for CAP's volunteer surveyors is currently optional.[Footnote 20] According to data provided by CAP, two- thirds of volunteer surveyors who had recently participated in a survey had no formal training in the 3 to 5 years preceding the survey. While full-time surveyors employed by other survey organizations conduct from 30 to about 200 surveys per year, CAP volunteer surveyors have much less experience conducting surveys because they only survey about one lab each year. CAP officials told us they plan to establish a mandatory training program for survey team leaders beginning in mid- 2006.[Footnote 21] However, the required training will take only 1 or 2 days. In contrast, state survey agency inspectors must complete 5 days of basic training, while COLA staff inspectors participate in a 5-week orientation program and an annual 20 hours of continuing education. CAP's method for staffing survey teams also raises concerns about the appearance of a conflict of interest. Typically, inspection team leaders are pathologists who direct other labs in the community, and the inspection team is comprised of several employees from the team leader's lab. In the event of differing opinions about survey findings, team members who are subordinates to the team leader may feel that they have no other recourse than to follow the team leader's instructions-- such as downgrading the record of an inspection finding to a less serious category. Recognizing that team members' objectivity may be compromised in this situation, CAP's revised conflict of interest policy instructs all parties to be cautious to retain objectivity in fact finding throughout the inspection process. Lab Workers Who File Complaints About Quality Problems in Lab Testing Not Afforded Whistle-blower Protections: Some lab workers may not be filing complaints about quality problems at their labs because of anonymity concerns or because they may not be familiar with filing procedures. Based on OSCAR data and data obtained from exempt-state programs and accrediting organizations for 2002 through 2004, few complaints were received about lab testing relative to the number of labs--significantly less than one complaint per lab per year.[Footnote 22] We found that lab workers may not know how to file a complaint. CAP experienced a significant increase in the number of complaints it received since October 2004 when it began requiring CAP-inspected labs to display posters on how to file complaints. Specifically, from October through December 2004, CAP received an average of 22 complaints per month, compared to an average of 11 complaints per month in the 9 months preceding the poster requirement.[Footnote 23] Because of the difficulty of protecting the anonymity of lab workers who file complaints, whistle-blower protections for such individuals are particularly important. Two of the three accrediting organizations we interviewed have whistle-blower protections--CAP and JCAHO.[Footnote 24] While officials from New York and Washington's exempt-state programs told us that whistle-blower laws in their states provide some protection for lab workers who file complaints, officials in most of the other 10 states we interviewed told us that they did not have any whistle-blower protections or were unable to identify specific protections that applied to lab workers in their state. Although there are no federal whistle-blower protections specifically for workers in labs covered by CLIA, legislation was introduced in 2005 to provide such protections.[Footnote 25] Lab Sanctions Are Rarely Imposed: Few labs were sanctioned by CMS from 1998 through 2004--even those with the same condition-level deficiencies on consecutive surveys--because many proposed sanctions are never imposed. Our analysis of CMS enforcement data from 1998 through 2004 found that while over 9,000 labs had sanctions proposed during these years, only 501 labs were sanctioned.[Footnote 26] This equates to less than 3 percent of the approximately 19,700 labs inspected by state survey agencies. Before sanctions go into effect, labs are given a grace period to correct condition-level deficiencies, unless the deficiencies involve an imminent and serious threat to patient health and a significant hazard to public health. Most labs correct the deficiencies within the grace period. CMS officials told us that it was appropriate to give labs an opportunity to correct such deficiencies within a prescribed time frame and thus avoid sanctions. However, the number of labs with the same repeat condition-level deficiencies from one survey to the next also raises questions about the overall effectiveness of the CLIA enforcement process. From 1998 through 2004, 274 labs surveyed by state survey agencies had the same condition-level deficiency cited on consecutive surveys and 24 of these labs had the same condition-level deficiency cited on more than two surveys.[Footnote 27] This analysis may understate the percentage of labs with repeat condition-level deficiencies because OSCAR data prior to 2004 no longer reflect about two-thirds of condition-level requirements and associated deficiencies at the time of those surveys. We found that only 30 of the 274 labs with repeat condition-level deficiencies had sanctions imposed--either principal, alternative, or both. With respect to accredited labs, from 1998 through 2004, less than 1 percent of accredited labs (81) lost their accreditation; few of these labs were subsequently sanctioned by CMS and many still participate in the CLIA program. Moreover, CMS did not sanction 3 labs that COLA concluded had cheated on proficiency testing by referring the samples to another lab to be tested.[Footnote 28] By statute, the intentional referral of samples to another lab for proficiency testing is a serious deficiency that should result in automatic revocation of a lab's CLIA certificate for at least 1 year.[Footnote 29] Based on our interviews, we found that the 3 labs were allowed to continue testing because they had initiated corrective actions; in effect, these labs were given an opportunity to correct a deficiency that appears to have required a loss of their CLIA certificate for at least 1 year. CMS Is Late in Ensuring CLIA Equivalency of Exempt States' and Accrediting Organizations' Inspection Requirements and Processes: We found that CMS has been late in determining that exempt states' and accrediting organizations' inspection requirements and processes are at least equivalent to CLIA's. Because CMS has not completed its equivalency reviews within required time frames, accrediting organizations and exempt state programs have continued to operate without proper approval.[Footnote 30] Equivalency reviews for CAP, COLA, JCAHO, and Washington due to be completed between November 1, 1997, and April 30, 2001, were an average of about 40 months late. In August 1995, CMS determined that New York's next equivalency review should be completed by June 30, 2001, but was over 4 years past due as of December 2005. Similarly, COLA's equivalency review was about 3 years past due. Furthermore, although federal regulations require CMS to review equivalency when an accrediting organization or exempt-state program adopts new requirements, CMS has not reviewed changes in the inspection requirements prior to use by these entities.[Footnote 31] As a result, such survey organizations may introduce changes that are inconsistent with CLIA requirements. For example, JCAHO made a significant change to its inspection requirements in January 2004; CMS did not begin an in-depth review of JCAHO's revised requirements until early 2005--over a year after they were implemented by JCAHO. According to CMS, its review has identified several critical areas where JCAHO standards are less stringent than those of CLIA. JCAHO acknowledged the need to make some adjustments to its revised requirements. CMS officials attributed delays in making equivalency determinations and reviewing interim changes to having too few staff. The CLIA program, located in CMS's Center for Medicaid and State Operations (CMSO), currently has approximately 21 full-time-equivalent positions compared to a peak of 29 such positions several years ago. As required by statute, the CLIA program is funded by lab fees and since its inception the program's fees have exceeded expenses. As of September 30, 2005, the CLIA program had a carryover balance of about $70 million--far more than required to hire an additional six to seven staff members. However, CMS officials told us that because the CLIA program staff are part of CMSO, they are subject to the personnel limits established for CMSO, regardless of whether or not the program has sufficient funds to hire more staff. CMS Validation Reviews Skip Some State Survey Agencies and Many Lack Independence: CMS validation reviews that are intended to evaluate lab surveys conducted by both states and accrediting organizations do not provide CMS with an independent assessment of the extent to which surveys identify all serious--that is, condition-level or condition-level equivalent--deficiencies. CMS requires its regional offices to conduct validation reviews of 1 percent of labs inspected by state survey agencies in a year.[Footnote 32] However, CMS does not specifically require that validations occur in each state. As a result, from 1999 through 2003, there were 11 states in which no validation reviews were conducted in multiple years. Without validating at least some surveys in each state, CMS is unable to determine if the states are appropriately identifying deficiencies. Many validation reviews occur at the same time a survey organization conducts its inspection and, in our view, the collaboration among the two teams during these simultaneous surveys prevents an independent evaluation. Seventy-five percent of validations of state lab surveys were conducted simultaneously from fiscal years 1999 through 2003.[Footnote 33] According to CMS officials, the large proportion of simultaneous validation reviews provides an opportunity for federal surveyors to share information with state surveyors, monitor their conformance with CLIA inspection requirements, and identify training and technical assistance needs. However, we found that such reviews do not provide an accurate assessment of state surveyors' ability to identify condition-level deficiencies. Of the 13 validation reviews that identified missed condition-level deficiencies, only 1 was a simultaneous review. Regarding validation reviews of accrediting organization's survey of labs, CMS officials were unable to tell us how many of the roughly 275 validation reviews conducted each year from fiscal year 1999 through fiscal year 2003 were simultaneous.[Footnote 34] However, JCAHO estimated that 33 percent of its validation reviews were conducted simultaneously. CMS officials told us that the agency's intent in instituting simultaneous reviews was for state and accrediting organization surveyors to share best practices, to promote understanding of each other's programs, and to foster accrediting organization improvement. In contrast, most of the state survey agency officials we interviewed told us that simultaneous validation reviews do not provide a realistic evaluation of the adequacy of accrediting organizations' inspection processes. Concluding Observations: Clinical labs play a pivotal role in the nation's health care system by diagnosing many diseases, including potentially life-threatening diseases, so that individuals receive appropriate medical care. Given this important role, lab tests must be accurate and reliable. Our work demonstrated that the oversight of clinical labs needs to be strengthened in several areas. Without standardized survey findings across all survey organizations, CMS cannot tell whether the quality of lab testing has improved or worsened over time or whether deficiencies are being appropriately identified. Using data to analyze activities across survey organizations can be a powerful tool in improving CMS oversight of the CLIA program, yet CMS has not taken the lead in ensuring the availability and use of data from survey organizations to help it monitor their performance. Furthermore, the agency is not requiring that labs participate in proficiency testing on a quarterly basis, as required by CLIA. More broadly, CMS and survey organization oversight of the lab survey process is not adequate to enforce CLIA requirements. Educating labs to ensure high-quality testing should complement but not replace the enforcement of CLIA inspection requirements. Labs with the same serious deficiencies on consecutive surveys often escape sanctions, even though Congress authorized alternative sanctions to give CMS more flexibility to achieve lab compliance. Without the threat of real consequences, labs may not be sufficiently motivated to comply with CLIA inspection requirements. By allowing validation reviews to occur simultaneously with surveys and permitting some states to go without validation reviews over a period of several years, CMS is not making full use of this oversight tool. Moreover, independent validation reviews of accrediting organization surveys are critical because CMS has not conducted equivalency reviews within the time frames it established. The recommendations we have made would help CMS to consistently identify and address lab quality problems. Mr. Chairman, this concludes my prepared remarks. I would be happy to answer any questions that you or other Members of the Subcommittee may have. Contact and Acknowledgments: For further information regarding this statement, please contact Leslie G. Aronovitz at (312) 220-7600 or aronovitzl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found in the last page of this statement. Walter Ochinko, Assistant Director; Jenny Grover; Kevin Milne; and Michelle Rosenberg contributed to this statement. FOOTNOTES [1] Medicare is a federal health care program serving elderly and certain disabled individuals. [2] Pub. L. No. 100-578, 102 Stat. 2903. [3] We use the term "survey organizations" when referring collectively to state survey agencies, the two state CLIA-exempt programs, and accrediting organizations. [4] COLA was formerly known as the Commission on Office Laboratory Accreditation. [5] New York, Philadelphia, and Seattle. [6] California, Colorado, Connecticut, Idaho, Louisiana, Michigan, Nebraska, North Carolina, Pennsylvania, and South Carolina. [7] Pregnancy and blood sugar screenings are examples of such tests. [8] Surveyed labs must participate in an approved external proficiency testing program, which evaluates the accuracy of laboratory testing. Under this requirement, a lab purchases samples with unknown characteristics several times each year from an approved proficiency testing provider. The lab is required to test the samples with its routine patient testing, and the results are returned to the testing provider to be graded. A proficiency testing failure is defined as unsatisfactory performance on two consecutive or two out of three testing events. [9] CMS contracts with state survey agencies in the District of Columbia and 49 states (including New York but not Washington) to survey labs under CLIA requirements. [10] Unlike validation reviews of accrediting organization surveys, CMS refers to the validation of state surveys as Federal Monitoring Surveys. Because of their similar objective, we refer to all such surveys as validation reviews in this testimony. We refer to validation reviews that occur at the same time as the lab survey as simultaneous. Conversely, validation reviews that occur after the lab survey are referred to as independent validations. [11] For example, some condition-level requirements were reorganized and some were consolidated. [12] Although CMS reviews the requirements of exempt-state programs and accrediting organizations to ensure that they are at least equivalent to CLIA's, there is not necessarily a one-to-one match with CLIA requirements. Thus, one CLIA condition-level requirement may equal several accrediting organization requirements or vice versa. For example, CMS's condition-level requirement for successful lab participation in approved proficiency testing corresponds to at least 19 CAP, 3 COLA, and 4 JCAHO requirements. [13] However, JCAHO officials noted that in 2004, about 90 percent of the labs it surveyed had a deficiency in at least one requirement. JCAHO classifies all of its requirements as serious. [14] As a result, CAP plans to spend in excess of $9 million during 2006 and 2007 to develop an integrated data system that pulls together multiple factors--survey results, complaints, proficiency testing, findings of other inspection bodies, and changes in lab directors--to enable it to readily identify problem labs. [15] Pub. L. No. 100-578, � 2, 102 Stat. 2903, 2907-08, 42 U.S.C. � 263a(f)(3)(2000). [16] In its rationale, CMS noted that experts were divided on the appropriate frequency of proficiency testing and further justified the change by explaining that fewer events of proficiency testing would give laboratories more time to analyze the causes of test failures, thus enhancing the value of proficiency testing as an educational tool. [17] Although CLIA neither requires nor precludes an educational role for surveyors, the preamble to CMS's implementing regulation noted that surveys are intended, in part, to provide an opportunity for on-site education regarding accepted laboratory procedures. [18] Because of lab testing errors that led to women's deaths, Congress required a specific type of proficiency testing for individuals who interpret the results of Pap smear tests, which requires examining glass slides under a microscope. Although CLIA was enacted in 1988, CMS told us that cost, the inability to find a national testing provider, and other technical issues delayed establishing a Pap smear proficiency testing program until 2005. [19] As of November 2005, CAP also employed 11 full-time surveyors. [20] Currently, CAP volunteer surveyors are encouraged to participate in surveyor training at least once every 3 years. [21] Mandatory training for survey team members is targeted to begin in 2007. [22] Information about complaints is from OSCAR data and data obtained from exempt-state programs and accrediting organizations for 2002 through 2004. The modifications to OSCAR did not affect data on the number of complaints. The complaint information in OSCAR excludes complaints that do not require an on-site survey. [23] In September 2005, COLA also began requiring labs to display a complaints poster similar to CAP's. Neither CMS nor JCAHO plan to require a similar complaints poster. Effective July 2005, JCAHO required labs to educate staff on how to report concerns about lab quality to the Joint Commission but does not specify use of a poster to do so. [24] COLA does not have a formal whistle-blower policy. COLA officials told us that they promptly investigate all complaints, many of them from former lab employees, and keep the identity of the complainants anonymous. [25] H.R. 686, 109th Cong. (2005). [26] Since CMS data list only the number of labs with proposed sanctions by year, this number may double-count labs that had proposed sanctions in multiple years. [27] Thirty-three states and the District of Columbia had at least one lab with the same repeat condition-level deficiency. [28] A fourth lab was ultimately sanctioned for proficiency testing cheating by CMS but was allowed to continue testing for almost 2 years after having its accreditation revoked. [29] Pub. L. No. 100-578, � 2, 102 Stat. at 2911, codified at 42 U.S.C. � 263a(i)(4)(2000). [30] CMS must verify the equivalency of accrediting organizations and exempt-state programs, and by regulation, CMS requires such survey organizations to seek reapproval at least once every 6 years, or more frequently if deemed necessary. CMS establishes the time frames for when the next reapproval should occur, which have ranged from about 15 months to about 6 years. [31] See 42 C.F.R. � 493.573(a)(3)(2005). [32] In contrast, validation reviews of 5 percent of labs inspected by accrediting organizations during a year are conducted by state survey agency personnel. [33] These validation reviews include both exempt-state and state survey agency lab surveys. [34] CMS did not begin tracking this information until August 2003. GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to www.gao.gov and select "Subscribe to Updates." Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, D.C. 20548: Public Affairs: Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548.