Hospital Quality Data

Hospitals submit data on a series of quality measures to the Centers for Medicare & Medicaid Services (CMS) and receive scores on their performance. CMS instituted the Reporting Hospital Quality Data for Annual Payment Update Program (APU program) to collect the quality data from hospitals and report their rates on the measures on its Hospital Compare Web site. For hospital quality data to be useful to patients and other users, they need to be reliable, that is, accurate and complete. The Deficit Reduction Act of 2005 directed CMS to implement a value-based purchasing program for Medicare that beginning in fiscal year 2009 would adjust payments to hospitals based on factors related to the quality of care they provide. This statement provides information on (1) how hospitals collect and submit quality data to CMS and (2) how CMS works to ensure the reliability of the quality data submitted. This statement is based primarily on Hospital Quality Data: HHS Should Specify Steps and Time Frame for Using Information Technology to Collect and Submit Data (GAO-07-320, Apr. 25, 2007) and Hospital Quality Data: CMS Needs More Rigorous Methods to Ensure Reliability of Publicly Released Data (GAO-06-54, Jan. 31, 2006). In preparing these reports, GAO conducted case studies of eight hospitals, and reviewed documents of, and interviewed officials at CMS.

GAO reported in April 2007 that the eight case study hospitals visited used six steps to collect and submit quality data, two of which (steps 2 and 3) involved complex abstraction--the process of reviewing and assessing all relevant pieces of information in a patient's medical record to determine the appropriate value for each data element. The six steps were (1) identify patients for whom the quality data should be submitted, (2) locate needed information in the medical records, (3) determine the appropriate value for each data element, (4) transmit the data to CMS, (5) review reports to ensure acceptance of the data by CMS, and (6) supply copies of selected medical records to CMS for data validation. Several factors account for the complexity of the abstraction process (steps 2 and 3), including the content and organization of the medical record, the scope of information and clinical judgment required for certain data elements, and frequent changes by CMS in its data specifications. GAO's case studies also showed that existing information technology (IT) systems help hospitals gather some quality data but are far from enabling hospitals to automate the abstraction process. GAO reported in January 2006 that CMS had processes for ensuring the accuracy of the quality data submitted by hospitals but had no ongoing process for ensuring completeness of these data. To check accuracy, one CMS contractor electronically checks the data as they are submitted to the clinical warehouse. Another contractor conducts an independent audit by comparing the quality data submitted by a hospital from the medical records for a sample of five patients per quarter for each hospital to the quality data that the contractor reabstracts from the same medical records. The data are deemed to be accurate if there is 80 percent or greater agreement between these two sets of results. However, GAO also reported that CMS's determination as to whether hospitals met the accuracy standard was statistically uncertain for some hospitals because of the small number of records examined--five cases per quarter per hospital regardless of the hospital's size. In 2006 GAO also reported that CMS did not have an ongoing process for assessing the completeness of quality data submitted by hospitals and recommended that CMS take steps to improve its processes for ensuring the accuracy and completeness of the hospital quality data. CMS agreed the process needed improvement. For fiscal year 2008 and subsequent years, CMS required that hospitals attest each quarter to the completeness and accuracy of their data. Further, in a 2007 report to Congress that lays out a plan to implement a value-based purchasing program, CMS recognized the need to redesign the data infrastructure and validation process to support a value-based purchasing program by, for example, increasing the number of patient medical records sampled from selected hospitals.