Medicare Physician Feedback Program
CMS Faces Challenges with Methodology and Distribution of Physician Reports Gao ID: GAO-11-720 August 12, 2011In Process
CMS faces challenges incorporating resource use and quality measures for physician feedback reports that are meaningful, actionable, and reliable. CMS had difficulty measuring the resources used by physicians to treat specific episodes of an illness, such as a stroke or a hip fracture, and the quality measures it used in the program's most recent phase applied to a limited number of physicians. CMS must also make decisions to address several other methodological challenges with developing feedback reports: how to account for differences in beneficiary health status, how to attribute beneficiaries to physicians, how to determine the minimum number of beneficiaries a physician needs to treat to receive a report, and how to select physicians' peer groups for comparison. These decisions involve trade-offs; for example, a higher minimum case size requirement increases the reliability of the information in the reports, but it decreases the number of physicians eligible to receive one. While CMS has tested different approaches to measuring and comparing physician performance, methodological difficulties remain in developing feedback reports. CMS also faced challenges distributing feedback reports to physicians that its plans for improvement may not entirely address. In the most recent phase of the program, about 82 percent of physicians in CMS's sample were not eligible to receive a report after CMS's methodological decisions were applied. CMS plans to make a number of methodological changes in the next phase, but significantly increasing eligibility will continue to be challenging. The electronic distribution of feedback reports also presented multiple challenges that resulted in few physicians accessing their electronic reports in the most recent phase. Factors that may have contributed to this low access rate include CMS's difficulty in obtaining accurate contact information, burdensome methods for electronic distribution, and lack of a strong incentive for physicians to review their reports. CMS conducted limited follow-up with physicians for whom feedback reports were produced. CMS plans to use a new distribution method in a four-state region in the next reporting phase. GAO is recommending that CMS use methodological approaches that increase physician eligibility for reports, statistically analyze the impact of its methodological decisions on report reliability, identify and address factors that may have prevented physicians from reading reports, and obtain input from a sample of physicians on the usefulness and credibility of reports. CMS concurred with these recommendations.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: James C. Cosgrove Team: Government Accountability Office: Health Care Phone: (202) 512-7029GAO-11-720, Medicare Physician Feedback Program: CMS Faces Challenges with Methodology and Distribution of Physician Reports
This is the accessible text file for GAO report number GAO-11-720
entitled 'Medicare Physician Feedback Program: CMS Faces Challenges
with Methodology and Distribution of Physician Reports' which was
released on August 12, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to Congressional Committees:
August 2011:
Medicare Physician Feedback Program:
CMS Faces Challenges with Methodology and Distribution of Physician
Reports:
GAO-11-720:
GAO Highlights:
Highlights of GAO-11-720, a report to congressional committees.
Why GAO Did This Study:
The Medicare Improvements for Patients and Providers Act of 2008
(MIPPA) directed the Department of Health and Human Services (HHS) to
develop a program to give physicians confidential feedback on the
resources used to provide care to Medicare beneficiaries. In response,
HHS‘s Centers for Medicare & Medicaid Services (CMS) has established
and implemented the Physician Feedback Program by distributing
feedback reports to an increasing number of physicians that provided
data on resources used and the quality of care. MIPPA mandated that
GAO conduct a study of this program. To address this mandate, GAO
identified (1) methodological challenges CMS faces in developing
feedback reports and approaches CMS has tested to address them and (2)
challenges CMS faces in distributing feedback reports and CMS‘s plans
to address them. GAO interviewed CMS officials and representatives
from the program contractor and reviewed relevant documentation.
What GAO Found:
CMS faces challenges incorporating resource use and quality measures
for physician feedback reports that are meaningful, actionable, and
reliable. CMS had difficulty measuring the resources used by
physicians to treat specific episodes of an illness, such as a stroke
or a hip fracture, and the quality measures it used in the program‘s
most recent phase applied to a limited number of physicians. CMS must
also make decisions to address several other methodological challenges
with developing feedback reports: how to account for differences in
beneficiary health status, how to attribute beneficiaries to
physicians, how to determine the minimum number of beneficiaries a
physician needs to treat to receive a report, and how to select
physicians‘ peer groups for comparison. These decisions involve trade-
offs; for example, a higher minimum case size requirement increases
the reliability of the information in the reports, but it decreases
the number of physicians eligible to receive one. While CMS has tested
different approaches to measuring and comparing physician performance,
methodological difficulties remain in developing feedback reports.
CMS also faced challenges distributing feedback reports to physicians
that its plans for improvement may not entirely address. In the most
recent phase of the program, about 82 percent of physicians in CMS‘s
sample were not eligible to receive a report after CMS‘s
methodological decisions were applied (see figure). CMS plans to make
a number of methodological changes in the next phase, but
significantly increasing eligibility will continue to be challenging.
The electronic distribution of feedback reports also presented
multiple challenges that resulted in few physicians accessing their
electronic reports in the most recent phase. Factors that may have
contributed to this low access rate include CMS‘s difficulty in
obtaining accurate contact information, burdensome methods for
electronic distribution, and lack of a strong incentive for physicians
to review their reports. CMS conducted limited follow-up with
physicians for whom feedback reports were produced. CMS plans to use a
new distribution method in a four-state region in the next reporting
phase.
Figure: Number of Physicians Excluded from Receiving Feedback Reports,
2010:
[Refer to PDF for image: illustration]
All sampled physicians:
CMS began with a sample of 9,189 individual physicians affiliated with
the 36 physician groups.
Methodological requirements for a feedback report:
At least 30 attributed beneficiaries for the resource use measures –
only 2,205 physicians met this requirement;
At least 11 attributed beneficiaries for at least one of the quality
measures – only 2,661 physicians met this requirement;
Only 1,733 physicians met both requirements;
At least 30 individual physicians practicing in the same medical
specialty and geographic area for a peer group – only 1,645 physicians
met this requirement.
Source: GAO analysis of CMS and contractor data.
[End of figure]
What GAO Recommends:
GAO is recommending that CMS use methodological approaches that
increase physician eligibility for reports, statistically analyze the
impact of its methodological decisions on report reliability, identify
and address factors that may have prevented physicians from reading
reports, and obtain input from a sample of physicians on the
usefulness and credibility of reports. CMS concurred with these
recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-11-720] or key
components. For more information, contact James C. Cosgrove at (202)
512-7114 or cosgrovej@gao.gov.
[End of section]
Contents:
Letter:
Background:
CMS Tested Various Approaches but Still Faces Several Methodological
Challenges in Developing Physician Feedback Reports:
CMS's Plans for Improvement May Not Fully Address Challenges in
Distributing Reports to Physicians:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Comments from the Department of Health and Human Services:
Appendix II: GAO Contact and Staff Acknowledgments:
Table:
Table 1: Percentage of Individual Physicians and Physician Groups
Eligible for Select Resource Use and Quality Performance Measures on
Phase II Feedback Reports, 2010:
Figure:
Figure 1: Number of Individual Physicians Excluded from Phase II
Feedback Reports Based on CMS's Methodological Criteria, 2010:
Abbreviations:
AHRQ: Agency for Healthcare Research and Quality:
BCSSI: Buccaneer Computer Systems and Services, Inc.
CBO: Congressional Budget Office:
CMS: Centers for Medicare & Medicaid Services:
CTS: Community Tracking Study:
E&M: evaluation and management:
GEM: Generating Medicare Physician Quality Performance Measurement
Results:
HCC: Hierarchical Condition Categories:
HEDIS®: Healthcare Effectiveness Data and Information Set:
HHS: Department of Health and Human Services:
IACS: Individuals Authorized Access to CMS Computer Services:
MAC: Medicare Administrative Contractor:
MedPAC: Medicare Payment Advisory Commission:
MIPPA: Medicare Improvements for Patients and Providers Act of 2008:
NCQA: National Committee for Quality Assurance:
NPPES: National Plan and Provider Enumeration System:
NQF: National Quality Forum:
PECOS: Provider Enrollment, Chain, and Ownership System:
PPACA: Patient Protection and Affordable Care Act:
PQRS: Physician Quality Reporting System:
UPIN: Unique Physician Identification Number:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
August 12, 2011:
The Honorable Max Baucus:
Chairman:
The Honorable Orrin G. Hatch:
Ranking Member:
Committee on Finance:
United States Senate:
The Honorable Fred Upton:
Chairman:
The Honorable Henry A. Waxman:
Ranking Member:
Committee on Energy and Commerce:
House of Representatives:
The Honorable Dave Camp:
Chairman:
The Honorable Sander M. Levin:
Ranking Member:
Committee on Ways and Means:
House of Representatives:
In recent years, we and other federal fiscal experts--including the
Congressional Budget Office (CBO) and the Medicare Trustees--have
noted the rise in Medicare spending and the serious long-term
financial challenges the program faces.[Footnote 1] Physicians play a
central role in the generation of Medicare expenditures both through
the services they provide and the services they order, including
hospital admissions, diagnostic tests, and referrals to other
physicians. There is evidence that not all of these services may be
necessary or appropriate, and that greater spending does not
necessarily result in better health outcomes. As a result,
policymakers have been exploring methods to reduce costs and encourage
physicians to practice efficiently--that is, to provide and order only
those services that are necessary, sufficient, and appropriate to meet
a beneficiary's health care needs.
Efficiency may be encouraged by physician profiling, which measures
and compares a physician's performance to a benchmark, such as the
performance of his or her peers. Certain public and private health
care purchasers routinely profile physicians in their networks and use
the results for a number of purposes, including developing physician
"report cards" or feedback reports and placing physicians in tiered
networks that can be used to steer patients toward the most efficient
providers. We and others have recommended that the Centers for
Medicare & Medicaid Services (CMS), the agency within the Department
of Health and Human Services (HHS) that administers the Medicare
program, profile physicians and provide them with feedback on their
use of health care resources to help identify and reduce overuse of
Medicare services.[Footnote 2] In addition to profiling physicians on
the resources used to provide care to Medicare beneficiaries, they can
also be profiled on the quality of that care. Some specialty societies
have called for the inclusion of quality measures in feedback reports
and cautioned that focusing solely on costs could create a
disincentive to providing appropriate, high-quality care.
The Medicare Improvements for Patients and Providers Act of 2008
(MIPPA) required HHS to establish and begin implementing by January 1,
2009, a Physician Feedback Program that would include distribution of
confidential feedback reports to physicians on the resources used to
provide care to Medicare beneficiaries.[Footnote 3] MIPPA gave HHS the
flexibility to apply the program to certain types of physicians, such
as those who treat conditions that have high costs, and also provided
flexibility on whether to provide reports to physician groups and
whether to include information on quality. Because developing feedback
reports requires a number of methodological decisions, such as
selecting performance measures that accurately reflect physicians'
resource use and quality of care, CMS has implemented the program in
phases by testing different approaches for developing feedback reports
and distributing reports to a small number of physicians and physician
groups.[Footnote 4]
The Patient Protection and Affordable Care Act (PPACA), which was
enacted in 2010, directed HHS to adjust Medicare payments to
physicians based on the quality of care provided compared to the cost
using a "value-based payment modifier."[Footnote 5] HHS is directed to
begin paying a limited group of physicians and physician groups
differentially using the payment modifier on January 1, 2015, and all
physicians and physician groups by January 1, 2017.[Footnote 6] The
law also states that HHS is to coordinate the Physician Feedback
Program with the value-based payment modifier.[Footnote 7] CMS has
said that it intends to use the quality and cost measures from the
Physician Feedback Program to develop the payment modifier and plans
to distribute at least one feedback report to physicians before paying
them differentially based on their performance.
MIPPA mandated that GAO conduct a study of the Physician Feedback
Program and report on our findings no later than March 1, 2011.
[Footnote 8] To respond to this requirement, we conducted a series of
briefings for congressional staff on our preliminary findings
beginning in February 2011. This report contains information we
provided during those briefings as well as additional information.
Specifically, we (1) identified methodological challenges CMS faces in
developing physician feedback reports and the approaches CMS has
tested to address them and (2) identified challenges CMS faces in
distributing physician feedback reports and CMS's plans to address
them.
To address these objectives, we interviewed relevant CMS officials and
representatives from Mathematica Policy Research, Inc. (Mathematica),
the contractor that assisted with the development and testing of
different methodologies and distribution methods for the Physician
Feedback Program.[Footnote 9] We reviewed internal agency reports and
relevant studies, including reports by CMS contractors and the
Medicare Payment Advisory Commission (MedPAC), summaries of comments
provided by physicians who received feedback reports from CMS, and
public comments submitted by medical specialty societies and other
stakeholders in response to the portion of CMS's 2011 proposed
physician fee schedule rule related to the Physician Feedback Program.
[Footnote 10] In addition, we attended a CMS listening session on the
Physician Feedback Program, at which representatives of medical
specialty societies and other stakeholders commented on the
methodological approaches CMS is considering in developing feedback
reports. We limited our study to challenges with feedback report
methodology and distribution as our initial audit work indicated that
these were the primary challenges faced by the agency in its
implementation of the Physician Feedback Program. Our work is based on
the most current information available as of June 7, 2011.
We conducted this performance audit from June 2010 through August 2011
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
Physicians can be profiled on the health care they provide to Medicare
beneficiaries using measures in two performance dimensions: the
resources used to provide care to beneficiaries and the quality of
that care. CMS has established goals and made progress in developing
its Physician Feedback Program.
Resource Use Measures:
Resource use can be measured using two methods: the per capita method
and the per episode method. The per capita method measures the
resources used by a physician to treat his or her Medicare
beneficiaries over a fixed period of time. By definition, it is a
comprehensive measure of a physician's practice patterns because it
includes all health care resources used and is generally considered
more straightforward than the per episode method to measure and
understand.
The per episode method measures the resource use associated with
treating a specific episode of an illness in a beneficiary--for
example, a stroke or a hip fracture. An episode of care may refer to
all services related to a health condition with a given diagnosis from
a patient's first encounter with a health care provider through the
completion of the last encounter related to that condition, including
postacute services such as home health, skilled nursing, and
rehabilitation.[Footnote 11] Since this method provides condition-
specific results, it may provide more useful, or "actionable,"
feedback to physicians. Per episode costs are generally considered
more difficult to measure than per capita costs since it can be
challenging to determine whether a particular health care service
should be grouped to one episode of care or another. Per episode costs
may be determined using "episode groupers," which are software
programs that use diagnosis codes to assign claims to clinically
distinct episodes of care.[Footnote 12]
Using both the per capita and per episode methods may more fully
capture differences in resource use among physicians. For example, in
a 2006 report, MedPAC found that beneficiaries in Miami had
significantly lower per episode costs for coronary artery disease than
beneficiaries in Minneapolis, suggesting that Miami physicians were
providing more efficient care for coronary artery disease. However,
MedPAC noted that because the beneficiaries in Miami had more episodes
of care for this disease, physicians in Miami actually used more
health care resources in total to treat their coronary artery disease
beneficiaries than physicians treating similar patients in
Minneapolis. In this case, the per capita method and the per episode
method together would provide a more complete picture of physicians'
resource use than either method by itself.
Quality Measures:
Health care quality measures can be used to evaluate how well health
care is delivered, and information obtained from such measures can
promote accountability among physicians. Quality measures can be
classified as process or outcome measures.[Footnote 13] Process
measures assess whether appropriate clinical practices, such as
screening and diagnosis, were followed. An example of a process
measure is whether a patient with high blood pressure received
appropriate medication. Outcome measures assess a patient's health
status after receiving health care services. An example of an outcome
measure is tracking the percentage of patients who were diagnosed with
high blood pressure and whose blood pressure was adequately controlled
during the measurement year.
Efforts are under way by a range of organizations, including CMS and
the National Committee for Quality Assurance (NCQA), to develop
measures of physician quality, and by the National Quality Forum (NQF)
to endorse the quality measures developed by others.[Footnote 14] For
example, NCQA created the Healthcare Effectiveness Data and
Information Set (HEDIS)®, which is a tool used by over 90 percent of
health plans in the nation and includes measures of both health plan
and physician performance.[Footnote 15] CMS has developed the
Physician Quality Reporting System (PQRS), which is a quality
reporting program that provides an incentive payment to professionals
who satisfactorily report data on quality measures for covered
professional services furnished during a specified reporting period.
CMS also contracted with Masspro, a quality improvement organization
for Massachusetts, to calculate performance rates for the Generating
Medicare Physician Quality Performance Measurement Results (GEM)
project. The GEM project used 2006 and 2007 Medicare administrative
claims data to generate performance rates for 12 process measures that
were drawn from HEDIS®, such as persistence of beta blocker treatment
after a heart attack.
CMS Physician Feedback Program Goals and Development:
CMS established the Physician Feedback Program in 2008 with the goal
of encouraging higher-quality and more efficient medical practice and
creating a transparent process for developing meaningful, actionable,
and fair physician performance indicators that could later be used in
CMS's value-based purchasing initiative. Feedback reports can help
ensure quality health care and control costs in three ways. First, the
feedback reports are intended to be educational by providing useful
information to physicians on how their resource use and quality of
care compare to their peers'. Second, the reports are intended to be
actionable by helping physicians identify and develop strategies for
improving quality and reducing costs in their practices. Third, the
reports are intended to help physicians become familiar with the
resource use and quality measures that the agency plans to use to
adjust their Medicare reimbursement under the value-based payment
program. CMS intends to distribute at least one feedback report to
physicians before paying them differentially under the value-based
modifier.
CMS has implemented the program in phases by distributing feedback
reports to an increasing number of physicians in selected metropolitan
areas. In each phase, CMS conducted pretesting to obtain physicians'
reactions to the methodology and format of mock feedback reports,
distributed feedback reports populated with actual performance data,
and followed up with a sample of the profiled physicians to obtain
their input on the reports. In Phase I, CMS distributed feedback
reports to 239 physicians who practiced in one of 12 metropolitan
areas.[Footnote 16] These reports were distributed in April and August
2009, and included information about physicians' resource use but not
their quality of care. In Phase II, CMS expanded the program to
distribute feedback reports to 36 physician groups and to 1,641
individual physicians who practiced within these groups from the same
12 metropolitan areas used in Phase I.[Footnote 17] Phase II reports
were produced in November 2010, and included resource use measures and
selected quality measures as well as information on beneficiaries'
hospital admissions. In addition, the reports to physician groups
included hospitalization rates for ambulatory care sensitive
conditions--acute conditions for which effective outpatient care could
have prevented complications or more severe disease. The reports also
contained the average per capita costs of treating Medicare
beneficiaries, as well as per capita costs by specific categories of
service, such as laboratory tests and imaging services. In addition,
the reports provided summary information about the average annual cost
of treating a subset of Medicare beneficiaries with selected common
chronic conditions: congestive heart failure, chronic obstructive
pulmonary disease, coronary artery disease, diabetes, and prostate
cancer.
CMS plans to continue to develop feedback reports and distribute them
to an increasing number of physicians and physician groups. It plans
to distribute Phase III reports to about 20,000 physicians in late
2011, and intends to provide feedback reports to all physicians and
physician groups by 2017.
CMS Tested Various Approaches but Still Faces Several Methodological
Challenges in Developing Physician Feedback Reports:
CMS faces challenges in selecting resource use and quality performance
measures that make feedback reports meaningful, reliable, and
actionable. In addition, the agency faces trade-offs in making other
key methodological decisions concerning risk adjustment, attribution
of beneficiaries to physicians, minimum case size, and peer group
selection. While CMS has tested different approaches to developing
feedback reports, challenges remain in making methodological decisions
that will enable CMS to accomplish its program goals.
CMS Tested Approaches to Measuring Physician Resource Use and Quality
but Still Faces Challenges:
Measuring resource use. CMS intends to use both per capita and per
episode methods to measure physicians' resource use, but it faces
particular challenges in determining per episode costs for the
Medicare population. In Phase I, CMS tested two commercially available
episode groupers, but found that these groupers had the following
shortcomings when used with Medicare claims data:
* Because of the prevalence of comorbidities in the Medicare
population, a beneficiary can be treated for several different
conditions concurrently, and it was difficult for the groupers to
determine which services belonged with a given episode.
* Because diagnosis coding used for different Medicare claim types was
inconsistent, claims from different sources were not always linked to
the same episode of care, even when they appeared to be clinically
related. For example, hospital, physician, and skilled nursing
facility claims have slightly different diagnostic information.
* Because it was difficult to identify the appropriate beginning and
end of an episode involving a chronic condition, the commercial
groupers did not work well to create episodes of care for the Medicare
population, since a significant portion of Medicare beneficiaries have
chronic conditions.
CMS concluded that per episode measurements included in Phase I
reports were inaccurate, and discontinued use of the commercial
groupers. Some medical specialty societies and other stakeholders
commended this decision.
CMS intends for shortcomings to be addressed by the Medicare-specific
episode grouper under development. In September 2010, CMS awarded four
contracts to develop a Medicare-specific episode grouper.[Footnote 18]
CMS plans to select a grouper developed under one of these four
contracts for future feedback reports.[Footnote 19] However, it is not
clear that all the problems identified with the commercial groupers
can be solved by a Medicare-specific grouper and the timeline for its
development is challenging.
In Phase II feedback reports, CMS elected to provide information on
resource use for beneficiaries with five high-cost, high-volume
chronic conditions. Because episode measures were not available, it
used per capita measures as proxies for per episode costs for patients
with diabetes, congestive heart failure, coronary artery disease,
chronic obstructive pulmonary disease, and prostate cancer. These
proxies included all the resources used to treat beneficiaries with
these select chronic conditions, regardless of whether the resource
use was related to that specific condition. CMS officials stated that
these proxies were adequate substitutes for episode-based cost
measures for these chronic conditions. In Phase III reports, CMS plans
to provide per capita information on subgroups with the same chronic
conditions as in Phase II with the exception of prostate cancer.
[Footnote 20]
Measuring quality. CMS faces the challenge of incorporating into its
feedback reports quality measures that are available, apply to
specialists, and provide information on patient outcomes.[Footnote 21]
Phase I reports did not contain quality measures. In Phase II, CMS
included 12 GEM measures in feedback reports.[Footnote 22] These
measures have the advantage of being readily available because they
are based on claims data.[Footnote 23] For Phase III, CMS is
considering 28 claims-based quality measures, which are endorsed by
NQF. These 28 measures, most of which are HEDIS® measures, were vetted
by an interagency committee composed of medical officers and other
internal experts who reviewed the specifications of each measure,
including whether the measure was an appropriate reflection of
physician care and whether it was evidence based.
While a number of quality measures available to CMS for use in
feedback reports are applicable to primary care physicians, there are
fewer measures for specialists. For example, the GEM measures used in
Phase II reports are only applicable to primary care physicians and a
limited number of specialists, such as cardiologists. In addition, the
28 measures CMS is considering for Phase III reports are, as a whole,
mostly applicable to primary care physicians, although individual
measures apply to certain specialists. Some stakeholders have
encouraged CMS to work with specialty societies to develop adequate
quality measures. CMS officials stated that while the agency is
willing to work with these specialty societies to ensure that selected
measures accurately reflect physicians' practices, CMS prefers to use
NQF-endorsed quality measures and many of the measures that specialty
societies have created have not yet achieved NQF endorsement. In
addition, CMS anticipates using PQRS measures that are applicable to
specialists, but according to CMS officials, it has not done so yet
because of limitations with the PQRS program, such as low physician
participation rates.[Footnote 24] CMS officials said that PQRS has
measures that are applicable to every type of physician, and the
agency is working to increase physician participation in PQRS, which
is currently voluntary. They expect program participation rates to
increase when, in 2013, CMS plans to begin penalizing physicians who
fail to report PQRS measures.[Footnote 25]
In addition, the GEM measures CMS used for Phase II reports are
process measures, which show whether a physician followed generally
accepted recommendations for clinical practice but may not reflect the
impact of the health care services on the health status of a
beneficiary. CMS officials have stated that although there is a need
to evaluate physician quality of care based on outcome measures, there
are currently few suitable measures. NQF has also stated that there is
a need to develop additional outcome quality measures, and funding of
$75 million is authorized for this in each of fiscal years 2010
through 2014. In addition, CMS officials stated that PQRS contains a
number of clinical outcome measures, and it is likely that moving
forward physician feedback reports will include these PQRS outcome
measures.
Other Key Methodological Decisions Involve Trade-offs, and CMS Has
Tested Different Approaches to Inform These Decisions:
Determining risk adjustment factors. CMS faces trade-offs in deciding
which factors to use for risk adjustment, which accounts for
differences outside the physician's control, such as beneficiary
health status. Because sicker beneficiaries are expected to use more
health care resources than healthier beneficiaries, the health status
of physicians' beneficiaries must be taken into account to make
meaningful comparisons among physicians. Without risk adjusting
resource use, physicians who treat sicker beneficiaries could appear
to use resources less efficiently than their peers in their feedback
reports. CMS used the Hierarchical Condition Categories (HCC) model to
risk adjust per capita resource use in the Phase I and Phase II
feedback reports. This model was originally developed for risk
adjustment in Medicare managed care. The HCC model used in Phase II
feedback reports is a method of adjusting for the expected resource
use of Medicare beneficiaries based on the health conditions they
experienced during the previous year and other factors, such as gender
and age.
There are trade-offs involved in determining whether to use a
prospective or concurrent risk adjustment model. A prospective model
uses risk factors from a previous period to predict physicians'
spending for a future period. A prospective model works well for some
health conditions, such as chronic conditions, which are accurate
predictors of health spending not only in the current year, but also
in future years. Conversely, a concurrent model uses factors from the
current period to adjust health spending for that period. The
concurrent model may risk adjust health care costs incurred in the
current year more fully by including acute conditions, such as a
broken leg, as well as acute exacerbations of chronic illnesses, such
as hospitalizations resulting from uncontrolled diabetes.[Footnote 26]
However, it may be appropriate to categorize the expenditures
associated with some complications as part of the physician's
performance, as opposed to factors outside of the physician's control
that require risk adjustment. For example, if a beneficiary needed to
be hospitalized because of poorly managed diabetes, it could be
appropriate to hold the physician accountable for those costs. CMS
used a prospective model in Phase II feedback reports.
CMS must also decide which factors, if any, should be added to the HCC
model. Although CMS officials believe the HCC risk adjustment model
adequately risk adjusted per capita costs, some stakeholders have
questioned CMS's use of the HCC model and have urged CMS to adjust for
additional factors that affect costs that CMS did not include. These
factors include some socioeconomic indicators, patient noncompliance,
and care setting. Some medical specialty societies and other
stakeholders have stated that if CMS does not risk adjust physician
resource use adequately, physicians could be discouraged from treating
atypical or disadvantaged populations that may be more costly to
treat. Although risk adjusting for additional factors could help
address these concerns, there may be a case for not including them.
For example, noncompliance with physicians' instructions may suggest
that physicians have not adequately educated their patients on the
importance of compliance. CMS officials also explained that they do
not want to adjust for factors that can provide meaningful information
about differences in practice patterns. CMS officials said that they
plan to continue using the HCC model to risk adjust per capita costs
in Phase III.
Selecting an attribution method. CMS faces trade-offs in determining
how to assign responsibility, or "attribute" beneficiaries' care to
physicians, in a way that promotes program goals. Program goals
include maximizing the number of physicians eligible for feedback
reports and encouraging care coordination, while also ensuring that
physicians are not held accountable for care they did not provide or
influence. Medicare fee-for-service beneficiaries may seek care from
any Medicare provider and often receive care from several physicians
and other providers. This makes it difficult to attribute
responsibility for all of the health care provided. Attributing care
to the physician who directly provided it may appear to be
straightforward, but it may not adequately reflect relative
responsibility for that care. For example, individual physicians may
have control over some costs directly incurred by another physician by
referring beneficiaries to specialists. Physicians may also indirectly
affect other health care costs by exercising their judgment regarding
hospital and postacute care decisions. As a result, determining to
whom a beneficiary's care should be attributed is an important
methodological decision.
In Phase I, CMS tested two attribution methods--a single and a
multiple provider attribution method.
* A single provider attribution method holds one physician responsible
for all of a beneficiary's care. This method is designed to identify
the principal "decision maker," such as the beneficiary's primary care
physician, and holds this physician responsible for all care provided,
including referrals and services provided by other physicians. The
single provider method CMS tested attributed a beneficiary's entire
cost of care to the single physician who provided the most evaluation
and management (E&M) services that the beneficiary received.
* A multiple provider attribution method holds more than one physician
responsible for the care provided to a beneficiary. This method
assumes that any one physician is unlikely to have complete
responsibility for all of that care. The multiple provider method CMS
tested held all physicians who billed for at least 10 percent of a
beneficiary's E&M costs partially responsible for that beneficiary's
care by attributing resource use in proportion to the amount of care
provided by a given physician.
In Phase II, CMS officials used a single provider attribution method.
The agency generally prefers single provider attribution, believing
that it encourages physicians to coordinate care. However, CMS has not
provided evidence that using a single provider attribution method
would lead to increased coordination, and physicians may not accept
this method as a credible way to attribute costs. According to
Mathematica officials, physicians profiled in Phase I generally
preferred the multiple provider attribution method. These physicians'
comments reflected concerns that it was unfair to attribute other
providers' resource use to them. Furthermore, most of the physicians
and other stakeholders who provided comments to Mathematica during
pretesting in Phase II thought it inappropriate to be held
accountable, even partially, for care provided by other physicians.
Both specialists and primary care physicians told Mathematica that
they did not have control over how another provider treated a
beneficiary. Specialists noted that they treated beneficiaries for
certain conditions and would not have knowledge of or be responsible
for care unrelated to those conditions. Similarly, Mathematica
reported that primary care physicians felt they had little control
over the care provided by the specialists to whom they referred
beneficiaries.
Despite physicians' concerns about being held responsible for care
they did not directly provide, they do have indirect control over some
costs incurred by other providers, such as referrals to specialists
and decisions about hospitalizations. Given that there is no
definitive way to determine which costs a physician was indirectly or
directly responsible for, a multiple provider attribution method may
be the more reasonable way to attribute costs. For example, the
multiple provider method CMS tested in Phase I held physicians
accountable for a proportion of the total care provided to a
beneficiary. Under this method, a physician who billed for 70 percent
of a beneficiary's total E&M services was assigned 70 percent of the
total Medicare resources used by that beneficiary--including office
visits, hospitalizations, skilled nursing facility stays, and
diagnostic tests and procedures.
A multiple provider attribution approach also increases the number of
physicians potentially eligible to receive feedback reports. Because
multiple provider attribution holds more than one physician
accountable for a beneficiary's care, more physicians will have
patients attributed to them, thus increasing the number of physicians
eligible for feedback reports. CMS officials recognize that using a
single attribution method will not allow all physicians to be eligible
to receive a report, and noted that it is likely that some Phase III
reports will use a multiple provider attribution method to assign
resource use to physicians.[Footnote 27]
CMS set a threshold for the minimum amount of care that a physician or
physician group needed to provide in order to be assigned
responsibility for all or part of that beneficiary's care. For
example, in Phase II, individual physicians needed to bill for at
least 20 percent of a beneficiary's total E&M costs, and physician
groups needed to bill for at least 30 percent of the total E&M costs
in order to be assigned responsibility for that beneficiary's care.
The minimum threshold was intended to reduce the likelihood that
physicians and groups would be assigned responsibility for
beneficiaries for whom they provided only minimal care. CMS is
considering setting a lower threshold in Phase III to increase the
number of physicians eligible to receive reports.
Determining minimum case size. CMS faces a challenging trade-off in
determining the minimum number of Medicare beneficiaries or episodes
of care a physician must have to produce reliable information without
excluding a large number of physicians--those without enough
beneficiaries or episodes--from receiving a report. A higher minimum
increases the reliability of the information, but decreases the number
of physicians eligible to receive a report. In contrast, decreasing
the minimum case size increases the number of physicians receiving
reports but reduces reliability.
In Phase I, CMS conducted a statistical reliability test to determine
the minimum number of episodes a physician needed to be eligible for a
feedback report. Reliability indicates how confidently one can
classify a physician's performance relative to that of his or her
peers. Estimates for this test range from zero to one, with an
estimate above 0.8 generally considered a strong indicator of
reliability. CMS used an estimate of 0.5--which is considered a
moderate level of reliability--to help ensure that enough physicians
would be eligible for Phase I reports.[Footnote 28] However, few
physicians met the minimum case size requirements for certain
episodes, such as acute myocardial infarctions, even when using this
moderate level of reliability. CMS did not conduct a reliability test
to determine the minimum number of beneficiaries a physician must
treat for per capita cost measurement.
In Phase II, CMS provided feedback reports to physicians with at least
30 Medicare beneficiaries attributed to them. CMS did not conduct
reliability tests for this estimate, stating that a minimum case size
of 30 is generally accepted in the research community. However, as
some stakeholders have noted, the appropriate minimum case size may
vary by condition, suggesting that CMS should instead use a measure of
reliability or precision to establish the appropriate case size.
[Footnote 29] For example, Phase II reports contained resource use
information for five high-cost, high-volume chronic conditions, and it
is likely that different minimum case sizes were needed to generate
reliable information for different conditions, such as diabetes and
coronary artery disease. CMS officials noted that minimum case size is
a major factor in excluding physicians from receiving feedback
reports. CMS officials have considered reducing the minimum case size
from 30 to 20 beneficiaries for Phase III reports. Officials analyzed
the potential effect of this change on individual physicians' per
capita resource use rankings, and found that nearly all physicians
were ranked in the same quartile when the case size was lowered from
30 to 20. According to CMS officials, this change would increase the
number of physicians eligible to receive Phase III reports by about 10
percent.
Selecting peer groups for comparisons. CMS faces trade-offs in
balancing stakeholders' preferences that feedback reports compare
physicians only to those most like themselves--that is, peer groups
representing narrow subspecialties or limited geographic areas--with
the need to establish a minimum peer group size that is large enough
to make statistically significant comparisons.
Individual-level feedback reports distributed in Phases I and II
contained two peer group comparisons: (1) physicians in the same
specialty in the same metropolitan area and (2) physicians in the same
specialty across all 12 metropolitan areas, which was meant to serve
as a proxy for a nationwide comparison.[Footnote 30] Some stakeholders
have encouraged CMS to compare physicians within a limited geographic
area. However, if a large number of physicians in a limited geographic
area were practicing inefficiently, a nationwide sample might be
needed to identify the inefficiencies.
In addition, some medical specialty societies and other stakeholders
urged CMS to compare physicians only within narrow subspecialties. For
example, the American Urological Association noted that surgeons with
active surgery practices are substantially different from those who
engage primarily in medical management of urological conditions, and
comparisons that do not differentiate between these distinct types of
physicians are not meaningful to physicians and do not promote
learning and improvement.
However, if CMS were to identify and compare physicians in smaller
subspecialties, it would face the challenge of ensuring that the peer
group size was large enough to make meaningful comparisons across
physicians. In Phase I, CMS did not impose a minimum peer group size,
but in Phase II it imposed a minimum peer group size of 30 physicians.
However, because not all individual physicians had peer groups
consisting of 30 physicians practicing in the same geographic area and
in the same specialty, some physicians received a report that did not
contain information on all performance measures. CMS officials said
they may use a minimum peer group size of 15 for Phase III feedback
reports.
CMS's Plans for Improvement May Not Fully Address Challenges in
Distributing Reports to Physicians:
The majority of sampled physicians were not eligible to receive a
Phase II report after CMS's methodological decisions were applied. CMS
officials plan to revise their methodology to increase eligibility for
Phase III reports, but significantly increasing the number of
physicians who are eligible will be challenging. Further, CMS faced
multiple challenges with the electronic distribution of feedback
reports to eligible physicians, and as a result, few physicians
accessed their reports. CMS officials plan to use a new distribution
method for Phase III reports.
Few Sampled Physicians Were Eligible to Receive a Feedback Report;
Significantly Increasing Eligibility Will Continue to Be Challenging:
Over 80 percent of CMS's initial sample of 9,189 physicians were
ineligible to receive a Phase II feedback report after CMS's
methodological decisions, such as minimum case size requirements, were
applied. To identify physicians for the Phase II reports, CMS began
with a sample of 9,189 individual physicians affiliated with 36
physician groups.[Footnote 31] To be eligible for a Phase II report,
individual physicians needed to meet CMS's criteria by having the
following:
* At least 30 Medicare beneficiaries attributed to them to meet the
minimum case size requirement for per capita resource use measures. Of
the 9,189 physicians in the original sample, 2,205 (24 percent) had at
least 30 beneficiaries attributed to them.
* At least 11 Medicare beneficiaries attributed to them who were
eligible for 1 or more of the 12 GEM quality measures. Of the 9,189
physicians in the original sample, 2,661 physicians (29 percent) had
at least 11 beneficiaries attributed to them who were eligible for at
least 1 of the 12 GEM quality measures.
* A sufficient number of attributed beneficiaries for both the per
capita resource use and GEM quality measures. Of the 9,189 physicians
in the original sample, 1,733 physicians (19 percent) had a sufficient
number of beneficiaries attributed to them for the per capita resource
use and GEM quality measures.
* At least 30 individual physicians in the same medical specialty and
geographic area for a peer group.[Footnote 32] Of the remaining 1,733
individual physicians, 1,645 physicians had a peer group of at least
30 individual physicians.[Footnote 33]
Figure 1 shows the number of physicians excluded by each criterion.
Figure 1: Number of Individual Physicians Excluded from Phase II
Feedback Reports Based on CMS's Methodological Criteria, 2010:
[Refer to PDF for image: illustration]
All sampled physicians:
CMS began with a sample of 9,189 individual physicians affiliated with
the 36 physician groups.
Case size and peer group requirements:
To be eligible for a report in the second phase of the program,
sampled physicians needed to meet CMS criteria by having the following:
At least 30 attributed beneficiaries for the resource use measures –
only 2,205 physicians met this requirement;
At least 11 attributed beneficiaries for at least one of the quality
measures – only 2,661 physicians met this requirement;
Only 1,733 physicians met both requirements;
At least 30 individual physicians practicing in the same medical
specialty and geographic area for a peer group – only 1,645 physicians
met this requirement.
Source: GAO analysis of CMS and contractor data.
[End of figure]
CMS's methodological criteria also excluded many specialists from
receiving feedback reports. Over 90 percent of Phase II reports were
created for generalists, such as internal medicine or family practice
physicians. The single provider attribution method used by CMS--which
assigned a beneficiary to the single physician who billed for the
greatest number of E&M services for the beneficiary--limited the
number of specialists eligible for a report, since specialists often
provide fewer but more expensive E&M services to beneficiaries than
generalists. Physicians also needed to have at least one GEM quality
measure to receive a Phase II report, but the GEM measures were only
applicable to a limited number of specialists, such as cardiologists
and nephrologists.
In addition, many of the 1,641 physicians eligible to receive a Phase
II feedback report did not meet the methodological criteria needed to
receive information on all performance measures, such as resource use
for the five chronic condition subgroups or the 12 GEM quality
measures. For example, only 5 percent of the 1,641 physicians eligible
for Phase II reports were eligible to receive resource use information
for their beneficiaries with chronic obstructive pulmonary disease,
and none were eligible to receive this information for their
beneficiaries with prostate cancer. Similarly, none of the 1,641
physicians eligible for Phase II reports were eligible to receive
information for 3 of the 12 GEM quality measures. By contrast, the
majority of the 36 physician groups profiled received information on
all performance measures (see table 1).
Table 1: Percentage of Individual Physicians and Physician Groups
Eligible for Select Resource Use and Quality Performance Measures on
Phase II Feedback Reports, 2010:
Resource use for chronic condition subgroups:
Congestive heart failure:
Individual physicians: 14;
Physician groups: 100.
Chronic obstructive pulmonary disease:
Individual physicians: 5;
Physician groups: 100.
Diabetes:
Individual physicians: 37;
Physician groups: 100.
Coronary artery disease:
Individual physicians: 39;
Physician groups: 100.
Prostate cancer:
Individual physicians: 0;
Physician groups: 100.
GEM quality measures:
LDL screening for beneficiaries up to 75 years of age with diabetes:
Individual physicians: 72;
Physician groups: 100.
Eye exam (retinal) for beneficiaries up to 75 years of age with
diabetes:
Individual physicians: 71;
Physician groups: 100.
HbA1c testing for beneficiaries up to 75 years of age with diabetes:
Individual physicians: 71;
Physician groups: 100.
Medical attention for nephropathy for diabetics up to 75 years of age:
Individual physicians: 35;
Physician groups: 100.
LDL-C screening for beneficiaries up to 75 years of age with
cardiovascular conditions:
Individual physicians: 38;
Physician groups: 100.
Beta blocker treatment after heart attack:
Individual physicians: 0;
Physician groups: 83.
Persistence of beta blocker treatment after heart attack:
Individual physicians: 0;
Physician groups: 83.
Colorectal cancer screening for beneficiaries up to 80 years of age:
Individual physicians: 99;
Physician groups: 100.
Breast cancer screening for women up to 69 years of age:
Individual physicians: 71;
Physician groups: 100.
Annual monitoring for beneficiaries on persistent medications
(angiotensin-converting enzyme inhibitors or angiotensin receptor
blockers, digoxin, diuretics, and anticonvulsants):
Individual physicians: 95;
Physician groups: 100.
Antidepressant medication management (acute phase):
Individual physicians: 0;
Physician groups: 86.
Disease-modifying antirheumatic drug therapy in rheumatoid arthritis:
Individual physicians: 3;
Physician groups: 100.
Source: GAO analysis of CMS and contractor data.
[End of table]
As we stated earlier in this report, CMS is considering a number of
methodological changes in Phase III, such as using a multiple provider
attribution rule and lowering the minimum case size and peer group
requirements. While such changes could lead to a modest increase in
physician eligibility for Phase III reports, significantly increasing
eligibility--particularly for individual physicians with small case
sizes--will continue to be challenging.
Multiple Challenges with Distribution Resulted in Few Physicians
Accessing Their Electronic Feedback Reports:
CMS faced multiple challenges distributing Phase II feedback reports,
and as a result of these challenges, few physicians accessed their
reports. In November 2010, CMS mailed letters to 36 physician groups
and 1,641 individual physicians affiliated with those groups to notify
them that electronic feedback reports were available for their review.
However, as of March 2011--approximately 4 months later--less than 60
percent of physician groups and less than 10 percent of individual
physicians had accessed their reports electronically.[Footnote 34]
Major challenges with Phase II distribution were CMS's difficulty
obtaining physicians' contact information, methods of electronic
distribution that were burdensome for physicians, and lack of a strong
incentive for physicians to review the reports.
Contact information. The lack of a comprehensive database with
accurate names and addresses for physicians and physician groups made
it difficult for CMS to notify physicians and physician groups about
the availability of their feedback reports. Although reports in Phase
II were produced in electronic form, CMS mailed hard copy notification
letters to tell individual physicians and physician groups that an
electronic feedback report was available and to provide instructions
for accessing it. Because available databases contained incomplete or
conflicting contact information, CMS had to use multiple sources,
including Internet searches, to compile names and addresses--a process
that took approximately 5 months.[Footnote 35]
Despite CMS's efforts to obtain accurate contact information, some
individual physicians and physician groups did not receive a
notification letter and therefore did not know that a feedback report
was available to them. In follow-up phone calls, CMS found that 27 of
the 32 physician groups reached reported that they had not seen the
notification letter and could not verify whether it had been received.
Many of these physician groups reported that the notification letter
was not addressed to the most appropriate person within the group
practice, such as the director of quality assurance. CMS also called a
sample of 10 individual physicians to ask whether they had received
the notification letter. Of these physicians, 1 was retired, 1
reported not receiving the letter, and the remaining 8 had no memory
of receiving the letter. In addition, nearly 10 percent of the
notification letters mailed to individual physicians were marked
undeliverable and returned to CMS.
Distribution method. CMS's electronic distribution method for Phase II
reports was burdensome for some profiled physicians and physician
groups. CMS transitioned from hard copy distribution of feedback
reports in Phase I to electronic distribution in Phase II based on
physicians' complaints that the reports distributed in Phase I were
too long and cumbersome to manage in hard copy. According to CMS,
electronic distribution was meant to help physicians navigate the
reports. CMS used two methods to electronically distribute feedback
reports in Phase II--one for individual physicians and one for
physician groups.
Individual physicians were instructed in the notification letter to
contact their Medicare Administrative Contractor (MAC) to request a
copy of their feedback report.[Footnote 36] In a report to CMS,
Mathematica reported that finding contact information for the correct
MAC may not have been a straightforward process for physicians. For
example, the notification letter directed physicians to a directory
with toll-free phone numbers listed by state for all MAC contact
centers, requiring physicians to choose from several possible
numbers.[Footnote 37] Mathematica also reported that MAC customer
service representatives were not always aware of the feedback reports
or the process for distributing them to physicians. According to CMS's
estimate, the majority of individual physicians did not contact their
MACs to request their reports. In February 2011, CMS mailed hard
copies of the 1,596 feedback reports that had not yet been
electronically accessed by individual physicians.[Footnote 38]
In theory, the electronic distribution method for physician groups
should have been more straightforward since groups were instructed to
download their feedback reports from the Individuals Authorized Access
to CMS Computer Services (IACS) system, which is the same system used
to distribute PQRS reports. However, 8 of the 32 physician groups CMS
reached in its follow-up calls reported difficulty downloading their
reports from the IACS system. For example, some groups did not know
that they needed to register for an IACS account--a process that takes
approximately 10 business days to complete--while others reported not
being able to download their feedback reports even after logging onto
the IACS system. CMS subsequently e-mailed feedback reports directly
to those physician groups that had trouble downloading their reports
through IACS.
CMS officials recognized the limitations with the distribution method
for Phase II reports, and they plan to use a new distribution method
for Phase III reports. CMS currently plans to distribute reports to
20,000 individual physicians in one four-state region--Nebraska,
Missouri, Iowa, and Kansas. According to CMS officials, the MAC
serving this region has e-mail addresses for most physicians in the
area. CMS plans to e-mail Phase III reports directly to physicians in
this region, thereby avoiding the need to mail hard copy notification
letters. In addition to distributing reports to individual physicians
in the four-state region, CMS also plans to distribute Phase III
reports to 35 physician groups that have participated in the PQRS
group practice reporting option. CMS intends to e-mail feedback
reports to these 35 physician groups.
Incentive to access reports. Physicians did not have a strong
incentive to access their Phase II feedback reports. The notification
letter sent by CMS said that these reports were "for informational
purposes only" and that they would not affect physicians'
participation in the Medicare program or their Medicare payments. In
pretesting for Phase II, many physicians noted that they would be
unlikely to review a feedback report closely unless they had an
incentive to do so. CMS officials said that they did not want to
emphasize that the types of cost and quality measures contained in the
feedback reports could affect physicians' payments in the future
because they did not want the reports to sound threatening. Several
physician groups suggested that CMS send feedback reports for those
physicians affiliated with a group practice to the group's
administrator, noting that individual physicians generally contact
their administrators for guidance on such reports. In a report to CMS,
Mathematica also noted that medical directors or others with quality
oversight responsibilities in larger group practices would be more
receptive to feedback reports than individual physicians. They added
that these individuals are more familiar with the data used to create
feedback reports, and have more experience analyzing quality and cost
information for practice improvement.
CMS made follow-up calls to representatives of 15 of the 36 profiled
physician groups to obtain their input on the feedback reports, but it
conducted minimal follow-up with individual physicians. At the time
CMS attempted to follow up with individual physicians, only 4 had
contacted their MACs to request a feedback report. Three of these
physicians were unwilling to participate in a follow-up call about the
report, and 1 physician was unable to download the feedback report
that had been sent via e-mail. Similarly, CMS called a sample of 10
physicians who had not requested their feedback reports to ask why
they had not done so, and 8 of these physicians expressed no interest
in their reports.
Conclusions:
In light of concerns about the long-term fiscal challenges facing the
Medicare program, the Physician Feedback Program is an important
effort that could encourage more efficient medical practice as well as
higher-quality care. CMS has worked under challenging timelines to
test different approaches to feedback report methodology and
distribution. Initial phases of the program indicate that significant
changes will need to be made for the program to meet its goal of
producing reports with meaningful, actionable, and fair performance
measures that apply to the majority of Medicare physicians. CMS will
need to do more to solicit input and reactions from physicians and
physician groups on the methodology and distribution of reports while
the stakes are still relatively low--that is, before CMS begins paying
physicians based on their performance on the resource use and quality
measures included in the feedback reports beginning January 1, 2015.
In the first two phases, CMS tested different methodological
approaches to developing feedback reports; however, the majority of
physicians in the most recent phase were ineligible for a feedback
report once CMS's methodological criteria were applied. For example,
CMS used a single provider attribution method in the most recent
phase, believing that it may improve care coordination--but this
method limited physician eligibility, and there is limited evidence to
suggest that using this method would increase coordination. And while
we also agree with CMS's decision to include quality measures in
feedback reports, some physicians who would have been eligible to
receive information on their resource use were disqualified from
receiving a Phase II report because they were not eligible for at
least 1 of the 12 GEM quality measures. Further, none of the
individual physicians who were eligible for a Phase II report had
enough beneficiaries attributed to them to receive performance data on
all 12 quality measures. CMS did not face such sample size issues in
the feedback reports it developed for physician groups.
CMS has not conducted the rigorous statistical analysis it needs to
fully understand the impact of its methodological decisions on
reliability. For example, CMS used a minimum case size of 30
beneficiaries for Phase II reports, but did not conduct reliability
testing to determine this number. The results of such testing can and
should influence how CMS ultimately uses the information. Lower levels
of reliability may be acceptable if feedback reports remain
confidential and are used solely for educational purposes. However,
since CMS ultimately intends to pay physicians based on their
performance as measured in the feedback reports, it must be reasonably
confident that these measures reflect real differences in medical
practice. It will also be difficult for CMS to obtain physician and
stakeholder buy-in if it does not clearly demonstrate that its
performance measures are reliable and robust.
Furthermore, CMS faces challenges distributing feedback reports to
physicians and physician groups that are eligible to receive them. CMS
transitioned to electronic distribution based on physicians'
complaints that hard copy reports were too long and cumbersome, yet
few physicians accessed their Phase II reports electronically.
Moreover, CMS conducted limited follow-up with profiled physicians to
obtain their input on the feedback reports. As a result, the agency
missed an important opportunity to increase physician engagement in
the program and to ensure that their concerns are addressed while the
program is still in its infancy.
Recommendations for Executive Action:
In order to develop feedback reports that are more reliable, credible,
accessible, and applicable to a greater number of Medicare physicians,
we recommend that the Administrator of CMS take the following four
actions:
* Use methodological approaches that increase the number of physicians
eligible to receive a report, such as:
- multiple provider attribution methods, which could also enhance
credibility of the reports with physicians, and:
- distributing feedback reports that include only resource use
information, if quality information is unavailable.
* Conduct statistical analyses of the impact of key methodological
decisions on reliability.
* Identify factors that may have prevented physicians from accessing
their reports and, as applicable, develop strategies to improve the
process for distributing reports and facilitating physicians' access
to them.
* Obtain input from a sample of physicians who received feedback
reports on the usefulness and credibility of the performance measures
contained in the reports and consider using this information to revise
future reports.
Agency Comments and Our Evaluation:
We received written comments on a draft of this report from CMS, which
are reprinted in appendix I. CMS concurred with our recommendations
and identified actions agency officials are taking to implement them.
These actions include refining the attribution methodology to increase
the number of physicians receiving feedback reports in Phase III,
analyzing the number of cases required to reliably measure quality and
make credible comparisons, developing new strategies for distributing
feedback reports, and obtaining input from individual physicians and
physician groups about the information contained in the feedback
reports. If these actions are implemented in accordance with our
recommendations, CMS will be better positioned to meet its goals and
objectives for the Physician Feedback Program. CMS also provided
technical comments, which we incorporated as appropriate.
We are sending copies of this report to the Administrator of CMS and
relevant congressional committees. The report also will be available
at no charge on the GAO website at [hyperlink, http://www.gao.gov].
If you or your staffs have any questions about this report, please
contact me at (202) 512-7114 or cosgrovej@gao.gov. Contact points for
our Offices of Congressional Relations and Public Affairs may be found
on the last page of this report. GAO staff members who made major
contributions to this report are listed in appendix II.
Signed by:
James C. Cosgrove:
Director, Health Care:
[End of section]
Appendix I: Comments from the Department of Health and Human Services:
Department Of Health & Human Services:
Office Of The Secretary:
Assistant Secretary for Legislation:
Washington, DC 20201:
July 29, 2011:
James Cosgrove:
Director, Health Care:
U.S. Government Accountability Office:
441 G Street N.W.
Washington, DC 20548:
Dear Mr. Cosgrove:
Attached are comments on the U.S. Government Accountability Office's
(GAO) draft report entitled: "Medicare Physician Feedback Program: CMS
Faces Challenges with Methodology and Distribution of Physician
Reports" (GAO 11-720).
The Department appreciates the opportunity to review this draft report
prior to publication.
Sincerely,
Signed by:
Jim R. Esquea:
Assistant Secretary for Legislation:
Attachment:
[End of letter]
General Comments Of The Department Of Health And Human Services (HHS)
On The Government Accountability Office's (GAO) Draft Report Entitled,
"Medicare Physician Feedback Program: CMS Faces Challenges With
Methodology And Distribution Of Physician Reports (GAO-11-720):
The Department appreciates the opportunity to review and comment on
this draft report, which is a factual synopsis of Phase I and Phase II
of the Physician Feedback Program, formerly called the Physician
Resource Use Measurement and Reporting Program.
This program was established as a confidential feedback program as
required by the Medicare Improvements for Patient and Providers Act of
2008 (MIPPA). The program was further modified as a result of the
Affordable Care Act in 2010, which also requires a value-based payment
modifier that provides for differential payments to specified
physicians based on the quality of care furnished compared to cost.
The value-based payment modifier would be applied to specified
physicians beginning in 2015 and all physicians starting in 2017.
Hence, we expect that this program, which started small with
flexibility in the number and content of reports disseminated, will
quickly evolve to affect physician payment nationwide.
GAO Recommendations:
In order to develop feedback reports that are more reliable, credible,
accessible, and applicable to a greater number of Medicare physicians,
we recommend that the Administrator of the Centers for Medicare and
Medicaid Services (CMS):
* use methodological approaches that increase the number of physicians
eligible to receive a report, such as:
- multiple provider attribution methods, which could also enhance
credibility of the reports with physicians, and;
- distributing feedback reports that include only resource use
information, if quality information is unavailable;
CMS Response:
We concur with this recommendation. As we discussed with the GAO, we
are planning in Phase III to refine the attribution models we used in
Phase II, and to test new ones. As a result, the number of physicians
receiving a report in Phase III will increase. This attribution
methodology will permit us to distribute feedback reports that include
only resource use information if quality information is unavailable.
* conduct statistical analyses of the impact of key methodological
decisions on reliability;
CMS Response:
We concur with this recommendation. We will be analyzing the results
in these reports for reliability as well as analyzing the number of
cases required to reliably measure quality and make credible
comparisons.
* identify factors that may have prevented physicians from accessing
their reports and as applicable, develop strategies to improve the
process for distributing reports and facilitating physicians' access
to them.
CMS Response:
We concur with this recommendation. We are working to develop new
strategies for distributing reports that will improve physician
access. To this end, for Phase III, we are working with the
Jurisdiction 5 (J5) Medicare Administrative Contractor (MAC) which
serves the four-State region of Nebraska, Kansas, Missouri, and Iowa.
This MAC has an advanced communications portal which allows robust
communications between the MAC and physicians in these States. In
particular, the MAC has e-mail information on a large number of the
physicians it serves and we expect to use this list to provide
individual physicians with their feedback reports. We are also working
to develop an enterprise-wide solution that could be used to reach all
physicians nationwide to provide feedback reports.
* obtain input from a sample of physicians who received feedback
reports on the usefulness and credibility of the performance measures
contained in the reports and consider using this information to revise
future reports.
CMS Response:
We concur with this recommendation. We will be working closely across
CMS, with J5 MAC, and with State stakeholders to inform physicians in
these four States that they will be receiving the feedback reports.
Through these mechanisms we will reach out to these physicians about
the information contained within the feedback reports, how the reports
can help them understand the quality of care their Medicare patients
receive and the resources used to provide this care. We will also
obtain feedback from the recipients of the group reports. With both
individual and group report recipients, we will discuss the importance
of these reports as beginning to provide the information building
blocks that could be used to calculate their value modifier. We
anticipate that this outreach will help spur physicians to participate
in focus groups following report dissemination to discuss these issues.
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
GAO Contact:
James C. Cosgrove, (202) 512-7114 or cosgrovej@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Phyllis Thorburn, Assistant
Director; William A. Crafton; Cathleen Hamann; Julian Klazkin; Amanda
Pusey; Jessica C. Smith; and Rachael Wojnowicz made key contributions
to this report.
[End of section]
Footnotes:
[1] Medicare is the federally financed health insurance program for
persons aged 65 and over, certain individuals with disabilities, and
individuals with end-stage renal disease. Medicare Part A covers
hospital and other inpatient stays. Medicare Part B covers physician,
outpatient hospital, home health, and other services. Medicare Parts A
and B are known as original Medicare or Medicare fee-for-service.
[2] Resource use can be defined as the costs to the Medicare program,
including those contributions by Medicare beneficiaries, such as co-
payments and deductibles. See GAO, Medicare: Focus on Physician
Practice Patterns Can Lead to Greater Program Efficiency, [hyperlink,
http://www.gao.gov/products/GAO-07-307] (Washington, D.C.: Apr. 30,
2007); GAO, Medicare: Per Capita Method Can Be Used to Profile
Physicians and Provide Feedback on Resource Use, [hyperlink,
http://www.gao.gov/products/GAO-09-802] (Washington, D.C.: Sept. 25,
2009); Medicare Payment Advisory Commission, Report to the Congress:
Medicare Payment Policy (Washington, D.C.: Mar. 2005), 142; and CBO,
Medicare's Payments to Physicians: Options for Changing the
Sustainable Growth Rate (Washington, D.C.: Mar. 1, 2007), 16-17.
[3] Pub. L. No. 110-275, § 131(c), 122 Stat. 2494, 2526.
[4] CMS has distributed feedback reports to various health care
providers--primarily physicians--as well as nurse practitioners and
physician's assistants. For this report, we refer to providers as
physicians and provider groups as physician groups.
[5] Pub. L. No. 111-148, § 3007, 124 Stat. 119, 373-376 (codified at
42 U.S.C. § 1395w-4(p)).
[6] 42 U.S.C. § 1395w-4(p)(4)(B)(iii).
[7] 42 U.S.C. § 1395w-4(p)(9).
[8] Pub. L. No. 110-275, § 131(c)(2), 121 Stat. 2494, 2527.
[9] Throughout this report, we generally attribute the analysis and
actions taken by Mathematica to CMS.
[10] See Medicare Program; Payment Policies Under the Physician Fee
Schedule and Other Revisions to Part B for CY2011; Proposed Rule, 75
Fed. Reg. 40040 (July 13, 2010). We identified 42 letters related to
the Physician Feedback Program that were submitted in response to the
proposed rule via the website [hyperlink, http://www.regulations.gov].
Sixteen of the letters we reviewed were from medical specialty
societies; 25 of the letters were from other stakeholders, such as the
National Business Group on Health; and 1 letter was from a medical
specialty society and one other organization that commented jointly.
[11] For chronic conditions, which do not have clearly defined start
or end dates, episodes of care may be measured over a specified time
period, such as on a 12-month basis.
[12] PPACA requires that CMS develop a Medicare-specific episode
grouper by January 1, 2012. Pub. L. No. 111-148, § 3003(a)(4), 124
Stat. 119, 366-8 (codified at 42 U.S.C. § 1395w-4(n)(9)(A)).
[13] Other types of measures can also be used to evaluate the quality
of care, such as tracking a patient's experience with health care
services.
[14] NQF is a nonprofit organization that fosters agreement on
national standards for measuring and public reporting of health care
performance data. NCQA is a national nonprofit organization that
develops health care quality and performance standards and accredits
health plans, physicians, and other health care providers.
[15] HEDIS® is a group of standardized measures used to measure
clinical performance in areas such as medication use, control of high
blood pressure, breast cancer screening, immunization, and
comprehensive diabetes care.
[16] The areas were those included in an ongoing Community Tracking
Study (CTS) being conducted by a research organization, the Center for
Studying Health System Change. The CTS sites were designated because
they provide a random sample of communities that represent different
geographic areas, populations, physician and health care market
structures, patterns of Medicare spending, and experience with public-
or private-sector performance measurement. They were Boston,
Massachusetts; Cleveland, Ohio; Greenville, South Carolina;
Indianapolis, Indiana; Lansing, Michigan; Little Rock, Arkansas;
Miami, Florida; Northern New Jersey; Orange County, California;
Phoenix, Arizona; Seattle, Washington; and Syracuse, New York.
[17] Physician groups were selected based on the following criteria:
that they have at least 5,000 Medicare beneficiaries in 2007 and at
least one physician who participated in the PQRS program since it
began in 2007.
[18] PPACA requires CMS to seek endorsement of the episode grouper by
the consensus-based entity that has a contract for performance
measurement under the Medicare program. Currently, that contract is
with NQF. See Pub. L. No. 111-148, § 3003(a)(4), 124 Stat. 119, 366-7
(codified at 42 U.S.C. § 1395w-4(n)(9)(A)(iv)); 42 U.S.C. §
1395aaa(a)(1).
[19] Two contracts were awarded to make the existing commercially
available software more usable for the Medicare population, and two
contracts were awarded to have a new episode grouper constructed.
[20] CMS officials explained that they found that prostate cancer was
rarely reported in their sample.
[21] Including quality measures in feedback reports is optional. 42
U.S.C. § 1395w-4(n)(1)(A)(iii).
[22] Phase II feedback reports also provided a link to CMS's Hospital
Compare, Nursing Home Compare, and Home Health Compare websites to
provide information on the quality of the hospitals used by the
physician's beneficiaries and nursing homes and home health agencies
in the physician's metropolitan area. In addition, physician group
feedback reports contained six ambulatory care sensitive conditions,
which are medical conditions for which timely and coordinated
outpatient care could have prevented the need for hospitalization.
These include congestive heart failure and dehydration.
[23] A recent Agency for Healthcare Research and Quality (AHRQ) report
also noted that other beneficial aspects of administrative data,
including claims data, are that they are relatively inexpensive to
acquire in electronic formats, coded by health information
professionals using accepted coding systems, and drawn from large
populations and therefore more representative of the populations of
interest. However, the report states that administrative data are
limited in that because most administrative data are intended for
financial management rather than quality assessment, they contain
varying degrees of clinical detail and are often limited in content,
completeness, timeliness, and accuracy. Patrick Romano, Peter Hussey,
and Dominique Ritley, Selecting Quality and Resource Use Measures: A
Decision Guide for Community Quality Collaboratives, Final Contract
Report (prepared by the University of California and RAND Corporation,
under contract No. 08003967), AHRQ Publication No. 09(10)-0073
(Rockville, Md.: AHRQ, May 2010).
[24] CMS considered, but decided not to include, measures from the
PQRS program in Phase II feedback reports because of current
limitations, such as physicians' low participation rate in the program
and because physicians had flexibility to choose which measures to
report under PQRS. CMS officials stated that as a result of these
limitations, it would have been difficult to make meaningful
comparisons using PQRS measures.
[25] CMS officials stated that they also plan to include measures from
the Health Information Technology for Economic and Clinical Health Act
in future feedback reports.
[26] CMS has used a concurrent risk adjustment model for its Physician
Group Practice Demonstration.
[27] CMS officials explained that one way to increase physician
eligibility is to use more than one attribution rule in future
physician feedback reports; for example, the attribution rule CMS uses
could vary by physician specialty.
[28] The minimum number of episodes required varied by physician
specialty and condition.
[29] A measure of reliability or precision could include, for example,
a confidence interval.
[30] In Phase II, physician groups were compared to other physician
groups in 12 metropolitan areas.
[31] CMS's initial sample consisted of individual physicians who were
affiliated with 1 of the 36 physician groups in 2007, the year of
Medicare claims data from which the performance measures in the
reports were derived; practiced in 1 of the 12 metropolitan areas
selected for Phase II report distribution; were considered eligible
for beneficiary attribution based on select criteria; and had a valid
Unique Physician Identification Number (UPIN) in 2007. The UPIN has
been changed to the National Provider Identifier.
[32] For example, a cardiologist practicing in Miami, Florida, had to
have a peer group of at least 30 other cardiologists in Miami with at
least 30 attributed beneficiaries for the resource use measure and at
least 11 attributed beneficiaries for at least one GEM quality measure
relevant to cardiologists, such as the percentage of patients
receiving beta blocker treatment after a heart attack.
[33] Four of the 1,645 physicians were disqualified because CMS could
not identify their National Provider Identifier or could not locate a
verifiable address; as a result, CMS created Phase II feedback reports
for 1,641 individual physicians.
[34] As of March 2011, 20 of 36 profiled physician groups had logged
onto the Individuals Authorized Access to CMS Computer Services (IACS)
system. Because this database is used for a number of purposes, CMS
was unable to determine how many of these groups actually downloaded
their Phase II feedback reports.
[35] These sources include the Provider Enrollment, Chain, and
Ownership System (PECOS) database; the National Plan and Provider
Enumeration System (NPPES) database; and the IACS system. CMS intended
to use PECOS as the primary source of contact information but found
multiple mailing addresses listed for individual physicians and
physician groups. PECOS also did not clearly identify the most
appropriate contact person within a physician group, such as the
director of quality assurance. As a result, CMS used other sources in
order to obtain contact information, such as NPPES and IACS, but in
some cases, the names and addresses listed in the other sources did
not match any of the information listed in PECOS. CMS ultimately
developed decision rules to select contact information for individual
physicians and physician groups from competing sources, and in some
cases, relied on Internet searches.
[36] Once contacted, the MAC verified the identity of the requesting
physician and then forwarded the request to Buccaneer Computer Systems
and Services, Inc. (BCSSI), a CMS contractor. BCSSI then e-mailed the
feedback report to the physician.
[37] According to CMS officials, the MAC directory lists several
telephone numbers for the convenience of their customers. CMS
officials reported that physicians can sort the directory to find the
correct MAC contact number in their state.
[38] CMS mailed hard copy feedback reports to the 1,596 physicians who
had not accessed their electronic reports by January 2011.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
