Medicare
Per Capita Method Can Be Used to Profile Physicians and Provide Feedback on Resource Use Gao ID: GAO-09-802 September 25, 2009The Medicare Improvements for Patients and Providers Act of 2008 directed the Secretary of Health and Human Services to develop a program to give physicians confidential feedback on the Medicare resources used to provide care to Medicare beneficiaries. GAO was asked to evaluate the per capita methodology for profiling physicians--a method which measures a patient's resource use over a fixed period of time and attributes that resource use to physicians--in order to assist the Centers for Medicare & Medicaid Services (CMS) with the development of a physician feedback approach. In response, this report examines (1) the extent to which physicians in selected specialties show stable practice patterns and how beneficiary utilization of services varies by physician resource use level; (2) factors to consider in developing feedback reports on physicians' performance, including per capita resource use; and (3) the extent to which feedback reports may influence physician behavior. GAO focused on four medical specialties and four metropolitan areas chosen for their geographic diversity and range in average Medicare spending per beneficiary. To identify considerations for developing a physician feedback system, GAO reviewed the literature and interviewed officials from health plans and specialty societies. Further, GAO drew upon literature and interviews to develop an illustration of how per capita measures could be included in a physician feedback report.
Using 2005 and 2006 Medicare claims data and a per capita methodology, GAO found that specialist physicians showed considerable stability in resource use despite high patient turnover. This stability suggests that per capita resource use is a reasonable approach for profiling specialist physicians because it reflects distinct patterns of a physician's resource use, not the particular population of beneficiaries seen by a physician in a given year. GAO also found that our per capita method can differentiate specialists' patterns of resource use with respect to different types of services, such as institutional services, which were a major factor in beneficiaries' resource use. In particular, patients of high resource use physicians used more institutional services than patients of low resource use physicians. GAO identified four key considerations in developing feedback reports on physician performance. To illustrate how per capita measures could be included in a physician feedback report, we developed a mock report containing three types of per capita measures. Although the literature suggested that feedback alone has no more than a moderate influence on physicians' behavior, the potential influence of feedback from CMS on Medicare costs may be greater, in part because of the relatively large share of physicians' practice revenues that Medicare typically represents. CMS reviewed a draft of this report and broadly agreed with our findings.
GAO-09-802, Medicare: Per Capita Method Can Be Used to Profile Physicians and Provide Feedback on Resource Use
This is the accessible text file for GAO report number GAO-09-802
entitled 'Medicare: Per Capita Method Can Be Used to Profile Physicians
and Provide Feedback on Resource Use' which was released on
October 26, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Health, Committee on Ways and
Means, House of Representatives:
United States Government Accountability Office:
GAO:
September 2009:
Medicare:
Per Capita Method Can Be Used to Profile Physicians and Provide
Feedback on Resource Use:
GAO-09-802:
GAO Highlights:
Highlights of GAO-09-802, a report to the Chairman, Subcommittee on
Health, Committee on Ways and Means, House of Representatives.
Why GAO Did This Study:
The Medicare Improvements for Patients and Providers Act of 2008
directed the Secretary of Health and Human Services to develop a
program to give physicians confidential feedback on the Medicare
resources used to provide care to Medicare beneficiaries. GAO was asked
to evaluate the per capita methodology for profiling physicians”a
method which measures a patient‘s resource use over a fixed period of
time and attributes that resource use to physicians”in order to assist
the Centers for Medicare & Medicaid Services (CMS) with the development
of a physician feedback approach. In response, this report examines (1)
the extent to which physicians in selected specialties show stable
practice patterns and how beneficiary utilization of services varies by
physician resource use level; (2) factors to consider in developing
feedback reports on physicians‘ performance, including per capita
resource use; and (3) the extent to which feedback reports may
influence physician behavior. GAO focused on four medical specialties
and four metropolitan areas chosen for their geographic diversity and
range in average Medicare spending per beneficiary. To identify
considerations for developing a physician feedback system, GAO reviewed
the literature and interviewed officials from health plans and
specialty societies. Further, GAO drew upon literature and interviews
to develop an illustration of how per capita measures could be included
in a physician feedback report.
What GAO Found:
Using 2005 and 2006 Medicare claims data and a per capita methodology,
GAO found that specialist physicians showed considerable stability in
resource use despite high patient turnover. This stability suggests
that per capita resource use is a reasonable approach for profiling
specialist physicians because it reflects distinct patterns of a
physician‘s resource use, not the particular population of
beneficiaries seen by a physician in a given year. GAO also found that
our per capita method can differentiate specialists‘ patterns of
resource use with respect to different types of services, such as
institutional services, which were a major factor in beneficiaries‘
resource use. In particular, patients of high resource use physicians
used more institutional services than patients of low resource use
physicians.
GAO identified four key considerations in developing feedback reports
on physician performance (see table).
Table: Key Considerations in Developing Physician Feedback Reports:
General considerations: Report content; Examples of specific
considerations: Types of measures, comparative benchmarks.
General considerations: Report design; Examples of specific
considerations: Length, organization, graphics.
General considerations: Report dissemination; Examples of specific
considerations: Which physicians should receive reports, frequency of
reporting, hardcopy versus electronic dissemination.
General considerations: Transparency; Examples of specific
considerations: Information about purpose, methods, data.
Source: GAO.
[End of table]
To illustrate how per capita measures could be included in a physician
feedback report, we developed a mock report containing three types of
per capita measures.
Although the literature suggested that feedback alone has no more than
a moderate influence on physicians‘ behavior, the potential influence
of feedback from CMS on Medicare costs may be greater, in part because
of the relatively large share of physicians‘ practice revenues that
Medicare typically represents.
CMS reviewed a draft of this report and broadly agreed with our
findings.
View [hyperlink, http://www.gao.gov/products/GAO-09-802] or key
components. For more information, contact A. Bruce Steinwald at (202)
512-7114 or steinwalda@gao.gov.
[End of section]
Contents:
Letter:
Background:
Per Capita Profiling Method Shows Specialist Physicians' Practice
Patterns Relatively Stable Over 2 Years; Patients of High Resource Use
Physicians Used More Institutional Services Than Other Patients:
Research Literature, Health Insurers, and Specialists Identified
Considerations in Developing Physician Feedback Reports on Resource
Use:
Potential Influence of Feedback Regarding Medicare Costs on Physician
Behavior Is Uncertain:
Concluding Observations:
Agency and Professional Association Comments and Our Evaluation:
CMS Comments:
AAOS and ACC comments:
Appendix I: Methodology:
Appendix II: Comments from the Centers for Medicare & Medicaid
Services:
Appendix III: GAO Contact and Staff Acknowledgments:
Bibliography:
Tables:
Table 1: Selected Events Preceding CMS Providing Physicians Feedback on
Their Medicare Resource Use:
Table 2: Average Stability of Physicians' Resource Use by Specialty--
Averaged Across Four Metropolitan Areas, 2005-2006:
Table 3: Average Stability of Physicians' and Beneficiaries' Resource
Use by Metropolitan Area--2005-2006:
Table 4: Key Considerations in Developing Physician Feedback Reports:
Table 5: Comments and Suggestions for Designing Physician Feedback
Reports:
Table 6: How the Nation's Five Largest Health Insurers Make Patient-
Level Profiling Data Available to Physicians for Review and Appeal:
Figures:
Figure 1: Stability of Medicare Beneficiaries' and Specialist
Physicians' Resource Use--Averaged Across Four Metropolitan Areas and
Four Physician Specialties, 2005-2006:
Figure 2: Share of Total Medicare Expenditures per Beneficiary for
Services Provided by Physicians to their Patients, Institutional
Services, and All Other Services--Practice Average Across Four
Specialties in Four Metropolitan Areas, 2006:
Figure 3: Mock Physician Feedback Report Illustrating Per Capita
Measures:
Figure 4: Example of a Hover in a Mock Physician Feedback Report:
Figure 5: Beneficiary Resource Use by Health Category for Quintiles of
Physician Resource Use--Four Specialties in Four Metropolitan Areas,
2006:
Abbreviations:
CBO: Congressional Budget Office:
CBSA: Core-Based Statistical Area:
CMS: Centers for Medicare & Medicaid Services:
FFS: fee-for-service:
HCC: Hierarchical Condition Category:
HHS: Department of Health and Human Services:
MedPAC: Medicare Payment Advisory Commission:
MIPPA: Medicare Improvements for Patients and Providers Act of 2008:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
September 25, 2009:
The Honorable Pete Stark:
Chairman:
Subcommittee on Health:
Committee on Ways and Means:
House of Representatives:
Dear Mr. Chairman:
In recent years, evidence has mounted that the Medicare program is
unsustainable in its present form.[Footnote 1] Because of rising health
care costs and the aging of baby boomers into eligibility for Medicare,
future program spending is projected to consume an increasing share of
the government's resources. In their 2009 annual report, the Medicare
Trustees projected that Medicare expenditures, which reached $468
billion in 2008, will increase in future years at a faster pace than
the overall economy, rising from 3.2 percent of gross domestic product
in 2008 to 11.4 percent by 2083.
Physicians play a central role in the generation of health care
expenditures, through both the services they provide and the services
they order, including hospital admissions, diagnostic tests, and
referrals to other physicians. The evidence suggests that some of the
spending for services provided and ordered by physicians may not be
warranted. For example, the wide variation in Medicare spending for
physician services--unrelated to beneficiary health status or outcomes--
indicates that health needs alone do not determine spending.
Consistent with physicians' central role in providing and ordering
services and their influence on the amount of spending for patient
services, physician groups, insurers, and Medicare officials have
turned to profiling as a possible tool to help identify and contain
overuse of services and the resulting high expenditures. In profiling,
the resource use of a physician's:
patients is compared to a benchmark[Footnote 2]. In our previous report
on profiling, Medicare: Focus on Physician Practice Patterns Can Lead
to Greater Program Efficiency,[Footnote 3] we profiled generalist
physicians and found that in each of the 12 metropolitan areas we
studied there were physicians who, relative to their peers in the same
area, treated a disproportionate share of overly expensive patients. In
that report we used a profiling methodology known as per capita, which
measures per patient resource use for a defined population over a fixed
period of time and attributes that resource use to physicians. We
recommended that the Administrator of the Centers for Medicare &
Medicaid Services (CMS) develop a profiling system to identify
individual physicians with inefficient practice patterns and provide
incentives for physicians to improve the efficiency of care they
provide.[Footnote 4] In our subsequent testimony on physician feedback
to the Subcommittee on Health of the House Ways and Means Committee we
stated that providing feedback to physicians on their practice patterns
could be a promising step toward encouraging efficiency in
Medicare.[Footnote 5] The Medicare Payment Advisory Commission (MedPAC)
has also recommended providing feedback to physicians on their resource
use.[Footnote 6] In its reports, MedPAC has explored an episode-based
profiling methodology, which measures resource use for treating a
particular episode of illness--for example, a stroke or heart attack--
and attributes that resource use to physicians.
Following the issuance of our report and subsequent testimony, Congress
passed the Medicare Improvements for Patients and Providers Act of 2008
(MIPPA), which directed the Secretary of Health and Human Services
(HHS) to develop a program to provide physicians confidential feedback
on the Medicare resources used to provide care to Medicare
beneficiaries.[Footnote 7] MIPPA gave HHS the flexibility to measure
resource use on a per capita basis, an episode basis, or both. In
response to this mandate, CMS is currently testing both profiling
methodologies in its Physician Resource Use Measurement and Reporting
Program. MIPPA also directed us to submit a report to Congress on CMS's
physician feedback program by 2011.
In your letter of August 22, 2007, you pointed out that both the per
capita and episode-based methods could be used to identify inefficient
physicians, but noted that less is known about the per capita method.
At that time, you asked us to evaluate the per capita method for
profiling physicians in order to assist CMS with the development of a
physician feedback approach for Medicare. This report explores the use
of a per capita method to profile physicians based on their patients'
level of resource use, and discusses the development and influence of
feedback reports. Specifically, this report examines (1) the extent to
which physicians in selected specialties show stable practice patterns
and how beneficiary utilization of services varies by physician
resource use level; (2) factors to consider in developing feedback
reports on physicians' performance, including per capita resource use;
and (3) the extent to which feedback reports may influence physician
behavior.
We focused our analysis on four diverse specialties--a medical
specialty (cardiology), a diagnostic specialty (diagnostic radiology),
a primary care specialty (internal medicine), and a surgical specialty
(orthopedic surgery); and four metropolitan areas--Miami, Fla.;
Phoenix, Ariz.; Pittsburgh, Pa.; and Sacramento, Calif.[Footnote 8] We
chose these areas for their geographic diversity, range in average
Medicare spending per beneficiary, and number of physicians in each of
the four specialties. We limited our study to physicians who
participate in Medicare fee-for-service (FFS). Our results are not
generalizable to other geographic areas and specialties.
To measure beneficiaries' resource use, we first adjusted for
beneficiaries' health conditions, because sick beneficiaries are
expected to use more resources than healthy beneficiaries. Using
Hierarchical Condition Category (HCC) and expenditure data obtained
from CMS, we estimated a risk adjustment model that uses the same 70
HCCs as the model CMS uses to set managed care capitation rates. HCCs
are a way of summarizing an individual's diagnoses into major medical
conditions, such as vascular disease or severe head injury.[Footnote 9]
Given beneficiaries' HCCs during the year, we used our model to
estimate Medicare's expected annual expenditures for services provided
to the beneficiaries in our study. Based on these expected
expenditures, we placed beneficiaries into 25 discrete risk categories.
Within each risk category and metropolitan area, we ranked
beneficiaries from 1 to 100 by their total annual Medicare FFS
expenditures such that the average beneficiary in a given risk category
and metropolitan area had a rank of 50.[Footnote 10] We used this rank
as our risk-adjusted measure of beneficiaries' resource use.
Our measure of physicians' resource use is derived from the resource
use of their patients.[Footnote 11] For all physicians in our study, we
calculated the average rank of their patients. We then used this
average to rank physicians within the same metropolitan area and
specialty on a scale of 1 to 100. This measure reflects how expensive a
physician's patients are compared to the patients of other physicians
in the same specialty and area after adjusting for differences in
patient health status.
To examine the stability of physicians' resource use from a year-to-
year perspective, we analyzed data for 2005 and 2006.[Footnote 12] We
divided physicians' and beneficiaries' resource use into quintiles and
examined which physicians and beneficiaries stayed in the same resource
use quintile from 2005 to 2006 and which ones did not.[Footnote 13] We
also examined the degree of turnover in the patients seen by physicians
between 2005 and 2006. In addition, we used the physician quintiles to
examine how beneficiary utilization of selected services in 2006 varied
by physician resource use quintile[Footnote 14].
We concluded that the information on Medicare claims that we used in
this report was sufficiently reliable for the purpose of our analysis,
because it is a record of Medicare's payments to health care providers.
We obtained beneficiaries' FFS expenditures from claims information,
and we used data from CMS files containing enrollment and institutional
status in order to determine whether beneficiaries were eligible for
our study. CMS provided us with a file containing beneficiaries' HCCs,
which we used to estimate their expected expenditures. We obtained
physicians' specialties from Medicare physician files that CMS uses to
administer the program and set payment rates. CMS and its contractors
closely monitor these files, so they are generally considered reliable.
In addition, we interviewed relevant CMS officials concerning the data
and consulted data documentation maintained by CMS. We consider the
data sufficiently reliable for our purposes.
To determine factors to consider in developing reports to provide
feedback to physicians on their performance, including their per capita
resource use, and the extent to which feedback reports may influence
physician behavior, we reviewed selected literature and interviewed
experts.[Footnote 15] To identify relevant literature, we searched 31
databases, including MEDLINE and Science Citation Index, using terms
such as "physician performance feedback," for journal articles and
other documents published between January 1, 2000, and February 13,
2009. From reference lists in documents identified during that search,
we identified additional documents that met our criteria. We selected
for review three types of documents: (1) meta-analyses, reviews, or
scans of the literature on the effectiveness of providing performance
feedback to physicians; (2) evaluations of various efforts to provide
performance feedback to physicians; and (3) documents that provided
guidance from experts on methods for providing performance feedback to
physicians. In addition to reviewing selected literature, we conducted
interviews with officials of four specialty societies to identify
specialty-specific perspectives and concerns, and to solicit officials'
comments on a mock feedback report we designed. We also conducted
interviews with officials of the five health insurers with the highest
revenues in 2007 about their experiences with feedback reports.
[Footnote 16]
There are several limitations to our findings. Our findings cannot be
generalized to other areas or specialties. We also restricted our scope
to individual physicians and did not analyze group practices. Most
importantly, we did not pilot our mock report, which illustrates how
per capita measures could be included in a physician feedback report,
or test it by giving physicians feedback based on actual resource use.
Consequently, we are unable to evaluate how helpful it would be to
physicians and, particularly, whether it has potential for increasing
physicians' efficiency.
We conducted our work from February 2008 to September 2009 in
accordance with all sections of GAO's Quality Assurance Framework that
are relevant to our objectives. The framework requires that we plan and
perform the engagement to obtain sufficient and appropriate evidence to
meet our stated objectives and to discuss any limitations in our work.
We believe that the information and data obtained, and the analysis
conducted, provide a reasonable basis for any findings and conclusions.
Background:
We, MedPAC, and the Congressional Budget Office (CBO) have all
suggested that CMS profile physician resource use and provide feedback
to physicians as a step toward improving the efficiency of care
financed by Medicare. In July 2008, Congress passed MIPPA,[Footnote 17]
which directed the Secretary of HHS to establish a program by January
1, 2009, to provide physicians confidential feedback on the Medicare
resources used to provide care to beneficiaries. MIPPA gave HHS the
flexibility to measure resource use on a per capita basis, an episode
basis, or both. In response to the MIPPA mandate, CMS is pursuing its
Physician Resource Use Measurement and Reporting Program. (See table
1.)
Table 1: Selected Events Preceding CMS Providing Physicians Feedback on
Their Medicare Resource Use:
March 2005:
MedPAC, in its report to Congress, recommended that the Secretary of
HHS should use Medicare claims data to measure fee-for-service
physicians' resource use and share results with physicians
confidentially to educate them about how they compare with aggregated
peer performance.[A]
June 2006:
MedPAC, in its report to Congress, stated that it is important to use a
per capita profiling methodology in conjunction with an episode-based
profiling methodology in order to get a complete picture of resource
use.[B]
March 2007:
CBO, in its testimony to the Committee on Finance, United States
Senate, stated that physicians participating in fee-for-service
Medicare could be required or encouraged to participate in a program
that would provide physicians feedback on how their practice patterns
compared to their peers as a step toward encouraging more efficient
care.[C]
April 2007:
GAO, in its report to Congress, recommended that CMS develop a
physician profiling system that included feedback and incentives as
part of a package of reforms to improve the efficiency of care financed
by Medicare.[D]
May 2007:
GAO, in its testimony to the Subcommittee on Health, House Committee on
Ways and Means, stated that providing feedback to physicians on their
practice patterns could be a promising step toward encouraging
efficiency in Medicare.[E]
July 2008:
Congress passed the Medicare Improvements for Patients and Providers
Act of 2008, which mandated that the Secretary of HHS establish a
program to provide physicians confidential feedback on the Medicare
resources used to provide care to beneficiaries.[F]
April 2008 to Present:
CMS began a phased implementation of its Physician Resource Use
Measurement and Reporting Program which, in Phase I, has disseminated
approximately 310 Resource Use Reports to physicians in 13 areas. The
program is exploring both per capita and episode-based
methodologies.[G]
Source: GAO.
[A] MedPAC, Report to the Congress: Medicare Payment Policy
(Washington, D.C.: March 2005), 142.
[B] MedPAC, Report to the Congress: Increasing the Value of Medicare
(Washington, D.C.: June 2006), xvi.
[C] CBO, Medicare's Payments to Physicians: Options for Changing the
Sustainable Growth Rate (Washington, D.C.: March 1, 2007), 16-17.
[D] GAO, Medicare: Focus on Physician Practice Patterns Can Lead to
Greater Program Efficiency, GAO-07-307 (Washington, D.C.: April 30,
2007), 22.
[E] GAO, Medicare: Providing Systematic Feedback to Physicians on their
Practice Patterns Is a Promising Step Toward Encouraging Physician
Efficiency, GAO-07-862T (Washington, D.C.: May 10, 2007).
[F] Medicare Improvements for Patients and Providers Act of 2008
(MIPPA), Pub. L. No. 110-275, §131(c), 122 Stat. 2494, 2520-25.
[G] Federal Register, vol. 74, Physician Resource Use Measurement and
Reporting Program, no. 132 (Washington, D.C.: July 13, 2009), 33589-
33591.
[End of table]
Key Decisions in Physician Profiling:
When profiling physicians on their resource use, five key decisions
must be made:
* Which resource use measurement methodology to use. There are two main
profiling methodologies: per capita and episode-based. Using both types
of measures of resource use may provide more meaningful results by more
fully capturing the relevant characteristics of a physician's practice
patterns.
* How to account for differences in patient health status. Accounting
for differences in patient health status, a process sometimes referred
to as risk-adjustment, is an important and challenging aspect of
physician profiling. Because sicker patients are expected to use more
health care resources than healthier patients, we believe the health
status of patients must be taken into account to make meaningful
comparisons among physicians. There are various risk-adjustment methods
and the suitability of a given method will depend on characteristics of
the physicians to be profiled and their patients.
* How to attribute resource use to physicians. Important attribution
decisions include whether to assign a patient's resource use to the
single physician who bears the greatest responsibility for the resource
use, to all physicians who bore any responsibility, or to all
physicians who met a given threshold of responsibility, such as
providing a certain percentage of the expenditures or volume of
services. A single attribution approach may not be applicable for all
types of measures or for all types of physician specialties.
* What benchmark(s) to use. Physician profiling involves comparing
physicians' resource use to a benchmark. There are differing opinions
on what are the most appropriate and meaningful comparative benchmarks.
* How to determine what is a sufficient sample size to ensure
meaningful comparisons. The feasibility of using resource use measures
to compare physicians' performance depends, in part, on two factors:
the availability of enough data on each physician to compute a resource
use measure and a sufficient number of physicians to provide meaningful
comparisons. It is important to calculate resource use measures only
for physicians with sufficient sample sizes in order to address
concerns that a physician's profile may be distorted by a few aberrant
cases. There is no consensus on what sample size is adequate to ensure
meaningful measures.
CMS's Resource Use Measurement and Reporting Program:
Responding to the MIPPA mandate to establish a physician feedback
program by January 1, 2009, CMS began in April 2008 to develop its
program for reporting to physicians on their resource use. In the first
phase of the program, CMS identified eight priority conditions and
disseminated approximately 310 Resource Use Reports to physicians in
selected specialties who practiced in one of 13 geographic areas. The
reports generally included both per capita and episode-based resource
use measures that were calculated according to five different
attribution rules. The reports also contained multiple cost benchmarks
relative to physicians in the same specialty and geographic area. In
Phase II, CMS is proposing to expand the program by adding quality
measures and reporting on groups of physicians as a mechanism for
addressing small sample size issues.
Per Capita Profiling Method Shows Specialist Physicians' Practice
Patterns Relatively Stable Over 2 Years; Patients of High Resource Use
Physicians Used More Institutional Services Than Other Patients:
Using a per capita profiling method, we found that from 2005 to 2006,
specialist physicians showed considerable stability in their practice
patterns, as measured by resource use--greater stability than their
patients, despite high patient turnover. We also found that our per
capita method can differentiate specialists' patterns of resource use
with respect to different types of services, such as institutional
services,[Footnote 18] which were a major factor in beneficiaries'
resource use. In particular, patients of high resource use physicians
used more institutional services than patients of low resource use
physicians.
Specialist Physicians' Resource Use More Stable Than Beneficiaries'
Resource Use:
Using a per capita method to profile specialist physicians, we found
that their practice patterns, as measured by the level of their
resource use, was relatively stable over 2005 and 2006 by comparison
with individual beneficiaries' resource use (see figure 1).[Footnote
19] This is true despite the fact that our measure of physicians'
resource use is derived from their patients' resource use and that the
specific patients whom physicians see are not always the same from year
to year. Among the physicians we studied, less than one-third of
patients seen by study physicians in 2005 were also seen by the same
physician in 2006. This stability suggests that per capita resource use
is a reasonable approach for profiling physicians, because it reflects
distinct patterns of a physician's resource use, not the particular
population of beneficiaries seen by a physician in a given year.
We divided both physician and beneficiary resource use into five groups
of approximately equal size (quintiles) and found that, on average
across the four metropolitan areas and four specialties, 58 percent of
physicians and 30 percent of beneficiaries were in the same quintile of
resource use in 2005 and 2006. The pattern was even more pronounced for
the top resource use quintile: 72 percent of physicians and 35 percent
of beneficiaries remained in that quintile. If the level of physicians'
and beneficiaries' resource use was purely random, only 20 percent
would be expected to have remained in the same quintile.
Figure 1: Stability of Medicare Beneficiaries' and Specialist
Physicians' Resource Use--Averaged Across Four Metropolitan Areas and
Four Physician Specialties, 2005-2006:
[Refer to PDF for image: vertical bar graph]
Resource use remained in same quintile[A] both years:
Physicians: 58%;
Beneficiaries: 30%.
Resource use remained in top quintile[A] both years:
Physicians: 72%;
Beneficiaries: 35%.
Percentage expected by chance: 20%.
Source: GAO analysis of Medicare claims data.
Note: The specialist physicians include cardiologists, diagnostic
radiologists, internists, and orthopedic surgeons in Miami, Fla.;
Phoenix, Ariz.; Pittsburgh, Pa.; and Sacramento, Calif.
[A] Beneficiaries and physicians are divided into five ascending groups
of nearly equal size based on the level of their resource use.
[End of figure]
We also examined the stability of physicians' resource use by specialty
and found a similar pattern, although not to the same extent in all
specialties. The average percentage of physicians who were in the same
resource use quintile in 2005 and 2006 ranged from 48 percent for
orthopedic surgeons to 60 percent for internists. Resource use in the
top quintile was more stable and ranged from 69 percent for diagnostic
radiologists to 74 percent for internists. (See table 2.)
Table 2: Average Stability of Physicians' Resource Use by Specialty--
Averaged Across Four Metropolitan Areas, 2005-2006:
Physician Specialty: Cardiology;
Average percentage remaining in same quintile[A]: 59;
Average percentage remaining in the top quintile[A]: 71.
Physician Specialty: Diagnostic radiology;
Average percentage remaining in same quintile[A]: 58;
Average percentage remaining in the top quintile[A]: 69.
Physician Specialty: Internal medicine;
Average percentage remaining in same quintile[A]: 60;
Average percentage remaining in the top quintile[A]: 74.
Physician Specialty: Orthopedic surgery;
Average percentage remaining in same quintile[A]: 48;
Average percentage remaining in the top quintile[A]: 70.
Source: GAO analysis of Medicare claims data.
Note: The four metropolitan areas are Miami, Fla.; Phoenix, Ariz.;
Pittsburgh, Pa.; and Sacramento, Calif.
[A] Physicians are divided into five ascending groups of nearly equal
size based on the level of their resource use.
[End of table]
In each of the four metropolitan areas, physicians showed greater
stability in their resource use than individual beneficiaries, although
the percentages varied. For example, the percentage of physicians
remaining in the top quintile ranged from 68 percent in Phoenix to 76
percent in Miami. For beneficiaries, the percentage in the top quintile
ranged from 31 percent in Phoenix to 39 percent in Miami. (See table
3.)
Table 3: Average Stability of Physicians' and Beneficiaries' Resource
Use by Metropolitan Area--2005-2006:
Metropolitan Area: Miami;
Physicians[A]: Average percentage remaining in same quintile[B]: 62;
Physicians[A]: Average percentage remaining in same quintile[B]: 76;
Beneficiaries: Average percentage remaining in same quintile[B]: 31;
Beneficiaries: Average percentage remaining in the top quintile[B]: 39.
Metropolitan Area: Phoenix;
Physicians[A]: Average percentage remaining in same quintile[B]: 56;
Physicians[A]: Average percentage remaining in same quintile[B]: 68;
Beneficiaries: Average percentage remaining in same quintile[B]: 29;
Beneficiaries: Average percentage remaining in the top quintile[B]: 31.
Metropolitan Area: Pittsburgh;
Physicians[A]: Average percentage remaining in same quintile[B]: 52;
Physicians[A]: Average percentage remaining in same quintile[B]: 70;
Beneficiaries: Average percentage remaining in same quintile[B]: 30;
Beneficiaries: Average percentage remaining in the top quintile[B]: 32.
Metropolitan Area: Sacramento;
Physicians[A]: Average percentage remaining in same quintile[B]: 58;
Physicians[A]: Average percentage remaining in same quintile[B]: 71;
Beneficiaries: Average percentage remaining in same quintile[B]: 30;
Beneficiaries: Average percentage remaining in the top quintile[B]: 32.
Source: GAO analysis of Medicare claims data.
[A] Cardiologists, diagnostic radiologists, internists, and orthopedic
surgeons.
[B] Beneficiaries and physicians are divided into five ascending groups
of nearly equal size based on the level of their resource use.
[End of table]
The greater stability of physicians' resource use compared to
beneficiaries' resource use could be due to their individual practice
styles, as well as to a range of other factors, such as participation
in formal or informal referral networks. These networks have a range of
providers, including other physicians, who treat their patients and
refer them for treatment, testing, and admissions to hospitals.
Beneficiary Use of Institutional Services Varies by Physician Resource
Level:
Beneficiaries seen by high resource use physicians generally were
heavier users of institutional services than those seen by lower
resource use physicians, and institutional services accounted for more
than one-half of total patient expenditures. This pattern was
consistent across three of the four specialties we studied, with
orthopedic surgery being the exception.
Institutional services were the major driver of Medicare expenditures
for beneficiaries in physicians' practices, accounting on average for
54 percent of expenditures. Services provided by a particular physician
in our study directly to that physician's patients accounted for only 2
percent of total expenditures or about $350 for each beneficiary in a
physician's practice. All other services--those provided by other
physicians, home health care, hospice care, outpatient services, and
durable medical equipment--accounted for the remaining 44 percent of
expenditures. (See figure 2.)
Figure 2: Share of Total Medicare Expenditures per Beneficiary for
Services Provided by Physicians to their Patients, Institutional
Services, and All Other Services--Practice Average Across Four
Specialties in Four Metropolitan Areas, 2006:
[Refer to PDF for image: pie-chart]
Services provided by a particular physician to his or her patients: 2%;
Institutional services[A]: 54%;
All other services[B}: 44%.
Source: GAO analysis of CMS claims data.
Note: The percentages shown are the average share of Medicare
expenditures for the beneficiaries in the practices of cardiologists,
diagnostic radiologists, internists, and orthopedic surgeons in Miami,
Fla.; Phoenix, Ariz.; Pittsburgh, Pa.; and Sacramento, Calif., with a
minimum of 100 Medicare patients in their practice.
[A] Institutional services include inpatient hospital and skilled
nursing care.
[B] Part B physician and supplier services (other than those provided
directly by the physician), home health care, hospice care, outpatient
hospital services, and durable medical equipment.
[End of figure]
Expenditures for institutional services for a physician's patients grew
as the level of physician resource use increased. Dividing the level of
physician resource use into quintiles, we examined the relationship of
physicians' resource use and expenditures for services provided to
their patients. Average expenditures for institutional services
increased more steeply by physician resource quintile than expenditures
for all other services.[Footnote 20]
The four specialties all exhibited this pattern of increasing
beneficiary expenditures for institutional services accompanying
increasing physician resource use, although for orthopedic surgery the
increase was small. The increase in average beneficiary expenditures
for all other services that accompanied increasing physician resource
use was similar for three of the four specialties and was steeper for
internal medicine.
We also examined the average number of physicians seen by the Medicare
beneficiaries we studied and found that it was positively associated
with increasing physician resource use. Overall, the number of
physicians seen increased from an average of about 13 physicians per
beneficiary in the lowest quintile of resource use to more than 23 in
the highest. The increase in the number of physicians seen was
accompanied by an increase in average beneficiary expenditures for
institutional services that was steeper than the rise in other
services.
Research Literature, Health Insurers, and Specialists Identified
Considerations in Developing Physician Feedback Reports on Resource
Use:
Through our review of selected literature and interviews with officials
of health insurance companies, specialty societies, and profiling
experts, we identified several key considerations in developing reports
to provide feedback to physicians on their performance, including their
per capita resource use. We also drew on information from these sources
to develop an example of how per capita measures could be presented in
a physician feedback report.
Key Considerations in Developing Physician Feedback Reports Include
Content, Design, Dissemination Strategy, and Transparency:
We identified four key considerations in developing reports to provide
feedback to physicians (see table 4).
Table 4: Key Considerations in Developing Physician Feedback Reports:
General considerations: Report content;
Examples of specific considerations: Types of measures, comparative
benchmarks.
General considerations: Report design;
Examples of specific considerations: Length, organization, graphics.
General considerations: Report dissemination;
Examples of specific considerations: Which physicians should receive
reports, frequency of reporting, hardcopy versus electronic
dissemination.
General considerations: Transparency;
Examples of specific considerations: Information about purpose,
methods, data.
Source: GAO analysis.
[End of table]
Report Content:
Our review of selected literature suggested that a physician feedback
report should contain three basic elements: an explanation of the
information contained in the report (which we will discuss in the
context of transparency), measures describing the performance of the
physician or physicians to whom the report is directed, and comparative
benchmarks.
Measures. Both the selected literature we reviewed and the officials we
interviewed supported including measures of quality along with measures
of cost, and ensuring that measures are actionable by providing
information that can help physicians improve their performance. The
officials we interviewed were divided as to whether these measures
should reflect physicians' performance at the individual level or the
group level.
* Quality measures. All five of the insurers we contacted were
profiling physicians in terms of quality and cost, and four of the five
had adopted a model code for physician ranking programs that called for
rankings to be based on quality as well as cost.[Footnote 21] Most of
the specialty society officials we interviewed also called for the
inclusion of quality measures in physician feedback reports, and some
cautioned that focusing solely on costs could create perverse
incentives--for example, encouraging physicians to reduce
inappropriately the level of care provided to patients. The lack of
widely accepted, claims-based quality measures for some specialties has
limited the number of specialties some insurers profile. For example,
at the time of our interview, one insurer was profiling physicians in
only one specialty (cardiology) while planning to begin profiling other
specialties within a year.
* Actionable measures. According to one research report we reviewed,
little research has been done to determine how the reporting of global
scores--such as an overall per capita cost rank--influences physician
behavior,[Footnote 22] but experts on physician profiling and a broad
array of stakeholders, including physicians and insurance company
officials, agreed that performance data should be disaggregated into
enough categories to enable physicians to identify practice patterns to
change. According to some profiling experts, resource use reports must
pinpoint physicians' overuse and misuse of resources, and identify
practices that add costs but do not improve desired outcomes.
Similarly, specialty society officials we interviewed emphasized the
importance of including measures that focus on areas in which the
physician has control.
* Individual versus group measures. Another measurement consideration
is whether physicians in group practices should be profiled as
individuals or as a group. The insurers we contacted took varying
approaches. In some cases, the approach was driven by contracting
arrangements, with insurers constructing group profiles for physicians
with whom they had group contracts. One insurance company official
pointed out that profiling at the group level allows more physicians to
be profiled, as it increases the data available to construct a profile.
Another official advocated profiling at the individual level because he
believes physicians are more interested in assessments of individual
performance. Officials of the four specialty societies generally saw
some merit to both approaches, but some underscored the difficulty of
identifying group affiliations or noted that groups are not necessarily
homogeneous enough for a group assessment to be appropriate.
Comparative benchmarks. One consideration addressed by multiple
publications we reviewed was the kind of benchmark to which physicians'
performance should be compared. For example, a physician's performance
may be compared to (1) an evidence-based standard, (2) a standard based
on professional judgment, such as the consensus standards endorsed by
the National Quality Forum, or (3) to a statistical norm, such as the
average for a physician's peers locally or nationally.
Although studies we reviewed offered conflicting evidence as to whether
including peer comparisons in physician feedback reports increases
their effectiveness, some profiling experts and specialty society
officials believe comparative information is useful and of interest to
physicians. In the literature we reviewed, for example, one profiling
expert suggested that such comparisons can motivate behavior change by
taking advantage of physicians' desire to perform at least as well as
their peers; another stated that performance statistics are not
meaningful to physicians without peer comparisons.
A physician's peer group can be defined in various ways. According to
one study, some organizations that provide performance feedback to
physicians have found comparisons within specialty and locality most
useful to and most frequently requested by physicians.[Footnote 23]
Representatives of some of these organizations said physicians find
local information more relevant because it reflects the practice
patterns of their geographic area. All five insurers we contacted
compare physicians to others in the same market and specialty; one of
the five also compares physicians to peers nationwide on some measures.
In contrast, officials of all four specialty societies recommended
comparisons at the national level, with officials of one society
stating that there is no scientific basis for regional variations in
practice patterns.[Footnote 24] There was less agreement about whether
physicians should be compared to others in their specialty or to a more
narrowly defined group. Officials of one specialty society advocated
comparisons at the subspecialty level in recognition of the variation
in resource use patterns among subspecialists. Another official pointed
out that such comparison groups could be difficult to define because
physicians in some specialties tend to have multiple subspecialties.
Because views differ on appropriate comparison groups, one hospital-
owned healthcare alliance plans to incorporate in its physician reports
a customizable feature that will allow users to select the peer
comparison they wish to see.
Comparisons to physicians' own past performance (trend data) are
commonly presented in feedback reports, and the majority of physicians
surveyed in one study found these comparisons useful.
Report Design:
The selected literature we reviewed offered little hard evidence on how
feedback reports should be designed to engage physicians' interest or
to prove their comprehension of the material. However, researchers and
profiling experts offered some comments and suggestions based either on
their experience with clinical performance measurement or on an
analysis of the literature on consumer behavior and its possible
implications for physician reporting (see table 5).
Table 5: Comments and Suggestions for Designing Physician Feedback
Reports:
Topic: Amount of material and report length;
Comments:
* Effective reports do not necessarily provide a high level of detail;
* Detailed supporting data can be made available in a separate
drilldown section;
* Physician feedback reports can vary greatly in length depending on
the number of topics covered and the level of detail.
Topic: Organization;
Comments:
* The organization of the report may be more important than its length;
* All high-level summary information should be in one place so that
it's easy to absorb;
* Reports should move from gross measures to more refined;
* Spatial organization, through the use of headings and lists, is
critical for helping readers find information.
Topic: Graphics;
Comments:
* Visual formats provide the best methods for data interpretation and
are useful for highlighting the most important measures;
* Information can be conveyed visually in tables, graphs, and score
cards;
* Tables may be better to show specific numeric values, while graphs
may be better to display information for comparative purposes, because
they facilitate the organization of material into meaningful groups;
* A score card or summary-rating format consolidates data even further
than tables or graphs, using colors or symbols to help readers easily
identify successes as well as areas for improvement.
Source: GAO analysis of selected literature.
[End of table]
The amount and combination of material that should be included in a
single report is an important consideration. According to one
publication that summarized a review of multiple feedback reports, some
organizations issue separate reports on efficiency/cost and
effectiveness/clinical quality, in part to avoid diluting the impact of
either set of measures. Others believe a single report gives physicians
a more complete picture of their performance.
Officials of the three insurers we contacted that routinely issued
feedback reports to physicians said that their companies produced
summary reports, typically one to two pages in length, containing high-
level information, but also made more detailed information, such as
patient-level data, available to physicians. One insurer's summary
report consisted of one page of cost efficiency measures and one page
of effectiveness measures. The cost efficiency page presented average
cost per episode of care by service category for the physician and the
physician's peer group, as well as the ratio of the two, in both
tabular and graphic form. The effectiveness page presented process-of-
care measures for selected conditions, including cardiovascular disease
and asthma. Company officials said summary reports were limited to two
pages to accommodate physicians' attention spans and that the two sets
of measures were presented separately to discourage attempts to link
the two. Specialty society officials agreed reports should be short--
most proposed one to two pages--and strongly recommended that
information be presented graphically to the extent possible. One
official, noting that physicians are very visually oriented,
recommended feedback reports consisting mainly of easily understood
graphics.
The selected literature we reviewed, our interviews with specialty
society officials, and existing physician feedback reports suggested
reports can be kept short by segmenting some information into separate
documents--for example, a cover letter that explains the report's
purpose, a description of the profiling methodology, a set of
frequently asked questions, and a list of definitions.
Report Dissemination:
Some key considerations with respect to report dissemination are which
physicians should receive reports, how frequently to issue reports, and
whether to issue reports in hardcopy or electronically.
Which physicians should receive feedback reports. One major decision is
whether to issue reports to all physicians for whom performance
measures can be calculated or only to a subset who fail to meet certain
performance standards--a decision that may involve weighing reporting
costs against potential impacts. None of the studies we reviewed
directly addressed this issue, but all of the specialty society
officials we interviewed advised sending reports to all or nearly all
physicians, rather than just to poor performers. They gave several
reasons: to provide positive recognition to physicians who are
performing well; to avoid singling out certain physicians as poor
performers, especially on the basis of excess costs over which they
have little control; and to create opportunities for voluntary peer-to-
peer learning among physicians who are at different points along the
performance spectrum. Similarly, all three of the insurers that
routinely issued feedback reports sent them to all physicians for whom
they had performance measures.
Frequency of reporting. According to one book we reviewed,
organizations that provide feedback to physicians should do so more
than once a year to give physicians an opportunity to improve their
performance in a timely manner.[Footnote 25] However, because of the
time needed to gather sufficient data to identify trends and patterns
of performance, many organizations provide feedback no more than twice
a year. Of the two insurers that told us how frequently they issued
feedback reports, one did so annually and the other at least every 6
months. Officials of the latter company said the frequency of their
reporting was limited by the number of claims in their dataset and
suggested that CMS would not face the same limitations.
Hardcopy versus electronic dissemination. Reports can be disseminated
in hardcopy through various channels, such as the mail, or
electronically, through e-mail or a Web site. One literature scan we
reviewed cited certain advantages of electronic formats such as Web-
based applications. Specifically, they allow users to organize
information as they choose and are well suited to presenting data from
the general to the specific, which facilitates information processing.
Although this report noted some concerns about physicians' access to
the Internet, according to a report based on a national survey of
physicians in December 2002 and January 2003, almost all respondents
said they had Internet access, and most said they considered it
important for patient care.[Footnote 26]
Of the three insurers that routinely issued feedback reports, two
issued them electronically and one issued them in hardcopy. Officials
of the latter company said that staff typically hand-delivered the
reports to physicians during on-site visits in order to discuss the
results.[Footnote 27] Officials of most of the specialty societies we
contacted did not advocate one dissemination mode over the other, but
some noted that organizations that issue reports electronically must
confront certain challenges, such as ensuring that security features do
not make access difficult, addressing the lack of high-speed Internet
service in some areas, and determining whether to send reports by e-
mail or to instruct physicians to access them on the Internet.[Footnote
28] One specialty society official recommended using both modes of
dissemination to accommodate different preferences.
Transparency:
Both the selected literature we reviewed and our interviews with
officials from insurance companies and specialty societies underscored
the importance of ensuring transparency regarding the purpose of the
report and the methodology and data used to construct performance
measures.
Purpose. According to one literature scan, feedback reports should
explicitly state their purpose--for example, to reduce costs, improve
quality, or simply to provide information--and should highlight any
items for which the physician will be held accountable.[Footnote 29]
Methodology. Two important considerations are where to provide
information about methodology--whether in the report itself or through
some other mechanism, such as a Web page--and how much technical detail
to provide. Some of the insurers we contacted provide information on-
line about their profiling methodologies, including details about
measures, attribution of care to physicians, risk adjustment, and
statistical issues. In addition, some of the officials we interviewed
said that company staff will meet with physicians to explain the
profiling methodology, if requested. For example, officials of one
company said that it has on staff four profiling experts, mostly
nurses, in addition to about 20 medical directors who can answer
physicians' questions.
Specialty society officials we interviewed highlighted a potential
trade-off between providing enough information in the report to
persuade physicians of the validity of the measures and keeping the
report concise enough to maintain physicians' interest. All of the
officials we interviewed agreed that physicians should have access to
details about the methodology; some suggested this information might
best be disseminated through a Web site. Explaining how the data are
risk-adjusted to account for differences in physicians' patient
populations was cited by specialty society officials as particularly
important.
Data. Another consideration is ensuring transparency with regard to the
data used in profiling--making patient-level detail available so
physicians can reconcile performance measures with their own
information about their practices. All five of the health insurers we
contacted provided opportunities for physicians to examine patient-
level data and file appeals before results are made public, although
their processes or policies for doing so varied (see table 6).
Table 6: How the Nation's Five Largest Health Insurers Make Patient-
Level Profiling Data Available to Physicians for Review and Appeal:
Availability of patient-level data:
Insurer A: Generally e-mailed upon request;
Insurer B: Generally mailed upon request;
Insurer C: Accessible on-line to each physician;
Insurer D: Accessible on-line to each physician;
Insurer E: Hand-delivered to each physician group during site visits.
Window for review and appeal:
Insurer A: 90 days;
Insurer B: 45 days;
Insurer C: 60 days;
Insurer D: 45 days;
Insurer E: 45 days.
Source: GAO analysis of information provided by insurers.
[End of table]
Officials of one of the two insurers that made detailed data available
on-line said their company previously sent hardcopy reports to
physicians, but learned from medical office managers that they would
prefer an on-line format that could be manipulated to facilitate
physician comparisons. Officials of the other insurer said that their
company planned to make the data available in a manipulatable format
soon. Most of the specialty society officials we interviewed agreed
that patient-level data should be made available to physicians, but
some predicted that few physicians would access them. Two interviewees
suggested practice size would probably be a factor; one added that
physicians in smaller groups would likely lack the resources and skills
to analyze the data.
Per Capita Measures Can Be Presented in a Physician Feedback Report:
Drawing upon lessons culled from the literature and our interviews, we
developed a mock report that illustrates how per capita measures could
be included in a physician feedback report. Such a report could also
include other measures such as quality measures and episode-based
resource use measures. We included two types of per capita measures--
risk-adjusted cost ranks and risk-adjusted utilization rates--each
presented with local and national comparative benchmarks. To provide
further context, we also included per capita measures showing how the
average Medicare costs of patients the physician treated at least once
were distributed among service categories, and the percentage of those
costs that were for services directly provided by the physician to whom
the report is directed. We kept the mock report under two pages and
included minimal text, while ensuring transparency by indicating the
availability of methodology details and supporting data. To accommodate
physicians' differing dissemination preferences, we designed the mock
report to be available in both electronic and hardcopy formats. (See
figure 3.)
Figure 3: Mock Physician Feedback Report Illustrating Per Capita
Measures:
[Refer to PDF for image: illustration]
FFS Medicare: Physicians Report:
Physician Information:
Name: Dr. John Doe:
NPI: •••••••2487:
Area: Cityville.
Report Information:
Reporting Period: 01/01/2007 - 12/31/2007:
Specialty: Cardiology.
Overall Medicare Resource Use Measures:
How the average risk-adjusted Medicare costs of patients you treated at
least once compared to those of other cardiologists:
Average patient risk-adjusted cost rank:
U = your rank;
A = Area rank;
N = National rank.
All services:
U: 62;
A: 47;
N: 49.
Physician visits:
U: 59;
A: 45;
N: 51.
Physician procedures:
U: 57;
A: 47;
N: 51.
Imaging:
U: 56;
A: 45;
N: 47.
Laboratory:
U: 43;
A: 46;
N: 50.
Hospital inpatient:
U: 62;
A: 47;
N: 49.
Hospital outpatient:
U: 48;
A: 41;
N: 50.
Skilled nursing and home health:
U: 43;
A: 47;
N: 53.
How the average Medicare costs of patients you treated at least once
were distributed:
Payments to all providers and you:
All services:
All providers: 100% ($13,422) of total;
You: 11% ($1,449) of category.
Physician visits:
All providers: 10% ($1,342) of total;
You: 45% ($604) of category.
Physician procedures:
All providers: 7% ($939) of total;
You: 65% ($610) of category.
Imaging:
All providers: 6% ($805) of total;
You: 20% ($161) of category.
Laboratory:
All providers: 3% ($403) of total;
You: 18% ($73) of category.
Hospital inpatient:
All providers: 44% ($5,906) of total;
You: 0% ($0) of category.
Hospital outpatient:
All providers: 8% ($1,074) of total;
You: 0% ($0) of category.
Skilled nursing and home health:
All providers: 14% ($1,879) of total;
You: 0% ($0) of category.
Risk-adjusted utilization rates:
Hospitalizations (per 100 patients):
You: 58.3;
Cardiologists In Your Area: 41.3;
Cardiologists Nationwide: 46.0 .
Rehospitalizations (per 100 patients):
You: 8.6;
Cardiologists In Your Area: 4.5;
Cardiologists Nationwide: 5.1.
Evaluation & Management Visits (per patient):
You: 3.7;
Cardiologists In Your Area: 3.1;
Cardiologists Nationwide: 3.5.
Note: This figure is an illustration of how per capita measures could
be included as part of a physician feedback report, which could include
a cover letter, quality measures, and other resource use measures. All
of the data presented in the figure are hypothetical.
Source: GAO.
[End of figure]
Specialty society officials who vetted a draft of the mock report made
several recommendations. Some recommendations centered on taking
advantage of electronic capabilities, such as adding hovers to define
key terms (see figure 4), creating interactive features to let
physicians explore "what if" scenarios, and including links to
educational materials and specialty guidelines. Officials also
recommended adding information on pharmaceutical costs, a category we
did not include because not all beneficiaries are enrolled in a
Medicare Part D prescription drug plan.
Figure 5: Example of a Hover in a Mock Physician Feedback Report:
[Refer to PDF for image: illustration]
The electronic version of the feedback report could make use of
interactive features.
For example, the report could use ’hovers“ to display a short
definition of key terms in the document and a reference to a page with
more information.
To meet requirements for federal agencies to make electronic
information accessible to disabled individuals, the feedback report
could show all hover definitions on the last page of the document, if
it were printed.
Average patient risk-adjusted cost rank:
Hover definition:
A patient‘s risk adjusted cost rank is calculated by comparing the
patient‘s Medicare costs to all other Cityville patients with similar
risk scores and represents how unexpectedly expensive or inexpensive
the patient‘s Medicare-covered care was. Your rank is the average rank
of all patients you treated at least once. See Glossary for more
details.
Source: GAO.
[End of figure]
More generally, specialty society officials said that they particularly
liked the graphs and charts in our mock report. One official added that
our report was easier to understand than other reports he had seen and
that he thought it would get physicians' attention. Another official
commented how the presented per capita measures could give physicians
insight on the care their patients are receiving that they were not
previously aware of--a perspective other cost measures could not
provide. However, multiple officials said the measures as presented
were too broad to be actionable and might not seem relevant to
physicians, as most physicians feel responsible only for the costs of
services they directly order or provide, not for the total cost of
patients' care. Two officials suggested that these per capita measures
would have more value in health care systems that emphasized
coordination of care.
Potential Influence of Feedback Regarding Medicare Costs on Physician
Behavior Is Uncertain:
Our review of available literature on the effectiveness of physician
feedback suggests that feedback alone generally has no more than a
moderate influence on physician behavior. However, the potential
influence of feedback from CMS regarding Medicare costs is uncertain,
and may be greater than that of feedback from other sources, because
Medicare reimbursement typically represents a larger share of
physicians' practice revenues than that from other insurers.
In general, studies examining the effect of feedback on physicians'
behavior have found it to have a small to moderate effect.[Footnote 30]
Factors that appear to influence the effectiveness of feedback include
its source, frequency, and intensity. For example, one review of the
literature concluded that physicians were more likely to be influenced
by reports from a source they expected to continue monitoring their
performance. This review also found that repeated feedback over a
period of several years may be more likely to get physicians'
attention.[Footnote 31] Another review reported that the intensity of
the feedback appeared to influence its effectiveness. The review cited
individual, written feedback containing information about costs or
numbers of tests, but no personal incentives, as among the least
intensive, and therefore likely to be among the least effective
approaches.[Footnote 32]
Consistent with the literature we reviewed, most of the insurance
company officials we interviewed questioned whether providing
performance feedback to physicians would have a significant impact on
the physicians' behavior in the absence of other incentives. While all
five insurers profiled physicians, none used the results solely to
provide feedback.[Footnote 33] Officials of four of the five insurance
companies said that to affect physicians' behavior, profiling results
must be made public, thus influencing patients' choice of physicians,
or linked to monetary incentives, as in pay-for-performance
arrangements. However, officials of one company disagreed, stating that
feedback alone can affect physicians' behavior if the reports show how
they rank against their peers and make clear what behavior they need to
change to improve their efficiency. These officials also said that the
impact of feedback could depend on the size of physicians' practices
and whether they have the resources to review the reports and the
management structure to affect changes.
Whether the experiences of private insurers or the lessons from the
literature on the influence of feedback will hold in the case of the
Medicare program is uncertain. A survey conducted in 2004-2005 found
that, for most physicians, Medicare represented more than one-quarter
of practice revenue, and for 17 percent of physicians, the proportion
was more than one-half.[Footnote 34] Because physicians typically
contract with a dozen or more health insurance plans, few, if any, of
these plans are likely to represent as large a share of physicians'
practice revenue as Medicare. Hence, the impact of feedback from CMS
might be greater than that from other sources. In addition, one
profiling expert suggested that physicians might expect feedback from
CMS to be only the first step in efforts to influence physicians'
behavior--to be followed, for example, by public reporting of profiling
results. This perspective comports with recommendations in our earlier
report.[Footnote 35] Two interviewees said that providing feedback on a
confidential basis would be an appropriate first step. One said it
would allow time to test the profiling methodology and gauge
physicians' reactions; the other said it would provide an opportunity
for physicians to vet the measures and identify any errors.
Most of the specialty society officials predicted that feedback from
CMS would have a small to moderate effect on physician behavior,
similar to that described in the literature we reviewed, but some
officials offered suggestions for enhancing its effectiveness. Other
suggestions can be drawn from the literature we reviewed. These
suggestions included:
* providing advance notice of feedback reports (through presentations,
letters, or other communications) to help ensure that physicians open
and read the reports;
* working through credible intermediaries, such as medical societies or
locally prominent physicians, to assure physicians that the feedback
process is reasonable and legitimate;
* providing opportunities for physicians to discuss the reports through
videoconferences, teleconferences, or on-line discussion groups; and:
* offering in-person follow up, possibly drawing on the resources of
the Medicare Quality Improvement Organizations.[Footnote 36]
Involving physicians in the development of a feedback system may also
enhance its effectiveness. One literature scan concluded that physician
involvement in system design was vital for obtaining physician buy-in.
[Footnote 37] Information from insurers suggested that, although
physicians may not always be involved in initial development of
feedback systems, their feedback can prompt modifications. Some
insurance officials we interviewed described an iterative process
involving ongoing communication with physicians and continuous
modification of reports and systems. For example, officials of one
insurance company said that the company did not seek initial input from
physicians--in the belief that they would not have been able to provide
much input without a complete understanding of the data and
methodology--but took into account physicians' responses to earlier,
less formal systems. Officials of other companies described various
mechanisms for obtaining physicians' perspectives, including formal
physician advisory councils, regular meetings with officials of
national medical societies, and town hall meetings with physicians at
the local level.
Concluding Observations:
Profiling physicians to improve efficiency is used by some private
insurance companies and, at the direction of Congress, is being adopted
by the Medicare program. We believe that a per capita methodology is a
useful approach to profiling physicians on their practice efficiency
and could be part of a feedback program that could also include quality
measures and episode-based resource use measures.
Our findings are consistent with those of our previous report on
physician profiling in which, through analysis of physician practice
patterns, we determined that CMS could use profiling to improve the
efficiency of Medicare. Despite a more diverse mix of physician
specialties in our present analysis, and with certain exceptions noted
in our findings, we found substantial consistency in certain patterns
we observed across metropolitan areas and specialties. We also found
consistency across time in that physicians who showed high resource use
in one year tended to stay high in the subsequent year.
Agency and Professional Association Comments and Our Evaluation:
We provided a draft of this report to the HHS for comment and received
written comments from CMS, which are reprinted in appendix II. We also
solicited comments on the draft report from representatives of the
American Academy of Orthopaedic Surgeons (AAOS), the American College
of Cardiology (ACC), the American College of Physicians, and the
American College of Radiology. We received oral comments from the first
two.
CMS Comments:
Our draft report did not include any recommendations for CMS to respond
to. CMS broadly agreed with each of our three findings:
* CMS agreed that the per capita methodology is a useful approach to
measuring physicians' resource use and noted that per capita
measurement is one of the cost of care measures included in CMS's
Physician Resource Use Management and Reporting Program. CMS also
agreed that the consistency of our per capita measure across years is
an important finding and stated that the agency intends to examine
measure consistency in the ongoing administration of its program.
* CMS found the attention in our report to considerations for
developing a physician feedback system to be particularly helpful. CMS
listed several examples of how its program already addresses many of
these considerations and is in the process of addressing others. We
agree with CMS that some of the approaches described in our report
would require significant resources and recognize that CMS will need to
investigate how to balance the trade-offs between different approaches
in order to best leverage its resources.
* CMS agreed that physician feedback may have a moderate influence on
physician behavior. CMS further stated its commitment to developing
meaningful, actionable, and fair measurement tools for physician
resource use that, along with quality measures, will provide a
comprehensive assessment of performance. We continue to believe that
providing physicians feedback on their performance could be a promising
step toward encouraging greater efficiency in Medicare; however, we are
still concerned that efforts to achieve greater efficiency that rely
solely on physician feedback without financial or other incentives will
be suboptimal.
CMS also provided technical comments, which we incorporated as
appropriate.
AAOS and ACC comments:
The representatives of AAOS and ACC raised no major issues with regard
to the substance of the report. The AAOS representative said that the
report captured well the key aspects of physician profiling and the key
considerations in developing physician feedback reports. The ACC
representatives endorsed the overall approach of a feedback report
consisting of a high-level summary accompanied by additional sections
with greater detail and a separate document that explains the
methodology in detail. The representatives of both groups said that
physicians should be provided feedback on both quality and resource
use, but differed on whether they should be presented in the same
report. Both groups also stressed that physicians should only be
compared to physicians within their specialty or subspecialty.
Both the AAOS and the ACC representatives commented on the design of
our mock report. Both said that the measures of physician resource use
by type of service and the benchmark comparisons were easy to
understand. They had difficulty, however, in understanding a related
measure that shows the physician's share of payments by service
category. We did not alter our mock report in response to these
comments, but believe that the concerns they expressed should be taken
into account by organizations designing physician feedback reports.
The representatives of both groups stressed the importance of risk
adjustment in the measurement of physician resource use and suggested
that we include a fuller explanation of risk adjustment techniques in
our report. We did not expand our explanation of such techniques
because they are not the focus of this report; however, we acknowledge
the important role played by risk adjustment techniques in constructing
physician feedback reports on resource use.
As agreed with your office, unless you publicly announce the contents
of this report earlier, we plan no further distribution until 30 days
from the report date. At that time, we will send copies of this report
to the Acting Administrator of CMS, committees, and others. The report
will also be available at no charge on the GAO Web site at [hyperlink,
http://www.gao.gov.
If you or your staff have any questions, please contact me at (202) 512-
7114 or steinwalda@gao.gov. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last
page of the report. GAO staff who made major contributions to this
report are listed in appendix III.
Sincerely yours,
Signed by:
A. Bruce Steinwald:
Director, Health Care:
[End of section]
Appendix I: Methodology:
This appendix describes the per capita methodology that we used to
measure beneficiaries' and physicians' Medicare fee-for-service (FFS)
resource use. We focused our analysis on four diverse specialties: a
medical specialty (cardiology), a diagnostic specialty (diagnostic
radiology), a primary care specialty (internal medicine), and a
surgical specialty (orthopedic surgery). We included diagnostic
radiologists in our study because they are less amenable to episode
grouping, the major alternative to per capita profiling of physicians.
We limited our analysis to physicians in these specialties who
practiced in one of four areas: Miami, Fla.; Phoenix, Ariz.;
Pittsburgh, Pa.; and Sacramento, Calif.[Footnote 38] We chose these
areas for their geographic diversity, range in average Medicare
spending per beneficiary, and number of physicians in each of the four
specialties. Our results apply only to the four specialties in the four
metropolitan areas we studied.
To conduct our analysis, we obtained 2005 and 2006 Centers for Medicare
& Medicaid Services (CMS) data from the following sources: (1) Medicare
claims files that include data on physician, durable medical equipment,
skilled nursing, home health, hospice, and hospital inpatient and
outpatient services; (2) Denominator File, a database that contains
enrollment and entitlement status information for all Medicare
beneficiaries in a given year; (3) Hierarchical Condition Category
(HCC) files that summarize Medicare beneficiaries' diagnoses; (4) files
summarizing the institutional status of beneficiaries; and (5) Unique
Physician Identification Number Directory, which contains information
on physicians' specialties.
Adjustment for Differences in Patient Health Status:
In order to develop a resource use measure that accounts for
differences in health status between beneficiaries, we developed a risk
adjustment model that uses an individual's diagnoses during the year to
estimate the total Medicare FFS expenditures expected for the
individual in that year. As our inputs to the model, we used the same
70 HCCs as those in the model CMS uses to set managed care capitation
rates.[Footnote 39] HCCs are a way of summarizing an individual's
diagnoses into major medical conditions, such as vascular disease or
severe head injury.[Footnote 40] To estimate our model, we used HCC and
expenditure data for 2005 and 2006 five percent national samples of
Medicare FFS beneficiaries.[Footnote 41],[Footnote 42]
Methodology Used to Determine Beneficiaries' Resource Use:
For all Medicare FFS beneficiaries who received at least one service in
2005 or 2006 from a physician located in any of our four metropolitan
areas and who also did not meet our exclusion criteria (see footnote
5), we used our risk adjustment model to estimate their total expected
Medicare FFS expenditures. Based on their expected expenditures, we
placed beneficiaries into 1 of 25 discrete risk categories.[Footnote
43] The categories were ordered in terms of health status from
healthiest (category 1) to sickest (category 25). Next, within each
risk category and metropolitan area, we ranked beneficiaries from 1 to
100 by their total actual annual Medicare expenditures, such that the
average beneficiary in a given risk category and metropolitan area had
a rank of 50.[Footnote 44] We used this rank as our risk-adjusted
measure of beneficiary resource use.
To examine the stability of beneficiaries' resource use, we divided the
2005 and 2006 beneficiary populations into five ascending groups of
nearly equal size (quintiles) based on the level of their resource use.
[Footnote 45] We then identified beneficiaries in each of the four
metropolitan areas who saw a physician in their area in 2005 and again
in 2006. We measured the stability of beneficiaries' resource use as
the percentage of beneficiaries who remained in the same quintile in
2006 that they were in during 2005. In addition, we determined the
percentage of beneficiaries who remained in the highest resource
quintile.
Methodology Used to Determine and Compare Physicians' Resource Use:
For the purposes of this study, we defined a physician's practice as
all Medicare FFS beneficiaries who did not meet our exclusion criteria
and who had at least one evaluation and management visit with the
physician during the calendar year for cardiologists, internists, and
orthopedic surgeons, or who received any service from the physician for
diagnostic radiologists.[Footnote 46],[Footnote 47] To ensure that a
physician's resource use measure would not be overly influenced by a
few patients with unusually high or low Medicare expenditures, we
excluded physicians with small practices--those who treated fewer than
100 of the Medicare patients in our study during the year.[Footnote 48]
For all physicians, we calculated the average beneficiary resource use
rank of the patients in their practices,[Footnote 49] which ranged from
a low of 26.0 to a high of 91.8 in 2006. Next, within each metropolitan
area and specialty, we ranked physicians on the basis of this average
from 1 to 100 such that the average measure of physician resource use
was 50. We used this rank as our measure of physician resource use.
This measure reflects how expensive a physician's patients are compared
to the patients of other physicians in the same specialty and area
after adjusting for differences in patient health status. For example,
a cardiologist in Miami is only compared to other cardiologists in
Miami.
To examine physicians' resource use, we divided the physicians into
five ascending groups (quintiles) of nearly equal size based on the
measure of their resource use described above.[Footnote 50] In the same
manner as we measured the stability of beneficiaries' resource use, we
measured the stability of physicians' resource use by determining the
percentage of them who remained in the same physician resource use
quintile from 2005 to 2006. We also measured the degree of turnover in
the patients seen by physicians by computing the percentage of patients
seen in 2005 by each physician that were also seen by the same
physician in 2006.
We examined utilization patterns by physician resource use quintile by
decomposing the 2006 Medicare expenditures of physicians' patients into
those for institutional services (inpatient hospital and skilled
nursing care), those for services provided directly by the physician to
his or her patients, and those for all other services--outpatient
hospital, home health care, hospice care, durable medical equipment,
and all other Part B services of Part B providers and suppliers. We
also measured the number of physicians seen by a physicians' patients
by physician resource use quintile.
Although our measure of a beneficiary's resource use is independent of
the beneficiary's health status, there was an association between
physician resource use and the mix of healthy and sick patients in
physicians' practices--physicians who ranked high in terms of resource
use also treated a larger proportion of beneficiaries who were in poor
health than did physicians who ranked low in resource use. However, the
resource use of all their patients was also consistently higher than
that of low resource use physicians' patients regardless of patient
health status. Figure 5 shows the average resource use of beneficiaries
in five health status categories across the five physician resource use
quintiles.[Footnote 51] For example, patients in the healthiest
category who were treated by physicians in the highest resource use
quintile had an average resource use rank of 74, whereas similarly
healthy patients treated by physicians in the lowest quintile had
average resource use rank of 53. This ordering of the differences in
patient resource use by the level of physician resource use is repeated
across all health categories. It indicates that physicians have
consistent patterns of resource use with respect to all of their
patients, regardless of their patients' health status.
Figure 6: Beneficiary Resource Use by Health Category for Quintiles of
Physician Resource Use--Four Specialties in Four Metropolitan Areas,
2006:
[Refer to PDF for image: line graph]
Beneficiary health category[A]: 1 (healthiest);
Beneficiary resource use: 1 (low resource use): 53;
Beneficiary resource use: 2: 59;
Beneficiary resource use: 3: 63;
Beneficiary resource use: 4: 68;
Beneficiary resource use: 5 (high resource use): 74.
Beneficiary health category[A]: 2;
Beneficiary resource use: 1 (low resource use): 50;
Beneficiary resource use: 2: 56;
Beneficiary resource use: 3: 61;
Beneficiary resource use: 4: 66;
Beneficiary resource use: 5 (high resource use): 73.
Beneficiary health category[A]: 3;
Beneficiary resource use: 1 (low resource use): 49;
Beneficiary resource use: 2: 55;
Beneficiary resource use: 3: 59;
Beneficiary resource use: 4: 64;
Beneficiary resource use: 5 (high resource use): 72.
Beneficiary health category[A]: 4;
Beneficiary resource use: 1 (low resource use): 49;
Beneficiary resource use: 2: 53;
Beneficiary resource use: 3: 56;
Beneficiary resource use: 4: 59;
Beneficiary resource use: 5 (high resource use): 66.
Beneficiary health category[A]: 5 (sickest);
Beneficiary resource use: 1 (low resource use): 50;
Beneficiary resource use: 2: 52;
Beneficiary resource use: 3: 54;
Beneficiary resource use: 4: 57;
Beneficiary resource use: 5 (high resource use): 63.
Source: GAO analysis of CMS claims data.
Note: Beneficiary resource use is averaged across the cardiologists,
diagnostic radiologists, internists, and orthopedic surgeons in Miami,
Fla.; Phoenix, Ariz.; Pittsburgh, Pa.; and Sacramento, Calif. who met
our requirement for a minimum of 100 Medicare patients in their
practice.
[A] Each health category consists of 5 risk categories, which span the
following ranges of risk scores (r): r £ .5, .5 < r £ 1.0, 1.0 < r £
2.0, 2.0 < r £ 5.0, 5.0 < r £ 18.0. The first health category includes
the healthiest beneficiaries and comprises 43.8 percent of the study
population; the fifth includes the sickest beneficiaries and comprises
5.5 percent of the study population. The second, third, and fourth
health categories comprise, respectively, 17.3, 17.4, and 16.1 percent
of the study population.
[B] Physicians are divided into five ascending groups of nearly equal
size based on the level of their resource use, which is based on the
average level of resource use of their patients.
[End of figure]
The mix of healthy and sick patients in physicians' practices did not
affect the positive relationship we found between average institutional
expenditures per beneficiary and physician resource use level. Within
each beneficiary health category, the patients of high resource use
physicians had average institutional expenditures that exceeded those
of the patients of physicians with lower resource use. Similar analyses
showed that patient mix did not affect (1) the positive relationship
between physicians' resource use and the average number of physicians
seen by their patients, (2) the positive relationship between
physicians' resource use and expenditures for all other services
provided their patients, and (3) the steeper rise in the use of
institutional services by physicians' patients with increasing
physician resource use as compared to the rise in the use of all other
services.
[End of section]
Appendix II: Comments from the Centers for Medicare & Medicaid
Services:
Department Of Health & Human Services:
Office Of The Secretary:
Assistant Secretary for Legislation:
Washington, DC 20201:
September 11 2009:
A. Bruce Steinwald:
Director, Health Care:
U.S. Government Accountability Office:
441 G Street N.W.
Washington, DC 20548:
Dear Mr. Steinwald:
Enclosed are comments on the U.S. Government Accountability Office's
(GAO) report entitled: "Medicare: Per Capita Method Can Be Used to
Profile Physicians and Provide Feedback on Resource Use" (GAO-09-802).
The Department appreciates the opportunity to review this report before
its publication.
Sincerely,
Signed by:
Andrea Palm:
Acting Assistant Secretary for Legislation:
Enclosure:
[End of letter]
Department Of Health & Human Services:
Centers for Medicare and Medicaid Services:
Administrator:
Washington, DC 20201:
September 10, 2009:
To: A. Bruce Steinwald:
Director, Health Care:
From: [Signed by] Charlene Frizerra:
Acting Administrator:
Subject: Government Accountability Office's Draft Report: "Per Capita
Method Can Be Used to Profile Physicians and Provide Feedback on
Resource Use" (GAO-09-802):
The Centers for Medicare & Medicaid Services (CMS) appreciates the
opportunity to review and comment on the Government Accountability
Office's (GAO) draft report entitled "Per Capita Method Can Be Used to
Profile Physicians and Provide Feedback on Resource Use." We agree
that, given the role of physicians in total Medicare spending, there
are opportunities to increase the efficiency of the Medicare program by
measuring and reporting on physician resource use. In addition, we
found the attention in your report to considerations for developing a
physician feedback system to be particularly helpful.
As GAO notes, CMS was given the authority to administer a Physician
Resource Use Measurement and Reporting Program (the "program") by the
Medicare Improvements for Patients and Providers Act of 2008 (MIPPA).
CMS has implemented a fully operational program in 13 selected
geographic sites and has made many of the same conclusions that GAO
includes in this report.
The GAO concludes that the "per capita profiling method shows
specialist physicians' practice patterns are relatively stable over 2
years." Per capita measurement is one cost of care measure included in
CMS' program for physicians who met a minimum threshold of at least 20
patients. Measurement consistency across years is an important finding,
although CMS notes that the single touch attribution rule GAO used may
inadvertently inflate the measure's consistency. CMS intends to also
examine measure consistency in the ongoing administration of the
program. GAO further concludes that feedback reports should include
quality measures. CMS has included a regulatory proposal in the
Calendar Year 2010 Physician Fee Schedule Proposed Rule (74 FR 33591)
to include quality measures in the program. GAO also recommends that
performance data should be disaggregated into categories. We have
disaggregated cost data into several categories including inpatient,
outpatient, home health, and skilled nursing facility services, among
others. In the report, GAO acknowledges that feedback reports are
useful at both the individual physician level and the physician group
level. CMS has included a regulatory proposal (74 FR 33591) to include
both individual and group level feedback in the program.
Regarding report dissemination, GAO's environmental scanning advised
sending feedback reports to all physicians rather than just poor
performers. To date, we have disseminated reports to all physicians in
the 13 selected geographic-sites that meet a minimum threshold of
patients/episodes. Further, GAO recognized the need for both
dissemination of reports in hard copy and electronically. To date, we
have only disseminated hard copy reports, but CMS is actively pursuing
electronic dissemination of reports. GAO concluded that the methodology
used to compile the reports should be disseminated in a transparent
fashion, such as a public posting on a web site. GAO also recognized
that some private insurers have made significant investments in the
operation of feedback programs; for example one company is utilizing a
staff of 4 profiling experts and 20 medical directors to support
physicians' questions about the feedback. We note that these insurers
are dedicating many more resources per profiled physician than CMS
currently has available to it. CMS is investigating the feasibility of
these investments while noting that significant resources would likely
be needed to fund some approaches.
Beyond general conclusions, GAO also identified the following key
points:
1. Using both per capita and episode of care measures of resource use
may provide meaningful results by capturing the relevant
characteristics of a physician's practice pattern;
2. There are various risk adjustment methodologies, and the suitability
of a given method will depend on characteristics of the physicians to
be profiled and their patterns;
3. A single attribution approach may not be applicable for all types of
measures or for all types of physician specialties;
4. There are differing opinions as to what are the most appropriate and
meaningful comparative benchmarks; and;
5. There is no consensus on what sample size is adequate to ensure
meaningful measurement.
In summary, we applaud GAO's recognition of these five key areas in the
ongoing management of physician resource use feedback programs. We
agree with GAO's conclusion that physician resource use feedback
reports have a moderate influence on physician behavior and the per
capita methodology is a useful approach to measuring physicians in a
feedback program that could also include episode-based resource use and
quality measures. CMS is committed to developing meaningful,
actionable, and fair measurement tools for physician resource use that,
along with quality measures, will provide a comprehensive assessment of
performance under a physician value-based purchasing program.
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
A. Bruce Steinwald, (202) 512-7114, or steinwalda@gao.gov.
Staff Acknowledgments:
In addition to the contact named above, Phyllis Thorburn, Assistant
Director; Alison Binkowski; Nancy Fasciano; Richard Lipinski; Drew
Long; Jessica Smith; Maya Tholandi; and Eric Wedum made key
contributions to this report.
[End of section]
Bibliography:
Balas, E. Andrew, Suzanne Austin Boren, Gordon D. Brown, Bernard G.
Ewigman, Joyce A. Mitchell, and Gerald T. Perkoff. "Effect of Physician
Profiling on Utilization: Meta-analysis of Randomized Clinical Trials."
Journal of General Internal Medicine, vol. 11, no. 10 (1996): 584-590.
Beckman, Howard B., Anthony L. Suchman, Kathleen Curtin, and Robert A.
Greene. "Physician Reactions to Quantitative Individual Performance
Reports." American Journal of Medical Quality, vol. 21, no. 3 (2006):
192-199.
Beckman, Howard B., Thomas Mahoney, and Robert A. Greene. Current
Approaches to Improving the Value of Care: A Physician's Perspective.
The Commonwealth Fund. November 2007.
Hartig, J.R., and Jeroan J. Allison. "Physician Performance
Improvement: An Overview of Methodologies." Clinical and Experimental
Rheumatology, vol. 25, supplement 47 (2007): S50-S54.
Jamtvedt, Gro, Jane M. Young, Doris Tove Kristoffersen, Mary Ann
O'Brien, and Andrew D. Oxman. "Audit and Feedback: Effects on
Professional Practice and Health Care Outcomes." Cochrane Database of
Systematic Reviews, no. 2 (2006).
Jamtvedt, Gro, Jane M. Young, Doris Tove Kristoffersen, Mary Ann
O'Brien, and Andrew D. Oxman. "Does Telling People What They Have Been
Doing Change What They Do? A Systematic Review of the Effects of Audit
and Feedback." Quality & Safety in Health Care, vol. 15, no. 6 (2006):
433-436.
Kiefe, Catarina I., Jeroan J. Allison, O. Dale Williams, Sharina D.
Person, Michael T. Weaver, and Norman W. Weissman. "Improving Quality
Improvement Using Achievable Benchmarks for Physician Feedback: A
Randomized Controlled Trial." JAMA, vol. 285, no. 22 (2001): 2871-2879.
Marder, Robert J., Mark A. Smith, and Richard A. Sheff. "Changing
Physician Practice: Providing Physicians Useful Feedback." In Effective
Peer Review: A Practical Guide to Contemporary Design, 2nd ed., 153-
164. Marblehead, Mass.: HCPro, Inc., 2007.
Micklitsch, Christine N. and Theresa A. Ryan-Mitlyng. Physician
Performance Management: Tool for Survival and Success. Englewood,
Colo.: Medical Group Management Association, 1996.
Mold, James W., Cheryl A. Aspy, and Zsolt Nagykaldi. "Implementation of
Evidence-Based Preventive Services Delivery Processes in Primary Care:
An Oklahoma Physicians Resource/Research Network (OKPRN) Study."
Journal of the American Board of Family Medicine, vol. 21, no. 4
(2008): 334-344.
Nathanson, Philip. "Influencing Physician Practice Patterns," Topics in
Health Care Financing, vol. 20, no. 4 (1994): 16-25.
Pacific Business Group on Health. Advancing Physician Performance
Measurement: Using Administrative Data to Assess Physician Quality and
Efficiency. September 2005.
Paxton, E. Scott, Barton H. Hamilton, Vivian R. Boyd, and Bruce L.
Hall. "Impact of Isolated Clinical Performance Feedback on Clinical
Productivity of an Academic Surgical Faculty." Journal of American
College of Surgeons, vol. 202, no. 5 (2006): 737-745.
Teleki, Stephanie S., Rebecca Shaw, Cheryl L. Damberg, and Elizabeth A.
McGlynn. Providing Performance Feedback to Individual Physicians:
Current Practice and Emerging Lessons. RAND Health Working Paper
Series. July 2006.
Van Hoof, Thomas J., David A. Pearson, Tierney E. Giannotti, Janet P.
Tate, Anne Elwell, Judith K. Barr, and Thomas P. Meehan. "Lessons
Learned from Performance Feedback by a Quality Improvement
Organization." Journal for Healthcare Quality, vol. 28, no. 3 (2006):
20-31.
Veloski, Jon, James R. Boex, Margaret J. Grasberger, Adam Evans, and
Daniel B. Wolfson. "Systematic Review of the Literature on Assessment,
Feedback and Physicians' Clinical Performance: BEME Guide No. 7."
Medical Teacher, vol. 28, no. 2 (2006): 117-128.
[End of section]
Footnotes:
[1] Medicare is the federally financed health insurance program for
persons aged 65 and over, certain individuals with disabilities, and
individuals with end-stage renal disease. Medicare Part A covers
hospital and other inpatient stays. Medicare Part B covers physician,
outpatient hospital, home health, and other services. Medicare Parts A
and B are known as original Medicare or Medicare fee-for-service (FFS).
[2] Physicians can be profiled both in terms of the resources used in
providing care to their patients and in terms of the quality of that
care. In this report, we focus on profiling physicians on their
resource use, which can be measured in terms of utilization or
expenditures.
[3] See GAO, Medicare: Focus on Physician Practice Patterns Can Lead to
Greater Program Efficiency, [hyperlink,
http://www.gao.gov/products/GAO-07-307] (Washington, D.C.: April 30,
2007), 22.
[4] The Centers for Medicare & Medicaid Services (CMS) is the agency
within the Department of Health and Human Services (HHS) that oversees
Medicare.
[5] See GAO, Medicare: Providing Systematic Feedback to Physicians on
their Practice Patterns Is a Promising Step Toward Encouraging Program
Efficiency, [hyperlink, http://www.gao.gov/products/GAO-07-862T] (May
10, 2007).
[6] The Medicare Payment Advisory Commission (MedPAC) is an independent
congressional agency established by the Balanced Budget Act of 1997 to
advise Congress on issues affecting the Medicare program.
[7] See Medicare Improvements for Patients and Providers Act of 2008
(MIPPA), Pub. L. No. 110-275, §131(c), 122 Stat. 2494, 2520-27.
[8] These areas refer to the following Core-Based Statistical Areas
(CBSA), an umbrella term for micropolitan and metropolitan statistical
areas: Miami-Fort Lauderdale-Pompano Beach, Fla.; Phoenix-Mesa-
Scottsdale, Ariz.; Pittsburgh, Pa.; and Sacramento--Arden-Arcade--
Roseville, Calif. For CBSA definitions, see [hyperlink,
http://www.gao.gov/products/http://www.census.gov/population/www/metroar
eas/metroarea.html].
[9] Hierarchical Condition Categories (HCC) collapse the over 15,000
diagnosis codes into 189 clinically-meaningful condition categories
which are additionally grouped into hierarchies of increasing severity.
See appendix I.
[10] We did not include Part D (drug) expenditures because not all
beneficiaries are enrolled in a Medicare Part D prescription drug plan.
[11] Our measure of physicians' resource use therefore includes all
resources used by their patients, including those ordered by other
providers. Patients were assigned to a physician if they had at least
one evaluation and management visit with the physician during the
calendar year for cardiologists, internists, and orthopedic surgeons,
or if they received any service from the physician for diagnostic
radiologists. According to our definition of a physician's practice, a
beneficiary could belong to the practice of multiple specialists in our
study.
[12] These were the most recent data available when we began our study.
[13] We divided the physicians into five ascending groups (quintiles)
of nearly equal size based on the measure of their resource use.
[14] See appendix I for further discussion of our methodology.
[15] See the bibliography.
[16] We interviewed officials of four specialty societies: the American
Academy of Orthopaedic Surgeons, the American College of Cardiology,
the American College of Physicians, and the American College of
Radiology. We also interviewed officials of Aetna, Inc.; Cigna
Corporation; Humana, Inc.; UnitedHealthGroup, Inc.; and WellPoint, Inc.
[17] See Pub. L. No. 110-275, §131(c).
[18] For the purposes of this report, we defined institutional services
as hospital inpatient and skilled nursing facility services.
[19] We defined beneficiaries' resource use in terms of their resource
use compared to that of other beneficiaries with similar health
conditions. Physicians' resource use is derived from beneficiaries'
resource use. It is defined as the average resource use of those
Medicare beneficiaries in our study population whom the physician saw
compared to the average resource use of other physicians' Medicare
beneficiaries. To determine stability of beneficiaries' resource use,
we identified beneficiaries who were in our study population in both
2005 and 2006. To determine stability of physicians' resource use, we
identified physicians in the four specialties we studied who saw at
least one of the Medicare beneficiaries in our 2005 study population
and at least one beneficiary in our 2006 study population. We divided
physicians and beneficiaries into quintiles according to their resource
use. See appendix I.
[20] Increases in per beneficiary expenditures across the physician
resource use quintiles were accompanied by an increase in the average
risk score of beneficiaries for all the specialties. However, this
tendency did not significantly affect our physician resource use
measure, because the resource use of physicians in higher quintiles was
higher than that of physicians in lower quintiles for all of the
patients they saw, regardless of health status. See appendix I.
[21] The model code was developed in 2007 by the New York State
Attorney General's office in consultation with medical societies,
including the American Medical Association, and consumer groups. The
model code was developed during the course of an investigation by the
Attorney General's office into insurers' potentially deceptive steering
of patients to the least expensive physicians under the guise of
physician ranking programs. As of February 2009, the Attorney General's
office had settled with eight insurers, instituting reforms designed to
ensure that ranking programs are based on accurate and transparent
measures.
[22] Beckman, et al., "Current Approaches to Improve the Value of
Care," p. 9.
[23] Teleki, et al., p.7.
[24] Officials of two specialty societies also recommended state or
local comparisons.
[25] Marder et al., p. 162.
[26] Bennett, Nancy L., Linda L. Casebeer, Robert E. Kristofco, Sheryl
M. Strasser. "Physicians' Internet Information-Seeking Behavior." The
Journal of Continuing Education in the Health Professions, Vol. 24
(2004), pp. 31-38.
[27] At the time of our interview, this insurer was profiling and
providing feedback to physicians in only one specialty. However,
officials said that the company would continue to hand-deliver results
to all physicians even after it begins reporting to physicians in
additional specialties.
[28] When disseminating information electronically, federal agencies,
including CMS, must comply with requirements under Section 508 of the
Rehabilitation Act of 1973 (29 U.S.C. §794d), which requires that
federal employees and members of the public who are individuals with
disabilities have access to and use of the information that is
comparable to the access to and use of the information by federal
employees and members of the public who are not individuals with
disabilities.
[29] Teleki, et al., pp. 5-6.
[30] These studies varied in terms of the type of feedback provided
(verbal and/or written; directed to individuals or groups; delivered by
senior colleagues, professional standards review organizations, or
other sources), types of clinicians to whom the feedback was delivered
(physicians, dentists, nurses, or other providers), frequency and
duration of the feedback, the content, whether the feedback was
combined with other interventions, and the outcomes studied.
[31] Veloski, p. 125.
[32] Jamtvedt, "Audit and Feedback," pp. 5 and 24.
[33] Two used the information to assign ratings to physicians in the
provider directories made available to members, two used it to select
physicians for high-performance networks, and one used it for both
purposes.
[34] These data are weighted national estimates from the Community
Tracking Study Physician Survey conducted by the Center for Studying
Health System Change.
[35] See [hyperlink, http://www.gao.gov/products/GAO-07-307].
[36] Medicare Quality Improvement Organizations are private
organizations that contract with CMS to monitor and improve the care
delivered to Medicare beneficiaries in the 50 states, the territories,
and the District of Columbia.
[37] Although another review of the literature concluded that physician
involvement had little or no impact on the effectiveness of a system in
changing physician behavior, the researchers acknowledged that this
finding was unexpected and could be related to a lack of detail in the
studies they reviewed about the level of physicians' involvement.
[38] These areas refer to the following Core-Based Statistical Areas
(CBSA), an umbrella term for micropolitan and metropolitan statistical
areas: Miami-Fort Lauderdale-Pompano Beach, Fla.; Phoenix-Mesa-
Scottsdale, Ariz.; Pittsburgh, Pa.; and Sacramento--Arden-Arcade--
Roseville, Calif. For CBSA definitions, see [hyperlink,
http://www.census.gov/population/www/metroareas/metroarea.html].
[39] We also included one additional variable to represent
beneficiaries who did not have any of the included 70 HCCs.
[40] Hierarchical Condition Categories (HCCs) collapse the over 15,000
diagnosis codes into 189 clinically meaningful condition categories
which are additionally grouped into hierarchies of increasing severity.
If a beneficiary's diagnoses correspond to more than one condition
within a hierarchy, he or she is assigned only the most severe one.
[41] We derived our expenditure data from beneficiaries' Part A and
Part B Medicare FFS claims. We did not include Part D claims because
not all Medicare beneficiaries are enrolled in a Part D prescription
drug plan. We made two adjustments to the data used to estimate the
model: (1) we annualized the expenditures of beneficiaries who died
during the year and (2) we capped total annual expenditures for all
beneficiaries at $100,000 in order to reduce the effect of
beneficiaries with extreme values in the model's estimation.
[42] We excluded several types of beneficiaries: (1) those who were
institutionalized for more than 3 consecutive months during the year,
(2) those who were enrolled in a Medicare Advantage plan for any part
of the year, (3) those who were newly enrolled in Medicare, and (4)
those enrolled on the basis of having end-stage renal disease.
[43] We chose the break points for the risk categories based on
beneficiaries' risk scores--the ratio of their predicted cost to the
sample mean. The first 10 risk categories had intervals of 0.1, while
the subsequent 15 had intervals ranging from 0.2 to 4. We initially
specified 26 risk categories, but dropped the final one containing
beneficiaries with risk scores exceeding 18.0 because it contained less
than 120 beneficiaries in each year.
[44] We included expenditures from all claims submitted on the
beneficiary's behalf, including claims from locations outside the four
selected metropolitan areas and claims from all provider types
(hospital inpatient, outpatient, physician, durable medical equipment,
skilled nursing facility, home health, and hospice). We did not include
Part D prescription drug costs because not all Medicare beneficiaries
are enrolled in a Medicare Part D prescription drug plan.
[45] Each beneficiary resource use quintile includes 20 ranks such that
the first quintile consists of beneficiaries with ranks 1-20 and the
last quintile consists of beneficiaries with ranks 81-100.
[46] According to our definition of a physician's practice, a
beneficiary could belong to the practice of multiple specialists in our
study.
[47] We applied this criterion for diagnostic radiologists because they
typically do not have evaluation and management visits.
[48] We excluded 28 percent of the physicians in the four specialties
in 2005 and 29 percent in 2006 because they treated less than 100
Medicare patients a year. Our analyses included 5,890 physicians in
2005 and 5,828 in 2006.
[49] Our measure of physicians' resource use therefore includes all
resources used by their patients, including those ordered by other
providers.
[50] Each physician resource use quintile includes 20 ranks such that
the first quintile consists of physicians with ranks 1-20 and the last
quintile consists of physicians with ranks 81-100.
[51] The five health status categories collapse the 25 risk categories
into five broader health status categories. Each health status category
consists of 5 risk categories, which span the following ranges of risk
scores (r): r £ .5, .5 < r £ 1.0, 1.0 < r £ 2.0, 2.0 < r £ 5.0, 5.0 < r
£ 18.0. The first health category includes the healthiest beneficiaries
and comprises, on average, 27 percent of the Medicare patients seen by
the physicians in our study in 2006; the fifth includes the sickest
beneficiaries and comprises 16 percent of their Medicare patients. The
second, third, and fourth health categories comprise, respectively, 14,
18, and 25 percent of the physicians' Medicare patients.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
