Health Care Quality Measurement
The National Quality Forum Has Begun a 4-Year Contract with HHS Gao ID: GAO-10-737 July 14, 2010The Medicare Improvements for Patients and Providers Act of 2008 (MIPPA) directed the Department of Health and Human Services (HHS) to enter into a 4-year contract with an entity to perform five duties related to health care quality measurement and authorized $40 million from the Medicare Trust Funds for the contract. In January 2009, HHS awarded a contract to the National Quality Forum (NQF), under which HHS will reimburse NQF for its costs and pay additional fixed fees. Established in 1999, NQF is a nonprofit member organization that fosters agreement on national standards for measuring and public reporting of health care performance data. This is the first of two reports MIPPA requires GAO to submit on NQF's contract with HHS. In this report, which covers the first contract year--January 14, 2009, to January 13, 2010--GAO describes (1) the status of NQF's work on the five duties under MIPPA; (2) the costs and fixed fees NQF has reported; and (3) what NQF and HHS do in order to help ensure that NQF's reported costs are proper. GAO reviewed relevant MIPPA provisions and reviewed HHS and NQF documents, such as HHS's contract with NQF, monthly progress reports and invoices for the first contract year, and policies and other documents that describe how HHS and NQF review invoices. GAO also interviewed NQF and HHS officials responsible for implementing and overseeing the contract.
NQF has begun work for each of five duties required by MIPPA related to quality measures: (1) make recommendations on a national strategy and priorities; (2) endorse quality measures, which involves a process for determining which ones should be recognized as national standards; (3) maintain--that is, update or retire--endorsed quality measures; (4) promote electronic health records; and (5) report annually to Congress and the Secretary of HHS. As of January 13, 2010--the end of the first contract year--NQF's work for four MIPPA duties was in progress and it had completed its first annual report for the fifth duty. For example, NQF had begun the duties related to endorsement and maintenance by initiating the endorsement process for three projects HHS selected and by starting maintenance reviews for a set of measures of interest to or used by HHS. While NQF began work for each of the duties in the first contract year, HHS determines on an annual basis the work NQF will be expected to perform under the five duties each contract year. NQF reported costs and fixed fees totaling approximately $6.5 million for the first contract year, including direct and indirect costs as well as fixed fees. Specifically, NQF reported about $3.2 million in direct costs, or 49 percent of the total. These were costs specifically incurred for the NQF contract, such as direct labor for NQF employees. NQF also reported about $2.9 million in indirect costs, which cover additional items such as employee benefits and overhead. Finally, NQF reported about $360,000 in fixed fees for the first contract year. Over $5 million of the reported costs and fixed fees were incurred in the second half of the contract year. NQF and HHS rely on reviews of NQF invoices in order to help ensure that NQF's reported costs are proper. At NQF, officials told us that they review the invoices prior to submitting them to HHS and carry out other activities, such as using an electronic system to track labor hours, in order to help ensure that the costs they report in the invoices are proper. Like NQF, HHS relies on reviews of NQF invoices in order to help ensure NQF's reported costs are proper. These reviews are governed by HHS policies and procedures and by requirements applicable to federal contracts generally. While NQF has begun work under the MIPPA contract, it is too early for GAO to assess whether, or to what extent, NQF will be successful in carrying out the five MIPPA duties. This report describes NQF's work for the first of 4 contract years. In the remaining 3 years of the contract, HHS will determine on an annual basis specific work for NQF to complete under each of the five MIPPA duties. Therefore, it is not yet known exactly what work NQF will be expected to complete during the remainder of the contract period. GAO's second report, which is due in January 2012, will provide another opportunity to review NQF's performance and costs. HHS and NQF reviewed a draft of this report and provided technical comments, which GAO incorporated as appropriate.
GAO-10-737, Health Care Quality Measurement: The National Quality Forum Has Begun a 4-Year Contract with HHS
This is the accessible text file for GAO report number GAO-10-737
entitled 'Health Care Quality Measurement: The National Quality Forum
Has Begun a 4-Year Contract with HHS' which was released on July 14,
2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
July 2010:
Health Care Quality Measurement:
The National Quality Forum Has Begun a 4-Year Contract with HHS:
GAO-10-737:
GAO Highlights:
Highlights of GAO-10-737, a report to congressional committees.
Why GAO Did This Study:
The Medicare Improvements for Patients and Providers Act of 2008
(MIPPA) directed the Department of Health and Human Services (HHS) to
enter into a 4-year contract with an entity to perform five duties
related to health care quality measurement and authorized $40 million
from the Medicare Trust Funds for the contract. In January 2009, HHS
awarded a contract to the National Quality Forum (NQF), under which
HHS will reimburse NQF for its costs and pay additional fixed fees.
Established in 1999, NQF is a nonprofit member organization that
fosters agreement on national standards for measuring and public
reporting of health care performance data.
This is the first of two reports MIPPA requires GAO to submit on NQF‘s
contract with HHS. In this report, which covers the first contract
year”January 14, 2009, to January 13, 2010”GAO describes (1) the
status of NQF‘s work on the five duties under MIPPA; (2) the costs and
fixed fees NQF has reported; and (3) what NQF and HHS do in order to
help ensure that NQF‘s reported costs are proper.
GAO reviewed relevant MIPPA provisions and reviewed HHS and NQF
documents, such as HHS‘s contract with NQF, monthly progress reports
and invoices for the first contract year, and policies and other
documents that describe how HHS and NQF review invoices. GAO also
interviewed NQF and HHS officials responsible for implementing and
overseeing the contract.
What GAO Found:
NQF has begun work for each of five duties required by MIPPA related
to quality measures: (1) make recommendations on a national strategy
and priorities; (2) endorse quality measures, which involves a process
for determining which ones should be recognized as national standards;
(3) maintain”that is, update or retire”endorsed quality measures; (4)
promote electronic health records; and (5) report annually to Congress
and the Secretary of HHS. As of January 13, 2010”the end of the first
contract year”NQF‘s work for four MIPPA duties was in progress and it
had completed its first annual report for the fifth duty. For example,
NQF had begun the duties related to endorsement and maintenance by
initiating the endorsement process for three projects HHS selected and
by starting maintenance reviews for a set of measures of interest to
or used by HHS. While NQF began work for each of the duties in the
first contract year, HHS determines on an annual basis the work NQF
will be expected to perform under the five duties each contract year.
NQF reported costs and fixed fees totaling approximately $6.5 million
for the first contract year, including direct and indirect costs as
well as fixed fees. Specifically, NQF reported about $3.2 million in
direct costs, or 49 percent of the total. These were costs
specifically incurred for the NQF contract, such as direct labor for
NQF employees. NQF also reported about $2.9 million in indirect costs,
which cover additional items such as employee benefits and overhead.
Finally, NQF reported about $360,000 in fixed fees for the first
contract year. Over $5 million of the reported costs and fixed fees
were incurred in the second half of the contract year.
NQF and HHS rely on reviews of NQF invoices in order to help ensure
that NQF‘s reported costs are proper. At NQF, officials told us that
they review the invoices prior to submitting them to HHS and carry out
other activities, such as using an electronic system to track labor
hours, in order to help ensure that the costs they report in the
invoices are proper. Like NQF, HHS relies on reviews of NQF invoices
in order to help ensure NQF‘s reported costs are proper. These reviews
are governed by HHS policies and procedures and by requirements
applicable to federal contracts generally.
While NQF has begun work under the MIPPA contract, it is too early for
GAO to assess whether, or to what extent, NQF will be successful in
carrying out the five MIPPA duties. This report describes NQF‘s work
for the first of 4 contract years. In the remaining 3 years of the
contract, HHS will determine on an annual basis specific work for NQF
to complete under each of the five MIPPA duties. Therefore, it is not
yet known exactly what work NQF will be expected to complete during
the remainder of the contract period. GAO‘s second report, which is
due in January 2012, will provide another opportunity to review NQF‘s
performance and costs. HHS and NQF reviewed a draft of this report and
provided technical comments, which GAO incorporated as appropriate.
View [hyperlink, http://www.gao.gov/products/GAO-10-737] or key
components. For more information, contact Linda T. Kohn at (202) 512-
7114 or kohnl@gao.gov.
[End of section]
Contents:
Letter:
Background:
NQF Has Begun Work for Each of the Five Duties Required by MIPPA
Related to Quality Measures:
NQF Reported about $6.5 Million in Costs and Fixed Fees for the First
Contract Year:
NQF and HHS Rely on Reviews of NQF Invoices in Order to Help Ensure
That NQF's Reported Costs Are Proper:
Concluding Observations:
Agency and Other External Comments:
Appendix I: National Quality Forum's Endorsement Process and Example
Project:
Appendix II: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Description of Quality Measurement Duties as Specified in the
Medicare Improvements for Patients and Providers Act of 2008 (MIPPA):
Table 2: Procedures Required under HHS Policy When Reviewing Invoices
and Implementation of These Procedures for the NQF Contract:
Table 3: National Quality Forum's (NQF) Endorsement Process and
Example Project:
Figure:
Figure 1: National Quality Forum's (NQF) Costs and Fixed Fees Reported
for the First Year of the Contract with HHS:
Abbreviations:
APU program: Reporting Hospital Quality Data for Annual Payment Update
Program:
ASPE: Assistant Secretary for Planning and Evaluation:
CMS: Centers for Medicare & Medicaid Services:
CSAC: Consensus Standards Approval Committee:
FAR: Federal Acquisition Regulation:
HHS: Department of Health and Human Services:
MIPPA: Medicare Improvements for Patients and Providers Act of 2008:
NQF: National Quality Forum:
PPACA: Patient Protection and Affordable Care Act of 2010:
QDS: Quality Data Set:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
July 14, 2010:
Congressional Committees:
Health care quality measures are used to evaluate how health care is
delivered, and information obtained from such measures can promote
accountability among health care providers and help consumers make
informed choices about their care. The Department of Health and Human
Services (HHS) encourages use of quality measures through programs
that provide financial incentives to health care providers who
voluntarily collect and report information on certain quality
measures, which HHS then makes publicly available.[Footnote 1] For
example, as part of one program, HHS reported that in fiscal year
2009, almost all--96 percent--of eligible hospitals participating in
Medicare reported on their performance against certain quality
measures. Recent legislation requires HHS to implement additional
programs that will rely on quality measures, such as a pay-for-
performance program under which HHS will pay incentives to hospitals
based on their performance on selected quality measures.[Footnote 2]
The Medicare Improvements for Patients and Providers Act of 2008
(MIPPA) directed HHS to enter into a 4-year contract with an entity to
perform five duties related to health care quality measurement: (1)
make recommendations on a national strategy and priorities, (2)
endorse quality measures, (3) maintain endorsed quality measures, (4)
promote electronic health records, and (5) report annually to Congress
and the Secretary of HHS.[Footnote 3] MIPPA authorized $10 million per
year--$40 million in total--from the Medicare Trust Funds for this 4-
year contract, which covers the period from January 14, 2009, through
January 13, 2013. In addition, the Patient Protection and Affordable
Care Act (PPACA), which was enacted in March 2010, established
additional duties for the entity.
On January 14, 2009, HHS awarded the 4-year contract required by
MIPPA, after issuing a solicitation seeking competitive proposals,
[Footnote 4] to the National Quality Forum (NQF), a nonprofit
organization established in 1999 that fosters agreement on national
standards for measurement and public reporting of health care
performance data. NQF uses a process recognized under the National
Technology Transfer and Advancement Act of 1995 that grants quality
measures and other standards endorsed by consensus-based entities,
such as NQF, standing as national voluntary consensus standards.
[Footnote 5] NQF uses its process to evaluate available quality
measures to determine which ones are qualified to be endorsed--that
is, recognized--as national standards. NQF-endorsed quality measures
have been used by HHS in its quality measurement programs.[Footnote 6]
In 2008, prior to receiving the contract award, NQF's revenue from all
sources was approximately $10 million. NQF staff told us that while
NQF has previously received funding from HHS for some of its work
related to quality measures, the $10 million per year authorized by
MIPPA for the contract is larger than previous funding.
HHS's 4-year contract with NQF is a cost-plus-fixed-fee contract,
under which HHS will pay NQF for its costs and additional fixed fees
for its services.[Footnote 7] The Federal Acquisition Regulation (FAR)
[Footnote 8] provides that cost-plus-fixed-fee and other types of cost-
reimbursement contracts may only be used when the contractor's
accounting system is adequate for determining costs under the contract
and appropriate government surveillance during performance will
provide reasonable assurance that efficient methods and effective cost
controls are used. For the purposes of this report, we refer to costs
that are allowable under the contract as "proper."[Footnote 9]
MIPPA required GAO to study the performance of and costs incurred by
NQF under its 4-year contract with HHS and submit a first report by
July 14, 2010, and a second report by January 14, 2012.[Footnote 10]
This first report covers the first contract year that began January
14, 2009, and ended January 13, 2010. In this report, we describe (1)
the status of NQF's work on the five duties related to health care
quality measurement required under MIPPA, (2) the costs and fixed fees
that NQF has reported under its contract, and (3) what NQF and HHS do
in order to help ensure that NQF's reported costs are proper.
To describe the status of NQF's work on duties related to health care
quality measurement required under MIPPA, we focused our review on the
status of NQF's work related to the five MIPPA duties as of the end of
the first contract year, January 13, 2010. We reviewed relevant
provisions in MIPPA, and HHS and NQF documents related to implementing
health care quality measurement duties in MIPPA. Specifically, we
reviewed HHS's contract with NQF and NQF's 2009 annual work plan,
which established specific activities for implementing these duties as
well as scheduled time frames for the activities. We also reviewed the
monthly progress reports NQF is required to submit to HHS on its
efforts for the first contract year, and we reviewed NQF's first
annual report to HHS and Congress. We interviewed NQF officials
responsible for implementing the contract and HHS officials
responsible for managing the contract and overseeing NQF's
performance. For NQF activities in progress at the end of the first
contract year, we gathered information about their planned completion
dates as of January 13, 2010. Our finding is limited to the duties
established under MIPPA and does not include additional duties
mandated by PPACA, which was enacted after the end of the first
contract year.
To describe the costs and fixed fees that NQF has reported under its
contract, we reviewed NQF invoices submitted to HHS for the first
contract year--January 14, 2009, through January 13, 2010. These
invoices include the amounts of costs and fixed fees reported by NQF.
We also reviewed NQF's monthly progress reports to HHS for the first
contract year. We interviewed NQF officials responsible for reviewing
and approving the costs and fixed fees submitted to HHS under the
contract. We also interviewed HHS officials responsible for reviewing
NQF's costs and fixed fees reported under the contract. Based on our
review of relevant documents and interviews with NQF and HHS
officials, we determined that the reported costs and fixed-fee data
were sufficiently reliable for the purposes of this report.
To describe what NQF and HHS do in order to help ensure that NQF's
reported costs are proper, we interviewed NQF and HHS officials and
reviewed relevant policies and procedures. For NQF, we interviewed
officials about their process for reviewing NQF invoices submitted to
HHS and about the other policies and procedures that NQF has in order
to help ensure that the costs they report to HHS in the invoices are
proper. We focused our discussions on policies and procedures related
to employee labor costs and payments to contractors and consultants,
which are the majority of NQF's direct costs. We also reviewed the
invoices that NQF submitted to HHS during the first contract year for
evidence of NQF approval by NQF officials. Additionally, we examined
the files that NQF maintained on its subcontracts to review
documentation for the eight subcontractors and consultants that
performed work related to the HHS contract in the first contract year.
We compared documentation that NQF maintains for the eight
subcontractors and consultants with the requirements in NQF's January
2010 procurement policy. For HHS, we interviewed officials responsible
for reviewing NQF invoices, the project officer and the contracting
officer for the NQF contract. We identified requirements in relevant
HHS policies and procedures as well as relevant federal contracting
requirements for oversight of cost-reimbursement contracts. We
interviewed HHS officials about how they review NQF invoices. We also
reviewed documentation in the file HHS maintained on the NQF contract
for the first contract year to describe whether HHS officials followed
the invoice review procedures they explained to us. Additionally, we
reviewed documentation in the NQF contract file related to NQF's use
of eight subcontractors and consultants that performed work under the
contract for the first contract year. Our work was limited to
describing what NQF and HHS do to help ensure that NQF's reported
costs were proper and did not include a determination of whether the
costs were proper.
We conducted this performance audit from September 2009 through June
2010 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
NQF is a nonprofit organization established in 1999 that fosters
agreement on national standards for measurement and public reporting
of health care performance data. Its membership includes more than 400
organizations that represent multiple sectors of the health care
system, including providers, consumers, and researchers.[Footnote 11]
NQF uses a consensus development process to evaluate and endorse
consensus standards, including quality measures, best practices,
frameworks, and reporting guidelines. NQF has endorsed over 600
quality measures in 27 areas, such as cancer and diabetes. NQF
endorses quality measures developed by other organizations, such as
the Joint Commission, the National Committee for Quality Assurance,
and the American Medical Association, rather than developing quality
measures itself. HHS has used a number of NQF-endorsed measures in
initiatives to promote quality measurement, and NQF continues to
endorse quality measures separate from this contract.
Duties Established in MIPPA:
MIPPA established five duties related to the use of quality measures.
See table 1 for a description of the duties.
Table 1: Description of Quality Measurement Duties as Specified in the
Medicare Improvements for Patients and Providers Act of 2008 (MIPPA):
MIPPA duties: Make recommendations on national strategy and priorities;
Description of MIPPA duties:
1. (1) The entity shall synthesize evidence and convene key
stakeholders to make recommendations, with respect to activities
conducted under this Act, on an integrated national strategy and
priorities for health care performance measurement in all applicable
settings. In making such recommendations, the entity shall:
(A) ensure that priority is given to measures:
i. that address the health care provided to patients with prevalent,
high-cost chronic diseases;
ii. with the greatest potential for improving the quality, efficiency,
and patient-centeredness of health care; and;
iii. that may be implemented rapidly due to existing evidence,
standards of care, or other reasons; and;
(B) take into account measures that:
i. may assist consumers and patients in making informed health care
decisions;
ii. address health disparities across groups and areas; and;
iii. address the continuum of care a patient receives, including
services furnished by multiple health care providers or practitioners
and across multiple settings.
MIPPA duties: Endorsement of measures;
Description of MIPPA duties:
(2) The entity shall provide for the endorsement of standardized
health care performance measures. The endorsement process under the
preceding sentence shall consider whether a measure:
(A) is evidence-based, reliable, valid, verifiable, relevant to
enhanced health outcomes, actionable at the caregiver level, feasible
to collect and report, and responsive to variations in patient
characteristics, such as health status, language capabilities, race or
ethnicity, and income level; and;
(B) is consistent across types of health care providers, including
hospitals and physicians.
MIPPA duties: Maintenance of measures;
Description of MIPPA duties:
(3) The entity shall establish and implement a process to ensure that
measures endorsed under the second duty are updated (or retired if
obsolete) as new evidence is developed.
MIPPA duties: Promotion of the development of electronic health
records;
Description of MIPPA duties:
(4) The entity shall promote the development and use of electronic
health records that contain the functionality for automated
collection, aggregation, and transmission of performance measurement
information.
MIPPA duties: Annual report to Congress and the Secretary of Health
and Human Services;
Secretarial publication and comment;
Description of MIPPA duties:
(5)(A) The entity shall submit to Congress and the Secretary, by not
later than March 1 of each year (beginning with 2009), a report
containing a description of:
i. the implementation of quality measurement initiatives under this
Act and the coordination of such initiatives with quality initiatives
implemented by other payers;
ii. the recommendations made under the first duty; and;
iii. the performance by the entity of the duties required under the
contract entered into with the Secretary under subsection (a);
(B) not later than 6 months after receiving a report under
subparagraph (A) for a year, the Secretary shall:
i. review such report; and;
ii. publish such report in the Federal Register, together with any
comments of the Secretary on such report.
Source: GAO summary of MIPPA duties prior to amendments made by the
Patient Protection and Affordable Care Act (PPACA).
[End of table]
NQF Contract:
For the NQF contract, HHS selected a cost-plus-fixed-fee contract--
NQF's first cost-reimbursement contract. Under the cost-plus-fixed-fee
contract, HHS will reimburse NQF for costs incurred under the contract
in addition to a fixed fee that is paid regardless of other costs.
Cost-plus-fixed-fee contracts are used for efforts such as research,
design, or study efforts where cost and technical uncertainties exist
and it is desirable to retain as much flexibility as possible in order
to accommodate change. However, this type of contract provides only a
minimum incentive to the contractor to control costs. As we reported
in 2009, these contracts are suitable when the cost of work to be done
is difficult to estimate and the level of effort required is unknown.
[Footnote 12]
Under the FAR, cost-reimbursement contracts may only be used when the
contractor's accounting system is adequate for determining costs under
the contract to help prevent situations where contractors bill the
government for unallowable costs. One method an agency can use to
determine if an accounting system is adequate is to perform a preaward
survey of a potential contractor's accounting system prior to awarding
a contract.[Footnote 13] This review serves as a key control to
determine whether the potential contractor has an adequate accounting
system in place to accurately and consistently record costs and submit
invoices for costs. HHS conducted two preaward surveys of NQF's
accounting system. HHS's initial review, in November 2007, found that
NQF's accounting system was inadequate because the system could not
identify and separate unallowable costs, among other issues. NQF
subsequently replaced its accounting system, and a second HHS review
in November 2008 found that the system was adequate.
Under the FAR, contracts are to contain provisions for agency approval
of a contractor's subcontracts.[Footnote 14] HHS's contract with NQF
contains this provision and also requires the approval of consultants.
This review requires appropriate support documentation provided by the
contractor to the agency, including a description of the services to
be subcontracted, the proposed subcontract price, and a negotiation
memo that reflects the principal elements of the subcontract price
negotiations between the contractor and subcontractor.
Two HHS components are principally responsible for administering the
NQF contract: the office of the Assistant Secretary for Planning and
Evaluation (ASPE) and the Centers for Medicare & Medicaid Services
(CMS)--an operational division within HHS.[Footnote 15] To conduct
oversight of the NQF contract, HHS assembled staff in these two units
with experience in acquisitions, contract management, and program
management. Specifically, the project officer for the NQF contract,
responsible for program management and performance assessment, is a
representative of ASPE.[Footnote 16] The contracting officer for the
NQF contract, responsible for administering the contract, is a
representative of CMS.[Footnote 17] The contracting officer and
project officer should perform a comprehensive review of contractor
invoices to determine if the contractor is billing costs in accordance
with the contract terms and applicable government regulations.
NQF Has Begun Work for Each of the Five Duties Required by MIPPA
Related to Quality Measures:
As of January 13, 2010--the end of the first year of HHS's 4-year
contract with NQF to implement the MIPPA duties--NQF had begun work
for each of the five duties required by MIPPA related to health care
quality measures: (1) make recommendations on a national strategy and
priorities; (2) endorse quality measures; (3) maintain endorsed
quality measures; (4) promote electronic health records; and (5)
report annually to Congress and the Secretary of HHS. While NQF began
work for each of the duties in the first contract year, HHS determines
on an annual basis the specific work NQF will be expected to perform
under the five MIPPA duties in each contract year.
Recommendations on a National Strategy and Priorities for Quality
Measurement. NQF has taken steps to begin the duty of making
recommendations on a national strategy and priorities for quality
measurement. In October 2009, NQF established a committee of
stakeholders that is expected to develop recommendations about a
national strategy and priorities for quality measurement. NQF
published the recommended priorities in May 2010. The committee's
recommendations are expected to be based on a synthesis of evidence
that NQF has collected, using a subcontractor, on 20 conditions that
account for the majority of Medicare's costs.[Footnote 18] The
subcontractor collected evidence on existing quality measures for
these conditions and identified gaps where quality measures did not
exist. The subcontractor also collected evidence related to each
condition, such as information on each condition's prevalence,
treatment costs, variability in providers' treatment of the condition,
disparities in treatment for patients with the condition, and
potential to improve quality of care for the condition. The committee
is expected to consider this evidence when developing recommendations
on a national strategy and priorities for quality measurement. Under
PPACA, NQF's recommendations[Footnote 19] on a national strategy and
priorities must be considered by HHS when it develops a national
strategy for quality improvement, which HHS is required to submit to
Congress by January 1, 2011.[Footnote 20]
Endorsement of Measures. NQF has taken steps to provide for the
endorsement of quality measures. Prior to its contract with HHS, NQF
established a process for endorsing quality measures. Under this
process, organizations that develop quality measures submit them to
NQF for consideration, in response to specific solicitations by NQF.
[Footnote 21] NQF forms a committee of experts from its member
organizations as well as other organizations and agencies to review
these quality measures against NQF-established criteria, such as the
usability and feasibility of the measure. After this committee
evaluates the measures against these criteria, NQF's process allows
for a period during which its member organizations and the public may
comment on the committee's recommendation for each measure. The
process also provides for a period for its member organizations to
vote on whether the measures should be endorsed by NQF as a national
standard. Ultimately, NQF's board of directors makes a final decision
on whether NQF should formally endorse the measures. (See appendix I
for a detailed description of NQF's endorsement process.)
In order to provide for the endorsement of quality measures under this
duty, NQF has taken several steps. Specifically, NQF initiated
projects and solicited measures to be endorsed using its process for
each of these projects. These projects relate to quality measurement
in nursing homes, patient safety, and patient outcomes, and are
scheduled to be completed between December 2010 and May 2011. In
addition to endorsing measures, NQF also hired a subcontractor to
evaluate its endorsement process and recommend ways to improve its
efficiency and effectiveness. The subcontractor's report and NQF's
approval of proposed enhancements to the process are due January 2011.
Maintenance of Endorsed Quality Measures. NQF has taken steps to
ensure that endorsed measures are maintained--that is, updated or
retired. Prior to its contract with HHS, NQF established a process for
maintenance of measures. According to NQF, once a quality measure has
been endorsed, updated information on the measure's specifications
should be submitted to NQF annually and the measure should be
comprehensively reviewed under the maintenance process every 3 years.
NQF's maintenance process is similar to NQF's endorsement process, in
that it involves a review of measures against NQF-established
criteria, a period for public comment, and a final decision by NQF's
board of directors. In order to implement this process under its
contract with HHS, NQF began maintenance reviews for 191 measures in
14 areas such as diabetes and cardiovascular care. The measures were
identified by HHS as being of interest to, or actually used by, HHS
programs. By the end of the first contract year, NQF had not
determined completion dates for maintenance of the 191 measures. As of
May 2010, maintenance of the 191 measures identified by HHS is
scheduled to be completed by the end of 2012.
Promotion of the Development and Use of Electronic Health Records. NQF
has taken steps towards completing the duty of promoting the
development and use of electronic health records for use in quality
measurement.[Footnote 22] As of January 13, 2010, NQF had begun to
implement a framework that defines a standardized set of data that
should be captured in patients' electronic health records. The
framework, known as the Quality Data Set (QDS), is intended to allow
data from electronic health records to be collected and used in
quality measurement. Implementation and maintenance of the QDS is
scheduled to continue through the end of the 4-year contract, which
ends January 13, 2013. To further promote the development and use of
electronic health records in quality measurement, NQF began additional
activities. For example, NQF established a panel of experts to
recommend additional capabilities to measure utilization. According to
NQF officials, efforts under this duty are scheduled for completion
between March 2010 and January 2013.
Annual Report to Congress and the Secretary of HHS. NQF submitted its
first annual report to Congress and the Secretary of HHS on March 1,
2009.[Footnote 23] HHS published this report, with its comments, in
the Federal Register on September 10, 2009. NQF submitted its second
annual report, which also covers activities it performed during the
first contract year, to Congress and the Secretary on March 1, 2010.
[Footnote 24]
While NQF has begun work for each of the duties in the first contract
year, HHS determines on an annual basis the specific work NQF will be
expected to perform under the five MIPPA duties each contract year.
Specifically, HHS gives direction for and then approves annual plans
that NQF develops. These plans can include work begun in prior
contract years that has not been completed. HHS can adjust work in the
annual plans in support of each of the five duties. For example, HHS
officials told us that in future contract years, they may select
additional projects for the endorsement of quality measures, and
additional measures for maintenance reviews.
NQF Reported about $6.5 Million in Costs and Fixed Fees for the First
Contract Year:
NQF reported costs and fixed fees totaling approximately $6.5 million
for the first year of its contract with HHS, which ended January 13,
2010.[Footnote 25] The amount NQF reported included direct and
indirect costs, as well as fixed fees. Direct costs, which are costs
incurred specifically for this contract, represented the largest
percentage--about $3.2 million, or 49 percent--of the amount NQF
reported (see figure 1). NQF's reported direct costs were largely
labor costs for NQF employees and payments to subcontractors and
consultants. In addition to direct costs, NQF reported about $2.9
million in indirect costs for the first contract year. Indirect costs
cover additional items, such as employee benefits, overhead, and
administrative costs.[Footnote 26] NQF calculates its indirect costs
based on a formula that takes into account an indirect-cost rate
approved by HHS and the amounts of certain direct costs.[Footnote 27]
For example, the formula estimates indirect costs such as employee
benefits by multiplying an indirect-cost rate by the amount of direct
costs for labor. Finally, in addition to its direct and indirect
costs, NQF reported fixed fees of approximately $360,000 during the
first contract year. HHS pays these fixed fees to NQF in addition to
reimbursing the organization for its costs.[Footnote 28]
Figure 1: National Quality Forum's (NQF) Costs and Fixed Fees Reported
for the First Year of the Contract with HHS:
[Refer to PDF for image: pie-chart]
Direct costs: $3,205,503: 49%;
Indirect costs: $2,939,901: 45%;
Fixed fee: $357,228: 6%;
Total costs: $6,502,631: 100%.
Source: GAO analysis of NQF data.
Note: Data are as of the end of the first contract year, January 13,
2010.
[End of figure]
Of the approximately $6.5 million in costs and fixed fees NQF reported
for the first contract year, most were incurred in the second half of
the contract year. Costs and fixed fees in the second half of the
contract year, from July 1, 2009, to January 13, 2010, totaled over $5
million.[Footnote 29] NQF staff told us that costs in the first half
of the contract year were primarily for activities such as development
of solicitations for subcontractors. Costs in the second half of the
contract year were primarily for activities related to quality
measurement, such as endorsement of quality measures and promotion of
electronic health records for use in quality measurement.
NQF and HHS Rely on Reviews of NQF Invoices in Order to Help Ensure
That NQF's Reported Costs Are Proper:
NQF reviews invoices and carries out other activities prior to
submitting them to HHS in order to help ensure that reported costs are
proper. HHS requires its officials to follow certain procedures when
reviewing these invoices.
NQF Reviews Invoices and Carries out Other Activities in Order to Help
Ensure That Its Reported Costs Are Proper:
NQF officials told us their organization has several ways to help
ensure that the contract costs it reports to HHS are proper. According
to NQF officials, invoices are electronically generated using NQF's
accounting system and then reviewed before submitting the invoices to
HHS for payment. These reviews are conducted by two senior staff--the
NQF Project Director, who manages the contract, and the Chief
Financial Officer. These officials meet to review costs reported in
each month's invoice. NQF officials told us that as part of their
reviews, the two officials compare the current month's invoice to the
previous month's invoice to identify discrepancies or cost trends that
seem unusual and that the officials investigate such discrepancies or
trends when necessary. After this review, the Chief Financial Officer
signs the invoice. During our review of NQF's invoices for the first
contract year, we found that the Chief Financial Officer signed the
invoices as the officials described to us.
In addition to the review of invoices, NQF officials described other
ways the organization helps to ensure that the costs it reports to HHS
are proper. In particular, NQF officials told us NQF uses an
electronic timesheet system in order to track employee labor hours.
[Footnote 30] NQF officials told us that the timesheet system allows
NQF employees to track their labor hours by project and have their
labor hours reviewed and approved by the appropriate NQF officials. In
addition to the timesheet system, NQF officials told us that their
organization established a written procurement policy in August 2009
and revised it in January 2010 to guide how they track other direct
costs--specifically, payments to subcontractors and consultants--that
are reported in NQF's invoices.[Footnote 31] NQF officials told us
that under its procurement policy, NQF officials are to obtain the
appropriate approval signatures for payments on invoices as well as
other payments for subcontractors and consultants once the services
have been received. Furthermore, according to the policy, NQF
officials are to document how key procurement decisions are made, such
as the basis for setting an award cost or price for a subcontractor or
consultant. Having a well-designed procurement policy can help reduce
the risk of inappropriate payments or pricing related to
subcontractors and consultants. During our review of NQF subcontractor
and consultant files for the period prior to January 2010--before NQF
revised its procurement policy--we found that NQF did not always
document approvals for subcontractor payment and did not document that
it had determined that its consultant pricing was reasonable.
HHS Requires Its Officials to Review NQF Invoices following Certain
Procedures in Order to Help Ensure That Reported Costs Are Proper:
Like NQF, HHS relies on reviews of NQF's invoices in order to help
ensure that reported costs are proper.[Footnote 32] Two HHS officials
assigned to oversee the NQF contract, the project officer and the
contracting officer, are responsible for these reviews.[Footnote 33]
When conducting their reviews, the two officials are required to
follow certain procedures established in HHS policies.[Footnote 34]
For example, under these policies, the project officer is required to
review NQF's invoices to determine whether billed services were
actually provided and are supported with adequate documentation.
Similarly, the contracting officer is required to review the invoices
to determine whether NQF's reported costs are consistent with its
contract, accurately calculated, and have adequate documentation. Both
officials are required to document when they approve invoices for
payment to NQF. When we reviewed HHS documentation and interviewed HHS
officials during the course of our work, we found that the contracting
officer and project officer had generally followed the review
procedures required by HHS policy.
Table 2 provides more detailed information on the procedures that the
project and contracting officers are required to follow when reviewing
NQF invoices. Table 2 also provides information we obtained from HHS
officials on how they implemented these requirements.
Table 2: Procedures Required under HHS Policy When Reviewing Invoices
and Implementation of These Procedures for the NQF Contract:
Required procedure for reviewing invoices:
1. The project officer must review each of the cost categories
reported in NQF's invoice to ensure that billed services were actually
received, that they were appropriate, and that they are adequately
supported with documentation submitted by NQF. The project officer can
recommend to the contracting officer disapproval of costs that do not
meet these criteria;
Implementation of procedure for the NQF Contract:
HHS officials told us that, as required, the project officer reviewed
each of the cost categories in NQF invoices with the project manager,
who works with the project officer to help provide technical direction
to NQF. Officials told us that the project officer recommended the
disapproval of certain costs to the contracting officer that were not
appropriate. Our review of HHS documentation showed that the project
officer questioned some of NQF's costs and recommended to the
contracting officer disapproval of certain costs for services NQF
should not have performed. For example, the project officer
recommended disapproval of costs that NQF had billed for work on a
project that had been placed on hold.
Required procedure for reviewing invoices:
2. The project officer is to document his or her approval of an
invoice for payment;
Implementation of procedure for the NQF Contract:
For the NQF contract, the project officer told us that she sends an e-
mail each month to the contracting officer to document her approval of
the invoice. We found this documentation during our review of the NQF
contract file.
Required procedure for reviewing invoices:
3. The contracting officer or the contract specialist, who provides
support to the contracting officer, is required to review the invoices
to determine if, among other things, (1) all costs are consistent with
the requirements of the contract, (2), all necessary supporting
documentation for costs are attached to the invoice, (3) all
calculations are correct and there are no obvious errors;
Implementation of procedure for the NQF Contract:
The contracting officer told us that he reviews NQF's invoices with
the assistance of the contract specialist to ensure that HHS pays only
for completed work that had been authorized by the project officer. In
particular, he stated that he reviews all costs reported in the
invoices to ensure they are consistent with the requirements of the
contract, and that he reviews supporting documentation for the costs
provided by NQF. HHS officials told us that because NQF invoices can
range from 100 to 200 pages, the contracting officer or the contract
specialist perform checks on a selection of costs within each invoice
to verify that calculations are correct. In addition, they may rely on
contract audits to determine if costs are proper.[A]
Required procedure for reviewing invoices:
4. Before approving costs associated with subcontractors and
consultants, the contracting officer must confirm that the use of each
subcontractor or consultant was approved[B];
Implementation of procedure for the NQF Contract:
The contracting officer told us that he confirmed that NQF requests to
use each subcontractor or consultant were approved. He stated that he
reviews the NQF requests to approve subcontractors and consultants
prior to reviewing the invoices. He also told us that he has
disapproved costs associated with subcontractors. In our review of NQF
and HHS's contract files, we found documentation of these reviews,
including disapprovals.
Required procedure for reviewing invoices:
5. The contracting officer or the contract specialist is required to
certify whether an invoice is approved for payment by signing it;
Implementation of procedure for the NQF Contract:
Our review of NQF invoices found no evidence of signatures indicating
approval.[C]
Source: GAO analysis of HHS data and interviews with HHS officials.
[A] As of February 2010, CMS officials told us that they had not
determined whether they will conduct an audit of final indirect-cost
rates each year or after the contract is complete in 2013.
Furthermore, a CMS official reported in March 2010 that the agency had
not determined whether CMS or another auditing entity, such as the
Defense Contract Audit Agency, would perform the audit.
[B] According to the NQF contract, NQF must submit requests to use
subcontractors and consultants to the contracting officer. The
contracting officer must review NQF's request for subcontract or
consultant approval and, while taking into consideration the project
officer's recommendation, advise the contractor of the decision to
consent to or dissent from the proposed subcontract or consultant
arrangement in writing.
[C] Reviewing invoices prior to payment is a preventative control that
may result in the identification of unallowable billings, especially
on cost-reimbursement contracts, before the invoices are paid, and a
signature provides evidence of review.
[End of table]
Concluding Observations:
While NQF has begun work in the first year of its contract for the
five duties related to quality measurement established by MIPPA, it is
too early for us to assess whether, or to what extent, NQF will be
successful in carrying out these duties. This report describes NQF's
work for the first of 4 contract years, and HHS has flexibility to
determine on an annual basis the specific work it expects NQF to
perform for each of the MIPPA duties. Therefore, it is not yet known
exactly what work NQF will be expected to complete during the
remainder of the contract period. In addition, other events related to
quality measurement, such as the completion of HHS's national strategy
for quality improvement, are expected to occur before the end of the 4-
year contract period and may have some influence on NQF's specific
work for the five MIPPA duties. Our second report will provide another
opportunity to review NQF's performance and costs.
Agency and Other External Comments:
We provided drafts of this report to HHS and NQF for comment. Both HHS
and NQF provided technical comments, which we incorporated as
appropriate.
We are sending copies of this report to the Secretary of Health and
Human Services and other interested parties. In addition, the report
will be available at no charge on GAO's Web site at [hyperlink,
http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-7114 or at kohnl@gao.gov. Contact points for
our Office of Congressional Relations and Office of Public Affairs can
be found on the last page of this report. Other major contributors to
this report are listed in appendix II.
Signed by:
Linda T. Kohn:
Director, Health Care:
List of Committees:
The Honorable Max Baucus:
Chairman:
The Honorable Charles E. Grassley:
Ranking Member:
Committee on Finance:
United States Senate:
The Honorable Tom Harkin:
Chairman:
The Honorable Michael B. Enzi:
Ranking Member:
Committee on Health, Education, Labor, and Pensions:
United States Senate:
The Honorable Henry A. Waxman:
Chairman:
The Honorable Joe Barton:
Ranking Member:
Committee on Energy and Commerce:
House of Representatives:
The Honorable Sander M. Levin:
Chairman:
The Honorable David Camp:
Ranking Member:
Committee on Ways and Means:
House of Representatives:
[End of section]
Appendix I: National Quality Forum's Endorsement Process and Example
Project:
The National Quality Forum (NQF) established its endorsement process
in 2000. NQF's process includes the nine steps described in table 3
below. The table also provides information on the endorsement process
as applied to a project to endorse a number of measures related to
home health care, such as measures on education provided to patients
and caregivers on medications for care and increases in the number of
pressure ulcers. This project was initiated prior to the NQF contract
with the Department of Health and Human Services (HHS) that was
required by the Medicare Improvements for Patients and Providers Act
of 2008. NQF announced a call for nominations for steering committee
members for this project in August 2008 and the final set of 20
endorsed measures was announced on March 31, 2009.
Table 3: National Quality Forum's (NQF) Endorsement Process and
Example Project:
Steps in NQF endorsement process:
1. Notice of Intent to Call for Measures for Endorsement Consideration;
At the beginning of a project where NQF seeks to endorse measures, NQF
usually issues a public notice of its intent to call for measures for
endorsement consideration. The notice includes a brief background on
the project and a statement on the scope of the project's activities;
Home health measures project dates and details:
NQF did not issue a notice of intent for this project because this
step was added to the process in April 2009, after the project's
completion.
Steps in NQF endorsement process:
2. Call for Nominations for Steering Committee Members; NQF issues a
call for nominations for experts to serve on a steering committee,
which will oversee the endorsement project. Any interested party can
submit nominations for the steering committee during this 30-day
period. NQF selects members of a steering committee based upon their
expertise, their potential contribution to the project, and the need
for input from a particular stakeholder perspective. Generally, a
steering committee is composed of individuals affiliated with NQF
member organizations, unless a necessary stakeholder perspective or
specific expertise is not available among NQF's membership;
Home health measures project dates and details: In August 2008, NQF
issued a call for nominations to serve on the steering committee for
this project. NQF selected 20 individuals representing the following
member organizations: 8 were from provider organizations (5 of which
were home health providers); 2 were from consumer advocacy groups;
5 were health professional organizations; 3 were from quality
measurement, research, and improvement groups; 1 was from a supplier
and industry group; and 1 was from a health plan.
Steps in NQF endorsement process:
3. Call for Measures for Endorsement Consideration; Approximately 14
days after the Intent to Call for Measures notice is issued, NQF
issues a formal call for submission of measures that are candidates
for endorsement. Any organization or agency, such as the Centers for
Medicare & Medicaid Services (CMS), can submit measures for
consideration during this 30-day period;
Home health measures project dates and details: Between September 15
and October 14, 2008, 57 measures were submitted to NQF, all from CMS.
Steps in NQF endorsement process:
4. Steering Committee Review of Measures for Endorsement Consideration;
After the end of the 30-day period for submission of measures, the
steering committee conducts a detailed review of all submitted
measures. The duration of the steering committee's review can vary
depending on the scope of the project, the number of standards under
review, and the relative complexity of the standards. Submitted
measures are evaluated against four criteria, but the measures must
meet the first of these criteria in order to be evaluated against the
remaining criteria. The four criteria are:
Importance to measure and report: extent to which a measure is
important for making significant gains in health care quality and for
improving health outcomes within a high-impact aspect of health care
where there is variation in or overall poor performance;
Scientific acceptability of measure properties: extent to which a
measure produces consistent (reliable) and credible (valid) results
about the quality of care;
Usability: extent to which intended audiences (e.g. consumers,
purchasers, providers, policymakers) can understand the results of the
measure and are likely to find them useful for decision making;
Feasibility: extent to which the required data are readily available,
retrievable without undue burden, and can be implemented for
performance measurement;
Based on its detailed evaluation, a steering committee can recommend
either that (1) a measure continue through the process toward possible
endorsement by NQF, or (2) a measure be returned for further
development and refinement;
Home health measures project dates and details: During October and
November 2008, the steering committee reviewed the 57 measures and
recommended endorsement of 22.
Steps in NQF endorsement process:
5. Member and Public Comment Period; After the steering committee
completes its initial review of the submitted candidate measures, a
draft of the committee's recommendations--or "draft report"--is posted
on the NQF Web site for review and comment by NQF member organizations
and the public. Member organizations have 30 days to comment on all
submitted measures and the public has 21 days to comment.
The comments are compiled by NQF staff and submitted to the steering
committee for consideration. A steering committee may revise its draft
report in direct response to these comments;
Home health measures project dates and details: Between December 16,
2008, and January 14, 2009, the recommended measures were posted for
comment, and 92 comments were submitted by a total of 22 individuals
and organizations. Twenty-four comments were from the public and 68
comments were from NQF member organizations. Based on these comments
and additional information received by the steering committee, the
committee revised its recommendation to only include 20 measures.
Steps in NQF endorsement process:
6. Member Voting; Member organizations have 30 days to vote on the
final version of the steering committee's recommendations for each
measure. Each NQF member organization may cast one vote in favor of or
against a steering committee's recommendations. A member organization
may also abstain from voting on a particular consensus development
project. Only measures that are approved will proceed to the next step
in the process;
Home health measures project dates and details: Between January 28 and
February 26, 2009, 20 measures were posted for voting. Fifty-eight
member organizations voted on each of the 20 measures. These
organizations included consumer advocacy groups; health plans;
health professional organizations; provider organizations;
purchasers; quality measurement, research, and improvement groups;
and supplier/industry groups.
Steps in NQF endorsement process:
7. Review of Measures by Consensus Standards Approval Committee (CSAC);
The CSAC, which is a subcommittee of NQF's Board of Directors, reviews
the measures under consideration for endorsement and voting results
prior to making a recommendation to the NQF Board of Directors about
endorsement of the measure. After detailed review of each measure, the
CSAC determines if consensus has been reached. In this context, NQF
considers consensus to mean that general agreement has been reached
across the various member organizations, such as consumers and health
care professionals and, if there are dissenters, that those opinions
have been taken into consideration during the review process. The CSAC
may seek further input from members if there is a lack of consensus.
The CSAC can recommend full endorsement, time-limited endorsement, or
denial of endorsement for a measure;
Home health measures project dates and details: On March 10, 2009, 20
measures were recommended by the CSAC.
Steps in NQF endorsement process:
8. Board of Directors Decision;
CSAC recommendations regarding endorsement are submitted to the Board
of Directors. The board can affirm or deny a CSAC decision;
Home health measures project dates and details: On March 31, 2009, 20
measures were endorsed by the Board of Directors.
Steps in NQF endorsement process:
9. Appeals; Any interested party may file an appeal with the NQF Board
of Directors of the decision to endorse a measure. An interested party
may not file an appeal regarding the decision to deny endorsement for
a measure; An interested party may file a concern about any measure
(whether endorsed or not endorsed) in the NQF endorsement process and
this concern will be reviewed by the CSAC;
Home health measures project dates and details: Between April 1 and
April 30, 2009, no appeals were filed.
Source: GAO analysis of NQF data.
Note: Data are from documents, the Web site, and information provided
during interviews.
[End of table]
[End of section]
Appendix II: GAO Contact and Staff Acknowledgments:
GAO Contact:
Linda T. Kohn, (202) 512-7114 or kohnl@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Will Simerl, Assistant
Director; La Sherri Bush; Helen Desaulniers; Krister Friday; Natalie
Herzog; Carla Lewis; Lisa Motley; Ruth S. Walk; Rasanjali Wickrema;
and William T. Woods made key contributions to this report.
[End of section]
Footnotes:
[1] These programs include the Medicare Physician Quality Reporting
Initiative for physicians and the Reporting Hospital Quality Data for
Annual Payment Update Program (APU program) for hospitals. For more
information on the APU program, see GAO, Hospital Quality Data: Issues
and Challenges Related to How Hospitals Submit Data and How CMS
Ensures Data Reliability, [hyperlink,
http://www.gao.gov/products/GAO-08-555T] (Washington, D.C.: Mar. 6,
2008).
[2] This program, known as a value-based purchasing program, is
required by the Patient Protection and Affordable Care Act (PPACA).
Pub. L. No. 111-148, 124 Stat. 119 (2010).
[3] Pub. L. No. 110-275, § 183, 122 Stat. 2494, 2583-86. The contract
may be renewed at the end of the 4-year period after a subsequent
bidding process.
[4] MIPPA required HHS to use full and open competition to enter into
the contract. HHS received only one proposal for the contract.
[5] See Pub. L. No. 104-113, 110 Stat. 775 (1996). The act directs
federal agencies and departments to use standards that are developed
or adopted by voluntary consensus standards bodies, such as NQF,
whenever possible.
[6] PPACA requires that HHS choose endorsed measures for certain
quality measurement programs if it is feasible and practical to do so.
[7] HHS obligated $10 million for the contract on the date of the
award and plans to increase the obligated amount each year of the
contract. Funds obligated but not actually paid for NQF's costs and
fees in a contract year remain available in subsequent contract years.
[8] 48 C.F.R. ch. 1. The FAR establishes uniform policies for
acquisition of supplies and services by executive agencies. Agency
acquisition regulations may implement or supplement the FAR.
[9] Cost principles applicable to contracts with nonprofit
organizations are set forth in the Office of Management and Budget
Circular A-122, the text of which is located at 2 C.F.R. pt. 230. See
48 C.F.R. § 31.702 (2009). Under these provisions, costs are allowable
if they are reasonable and allocable, consistent with any limitations
and applicable policies, accorded consistent treatment, determined in
accordance with generally accepted accounting principles, not counted
elsewhere, and adequately documented.
[10] MIPPA states that GAO's reports shall be submitted by 18 months
and 36 months, respectively, after the effective date of the contract.
[11] NQF classifies its membership as being composed of the following
groups: provider organizations, which include hospitals, pharmacies,
and other organizations (33 percent); health professional
organizations, such as those representing doctors, nurses, and
clinicians (20 percent); organizations that conduct research,
education, or initiatives to improve health care quality, measurement,
and reporting (16 percent); supplier/industry groups that provide
devices, medications, and other products (8 percent); public/community
health agencies (7 percent); consumer advocacy groups (7 percent);
purchasers, such as private organizations and government agencies (6
percent); and health plans and organizations involved in
administration of health insurance programs (4 percent). These
percentages add up to over 100 percent due to rounding. A list of
NQF's member organizations is available at its Web site, [hyperlink,
http://www.qualityforum.org/Membership/Membership_in_NQF.aspx].
[12] For more information on cost-plus-fixed-fee contracts, see GAO,
Contract Management: Extent of Federal Spending under Cost-
Reimbursement Contracts Unclear and Key Controls Not Always Used,
[hyperlink, http://www.gao.gov/products/GAO-09-921] (Washington, D.C.:
Sept. 30, 2009).
[13] An agency's examination is to determine whether an accounting
system is adequate. These reviews include a determination of whether
the accounting system can meet generally accepted accounting
principles and whether it provides for, among many things, proper
segregation of direct costs from indirect costs. See 48 C.F.R. §
53.301-1408, FAR Form 1408; 48 C.F.R. § 9.106-4. The scope of HHS's
review of NQF's accounting system was limited to determining whether
the design of the system was acceptable for accumulating costs under a
government contract.
[14] 48 C.F.R. §§ 44.204(a)(1), 52.244-2 (2009).
[15] Within CMS, the Office of Acquisition and Grants Management is
responsible for administering the NQF contract.
[16] The project officer serves as the technical representative of the
contracting officer, and provides technical direction to NQF for all
tasks described in the NQF contract. In addition, the project officer
monitors NQF's performance and reviews invoices for payment.
[17] The contracting officer enters into, administers, and terminates
government contracts. The contracting officer negotiates and prepares
contract documents, modifies terms or conditions of the contract, and
approves payment of invoices, among other tasks.
[18] The 20 conditions are acute myocardial infarction, Alzheimer's
disease and related disorders, atrial fibrillation, breast cancer,
cataract, chronic kidney disease, chronic obstructive pulmonary
disorder, colorectal cancer, congestive heart failure, diabetes,
endometrial cancer, glaucoma, hip/pelvic fracture, ischemic heart
disease, lung cancer, major depression, osteoporosis, prostate cancer,
rheumatoid arthritis and osteoarthritis, and stroke/transient ischemic
attack.
[19] According to NQF officials, these recommendations to HHS will be
based both on NQF's work under the contract and on other NQF
initiatives.
[20] Pub. L. No. 111-148, § 3011, 124 Stat. 119, 378-80 (2010).
[21] Examples of organizations that have developed measures and
submitted them to NQF include the Joint Commission, the National
Committee for Quality Assurance, and the American Medical Association.
[22] According to NQF, its efforts to promote the development and use
of electronic health records for use in quality measurement support
"meaningful use" of electronic health records under the American
Recovery and Reinvestment Act of 2009. The American Recovery and
Reinvestment Act authorizes CMS to provide reimbursement incentives
for eligible professionals and hospitals who are successful in
becoming "meaningful users" of electronic health records. The act
states that one factor in determining if a provider is a "meaningful
user" of electronic health records is whether it submits information
on quality measures selected by HHS. The act also states that, in
selecting these measures, HHS should give preference to measures
endorsed by NQF. Pub. L. No. 111-5, §§ 4101-4102, 123 Stat. 115, 469-
70, 479-80 (2009) (codified at 42 U.S.C. §§ 1395w-4(o), 1395ww(n)).
[23] NQF's first annual report to Congress and the Secretary of HHS
covers a 6-week period, January 14, 2009, to February 28, 2009.
[24] NQF's second annual report to Congress and the Secretary of HHS
covers the period March 1, 2009, through February 28, 2010, which
includes a portion of the first contract year. NQF's annual reports
can be found at [hyperlink,
http://www.qualityforum.org/projects/ongoing/hhs/].
[25] These are the costs and fixed fees that NQF reported for the
first contract year as of May 31, 2010.
[26] Employee benefits, known as fringe benefit costs, include items
such as annual leave and holiday pay. Overhead includes items such as
equipment rental and office supplies. Administrative costs, known as
general and administrative costs, include bank fees, dues and
subscriptions, and taxes.
[27] HHS approved provisional indirect rates for NQF to use during the
first contract year. These rates are intended to help ensure that
indirect costs are reasonable for the services provided and within
limits specified in the contract. The rates are provisional, which
means that they are used until final indirect-costs rates can be
established, generally at the end of the contractor's fiscal year. For
more information on provisional and final indirect cost rates, see
GAO, Centers for Medicare and Medicaid Services: Deficiencies in
Contract Management Internal Controls Are Pervasive, [hyperlink,
http://www.gao.gov/products/GAO-10-60] (Washington, D.C.: Oct. 23,
2009).
[28] NQF's contract requires that, in accordance with 48 C.F.R. §
52.216-8, the payment of the fixed fee be paid monthly until fee
payments reach 85 percent of the total amount of the fixed fee
authorized, and after they reach 85 percent HHS may withhold a reserve
up to 15 percent or $100,000, whichever is less. At such time, the
contracting officer may withhold further payment of the fee to protect
the government's interest.
[29] The increase in costs and fixed fees throughout the year is due
solely to increases in costs because NQF reported the same amount of
fixed fees each month.
[30] Labor costs represented NQF's largest category of direct costs
during the first contract year.
[31] Costs associated with subcontractors and consultants accounted
for over one-third of NQF's direct costs during the first contract
year.
[32] In addition to the review of invoices, HHS officials conducted
two preaward surveys of NQF's accounting system in 2007 and 2008 prior
to the start of the contract. As a result of these surveys, HHS found
the design of the contractor's accounting system to be adequate for
determining costs related to the contract. The FAR requires that this
determination be made prior to the start of the contract to help
ensure costs are proper and to reduce improper payments.
[33] Other HHS officials, such as the project manager and contract
specialist, provide invoice review support to the project officer and
contracting officer.
[34] These policies include those specified in the HHS Project Officer
manual and CMS's May 2008 invoice review policy.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
