Global Health
Global Fund to Fight AIDS, TB and Malaria Has Improved Its Documentation of Funding Decisions but Needs Standardized Oversight Expectations and Assessments Gao ID: GAO-07-627 May 7, 2007The Global Fund to Fight AIDS, Tuberculosis and Malaria has approved about $7 billion in grants to developing countries; the U.S. has contributed $1.9 billion. The State Department's Office of the Global AIDS Coordinator (OGAC) coordinates the U.S. government's overseas AIDS programs, with participation from the Department of Health and Human Services (HHS) and the U.S. Agency for International Development (USAID). In 2003, Congress directed GAO to report on the Global Fund every 2 years. This report assesses the Global Fund's (1) documentation of information used to support performance-based funding decisions, (2) progress in implementing a risk assessment model and early warning system, and (3) oversight of the performance of "local fund agents" (LFAs), which monitor grant progress in recipient countries. GAO reviewed the documentation for funding decisions and interviewed key officials.
Since our 2005 review, the Global Fund has improved its documentation for decisions to disburse funds and renew grants. The Global Fund now requires that fund portfolio managers more consistently document factors, such as grant ratings and contextual information that support disbursement and grant renewal decisions. Our current review of 80 grant disbursements and 45 grant renewal decisions confirmed that Global Fund grant files consistently contained explanations of the information used in its funding decisions. For example, all grant disbursement files in our sample contained a written narrative explaining the ratings that portfolio managers gave the grants, based in part on reports completed by grant recipients. Although we noted that many of the grant recipients' reports lacked some information needed for disbursement and renewal decisions, Global Fund officials said that portfolio managers obtain this information informally from the recipients or other stakeholders. The Global Fund did not implement the risk assessment model that it was developing at the time of our 2005 report, because it determined that the model did not accurately identify grant risk. To identify risks that may affect grant implementation, the organization currently relies on elements of its structures and processes, including its initial technical review, disbursement decision-making form, periodic grant ratings, oversight by country stakeholders, information from technical partners, and LFA oversight. Recognizing the need for a more comprehensive approach to risk management, the Global Fund has begun developing a risk assessment framework for the organization that includes an early alert and response system to address poorly performing grants. The Global Fund has limited access to the information it needs to manage and oversee LFAs because it does not require systematic assessments of LFAs' performance. As a result, the Global Fund has limited ability to determine the quality of LFAs' monitoring and reporting and to identify situations in which more oversight of LFAs' performance may be required. Previously, the Global Fund introduced a tool to assess LFA performance more systematically; however, this effort was unsuccessful, because use of the tool was not mandatory. Numerous sources raise concerns about the quality of grant monitoring and reporting provided by LFAs, particularly their ability to assess and verify recipients' procurement capacity and program implementation.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-07-627, Global Health: Global Fund to Fight AIDS, TB and Malaria Has Improved Its Documentation of Funding Decisions but Needs Standardized Oversight Expectations and Assessments
This is the accessible text file for GAO report number GAO-07-627
entitled 'Global Health: Global Fund to Fight AIDS, TB and Malaria Has
Improved Its Documentation of Funding Decisions but Needs Standardized
Oversight Expectations and Assessments' which was released on May 7,
2007.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
May 2007:
Global Health:
Global Fund to Fight AIDS, TB and Malaria Has Improved Its
Documentation of Funding Decisions but Needs Standardized Oversight
Expectations and Assessments:
GAO-07-627:
GAO Highlights:
Highlights of GAO-07-627, a report to congressional committees
Why GAO Did This Study:
The Global Fund to Fight AIDS, Tuberculosis and Malaria has approved
about $7 billion in grants to developing countries; the U.S. has
contributed $1.9 billion. The State Department‘s Office of the Global
AIDS Coordinator (OGAC) coordinates the U.S. government‘s overseas AIDS
programs, with participation from the Department of Health and Human
Services (HHS) and the U.S. Agency for International Development
(USAID). In 2003, Congress directed GAO to report on the Global Fund
every 2 years. This report assesses the Global Fund‘s (1) documentation
of information used to support performance-based funding decisions, (2)
progress in implementing a risk assessment model and early warning
system, and (3) oversight of the performance of ’local fund agents“
(LFAs), which monitor grant progress in recipient countries. GAO
reviewed the documentation for funding decisions and interviewed key
officials.
What GAO Found:
Since our 2005 review, the Global Fund has improved its documentation
for decisions to disburse funds and renew grants. The Global Fund now
requires that fund portfolio managers more consistently document
factors, such as grant ratings and contextual information that support
disbursement and grant renewal decisions. Our current review of 80
grant disbursements and 45 grant renewal decisions confirmed that
Global Fund grant files consistently contained explanations of the
information used in its funding decisions. For example, all grant
disbursement files in our sample contained a written narrative
explaining the ratings that portfolio managers gave the grants, based
in part on reports completed by grant recipients. Although we noted
that many of the grant recipients‘ reports lacked some information
needed for disbursement and renewal decisions, Global Fund officials
said that portfolio managers obtain this information informally from
the recipients or other stakeholders.
The Global Fund did not implement the risk assessment model that it was
developing at the time of our 2005 report, because it determined that
the model did not accurately identify grant risk. To identify risks
that may affect grant implementation, the organization currently relies
on elements of its structures and processes, including its initial
technical review, disbursement decision-making form, periodic grant
ratings, oversight by country stakeholders, information from technical
partners, and LFA oversight. Recognizing the need for a more
comprehensive approach to risk management, the Global Fund has begun
developing a risk assessment framework for the organization that
includes an early alert and response system to address poorly
performing grants.
The Global Fund has limited access to the information it needs to
manage and oversee LFAs because it does not require systematic
assessments of LFAs‘ performance. As a result, the Global Fund has
limited ability to determine the quality of LFAs‘ monitoring and
reporting and to identify situations in which more oversight of LFAs‘
performance may be required. Previously, the Global Fund introduced a
tool to assess LFA performance more systematically; however, this
effort was unsuccessful, because use of the tool was not mandatory.
Numerous sources raise concerns about the quality of grant monitoring
and reporting provided by LFAs, particularly their ability to assess
and verify recipients‘ procurement capacity and program implementation.
What GAO Recommends:
We recommend that the Secretaries of State and Health and Human
Services work with the Global Fund‘s Board Chair and Executive Director
to (1) establish standardized expectations for LFA performance and (2)
require systematic assessments of LFA performance and the collection
and analysis of performance data. The Departments of State and HHS,
USAID, and the Global Fund agreed with our findings and
recommendations.
[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-627].
To view the full product, including the scope and methodology, click on
the link above. For more information, contact David Gootnick at (202)
512-3149 (gootnickd@gao.gov).
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Global Fund Has Improved Documentation of Its Basis for Funding
Decisions:
Global Fund Did Not Implement Earlier Risk Model but Is Developing
Comprehensive Risk Framework That Includes Early Alert and Response
System:
Global Fund Has Limited Ability to Manage and Oversee Local Fund
Agents' Performance:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Other Sources Cited:
Appendix III: Comments from the Global Fund:
Appendix IV: Comments from Departments of State and Health and Human
Services and U.S. Agency for International Development:
Appendix V: GAO Contact and Staff Acknowledgments:
Table:
Table 1: Key Source Documents for Disbursement and Grant Renewal
Decisions:
Figures:
Figure 1: Global Fund Allocations between 2002 and 2006 by Geographical
Region and Disease:
Figure 2: Global Fund Allocations by Disease and Intervention Category:
Figure 3: Performance-based Funding Oversight by Global Fund
Secretariat, Grant Recipients, and LFAs:
Figure 4: Global Fund Disbursement Decision Process:
Figure 5: Key Elements of the Global Fund's Phase 2 Grant Renewal
Process:
Abbreviations:
CCM: country coordinating mechanism:
HHS: Department of Health and Human Services:
HIV/AIDS: human immunodeficiency virus/acquired immune deficiency
syndrome:
LFA: local fund agent:
OGAC: Office of the U.S. Global AIDS Coordinator:
TB: tuberculosis:
UNAIDS: Joint United Nations Programme on HIV/AIDS:
USAID: United States Agency for International Development:
WHO: World Health Organization:
United States Government Accountability Office:
Washington, DC 20548:
May 7, 2007:
The Honorable Joseph R. Biden, Jr.
Chairman:
The Honorable Richard G. Lugar:
Ranking Minority Member:
Committee on Foreign Relations:
United States Senate:
The Honorable Tom Lantos:
Chairman:
The Honorable Ileana Ros-Lehtinen:
Ranking Minority Member:
Committee on Foreign Affairs:
House of Representatives:
The Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global
Fund) was created to help finance the fight against three infectious
diseases that kill more than 6 million people each year. Established in
2002, the Global Fund is a private foundation intended as a partnership
among governments, civil society, the private sector, and affected
communities. From 2002 through February 2007, the Global Fund approved
grant proposals totaling about $7 billion to governmental and
nongovernmental entities to carry out disease intervention activities
in more than 100 countries. During this period, the U.S. government
contributed $1.9 billion--the largest amount provided to the Global
Fund by a single donor. The State Department's Office of the Global
AIDS Coordinator (OGAC) coordinates the U.S. government's overseas AIDS
programs, with participation from the Department of Health and Human
Services (HHS) and the U.S. Agency for International Development
(USAID).
The Global Fund uses a performance-based funding system designed to
encourage financial accountability and program progress and to link its
continued grant disbursements to achievement of these results. The
Global Fund contracts with "local fund agents" (LFA) to assess
recipients' capacity for grant implementation and verify recipients'
financial and program performance data. In June 2007, it plans to issue
applications for new contracts for LFA oversight services.
In 2003, Congress directed the Comptroller General to monitor and
evaluate projects supported by the Global Fund and to submit a report
every 2 years.[Footnote 1] In June 2005, we reported that the Global
Fund's grant documentation did not always show clear grounds for
performance-based funding decisions. [Footnote 2] These decisions can
include periodic disbursements as well as decisions to renew, suspend,
or terminate grants. We further noted that some of the grant
recipients' self-reported performance data were of poor quality or
incomplete. We also reported that the Global Fund was developing a risk
assessment model and early warning system to alert its fund portfolio
managers to the need to respond to poorly performing grants.[Footnote
3] In addition, we noted that LFAs' assessments of recipient
performance contained inconsistencies and that LFAs often lacked the
knowledge and experience needed for these assessments. In its response
to our 2005 report, the Global Fund generally concurred with our
findings and indicated that it was taking steps to improve its
structures and systems.
Following up on our 2005 review, this report assesses the Global Fund's
(1) documentation of the information used to support performance-based
funding decisions, (2) progress in implementing a risk assessment model
and early warning system, and (3) oversight of local fund agents'
performance.
To address these objectives, we met with key officials from the Global
Fund and reviewed Global Fund documents, including key source documents
from grant files. To assess the documentation of the information used
to support performance-based funding decisions, we analyzed two
separate random samples of grant decisions from the Global Fund. The
first sample consisted of 80 grants that were assessed for performance-
based funding disbursements between January 1 and October 1, 2006. The
second sample consisted of 45 grants that were assessed for phase 2
grant renewal.[Footnote 4] We projected our findings to the files of
all grants that received a disbursement or were assessed for grant
renewal during that period. To assess Global Fund progress in
implementing a risk assessment model and early warning system, we
reviewed international standards on risk management,[Footnote 5] met
with Global Fund secretariat officials, and reviewed Global Fund
documents. We also held discussions with the Global Fund's Inspector
General and three of its key technical partners.[Footnote 6] To assess
the Global Fund's oversight and measurement of LFAs' capacity and
performance, we met with each of the six LFA headquarters offices and
relevant secretariat personnel. In addition, we held structured
interviews with LFA teams in 12 countries via telephone. We also
obtained perspectives on the progress of the Global Fund from its
partners and outside experts. We conducted our work between May 2006
and March 2007 in accordance with generally accepted government
auditing standards. (See app. I for a detailed description of our scope
and methodology.)
Results in Brief:
The Global Fund has improved its documentation of information that
supports its performance-based funding decisions for disbursements and
grant renewals. Since our 2005 review, the Global Fund has updated its
documentation system to require that portfolio managers more
consistently document factors, such as grant ratings and contextual
information, on which they base their decision to continue funding a
grant.[Footnote 7] Our review of 80 grant disbursements and 45 phase 2
grant renewal decisions confirmed that Global Fund grant files
consistently contained explanations of the information used to make its
funding decisions.[Footnote 8] For example, 100 percent[Footnote 9] of
the disbursement and phase 2 files in our samples contained narratives
explaining the manager's rating of the grant's performance. We noted
that many reports completed by grant recipients lacked explanations
either for not meeting or for exceeding the grant's targets. However,
Global Fund officials said that fund portfolio managers obtain the
needed information informally from the recipients or other
stakeholders.
The Global Fund did not implement the risk assessment model that it was
developing at the time of our 2005 report, because it determined that
the model did not accurately identify grant risk. To identify risks
that may affect grant implementation, the organization currently relies
on elements of its structures and processes, including its initial
technical review, disbursement decision-making form, periodic grant
ratings, oversight by country stakeholders, information from technical
partners, and LFA oversight. Recognizing the need for a more
comprehensive approach to risk management the Global Fund has begun
developing a risk assessment framework for the organization that
includes an early alert and response system to address poorly
performing grants.
Because it does not require systematic assessments of LFAs'
performance, the Global Fund has limited access to the information it
needs to manage and oversee LFAs.[Footnote 10] Global Fund portfolio
managers set expectations for LFA performance informally and
individually, often resulting in, for example, variances in scopes of
work and definitions of satisfactory performance. Moreover, the Global
Fund does not require systematic assessment or reporting of LFAs'
performance; instead, managers provide feedback to LFAs informally.
Because of the lack of systematic assessment, the Global Fund has
limited ability to determine the quality of grant services provided by
LFAs and to identify situations in which more oversight of LFAs by the
Global Fund may be required. The Global Fund previously introduced a
tool to assess LFA performance more systematically. However, this
effort was unsuccessful because use of the tool was not mandatory.
Numerous sources raise concerns about the quality of grant oversight
provided by LFAs, particularly their ability to assess and verify
recipients' procurement capacity and program implementation.
To improve the Global Fund's ability to oversee the performance of its
local fund agents, we recommend that the Secretaries of State and HHS
work with the Global Fund's Board Chair and Executive Director to (1)
establish standardized expectations for LFA performance; and (2)
require systematic assessments of LFA performance and the collection
and analysis of performance data.
In responding to our draft report, the Departments of State and Health
and Human Services, the U.S. Agency for International Development, and
the Global Fund agreed with the report's findings, conclusions, and
recommendations. (See apps. III and IV for their comments.)
Background:
The Global Fund, a private foundation in Switzerland, was established
in 2002 as a funding mechanism to collect public and private
contributions to finance grants for the prevention and treatment of
HIV/AIDS, tuberculosis (TB), and malaria.[Footnote 11] About 96 percent
of the contributions to the Global Fund since its inception have come
from governments. Private sector support increased in 2006, notably
owing to a pledge of $500 million by the Bill and Melinda Gates
Foundation and receipt of almost $19 million through the (RED)™
initiative.[Footnote 12]
By December 2006, the Global Fund had completed six proposal rounds,
approved about $7 billion in grants, signed grant agreements worth over
$5 billion and disbursed more than $3 billion to grant recipients in
132 countries. The Global Fund also reports that by December 2006, its
grants enabled recipients to supply antiretroviral drugs to 770,000
people infected with HIV/AIDS, treat 2 million people infected with
tuberculosis, and distribute 18 million insecticide-treated nets to
combat malaria.
Global Fund Portfolio Distribution:
The Global Fund's underlying principles require, among other things,
that it maintain a balanced grant portfolio in terms of region,
disease, and type of intervention; operate transparently and
accountably; and use a simple, rapid, and innovative grant-making
process.[Footnote 13] As figure 1 shows, between 2002 and 2006, the
Global Fund distributed 55 percent of grant funds to sub-Saharan
Africa, with the remainder of the funding divided among other regions
of the world. Also during this time period, the Global Fund distributed
57 percent, 15 percent, and 27 percent of grant funds, respectively,
for HIV/AIDS, tuberculosis, and malaria programs. The remaining 1
percent funded projects to strengthen health systems.
Figure 1: Global Fund Allocations between 2002 and 2006 by Geographical
Region and Disease:
[See PDF for image]
Source: GAO analysis of Global Fund data.
[End of figure]
In 2005, the Global Fund began tracking the allocation of grant funds
in grant proposals according to the disease intervention categories of
prevention, care, and treatment. Figure 2 shows the Global Fund's
estimated distribution by disease and intervention category.
Figure 2: Global Fund Allocations by Disease and Intervention Category:
[See PDF for image]
Source: GAO based on Global Fund Round 5 data from 2006.
[A] For HIV/AIDS, "Other" includes grants for health system
strengthening, monitoring and evaluation, supportive environment,
HIV/TB collaborative activities, and other activities.
[B] For tuberculosis, "Other" includes grants for health system
strengthening, monitoring and evaluation, supportive environment, and
other activities.
[C] For malaria, "Other" includes grants for health system
strengthening, supportive environment, and other activities.
[End of figure]
Key Elements of the Global Fund's Structure:
The Global Fund's principles also require it to focus on performance by
linking its resources to the achievement of clear, measurable, and
sustainable results. To achieve this focus on performance-based
funding, the Global Fund relies primarily on oversight by, and
interaction among, Global Fund secretariat personnel,[Footnote 14]
grant recipients, and LFAs. (See fig. 3.)
Figure 3: Performance-based Funding Oversight by Global Fund
Secretariat, Grant Recipients, and LFAs:
[See PDF for image]
Sources: GAO(data); Map Resources (clip art).
[A] Fund portfolio managers also have formal relationships with grant
recipients and they interact with all stakeholders including LFA
central teams:
[End of figure]
* Global Fund secretariat personnel. Within the secretariat, the fund
portfolio managers are primarily responsible for reviewing grant
progress and making performance-based funding recommendations. Other
secretariat personnel are consulted during key stages in the grant life
cycle. Fund portfolio managers interact with the LFA country teams
providing oversight services for their grant, as well as with principal
recipients, subrecipients, technical partners, and other stakeholders
in the recipient countries. In addition to the country teams, staff
from the contracts department and the Global Fund's LFA manager
interact with LFA headquarters teams, known as the central teams,
regarding contracting and administrative matters and any problems
related to LFA country teams.
* Grant recipients. At the country level, the principal grant recipient
is responsible for receiving and implementing the grant.[Footnote 15]
During implementation, the principal recipient is responsible for
monitoring and evaluating the grant's effectiveness in accordance with
grant objectives and making sure that funds are properly accounted for.
The principal recipient is also responsible for overseeing the
activities of any subrecipients implementing grant activities.
* Local fund agents. The Global Fund currently contracts with LFA
headquarters central teams at six organizations[Footnote 16] to oversee
grants in 120 countries and multicountry programs.[Footnote 17] In the
majority of cases, the central team establishes interfirm agreements
with its affiliated offices within each country to perform LFA
services.[Footnote 18] The LFA assesses recipient capacity prior to
grant signing and performs verification and oversight duties throughout
the life of the grant. The LFA central teams typically contract with
external specialists for work outside the expertise of their
organization, especially during the initial assessments of recipient
capacity. For example, during the initial assessments, some
organizations contract with a procurement expert to review recipients'
capacity for procurement and supply chain management. The Global Fund
budgeted approximately $22 million--26 percent of the secretariat's
operating budget--to support the LFA structure for 2006 and spent about
an additional $320,000 on LFA-related functions, including personnel
costs related to LFA management, studies of the LFA model, and travel
to meet with LFAs.
Performance-Based Funding Process:
The Global Fund's performance-based funding process involves an initial
assessment of recipients' capacity to manage and implement grants,
periodic disbursement decisions based largely on reports by the grant
recipient and the LFA, and a phase 2 grant renewal review after about 2
years.
* Assessment of grant recipients' capacity. Prior to funding a grant,
the Global Fund considers, among other things, a grant recipient's
ability to (1) manage, evaluate, and report on grant-funded activities;
(2) manage and account for funds; and (3) procure goods and services.
LFAs assess prospective recipients' capacity to perform these
activities. Funding is conditioned on recipients' meeting performance
targets mutually agreed with Global Fund technical experts.
* Periodic disbursements. After the initial screening and approval
process, the grant recipient must produce quarterly or semi-annual
reports describing financial and program progress as well as a funding
request for the next disbursement period. The grant recipient sends
these reports to the LFA, which prepares a report verifying the
accuracy of the information, identifying potential grant problems,
rating grant performance, and recommending a disbursement amount. The
LFA then forwards the grant recipient's report and its own report to
the designated Global Fund portfolio manager. The fund portfolio
manager compares the financial information and data on program progress
with the grant's performance targets, taking into account information
such as the status of procurements and political or economic
constraints. The fund portfolio manager's team leader makes a funding
decision if the disbursement is less than $2 million.[Footnote 19]
Figure 4 illustrates the Global Fund's periodic disbursement decision
process.
Figure 4: Global Fund Disbursement Decision Process:
[See PDF for image]
Source: GAO (data); Map Resources (clip art).
[End of figure]
* Grant renewal process. A key milestone in the performance-based
funding approach is the phase 2 grant renewal review--a detailed
analysis of grant progress at about 2 years, required to continue
funding for up to the next 3 years. Similar to its process for
disbursement decisions, the Global Fund's grant renewal process
requires grant recipients to send documentation of financial and
program progress to the LFA, which verifies the information and
analyzes the phase 2 proposal. In a report to the Global Fund, the LFA
identifies potential grant problems, rates the performance of the
grant, and recommends the amount needed to fund up to the next 3 years
of the grant. Because phase 2 is the decision point for continuing the
grant to its 5-year completion, the process also includes an extensive
and detailed review of cumulative information by fund portfolio
managers, their supervisors, and evaluation and finance staff. The
phase 2 renewal process culminates in a panel discussion and final
recommendation to the board. Figure 5 illustrates key elements of the
Global Fund's phase 2 grant renewal process.
Figure 5: Key Elements of the Global Fund's Phase 2 Grant Renewal
Process:
[See PDF for image]
Sources: GAO (data); Map Resources (clip art).
[End of figure]
Table 1 outlines the key source documents related to disbursement
decisions and phase 2 grant renewals from electronic "grant files" that
we reviewed in conducting this study.
Table 1: Key Source Documents for Disbursement and Grant Renewal
Decisions:
Disbursement.
Reporting form: Progress Update and Disbursement Request;
Completed by: Grant recipient.
Reporting form: LFA Progress Review and Recommendation;
Completed by: LFA.
Reporting form: Disbursement Decision- Making Form;
Completed by: Global Fund secretariat.
Phase 2 renewal.
Reporting form: Request for Continued Funding;
Completed by: Country coordinating mechanism (CCM).
Reporting form: Grant Performance Report;
Completed by: Global Fund secretariat.
Reporting form: Phase 2 Request Assessment Report;
Completed by: LFA.
Reporting form: Grant Scorecard;
Completed by: Global Fund secretariat.
Source: GAO analysis based on Global Fund documentation.
[End of table]
In preparation for its retender of LFA contracts worldwide in mid to
late 2007, the secretariat will continue discussions on the key
principles that will potentially shape the design of the next LFA
model. According to a Global Fund official, they will take into
consideration findings from this review and two internal Global Fund
studies. They expect to issue the request for applications in late
June.
Global Fund Has Improved Documentation of Its Basis for Funding
Decisions:
The Global Fund has improved its documentation of information
supporting its performance-based funding decisions. For example, the
Global Fund's system now requires that portfolio managers consistently
document factors that may affect the decision to continue funding a
grant, especially when the grant falls short of its performance
targets. In addition, the Global Fund grant files that we reviewed
consistently contained explanations of funding decisions and contextual
data informing the decisions. Although we found many reports submitted
by grant recipients to the Global Fund do not provide explanations for
all unmet grant performance targets, according to Global Fund
officials, fund portfolio managers are able to obtain the needed
information directly from the grant recipients or other stakeholders.
Global Fund Has Improved Its Processes for Documenting the Basis for
Funding Decisions:
The Global Fund has developed a more systematic process for documenting
the basis for its funding decisions. In June 2005, we noted that the
Global Fund did not consistently document the reasons for its
determinations that recipients' performance warranted additional
funding for periodic disbursements or its recommendations regarding
grant renewals. We recommended that the Global Fund continue efforts to
clearly document the Global Fund's reasons for periodically disbursing
funds and renewing grant agreements. Subsequently, to increase the
transparency of its performance-based funding system, the organization
made several changes to its documentation process.
In late 2005, the Global Fund developed a disbursement decision-making
form to ensure that fund portfolio managers consistently show the basis
for each disbursement decision. The form requires, among other things,
* an explanation of the performance rating assigned to the grant,
addressing any discrepancy between ratings by the fund portfolio
manager and the LFA and verifying that the manager has considered any
problems raised by the LFA regarding data quality and reporting,
program progress, and expenditures;[Footnote 20] and:
* contextual information critical to the disbursement decision, such as
information demonstrating governance problems, grant management
weaknesses, currency fluctuations, or natural disasters in the country
or region where a funded project is being implemented.
The Global Fund also revised other forms to provide more systematic
reporting of grant program progress. These revisions included
improvements to the grant progress update and disbursement request, the
LFA progress review and recommendation report, and the phase 2 grant
scorecard.[Footnote 21] For example, the updated progress update form
contains a new requirement that LFAs comment on the reliability of
grant recipient data on target achievement. The updated grant scorecard
includes requests for data showing the timeliness and effectiveness of
grant recipients' use of the funds during phase 1 of grant
implementation. It also includes a new section where the secretariat
can record plans to materially change the grant in phase 2--for
example, adding or dropping goals or objectives because of lessons
learned during phase 1.[Footnote 22]
According to Global Fund officials, the changes to their system of
recording information about grant performance and problems have made
the decision process more systematic and transparent. For example,
Global Fund officials, including some fund portfolio managers, said the
new disbursement decision-making form provides managers a place to
systematically explain their ratings. Officials from OGAC and the World
Bank with whom we met said that the Global Fund's more consistent
documentation of the basis of its decisions has made the decisions
easier to understand.
Grant Files Demonstrate Improved Documentation Supporting Funding
Decisions:
Our review of random samples of disbursement and phase 2 grant files
indicated that most files systematically explain a number of key
factors used to support disbursement and grant renewal
decisions.[Footnote 23] In our 2005 report, we observed that the Global
Fund's grant files provided little explanation of disbursement
decisions, including decisions to disburse funds to recipients who
reported not meeting their targets. In addition, our 2005 report noted
that, according to Global Fund officials, many grant renewal decisions
were not supported by the information needed to assess whether funding
decisions were consistently based on grant performance.
In our current review, we found that Global Fund grant files
consistently contained explanations of ratings of grant performance
that were important to funding decisions, even when key stakeholders'
ratings and recommendations did not concur. Based on our samples, 100
percent[Footnote 24] of both disbursement and phase 2 files contained
narratives explaining the manager's rating of the grant's performance.
Most grant files also contained explanations for lack of concurrence in
ratings or recommendations by the various officials charged with
assessing grant performance.[Footnote 25] For example:
* In all 8 cases in which fund portfolio managers' phase 2 grant
ratings differed from those of the evaluation staff, the scorecards in
the grant files contained an explanation for the lack of
concurrence.[Footnote 26]
* In all 5 cases in which fund portfolio managers' phase 2 grant
renewal recommendations differed from those of the LFAs', the
scorecards in the grant files contained an explanation for the lack of
concurrence.
* In cases in which fund portfolio managers' assessments of grant
performance led them to disagree with the disbursement amount
recommended by the LFA or the amount requested by the grant recipient,
31 of 32 of those grant files included a narrative explaining the
discrepancy.[Footnote 27]
According to Global Fund officials, such explanations provide valuable
information that helps decision-makers better understand and consider
grant performance.
The Global Fund files that we reviewed also consistently explained
contextual information and other issues that may affect the grant
recipient's ability to meet agreed-upon targets. We found that an
estimated 96 percent of disbursement files in which portfolio managers
had specified contextual issues that negatively affected grant
performance included narratives explaining each issue. According to
Global Fund officials, such information is crucial to grant
disbursement and renewal decisions, especially if recipients have not
met many of the targets set at the grant's initiation. In addition, in
reviewing 43 disbursement files for grants whose recipients had not yet
met remedial requirements imposed by the Global Fund as conditions for
grant renewal, we found that 42 (98 percent) contained an explanation
of recipients' progress toward meeting the requirements.
Global Fund Informally Gathers Missing Data on Grant Recipients'
Performance:
Although recipients' formal documentation of grant performance remains
incomplete, Global Fund officials informally gather the missing
information that they consider necessary for making and documenting the
basis for funding decisions. In June 2005, we reported that recipient
progress reports varied in quality and that the limited monitoring and
evaluation capabilities of many recipients raised questions about the
accuracy of their reporting.[Footnote 28] Our current analysis found
that an estimated 39 percent of phase 2 grant files contained evidence,
identified by fund portfolio managers, of weaknesses in the reliability
of the grant recipient's data. In contrast to the consistent
documentation we found on forms completed by Global Fund staff, many
periodic performance reports completed by grant recipients lacked
explanations either for not meeting or for exceeding the grant's
targets. In those cases, according to LFAs, fund portfolio managers,
and other Global Fund officials, fund portfolio managers relied on
regular phone and e-mail contact with grant recipients or other
stakeholders to obtain the missing information needed for performance-
based funding decisions. Global Fund officials said that information
crucial to funding decisions is documented on the disbursement decision-
making form or phase 2 grant scorecard, even if it was not recorded by
grant recipients or LFAs in their formally required reports.
Global Fund portfolio managers said that recipients' explanations for
not meeting targets may influence the portfolio manager's
recommendation to withhold funds or, alternately, to continue to fund
the grant despite performance problems. According to Global Fund
officials and public health experts, improving grant recipients'
monitoring and evaluation capacity, as well as their ability to report
accurately and completely, will require long-term investment on the
part of the Global Fund and its partners.
Global Fund Did Not Implement Earlier Risk Model but Is Developing
Comprehensive Risk Framework That Includes Early Alert and Response
System:
The Global Fund did not implement the risk assessment model that it was
developing at the time of our 2005 report, because it determined that
the model did not accurately identify grant risk. The organization
relies on elements of its structures and processes, including its
initial technical review, disbursement decision-making form, periodic
grant ratings, CCM oversight, information from technical partners, and
LFA oversight, to identify potential risks. Recognizing the need to
establish a more comprehensive approach to risk management, the Global
Fund has begun developing a risk assessment framework for the
organization that includes an early warning and response system to
address poorly performing grants.
Global Fund Did Not Implement the Grant Risk Assessment Model Described
in the 2005 GAO Report:
The Global Fund did not implement the risk-assessment framework to
identify poorly performing grants that it was developing at the time of
our 2005 report.[Footnote 29] We reported that the organization had
devised a risk assessment model and an early warning system to identify
poorly performing grants and more systematically alert portfolio
managers when they needed to intervene.[Footnote 30] According to the
Global Fund, because it disburses grants in countries with varying
levels of economic development and health systems capacity, the risk
model was to incorporate quantitative data, such as grant size and
performance as well as country development and corruption data. Using
these quantitative indicators, the model would generate reports that
indicated potential problems, and portfolio managers would evaluate the
reports in light of contextual information. According to an official
who participated in developing the framework, it was not implemented
because it relied too heavily on quantitative indicators and did not
adequately account for contextual factors, leading to risk ratings that
were too inaccurate to be useful. For example, according to Global Fund
officials, in a test of the model, a particular grant was rated as low
risk based on the host country's development and corruption indicators.
However, the grant ultimately failed and was not renewed at the phase 2
grant renewal review.
Global Fund Applies Elements of Risk Management but Lack of Risk
Assessment Model Limits its Ability to Respond to Poorly Performing
Grants:
Although the Global Fund's grant oversight incorporates several
elements of risk management, the absence of a risk assessment model
limits its ability to respond methodically to poorly performing grants
and use available resources to increase oversight as needed. According
to Global Fund officials, the organization currently relies on existing
oversight mechanisms such as its initial technical review of grant
proposals, standardized decision forms, periodic grant ratings, CCM
oversight, technical partners' assistance, and LFAs' reviews to
identify and assess potential mismanagement, irregularities, or other
risks to successful grant implementation.
* Technical review. The technical review panel raises important
technical issues that grantees must address when presenting their
proposals to the panel. Proposals with a very high technical risk
profile generally are not recommended for funding.
* Disbursement decision-making form. Grant risk is mitigated by the
fund portfolio manager's use of the standard disbursement decision-
making form, which includes financial and program data from the
recipient that has been verified by the LFA.
* Rating grant progress for each disbursement request. Since 2006,
according to Global Fund officials, every progress and disbursement
request, after the first disbursement, has been graded on a scale of A,
B1, B2, or C. This information is posted on the Global Fund Website's
financial data base and included in grant performance reports that are
updated on the Global Fund website at each disbursement. This rating
system is a key tool for identifying grants at risk and for sharing
information with technical partners.
* Country coordinating mechanism (CCM). A key responsibility of the CCM
is to continue to monitor grant progress during implementation and
identify problems that may affect progress. In many countries, Global
Fund technical partners such as UNAIDS, WHO, and USAID are members of
CCMs.
* Partner assistance. Global Fund officials rely on technical partners
with staff based in recipient countries to provide contextual
information on grant performance, identify risks, and help respond to
problems that affect grant implementation.
* LFA reviews. Global Fund officials emphasized that LFAs identify
potential grant risks in their periodic verification of grant progress
and during the cumulative review and performance assessment for the
phase 2 renewal process.
The absence of a model to assess grant risk has limited the Global
Fund's ability to methodically identify poorly performing grants and
use available resources to increase oversight as needed. According to a
senior manager from an LFA with one of the largest grant portfolios,
because the Global Fund has not assessed grants as high, medium, or low
risk, LFA oversight has not differed substantially by grant risk level
and LFA oversight resources have not been used to optimal effect. For
example, although some grants may require more oversight because of
potential misuse of grant money or limited recipient capacity, LFA task
orders and fees are generally no different for potentially problematic
grants than for grants with few expected problems. Similarly, the
frequency of LFA visits to project sites for monitoring and data
verification has not been based on a systematic estimate of risk.
Global Fund Is Developing a Comprehensive Framework for Risk Management
and Has Established First Components:
Global Fund officials informed us in April 2007 that they are
developing a top-level framework for risk management that will cover
all aspects of the organization's strategic direction, including
addressing resources and skills at the secretariat level, developing a
model for systematically assessing grant risk, and fully implementing
an early alert and response system. Global Fund officials have
acknowledged the need for a quantitative model for ranking grants by
level of risk, supplemented by additional qualitative factors that may
change a grant's rating or ranking, to identify and mitigate grant risk
in a timely manner. The Global Fund recently organized a risk seminar
for directors and senior managers and engaged an international firm to
help them achieve consensus on a shared approach to comprehensive risk
management. We reviewed a draft outline of the guiding principles for
developing a risk management framework that the Global Fund developed
in February 2007. As of April 2007, the Global Fund is in varying
stages of progress in developing and implementing detailed risk
strategies.
According to Global Fund officials, the secretariat has put in place
the first components of its early alert and response system intended to
identify as early as possible grants that are performing poorly and
facing implementation difficulties. These first components include
posting the grant disbursement and renewal ratings on the Global Fund
Website and efforts to engage more technical assistance from partners
for poorly performing grants. Global Fund officials recognize that for
the system to work effectively principal recipients, CCMs,
stakeholders, and partners must share responsibility for tracking the
progress of grant implementation and take appropriate action when
progress is impeded. Global Fund officials stated that the early alert
and response system will eventually include documenting and
disseminating best practices, formalizing the system within the
secretariat and among country-level stakeholders and partners, and
increasing the engagement of partners in grant oversight.
Global Fund Has Limited Ability to Manage and Oversee Local Fund
Agents' Performance:
The Global Fund lacks the information it needs to manage and oversee
the LFAs because of a lack of systematic assessments of LFAs'
performance. Good practice suggests that organizations monitor their
contractors' performance during the course of interaction and
periodically evaluate performance based on clear expectations.
Assessment results should include consistent patterns of rewards,
penalties, or both. (See app. II for sources of these principles.)
Global Fund portfolio managers set expectations for LFA performance
informally and individually, frequently leading to inconsistent
expectations and resulting in challenges for both the LFAs and their
managers. Moreover, the Global Fund does not require formal assessments
or reports of LFA performance; instead, LFAs are assessed informally
and irregularly, and assessment results are not systematically
documented. As a result, the Global Fund has limited records of, and
data on, LFA performance. Although the Global Fund made a previous
attempt to assess LFAs more systematically, this effort was
unsuccessful. According to secretariat staff, key partners, and outside
experts, LFAs' performance and capacity vary.
Global Fund Has Not Standardized Its Expectations for Local Fund Agent
Performance:
Rather than establish standardized performance expectations for all
LFAs, the Global Fund portfolio managers establish expectations
informally for the quality of each LFA's performance. The Global Fund's
contracts with LFAs, as well as the guidelines that it provides,
outline the tasks that all LFAs must complete, such as considering a
principal recipient's programmatic and financial implementation
progress in relation to its plans. The contracts also contain general
statements about LFA performance; for example, a sample LFA contract
specifies that all tasks should be carried out to the satisfaction of
the Global Fund.[Footnote 31] However, Global Fund policy states--and
secretariat staff and 11 of the 12 LFA country teams that participated
in our structured interview confirmed--that the fund portfolio manager
is primarily responsible for defining satisfactory performance for his
or her designated LFAs. As of March 2007, the Global Fund had not
provided fund portfolio managers any specific training or written
guidance defining acceptable or good LFA performance, including the
quality of their reporting.[Footnote 32]
The lack of standardized performance expectations results in challenges
both for the LFAs and for Global Fund management.
* LFA challenges. Because fund portfolio managers set expectations
informally and individually, LFA country teams frequently encounter
different expectations from successive portfolio managers. For example,
among the LFA country teams who participated in our structured
interviews, 6 of 10 teams who had interacted with more than one fund
portfolio manager said that the managers' criteria for a high-quality
report had varied. According to the respondents, variations in fund
portfolio managers' expectations often involved differences in the
scope of work that they expected LFAs to complete. For instance,
members of one country team said that their previous fund portfolio
manager had wanted the team to focus primarily on financial data in its
reports, while the current manager wanted a focus on programmatic
issues. In another country, one fund portfolio manager wanted the team
to conduct on-site verification of grant subrecipients, while a
subsequent manager did not consider site visits a priority. In the
remaining four cases, country teams did not report substantial
variation between different fund portfolio managers. In one of these
cases, the team said that a previous manager established expectations
that subsequent managers used in turn, therefore providing continuity.
* Management challenges. The lack of standardized LFA performance
expectations also presents challenges among those responsible for
managing the LFAs. According to the manager of one LFA central team
with a large grant portfolio, variations in Global Fund expectations
made it difficult to establish best practices for its LFA country
teams. Likewise, several Global Fund officials noted that secretariat
staff have conflicting ideas about LFA performance criteria. In one
instance, according to Global Fund officials, a group of fund portfolio
managers were unable to agree on what constituted a high-quality
report. Fund portfolio managers and other key officials observed that
more standardization of expectations would be beneficial, although some
noted that this standardization should be balanced to allow them to
maintain flexibility to account for country context.
Global Fund Does Not Assess LFAs Systematically and Lacks Formal
Records of LFA Performance:
The Global Fund does not require formal, regular assessments of LFA
performance. As a result, it lacks LFA performance records and cannot
systematically track performance.
LFA Performance Assessment and Reporting Are Informal and Irregular:
Although Global Fund officials regularly review the reports they
receive from the LFAs and provide informal feedback to country teams,
they assess LFA performance informally and at irregular
intervals.[Footnote 33] All fund portfolio managers that we
interviewed, as well as 11 of the 12 LFA country teams that
participated in our structured interviews, said that fund portfolio
managers interact informally and frequently with country team members,
often orally or by e-mail. In addition, all but one country team said
that fund portfolio managers provide informal feedback on their
performance. For example, members of one team noted that they received
feedback from their fund portfolio manager for each report and
requested more information as necessary. In another case, the manager
provided feedback to the LFA central team, which passed the feedback to
the country team. A majority of the country teams reported receiving
feedback from their fund portfolio manager when he or she traveled to
the recipient country, at least once a year. Fund portfolio managers
told us that if they continually encounter problems with a country
team, they inform the Global Fund's LFA manager, who discusses the
issue with the central team; if this does not resolve the problem, the
contract may be terminated. A key Global Fund official stated that two
to three LFA country team subcontracts are terminated each year based
on information from fund portfolio managers on their poor performance.
Although LFA central teams provide some performance feedback to their
country teams, the scope and frequency of that feedback varies
depending on the LFA's structure. Ten of the LFA country teams that
participated in our structured interviews told us that LFA central
teams provide informal feedback about their performance. The central
team at two of the six LFA organizations reviews all products before
they reach the secretariat. At two other LFA organizations, reports are
reviewed only if it is deemed necessary, and in those cases, regional
coordinators or mentors may review the reports.[Footnote 34] At the
remaining two LFA organizations, the distinctions between the country
and central teams are less clear, because the reports are at least
partially written by central team members.
Lack of Assessment Records Limits Available Information on LFA
Performance:
The lack of formal assessments limits the Global Fund's access to the
information needed to manage LFAs' performance. The fund portfolio
managers we interviewed said that their frequent interaction with
country teams generally allows them to know which teams are performing
well and which teams need improvement. However, Global Fund officials
with LFA management responsibility stated that formalizing the process
would allow the agency to establish LFA performance records and
identify situations in which more oversight of LFAs by the Global Fund
may be required. A working group of Global Fund experts echoed these
concerns, reporting that despite extensive debate about the LFA model,
the Global Fund had conducted little systematic assessment of the
performance of individual LFA country teams.[Footnote 35]
Global Fund Previously Attempted to Implement Systematic LFA
Assessments:
A prior attempt to introduce a tool to make LFA assessments more
systematic was not successful, because use of the tool was not
mandatory and fund portfolio managers did not see it as a priority.
According to the Global Fund, the LFA manager conducted a pilot of an
evaluation checklist as a tool for fund portfolio managers to
systematically assess LFA performance. Secretariat staff stated that,
because Global Fund management did not require fund portfolio managers
to complete the checklist, very few portfolio managers used it. As of
early 2007, the LFA manager was developing a revised tool to routinely
collect staff input on LFAs. However, it is too early to determine
whether this tool, when finalized and made available to staff, will
capture the information needed to assess LFA performance. According to
the LFA manager, integrating this tool into existing tasks and making
it an explicit component of fund portfolio managers' job descriptions
would increase its use.
LFA Performance and Capacity Reportedly Vary:
Although Global Fund officials cited instances in which LFA country
teams performed adequately, secretariat staff, key partners, and
outside experts expressed concerns about some LFAs' capacity to oversee
grant recipients' programmatic and procurement activities. To
illustrate adequate LFA capacity and performance, secretariat staff
identified cases where LFA country teams have quickly identified
problems with grants. For example, the majority of fund portfolio
managers we interviewed noted that most LFA country teams performed
financial oversight competently.[Footnote 36] Similarly, a procurement
officer described a country team that focused on supply management
issues throughout the life of the grant and proved instrumental in
resolving a large corruption problem. One fund portfolio manager
described a country team that had enough knowledge of monitoring,
evaluation, and procurement to understand an LFA subcontractor's
initial assessment of recipient capacity and, therefore, could use this
information in subsequent reporting about grantee progress.
Conversely, secretariat staff also cited multiple instances in which
LFAs demonstrated lack of capacity that affected the conduct of their
oversight duties. In a recent Global Fund survey, secretariat employees
who identified themselves as having sufficient exposure to LFAs to form
an opinion reported mixed experiences regarding LFAs' technical program
and monitoring expertise.[Footnote 37] About half of the respondents
disagreed with the statement that LFAs with whom they had worked
provided "adequate" expertise on health and program issues. The
remainder was about equally divided between those who agreed that
health and program expertise was adequate and those who neither agreed
nor disagreed. Global Fund employees' assessments of the adequacy of
LFAs' expertise for monitoring and evaluation were similarly mixed. The
survey also provided evidence that the service quality varies greatly
among countries: about three-quarters of the staff agreed with the
statement that "within LFA firms there are substantial differences in
service quality across countries."
In addition, Global Fund partners and outside experts have raised
concerns about LFAs' oversight capacity. UNAIDS officials said that
they have serious doubts about the capacity of LFAs to monitor program
progress, specifically noting that LFAs have little understanding of
how to weigh and balance various performance targets that are not of
equal importance.[Footnote 38] For example, according to these
officials, an LFA that lacks strong technical program capacity might
not recognize that a recipient exceeding a simple performance target,
such as training staff, may be trying to compensate for poor
performance in more difficult target areas, such as administering drugs
to treat AIDS, TB, or malaria. Several other sources--a report
commissioned by the Global Fund, our 2005 report, and several reports
by an acknowledged expert on the Global Fund--also note problems with
LFA capacity. (See app. II for a list of sources.)
Conclusions:
Since our report in June 2005, the Global Fund has taken steps to
improve operations at its secretariat in Geneva while it continues to
confront complex issues facing grant implementation at the country
level. The Global Fund will continue to face important challenges,
including evaluating the effectiveness of its investment of the
billions of dollars it receives annually from the United States and
other donors and the impact of its programs over time. The Global Fund
has improved the transparency of its periodic disbursement and grant
renewal decisions by providing better documentation in its grant files.
Delays in developing a systematic approach to assessing grant risk have
left the Global Fund vulnerable to some of the risks associated with
implementing grants in developing countries. However, the Global Fund
board recognizes the importance of developing a more comprehensive
approach to risk management, and the secretariat has recently drafted
principles to guide its development of a risk framework for the
organization. In addition, the Global Fund's lack of a formal process
for setting expectations and assessing and analyzing LFA performance
constrains its ability to determine the quality of the LFAs' services,
track LFA performance over time, and address ongoing capacity issues.
The lack of a formal assessment process also limits the information
available to the Global Fund during its upcoming global retender of LFA
contracts.
Recommendations for Executive Action:
To enhance Global Fund operations, we recommend that the Secretaries of
State and HHS work with the Global Fund's Board Chair and Executive
Director to take the following two actions:
* establish standardized expectations for LFA performance, and:
* require systematic assessments of LFA performance and the collection
and analysis of LFA performance data to improve the management and
oversight of LFAs.
Agency Comments:
Written comments on a draft of this report from the Departments of
State and Health and Human Services and the U.S. Agency for
International Development are represented in the unified response
received from the Office of the U.S. Global AIDS Coordinator. The
Global Fund also provided written comments on the draft. Both entities
agreed with the report's findings, conclusions, and recommendations.
The Office of the U.S. Global AIDS Coordinator stated that the report
accurately reflected areas in which the Global Fund has made progress
as well as areas in which more work remains to be done. The Office's
response noted that the United States will urge the Board to adopt
concrete recommendations from several studies to improve the
accountability and transparency of Global Fund activities and will work
with the Global Fund to improve its oversight and risk assessment
mechanisms. The Global Fund stated that the report reinforced the
importance of measures it has begun taking to better measure local fund
agent performance. Regarding risk assessment, the Global Fund
emphasized the role of its country-based and global partners in helping
to track grant implementation and provide technical support as needed.
(See apps. III and IV for their comments.)
We are sending copies of this report to interested congressional
committees and will make copies available to others on request. In
addition, this report is available on GAO's Web site at
http://www.gao.gov.
If you or your staff have any questions, please contact me at (202) 512-
3149 or gootnickd@gao.gov. Contact points for Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. Key contributors to this report are listed in appendix V.
Signed by:
David M. Gootnick:
Director, International Affairs and Trade:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To assess the Global Fund's efforts to improve documentation of
performance-based-funding decisions, we analyzed the forms used to
collect data on grant performance and discussed with Global Fund
officials how the forms have changed since GAO's 2005 report. We met
with officials from all of the entities that have current contracts to
serve as local fund agents and with officials from the Office of the
Global AIDS Coordinator, UNAIDS, and the World Bank---the Global Fund's
key technical partners--to discuss how documentation of the Global
Fund's decisions has changed and whether it has improved.
To review information in the grant files used to support performance-
based funding decisions, we analyzed two separate simple random
probability samples of grant funding decisions using a separate data
collection instrument for each sample. The first sample consisted of 80
of the 348 grants that the Global Fund assessed for performance-based
funding disbursements between January 1 and October 1, 2006. The second
sample consisted of 45 of the 65 grants that the Global Fund assessed
for phase 2 grant renewal during the same time period.[Footnote 39] We
chose January 2006 as the starting point for our sample because Global
Fund officials told us that their new disbursement decision worksheet-
-created to address GAO's recommendation that they better document
their decisions--was fully implemented by the end of 2005. With our two
probability samples, each member of the study population had a nonzero
probability of being included, and that probability could be computed
for any member. Each sample case file was subsequently weighted in the
analysis to account statistically for all the case files in the
population, including the files that were not selected.
Because we followed a probability procedure based on random selections,
our samples are only one of a large number of samples that we might
have drawn (for both the grant disbursement and phase 2 file reviews).
Since each sample could have provided different estimates, we express
our confidence in the precision of our particular sample's results as
95 percent confidence intervals. These are intervals that would contain
the actual population values for 95 percent of the samples we could
have drawn. All percentage estimates provided in this report based on
our samples have 95 percent confidence intervals of within plus or
minus 13 percentage points.
For each grant in our disbursement sample, we requested the grant
recipient's Progress Update and Disbursement Request report, the Local
Fund Agent Recommendation report, and the Global Fund's Disbursement
Decision-Making Worksheet. For the phase 2 decision sample, we
requested the following documents for each grant:
* Phase 2 Grant Scorecard (the in-house version that goes to the
panel):
* Phase 2 Grant Scorecard (the panel's version that goes to the board):
* Documentation of the board's decision with any explanation or
justification of the decision (whether this is in the form of a
scorecard or some other format):
* Phase 2 Panel Minutes:
* Board Notes:
* Grant Performance Report:
* Annex A Attachments 2 and 3:
* Local fund agent Phase 2 Assessment Report:
We gathered the information from the documents for our data collection
instruments and verified the data that we used to support our findings.
Our file review consisted of a number of checks on the presence or
absence of an array of qualitative and contextual information that the
Global Fund forms specify as important for grant decision-making and
that therefore should be provided by those responsible for filling them
out. We did not evaluate the appropriateness or accuracy of the
judgments made on the basis of the narrative or contextual information
provided in the documents; we focused on the issue of whether the
contextual or narrative information provided was relevant to particular
judgments or decisions.[Footnote 40]
In our analysis of the disbursement files, while we examined
information contained in other documents, we focused mostly on the
Disbursement Decision-Making Form, due to the fact that this was the
key form that the Global Fund instituted for disbursement-related
decision-making since our last report. In this form, we checked for the
presence or absence of the following pieces of information:
* Fund portfolio manager/cluster explanation for differences in fund
portfolio manager/cluster and LFA ratings:
* Fund portfolio manager's explanation for performance rating:
* Fund portfolio manager's explanation for differences in funding
decision with the principal recipient request and LFA recommendation:
* Fund portfolio manager's check-offs on an array of grant decision-
making issues such as performance evaluation, financial considerations,
future and outstanding requirements, etc.
* Fund portfolio manager's explanation for array of contextual issues
(e.g., governance, general management, and external factors):
* Fund portfolio manager's explanation for required documents and
conditions precedent items:
In the phase 2 files we focused on the Grant Scorecard, as this is the
key Global Fund decision-making document for this phase, although we
performed checks on several other documents as well. We checked for the
presence or absence of the following pieces of information in the
following phase 2-related documents; all items are for the Grant
Scorecard unless otherwise noted:
* Evaluation unit's explanation for nonconcurrence with fund portfolio
manager/cluster rating:
* General explanation for difference between fund portfolio manager/
cluster recommendation and the LFA's recommendation:
* Evaluation unit's explanation for difference between fund portfolio
manager/cluster recommendation and the LFA's recommendation:
* General explanation for difference in the recommended amounts between
Global Fund financial review and cluster leader phase 2:
* General explanation for difference in the recommended amounts between
LFA and fund portfolio manager/cluster leader (i.e. the Global Fund)
phase 2:
* Fund portfolio manager/cluster identification of material issues with
reliability of principal recipient's data:
* Fund portfolio manager/cluster explanation for phase 2 rating:
* Fund portfolio manager/cluster explanation for phase 2
recommendation:
* (LFA phase 2 assessment) report-specific conditions identified by LFA
for 'Conditional Go' recommendations:
* Time-bound actions identified by portfolio manager/cluster for
'Conditional Go' recommendations:
To assess the Global Fund's progress in implementing a risk assessment
model and early warning systems for its grants, we followed up on the
risk assessment model and early warning system described in our 2005
report with Global Fund officials. In addition, we reviewed
international standards for risk that establish the criteria for
assessing risk developed by the Committee on Sponsoring Organizations
of the Treadway Commission. We also discussed the need for a risk
assessment model and early warning system with key Global Fund staff,
key Global Fund partners, UNAIDS, World Bank, and OGAC, and with the
Global Fund Inspector General and LFA officials. In addition, we
reviewed Global Fund board notes and the Global Fund's draft outline of
Guiding Principles for a Risk Management Framework. In their technical
review of our draft, Global Fund officials clarified the distinction
between the risk assessment model that was not implemented and the
current early alert and response system, and they provided additional
information on the current status of the early alert and response
system. We modified the risk assessment section of the draft based on
our analysis of this information.
To assess how the Global Fund oversees and measures the capacity and
performance of local fund agents, we established criteria for good
practices in performance assessment, consulted key reports and
documents, met with relevant personnel, conducted structured interviews
with LFA country teams, and analyzed the structured interview data
using a content analysis framework. We consulted GAO experts in
performance assessment and contracting, good practice documents from
the Government Performance and Results Act,[Footnote 41] performance-
based contracting literature, internal control literature, and training
development literature. (See app. II) To understand Global Fund
policies and requirements regarding LFA performance, we reviewed Global
Fund documentation, including guidelines for LFA work, the Global
Fund's operational policy manual, LFA manager terms of work, and LFA
framework contract forms, as well as relevant evaluative reports
published by the Global Fund and development experts. We meet with key
Global Fund officials in Geneva, Switzerland, including the fund
portfolio managers and members of the Portfolio Services and Projects
Team, representatives from all LFA central teams, selected LFA
subcontractors, and recognized experts on the Global Fund to discuss
the current practices and potential difficulties of the LFA system. We
also reviewed the methodology and results of a survey of Geneva-based
Global Fund secretariat staff who are users of LFAs conducted by a
Global Fund contractor in November of 2006.
Additionally, to support our review of issues related to LFA capacity
and performance, we conducted structured telephone interviews with a
nonprobability sample of 12 LFA country teams out of a total of 102 LFA
country teams worldwide.[Footnote 42] We selected country teams for
this sample based on criteria such as the size and amount of the grants
that the LFA oversaw, whether the LFA was located within the country,
and whether the LFA acted as a mentor to other LFA country teams. The
purpose of these 12 interviews was to obtain information on the views
of these important participants in the Global Fund process; we did not
obtain a representative perspective of the views of all 102 country
teams. We pre-tested our questions with two of our initial respondents
and refined our questions based on their input. We followed up via e-
mail with our respondents to obtain supplementary information. To
summarize the open-ended responses and developed categories for the
analysis, we first grouped interview responses by general categories,
based on our objectives, criteria, and inductive observation. Within
each of these general categories, we then further grouped the data into
subcategories based on the emerging themes. To ensure the validity and
reliability of our analysis, these subcategories were reviewed by a
methodologist, who proposed modifications. After discussion of these
suggestions, we determined a final set of subcategories. An analyst and
a methodologist independently tallied the number of respondents
providing information in each subcategory, compared their tallies, and
reconciled any variation.
[End of section]
Appendix II: Other Sources Cited:
Principles for Contractor Management:
* GAO, Strategic Issues Web pages on the Government Performance and
Results Act (GPRA):
* GAO, Internal Control Management and Evaluation Tool. GAO-01-1008G.
Washington D.C.: Aug. 2001.
* Office of Federal Procurement Policy, Seven Steps to Performance-
Based Services Acquisition, see: www.acqnet.gov/Library/OFPP/
BestPractices/pbsc:
* GAO, Contract Management: Guidance Needed for Using Performance-based
Service Contracting. GAO-02-1049. Washington, D.C.: Sept. 23, 2002.
Analyses of LFA Capacity and Performance:
* GAO, Contract Management: Guidance Needed for Using Performance-Based
Service Contracting. GAO-02-1049. Washington, D.C.: 2002.
* GAO, Internal Control Management and Evaluation Tool. GAO-01-1008G.
Washington, D.C.: 2001.
* Kruse, Stein-Erik, and Jens Claussen. Review of the Roles, Functions
and Performance of Local Fund Agents. Centre for Health and Social
Development and Nordic Consulting Group; 2004.
* Office of Federal Procurement Policy, Seven Steps to Performance-
Based Services Acquisition. See: www.acqnet.gov/Library/OFPP/
BestPractices/pbsc:
* Radelet, Steven et al. Challenges and Opportunities for the New
Executive Director of the Global Fund: Seven Essential Tasks. Global
Fund Working Group, Center for Global Development, Washington D.C.:
October 2006.
* Radelet, Steven. The Global Fund to Fight AIDS, TB, and Malaria:
Progress, Potential, and Challenges for the Future. Center for Global
Development, Washington D.C.: 2004.
[End of section]
Appendix III: Comments from the Global Fund:
Investing in our future:
The Global Fund To Fight AIDS, Tuberculosis and Malaria:
22 April 2007:
Mr David Gootnick:
Director, International Affairs and Trade Issues:
United States Government Accounting Office:
Washington DC 20548:
United States of America:
Dear Mr Gootnick:
Thank you for sharing your draft report on the Global Fund: "Global
Health: Global Fund to Fight AIDS, Tuberculosis and Malaria Improved
Documentation of Funding Decisions but Needs to Standardize Oversight
Expectations and Assessments." External analyses such as this report
are extremely useful. We welcome its findings as they provide feedback
and recommendations that will help the Global Fund to remain adaptive,
responsive and ultimately more successful in its fight against HIV/
AIDS, tuberculosis and malaria.
Documentation and Funding Decisions:
We acknowledge your recognition of the significant improvements made by
the Global Fund in the documentation of decisions to disburse funds and
renew grants since your 2005 report entitled "Global Health: The Global
Fund to Fight AIDS, Tuberculosis and Malaria is Responding. to
Challenges but Needs Better Information and Documentation for
Performance-based Funding."
As noted in your report, the Global Fund has significantly strengthened
its systematic documentation of funding decisions throughout a grant's
life-cycle. We are committed to continuously monitor and improve the
system as appropriate.
Risk Management Framework and Early Alert System:
The report reflects the progress made to date by the Global Fund to
improve grant oversight and grant-associated risk management. We have
recognized the need to develop a comprehensive risk management
framework for the Global Fund and work has commenced on this. As part
of this framework the Global Fund has already implemented the first
phase of an early alert and response system to address poorly
performing grants. We will work to improve the systems and tools for
use by the Secretariat and our in-country partners, as well as by our
global partners, to better respond to grants at risk.
We wish to emphasize the different yet critical roles and
responsibilities that country-based and global partners play in
tracking progress in Global Fund grant implementation and providing
technical support as needed. This dependence on in-country partners
arises from the founding principles that underpin the business model of
the Global Fund. The Global Fund was established as a financing not an
implementing entity with no country-based staff. The support of
partners is therefore essential for an effective response to address
grants at risk.
The Secretariat will continue to make all possible efforts to improve
its communication and information sharing, to ensure that the risk
information is more readily accessible and available to our partners at
the country level. We will also work with them to strengthen the
reciprocal, timely and effective flow of information from our partners
in the field.
Local Fund Agent's Management and Oversight:
The Secretariat agrees with the report's findings and recommendations
on the need to establish standardized expectations for Local Fund
Agent's (LFA) performance and to require the systematic assessment of
LFA performance. The quality of LFAs' work-products and performance is
periodically reviewed by individual Fund Portfolio Managers, and the
Secretariat provides feedback to LFAs. This assessment now needs to be
more transparent and systematic. We had already commenced work on
developing performance standards and measuring tools and thus report
reinforces the importance of that initiative. These standards will form
part of any new contractual agreement following the retendering of LFA
services due to commence in mid 2007.
The report also raises concerns about the ability of some current LFAs
to monitor procurement and program implementation. This is an area of
ongoing discussion with LFAs. The Secretariat is committed to ensure
that all future LFAs have access to high-quality procurement and
programmatic skills, to enable them to better deliver on the oversight
of grants performance.
Conclusion:
In conclusion, the Global Fund Secretariat thanks GAO for this report.
We appreciate the constructive and forward looking recommendations and
commit to their implementation.
We welcome the United States Congress's continued interest in, and
support for, the Global Fund. We value the significant contribution
that representatives of the United States make on the Global Fund Board
and the close collaboration we have at country level with the PEPFAR
program. We are also deeply grateful to the United States for its
significant financial contribution to the global fight against the
three pandemics.
Yours sincerely:
Signed by:
Helen Evans:
Executive Director Ad Interim:
[End of section]
Appendix IV: Comments from Departments of State and Health and Human
Services and U.S. Agency for International Development:
United States Department of State:
Assistant Secretary for Resource and Chief Financial Officer:
Washington, D.C. 20520:
Ms. Jacquelyn Williams-Bridgers:
Managing Director:
International Affairs and Trade:
Government Accountability Office:
441 G Street, N. W.
Washington, D.C. 20548-0001:
Apr 26 2007:
Dear: Ms. Williams-Bridgers:
We appreciate the opportunity to review your draft report, "Global
Health: Global Fund to Fight AIDS, T13 and Malaria Improved
Documentation for Its Funding Decisions but Needs Standardized
Expectations and Assessments," GAO Job Code 320416.
The enclosed Department of State comments are provided for
incorporation with this letter as an appendix to the final report.
If you have any questions concerning this response, please contact Mary
Jeffers, Point for Global Fund Issues, Office of the U.S. Global AIDS
Coordinator, at (202) 663-2586.
Sincerely,
Signed by:
Sid Kaplan (Acting):
cc: GAO--Audrey Solis:
S/GAC - Amb. Mark Dybul:
State/OIG - Mark Duda:
Department of State, Health and Human Services, and USAID Comments GAO-
07-627; Code 320416:
On behalf of the U.S. Departments of State (DOS) and Health and Human
Services (HHS), and the U.S. Agency for International Development
(USAID), the Office of the U.S. Global AIDS Coordinator (OGAC)
appreciates the opportunity to comment on the draft report from the
Government Accountability Office (GAO) entitled, "Global Fund to Fight
AIDS, TB and Malaria Has Improved Its Documentation of Funding
Decisions but Needs Standardized Oversight Expectations and
Assessments" (GAO-07-627).
OGAC concurs with the report's conclusions. We believe the report
accurately identifies areas in which the Global Fund Secretariat has
made great improvement, as well as areas in which there is still work
to be done. One of President George W. Bush's primary mandates for the
Global Fund at. its inception was that it award grants on the technical
merit of proposals and make disbursements based on subsequent grant
performance. As the Global Fund enters its sixth year of operations, it
is clear there is still significant variation in how well recipients
are able to implement and administer Global Fund grants. The issues GAO
addresses in this report go to the heart of the Global Fund's ability
to develop and use effective tools to respond to such performance
variability.
We have worked diligently on the Global Fund Board and with the Global
Fund Secretariat to address the issue of documentation of performance,
and are pleased the GAO sees a marked improvement in this area, as we
do ourselves. The GAO notes, however, that it did not evaluate the
appropriateness or accuracy of the judgments made by the Global Fund
Secretariat based on the documents the GAO reviewed. It also did not
comment on whether the Global Fund Secretariat appropriately adjusted
the budgets of grants to reflect material reprogramming in each grant.
In addition, we remain concerned by the paucity of information on the
performance of sub-recipients of Global Fund grants.
We have also worked consistently on the Global Fund Board to urge an
expanded and more consistently active role for Local Fund Agents
(LFAs), and we welcome the report's specific recommendations in this
area. We concur with the need for the Global Fund to establish
standardized expectations for LFA performance, and believe these
expectations should include a specific instruction to LFAs to look
beyond financial management and examine program management and the
delivery of quality interventions, including on-site verification of
sub-recipients' activities. The examination by LFAs of program
implementation and financing of grant activities, as a standard
practice, would provide critical information for the Secretariat to
make fully informed and consistent decisions about disbursements and
possible adjustments to grant budgets.
We would like to note that the Global Fund Secretariat has commissioned
a study of the LFA system, which it presented at the Meeting of the
Board in April 2007.
The U.S. Delegation urged our Board colleagues to adopt concrete
recommendations from this and other studies to improve the
accountability and transparency of Global Fund activities at the
country level.
Finally, we are very appreciative of the report's insights into the
Global Fund Secretariat's efforts to develop an early warning system
for grants that are experiencing problems. USAID, HHS, OGAC, and U.S.
Government country teams in the field all have mechanisms to provide
necessary technical assistance to Global Fund grants. Unfortunately,
the lack of a risk-assessment and early-warning system at the Global
Fund Headquarters in Geneva has hampered our ability to offer timely
technical assistance and support to grants that are experiencing
implementation bottlenecks. We plan to continue working directly with
the Global Fund Secretariat, as well as through multilateral groups
such as the Global Implementation Support Team (GIST), to help the
Global Fund refine and improve its oversight mechanisms, and to develop
an effective system to identify poorly performing grants quickly so as
to allow Global Fund grantees to use and fully benefit from these and
other technical-assistance resources.
Given that the U.S. Government has provided roughly one-third of all
total Global Fund contributions to date, the Coordinator's office takes
very seriously every aspect of the Global Fund's ability to use U.S.
taxpayer money in the most efficient and effective way possible. The
GAO Report has made a valuable contribution to our long-term efforts to
help the Global Fund achieve fully its enormous potential to aid in the
global fight against HIV/AIDS, tuberculosis and malaria.
Drafted: MJeffers, S/GAC:
Clearances:
RM/GAO: JSchinnick:
MMcche, S/GAC - OK:
TWalsh, S/GAC - OK:
WSteiger, HHS - OK:
Other clearances:
JKolker, S/GAC - OK:
GSteele, USAID - OK:
MWynne, HHS - OK:
JWright, USAID - OK:
PMamacos, S/GAC - OK:
ALion, S/GAC - OK:
CNeary, H - OK:
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
David Gootnick (202) 512-3140:
Staff Acknowledgments:
In addition to the person named above, Audrey Solis, Assistant
Director; Tom Zingale; Jeffrey Baldwin-Bott; Bruce Kutnick; Barbara
Steel-Lowney; David Dornisch; and Reid Lowe made key contributions to
this report. Kathryn Bernet, Antoine Clark, Etana Finkler, Hynek
Kalkus, and Tracy Guerrero provided technical support.
FOOTNOTES
[1] United States Leadership Against HIV/AIDS, Tuberculosis, and
Malaria Act of 2003, Pub. L. No. 108-25, § 202 (f), 117 Stat. 711, 727
(codified at 22 U.S.C. § 7622).
[2] The June 2005 report on the Global Fund was our first review in
response to the mandate. See GAO, Global Health: The Global Fund to
Fight AIDS, TB and Malaria Is Responding to Challenges but Needs Better
Information and Documentation for Performance-based Funding, GAO-05-639
(Washington, D.C.: June 2005). We also completed a report in 2003 on
the Global Fund at the request of the Chairman, Committee on
Appropriations, Subcommittee on Foreign Operations, Export Financing,
and Related Programs. See GAO, Global Health: Global Fund to Fight
AIDS, TB and Malaria Has Advanced in Key Areas, but Difficult
Challenges Remain, GAO-03-601 (Washington, D.C.: May 2003). This report
is our final response to the 2003 mandate.
[3] Risks faced by Global Fund grants include problems associated with
operating in countries with varying levels of economic development and
capacity, which can affect grant effectiveness.
[4] The Global Fund's Board approves grant proposals covering up to a 5-
year period and initially commits grants for the first 2-year phase of
the proposal period. Continued grant funding for phase 2 is conditioned
on program performance, based on a review of performance reports and
other information taken 20 months after the grant program's start date.
[5] See Committee of Sponsoring Organizations of the Treadway
Commission, Enterprise Risk Management --Integrated Framework Executive
Summary: 2004.
[6] As a financing entity, the Global Fund relies on partners to
provide technical support to grant recipients. Its key partners include
UNAIDS, OGAC, the World Bank, and the World Health Organization (WHO).
[7] According to the Global Fund's disbursement decision-making form,
contextual information critical to disbursement decisions includes
qualitative factors relating to governance, grant management, and
external factors. This form, together with the principal recipient's
report on the achievement of all individual targets and the LFA's
verification is the documented basis for funding decisions during
implementation.
[8] The Global Fund maintains a file for each grant that contains,
among other things, all reports relating to that grant's performance,
funding requests, and disbursement and phase 2 assessments and funding
decisions.
[9] For both samples the estimate was 100 percent. For each of these
estimates we are 95 percent confident that at least 94 percent of all
files in our population contain narratives explaining the manager's
rating of the grant's performance.
[10] Good practice suggests that organizations should monitor their
contractors' performance during the course of interaction and
periodically evaluate performance based on clear expectations.
Assessment results should include consistent patterns of rewards,
penalties, or both. (See app. II for sources of these criteria.)
[11] UNAIDS and WHO estimate that 39.5 million people were living with
HIV in 2006 and that 2.9 million people died of AIDS-related illnesses
in 2006. WHO estimates that malaria kills more than 1 million people
per year and that 124 million people in Africa live in areas that are
at risk for seasonal malaria epidemics. WHO also estimates that 1.7
million people die of TB each year.
[12] (RED)™ is a private sector initiative to provide financing for
Global Fund HIV/AIDS projects in Africa. Companies participating in the
initiative donate a portion of profits from the sale of (RED) branded
items.
[13] The Global Fund's key principles are to (1) operate as a financial
instrument, not an implementing entity; (2) make available and leverage
additional resources; (3) support programs that evolve from national
plans and priorities; (4) operate in a balanced manner with respect to
geographic regions, diseases, and healthcare interventions; (5) pursue
an integrated and balanced approach to prevention, treatment, care, and
support; (6) evaluate proposals through an independent review process;
and (7) operate in a transparent and accountable manner and employ a
simplified, rapid, and innovative grant-making process.
[14] Between 2003 and 2006 the Global Fund secretariat staff increased
from 63 to 240 employees.
[15] Government agencies are the principal recipients of 64 percent of
grant funds. Nonprofit development organizations and multilateral
organizations also act as principal recipients. In many cases the
principal recipient further distributes the funds to multiple
subrecipients to implement grant activities.
[16] As of 2007, the Global Fund had contracted with the following
organizations to perform LFA oversight duties: PricewaterhouseCoopers,
KPMG, Emerging Markets Group, Crown Agents, the Swiss Tropical
Institute, and the United Nations Office for Project Services.
PriceWaterhouseCoopers and KPMG together account for LFA services in 97
of 120 countries and regional programs.
[17] In addition to 113 country programs, LFAs oversee grants in 1
multicountry program in Africa 5 multicountry programs in Latin America
and the Caribbean, and 1 multicountry program in the Western Pacific.
[18] The specific responsibilities delegated to the LFA central team,
LFA country team, and outside experts sometimes vary from one LFA
organization to another. For example, of the two firms with the largest
portfolios, one uses a decentralized approach in which the majority of
the work occurs within the country team, and the other uses a
centralized approach in which the country teams perform the oversight
tasks, but all LFA products are vetted through the central team before
reaching the secretariat. Using a different approach, one of the
smaller LFA organizations has very limited presence within the
recipient countries; instead, it sends specialists into the country to
perform oversight activities as needed.
[19] A disbursement greater than $2 million requires a decision by the
secretariat's Director of Operations.
[20] Fund portfolio managers, with recommendations from LFAs, give
performance ratings that range from A (expected or exceeding
expectations), B1 (substantial achievements made), B2 (little progress
made), C (unacceptable).
[21] In preparation for the phase 2 grant renewal decision, Global Fund
staff prepare a summary of information from the grant recipient, LFA,
portfolio manager, performance evaluation team, and secretariat
financial staff on grant performance.
[22] The Global Fund calls such changes material reprogramming.
[23] All percentage estimates based on this sample provided in this
report have 95 percent confidence intervals of within plus or minus 13
percentage points. See appendix I for more information on our sampling
methodology.
[24] For both samples the estimate was 100 percent. For each of these
estimates we are 95 percent confident that at least 94 percent of all
files in our population contain narratives explaining the manager's
rating of the grant's performance.
[25] LFAs, portfolio managers, and evaluation staff make phase 2
funding recommendations (Go, Conditional Go, or No Go). The Global Fund
Board of Directors reviews these recommendations and makes the final
phase 2 recommendation to renew the grant, not to renew the grant, or
to renew the grant with conditions attached. Global Fund officials said
that differences between ratings and proposed funding levels are common
and do not necessarily indicate mistakes or poor judgment by LFAs,
portfolio managers, or staff from the evaluation team. They said, for
example, that after LFAs have completed their assessments, LFAs and
fund portfolio managers often receive new information about grant
performance that leads them to propose a different phase 2 amount than
originally recommended by the LFA.
[26] Fund portfolio managers and the evaluation staff gave the majority
of the phase 2 grants matching ratings. They concurred on about 73
percent of their phase 2 grant renewal recommendations.
[27] We estimated that in almost half of grant disbursement files, the
fund portfolio manager disagreed with the amount recommended by the LFA
or the amount requested by the grant recipient.
[28] GAO-05-639.
[29] According to international standards, risk assessment reduces
operational surprises and losses and enhances an organization's ability
to identify potential concerns and establish appropriate responses. See
Committee of Sponsoring Organizations of the Treadway Commission,
Enterprise Risk Management --Integrated Framework Executive Summary,
2004.
[30] GAO-05-639.
[31] Global Fund officials gave us a sample LFA contract and said that
its contracts with the six LFAs were very similar.
[32] The Global Fund secretariat began piloting training courses for
fund portfolio managers in 2006 and told us that a module on LFAs would
be added to this training series in the future.
[33] Good practice for overseeing contractors suggests that
contractors' fulfillment of established expectations should be
systematically measured. See appendix II.
[34] Mentors are individuals from country teams that the central team
has identified as high performers.
[35] Challenges and Opportunities for the New Executive Director of the
Global Fund - Seven Essential Tasks" by Steve Radelet et.al.
(Washington, D.C.: Oct. 2006).
[36] Of the 12 country teams that participated in our structured
interviews, almost all team members had financial expertise but limited
training in procurement or monitoring and evaluation.
[37] The Global Fund drafted a questionnaire that a contractor tested
and administered anonymously in a Web-based survey from November 9 to
16, 2006 to the approximately 120 Geneva-based Global Fund secretariat
staff who use LFA services. The 75 staff who responded rated their
agreement on more than 50 statements about LFA-related issues using a
five-point scale of "Strongly Disagree, Disagree, Neither Agree nor
Disagree, Agree, Strongly Agree" while being given the option on most
questions of not answering if they had "Insufficient experience to form
an opinion." We report responses for the 51 operations staff, all of
whom had direct contact with LFAs on a range of issues.
[38] The Global Fund categorizes targets into three levels of
complexity. The first two levels are considered to be "process"
indicators, and include measures such as "people trained" or "service
points supported." The third, more complex category, denotes "coverage"
indicators and measures the number of individuals receiving services.
The Global Fund is also considering the most appropriate way to measure
"impact" indicators.
[39] We did not consider grants for the disbursement sample that
received their first disbursement of funds during the period so our
results were not skewed by problems that commonly accompany grants when
they start.
[40] A limitation of our analysis was that we did not evaluate the
adequacy of the information provided, as determining adequacy would
have required us to make judgments requiring substantial additional
grant-specific information not contained in the grant files.
[41] Government Performance and Results Act of 1993, Pub. L. No. 103-
62, 107 Stat. 285.
[42] Results from our nonprobability sample cannot be used to
generalize to the overall population of LFA country teams because, in a
nonprobability sample, some elements of the population being studied
have no chance or an unknown chance of being selected as part of the
sample.
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site.
To have GAO e-mail you a list of newly posted products every afternoon,
go to www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548:
Public Affairs:
Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548:
