Foreign Assistance
State Department Foreign Aid Information Systems Have Improved Change Management Practices but Do Not Follow Risk Management Best Practices Gao ID: GAO-09-52R November 21, 2008In January 2006, the Secretary of State announced major changes in the U.S. government's process for directing and managing foreign assistance programs in a reorganization and revision of operational procedures commonly known as the F Process. These changes were intended to ensure the effective use of U.S. foreign assistance to meet broad U.S. foreign policy objectives and more fully align programs carried out by the Department of State and the U.S. Agency for International Development (USAID). To administer the F Process, the Secretary created the new Office of the Director of U.S. Foreign Assistance (State/F) with a Director of Foreign Assistance (DFA), who carries the rank of Deputy Secretary of State and also serves concurrently as USAID Administrator. To support State and USAID planning, budgeting, and reporting of foreign assistance under these reforms, State/F developed two new data information systems, the Foreign Assistance Coordination and Tracking System (FACTS) and FACTS Info. FACTS, which State/F began to develop in mid-2006, is a database used to collect foreign assistance planning and reporting data, including plans for implementing current-year appropriated budgets and performance planning and reporting data. FACTS Info, which State/F created in 2007, is a system used to aggregate, analyze, and report data on U.S. foreign assistance programs under the authority of the DFA. Although State and USAID are the only U.S. agencies currently using both systems, State/F expects the systems to eventually include data from other agencies involved in foreign assistance, including the Millennium Challenge Corporation and the Department of Treasury, among others. In connection with our ongoing work addressing the management of State/F and at Congressional request, we examined (1) the creation and development of FACTS and FACTS Info and (2) whether State/F is following best practices for configuration management--the process of establishing and maintaining control over changes made to a system--and risk management of FACTS and FACTS Info.
In conjunction with USAID, State/F developed FACTS I in December 2006 (initially known as FACTS) as a database used to collect foreign assistance planning and reporting data, including plans for implementing current-year appropriated budgets and performance planning and reporting data for State, USAID, and the over 25 other U.S. departments and agencies implementing U.S. foreign assistance programs. However, only programs under the authority of the DFA are currently detailed in FACTS. Based on a State/F survey of over 100 State and USAID users in 2007, as well as our interviews with State and USAID officials in Washington, D.C., and the six countries we visited, FACTS I was slow and unreliable during the first 2 years of the F Process. To remedy these problems, in February 2008, State/F hired a contractor to develop FACTS II, which was deployed worldwide in October 2008. State/F created FACTS Info to aggregate, analyze, and report data on U.S. foreign assistance programs under the authority of the DFA. During the initial pilot phase, which began in September 2007, FACTS Info was accessible to a limited number of State and USAID users; however, it is expected to expand its usage to additional State/F and USAID users when the pilot phase ends, currently planned for fall 2008. FACTS II and the ongoing pilot of FACTS Info have recently implemented new configuration management processes, but both lack adequate risk management procedures, such as formalized procedures to plan for foreseeable risks. State/F has taken steps to address these challenges, such as updating the project management plan and implementing change tracking software to address certain weaknesses, particularly to both systems' configuration management. However, as of October 2008, State/F had not fully implemented improvements to the systems' risk management; State officials noted that they plan to complete these improvements by December 2008. Because both FACTS II and FACTS Info lack formal processes for risk management, State/F cannot ensure that risks are identified, analyzed, tracked, and mitigated, increasing the likelihood that potential problems become actual problems. Moreover, State/F was unable to mitigate a key risk that led to problems with the development of FACTS II. Without improved risk management processes, risks may not be effectively managed. To help ensure that FACTS II and FACTS Info are implemented successfully and perform as designed, we are recommending that the Secretary of State direct the DFA to better utilize best practices for risk management procedures to both systems. In particular, we are recommending that the DFA (1) identify and develop a comprehensive list of system development risks for FACTS and FACTS Info, and (2) fully develop risk mitigation plans for FACTS and FACTS Info.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-09-52R, Foreign Assistance: State Department Foreign Aid Information Systems Have Improved Change Management Practices but Do Not Follow Risk Management Best Practices
This is the accessible text file for GAO report number GAO-09-53R
entitled 'Foreign Assistance: State Department Foreign Aid Information
Systems Have Improved Change Management Practices but Do Not Follow
Risk Management Best Practices' which was released on November 21, 2008.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
Washington, DC 20548:
November 21, 2008:
The Honorable Howard L. Berman:
Chairman:
Committee on Foreign Affairs:
House of Representatives:
The Honorable Robert Menendez:
Chairman:
Subcommittee on International Development and Foreign Assistance,
Economic Affairs, and International Environmental Protection:
Committee on Foreign Relations:
United States Senate:
The Honorable Daniel K. Akaka:
Chairman:
Subcommittee on Government Management, the Federal Workforce, and the
District of Columbia:
Committee on Homeland Security and Government Affairs:
United States Senate:
Subject: Foreign Assistance: State Department Foreign Aid Information
Systems Have Improved Change Management Practices but Do Not Follow
Risk Management Best Practices:
In January 2006, the Secretary of State announced major changes in the
U.S. government's process for directing and managing foreign assistance
programs in a reorganization and revision of operational procedures
commonly known as the F Process. These changes were intended to ensure
the effective use of U.S. foreign assistance to meet broad U.S. foreign
policy objectives and more fully align programs carried out by the
Department of State and the U.S. Agency for International Development
(USAID). To administer the F Process, the Secretary created the new
Office of the Director of U.S. Foreign Assistance (State/F) with a
Director of Foreign Assistance (DFA), who carries the rank of Deputy
Secretary of State and also serves concurrently as USAID Administrator.
To support State and USAID planning, budgeting, and reporting of
foreign assistance under these reforms, State/F developed two new data
information systems, the Foreign Assistance Coordination and Tracking
System (FACTS) and FACTS Info.[Footnote 1] FACTS, which State/F began
to develop in mid-2006, is a database used to collect foreign
assistance planning and reporting data, including plans for
implementing current-year appropriated budgets and performance planning
and reporting data. FACTS Info, which State/F created in 2007, is a
system used to aggregate, analyze, and report data on U.S. foreign
assistance programs under the authority of the DFA.[Footnote 2] (See
enclosure 1 for a complete list of foreign assistance accounts that
fall under the DFA's authority according to State.) Although State and
USAID are the only U.S. agencies currently using both systems, State/F
expects the systems to eventually include data from other agencies
involved in foreign assistance, including the Millennium Challenge
Corporation and the Department of Treasury, among others.
In connection with our ongoing work addressing the management of State/
F and at your request, we examined (1) the creation and development of
FACTS and FACTS Info and (2) whether State/F is following best
practices for configuration management--the process of establishing and
maintaining control over changes made to a system--and risk management
of FACTS and FACTS Info. In conducting this work, we reviewed FACTS and
FACTS Info system procurement, contract, development, performance, and
assessment documents from State and USAID. We also reviewed State/F's
configuration management and risk management procedures for consistency
with industry best practices.
We interviewed cognizant officials at State and USAID headquarters in
Washington, D.C., and at U.S. embassies and USAID missions, including
regional missions, in Ethiopia, Haiti, Jordan, Kenya, Peru, and
Ukraine.[Footnote 3] In particular, we conducted several interviews
with officials from State/F's Strategic Information, Monitoring and
Evaluation Unit, which is charged with implementing and overseeing both
FACTS and FACTS Info.
We conducted this performance audit from January 2008 through November
2008 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Results in Brief:
In conjunction with USAID, State/F developed FACTS I in December 2006
(initially known as FACTS) as a database used to collect foreign
assistance planning and reporting data, including plans for
implementing current-year appropriated budgets and performance planning
and reporting data for State, USAID, and the over 25 other U.S.
departments and agencies implementing U.S. foreign assistance programs.
However, only programs under the authority of the DFA are currently
detailed in FACTS. Based on a State/F survey of over 100 State and
USAID users in 2007, as well as our interviews with State and USAID
officials in Washington, D.C., and the six countries we visited, FACTS
I was slow and unreliable during the first 2 years of the F Process. To
remedy these problems, in February 2008, State/F hired a contractor to
develop FACTS II, which was deployed worldwide in October 2008. State/F
created FACTS Info to aggregate, analyze, and report data on U.S.
foreign assistance programs under the authority of the DFA. During the
initial pilot phase, which began in September 2007, FACTS Info was
accessible to a limited number of State and USAID users; however, it is
expected to expand its usage to additional State/F and USAID users when
the pilot phase ends, currently planned for fall 2008.
FACTS II and the ongoing pilot of FACTS Info have recently implemented
new configuration management processes, but both lack adequate risk
management procedures, such as formalized procedures to plan for
foreseeable risks. State/F has taken steps to address these challenges,
such as updating the project management plan and implementing change
tracking software to address certain weaknesses, particularly to both
systems' configuration management. However, as of October 2008, State/
F had not fully implemented improvements to the systems' risk
management; State officials noted that they plan to complete these
improvements by December 2008. Because both FACTS II and FACTS Info
lack formal processes for risk management, State/F cannot ensure that
risks are identified, analyzed, tracked, and mitigated, increasing the
likelihood that potential problems become actual problems. Moreover,
State/F was unable to mitigate a key risk that led to problems with the
development of FACTS II. Without improved risk management processes,
risks may not be effectively managed.
To help ensure that FACTS II and FACTS Info are implemented
successfully and perform as designed, we are recommending that the
Secretary of State direct the DFA to better utilize best practices for
risk management procedures to both systems. In particular, we are
recommending that the DFA (1) identify and develop a comprehensive list
of system development risks for FACTS and FACTS Info, and (2) fully
develop risk mitigation plans for FACTS and FACTS Info.
State provided written comments about a draft of this report, which we
have reprinted in enclosure II. State noted the steps it has taken to
begin implementing the recommendations and expects to complete those
steps in December 2008.
Background:
In January 2006, following a foreign assistance review, the Secretary
of State determined that U.S. foreign assistance programs were
fragmented among multiple State Department bureaus and offices, as well
as between State and USAID, which created potential redundancies and
complicated efforts to integrate foreign assistance with broader U.S.
foreign policy objectives. As a result, the Secretary concluded that
the U.S. government must align its activities more fully across the
State Department and USAID and within the State Department itself.
To address these challenges, State/F was created in 2006 to focus the
use of foreign assistance toward achieving the Secretary's
transformational diplomacy goal "to help build and sustain democratic,
well-governed states that will respond to the needs of their people,
reduce widespread poverty, and conduct themselves responsibly in the
international system."[Footnote 4] The DFA serves concurrently as the
USAID Administrator, with approval authority over all State and USAID
foreign assistance funding and programs, with the exception of Global
HIV/AIDS Initiative funds. (See enclosure 1 for a complete list of
foreign assistance accounts that fall under the DFA's authority
according to State.) As part of the integrated interagency planning,
coordination, and implementation of foreign assistance, State/F's major
responsibilities include, among other duties, the following:
* developing a coordinated U.S. government foreign assistance strategy
and directing the development of related country-specific multiyear
assistance strategies and annual operational plans;
* creating and directing consolidated policy, planning, budget, and
implementation mechanisms and staff functions required to provide
leadership to foreign assistance; and:
* providing guidance to foreign assistance delivered through other U.S.
agencies and entities, including the Millennium Challenge Corporation
and the Office of the U.S. Global AIDS Coordinator.
State/F created the Strategic Information, Monitoring and Evaluation
Unit to manage the information tools and products that support the
organization, including FACTS and FACTS Info. Prior to the creation of
State/F, many individual operating unit systems gathered information
and informed program management and budget formulation, but no system
existed to integrate and aggregate the information. The creation of
such a system was intended to provide uniform reporting across
countries, funding accounts, and agencies.
Configuration and Risk Management Best Practices:
The disciplines of configuration and risk management, along with other
system development practices, are important to help ensure that
projects are delivered on time, within budget, and with the promised
functionality. These practices should be formalized in such a way that
their processes are clearly defined, precise, systematized, and
documented.
According to the Software Engineering Institute,[Footnote 5] the
purpose of configuration management is to establish and maintain
control over changes made to a system. When changes or problems are
identified, configuration management activities can help determine
where the changes or problems have occurred and what steps were taken
to correct or implement them. The configuration management process
includes activities designed to identify the system's baseline
specifications and configuration and to track and control changes made
to the baseline. Configuration management activities can be divided
into several key areas:[Footnote 6]
* identifying the configuration of a system's baseline data;
* controlling changes made to the system;
* maintaining the integrity of the system's baselines; and:
* providing accurate status and current configuration data of the
system to developers, end users, and customers.
The purpose of risk management is to identify potential problems before
they occur. When problems are identified, risk-handling activities can
be planned and invoked as needed across the life of a project to
mitigate adverse impacts on objectives. Effective risk management
involves early and aggressive risk identification through the
collaboration and involvement of relevant stakeholders. Risk management
activities can be divided into several key areas:[Footnote 7]
* preparing for risk management;
* identifying and analyzing risks;
* mitigating risks, and:
* executive oversight.
State Developed and Implemented FACTS I and FACTS Info to Support the F-
Process, but Found FACTS I to Be Slow and Unreliable:
In conjunction with USAID, State/F developed FACTS I as a database to
collect foreign assistance planning and reporting data, including plans
for implementing current-year appropriated budgets and performance
planning and reporting data. According to numerous FACTS I users and
based on our review, FACTS I was slow and unreliable during the first 2
years of the F Process. Consequently, State/F developed FACTS II as a
successor to FACTS I. State/F also created the FACTS Info system to
aggregate, analyze, and report data on U.S. foreign assistance programs
under the authority of the DFA. During its initial pilot phase, which
began in September 2007, FACTS Info is accessible within State and
USAID to a limited number of users; however, State/F plans to expand
its usage to additional State/F and USAID users when the pilot phase
ends, currently planned for the fall of 2008.
Figure 1: FACTS and FACTS Info Data Flow Chart:
[Refer to PDF for image]
This figure is an illustration of the FACTS and FACTS Info Data Flow
Chart, as follows:
FACTS: Foreign Assistance Coordination and Tracking System.
State/F: State office charged with directing U.S. foreign assistance.
Input: State and USAID operating units in put data into FACTS;
State/F collects foreign assistance planning and reporting data from
operating units in FACTS.
Outputs:
* Reports on individual operating units;
* 1 year operational plans and performance reports.
Input: State/F loads the final operational plan and performance report
data into FACTS Info;
State/F also uploads budget data from State Mission Strategic Plans,
HIV/AIDS Country Operational Plans, and demographic data into FACTS
Info.
Outputs:
* Aggregated data: State/F aggregates operational plan data into FACTS
info;
* Analyzed data: State/F analyzes data in FACTS Info;
* Reports to Congress: State/F uses FACTS Info to create reports for
OMB, Congress, and others in State and USAID.
Sources: GAO analysis of Department of State data; Nova Development
(clip art).
[End of figure]
FACTS I Was Designed to Support Foreign Assistance Reform but Was Slow
and Unreliable:
In conjunction with USAID, State/F developed FACTS I as a database to
collect foreign assistance planning and reporting data, including plans
for implementing current-year appropriated budgets and performance
planning and reporting data. USAID awarded the contract, which cost a
total of $4.5 million, for the initial phase of FACTS, known as FACTS
I. According to USAID and State officials, both USAID and State/F
managed the contract. State/F's Strategic Information, Monitoring and
Evaluation Unit managed the development of FACTS I to combine the data
from the more than 25 U.S. government agencies involved in planning and
reporting on foreign assistance activities into one central data
system. However, only programs under the authority of the DFA are
currently detailed in FACTS. FACTS I was based on the system used by
the Office of the U.S. Global AIDS Coordinator to plan, budget, and
report on the President's Emergency Plan for AIDS Relief program. FACTS
I was launched in December 2006 and included foreign assistance program
information from both State and USAID. State and USAID operating units-
-which in 2007 included 128[Footnote 8] U.S. embassies and USAID
missions overseas, regional offices, and Washington bureaus involved in
the management and delivery of foreign assistance--used FACTS I to
enter, review, and submit their operational plans[Footnote 9] and
performance reports. In addition, State and USAID operating units used
the system to plan activities; retrieve data for operating unit
reporting; and report on specific operating unit programs to State and
USAID management, Congress, and others. State/F used the system to
review and approve the operational plan and performance report
submissions, to retrieve data for analysis and reporting, and to
respond to information requests.
A State/F survey and our field work indicate that users found FACTS I
was slow and unreliable during the first 2 years of the F Process. The
system was intended to be accessible to users at 128 operating units.
However, FACTS I faced several problems in the field. In particular, in
an internal State/F survey that had over 100 respondents and at the
posts we visited during our field work, individuals who used the system
described it as slow and tedious with frequent periods when data could
not be entered during peak usage times prior to work deadlines. For
example, one USAID employee told us that, to perform her other
responsibilities, she frequently entered data from her home computer
after normal working hours (weekends and evenings) because the system
was especially slow during normal working hours. Concluding that FACTS
"was marred by slowness, technical glitches, and unreliability," the
State/F survey concluded that State/F should correct the system's
problems and make it more user friendly.
To remedy the problems with FACTS I, in February 2008, State/F hired a
contractor for $1.9 million to develop a follow-on system known as
FACTS II.[Footnote 10] The contractor was required to (1) design,
build, test, and implement a data system that collects, aggregates, and
prepares reports on data needed for foreign assistance planning and
reporting; (2) maintain and operate FACTS II, including developing and
staffing a help desk facility to support users; (3) train and make
proficient the users of FACTS II; and (4) update and maintain FACTS I
until FACTS II was deployed. State/F deployed FACTS II worldwide to
improve FACTS I's ability to capture, report, and manage foreign
assistance planning and performance data in October 2008.
FACTS Info Pilot Is Launched to Support Foreign Assistance Reform:
FACTS Info is an information system created in-house[Footnote 11] by
two State/F employees and is used to aggregate, analyze, and report
data on U.S. foreign assistance programs under the authority of the
DFA. The system imports budget, program, and performance data from
FACTS I and embassy mission strategic plans (MSP) and is integral in
formulating the foreign operations portion of the congressional budget
justification and performance report. As of September 2008, FACTS Info
contained the following information:
* appropriations for fiscal years 2006 through 2008;
* operational plan data for fiscal years 2007 and 2008;
* performance report data for fiscal year 2007;
* congressional budget justification process data for fiscal year 2009;
and:
* budget formulation information from the fiscal year 2010 Mission
Strategic Plans.
FACTS Info has multiple uses. State/F has used FACTS Info primarily in
supporting the foreign assistance budget formulation process;[Footnote
12] conducting budget and performance analysis to develop funding and
strategic priorities; and reporting to Congress, OMB, and other
stakeholders on the progress of assistance programs. Other State and
USAID bureaus and offices that have access to the system may use FACTS
Info as a reporting and management tool--for example, in aggregating
and analyzing information by region, sector, funding account, or
special interest area for program planning and for preparing speeches,
briefs, and responses to queries. FACTS Info also serves as a data
source for reporting on the progress and funding of specific
congressional earmarks and presidential initiatives.
During the initial pilot phase, which began in September 2007 and is
expected to end in the fall of 2008, FACTS Info is accessible within
State and USAID to a limited number of users. FACTS Info is being
piloted at 42 overseas and domestic operating units at State and USAID.
After the trial period ends, State/F plans to expand access to the
FACTS Info system to other State and USAID operating units, as well as
other U.S. agencies, on an as-needed basis. Once fully operational,
FACTS Info is expected to be the DFA's primary data repository,
analysis tool, and reporting system.
FACTS and FACTS Info Are Beginning to Follow Configuration Management
Best Practices but Do Not Follow Risk Management Best Practices:
FACTS II and the ongoing pilot of FACTS Info have recently implemented
new configuration management processes, but both lack adequate risk
management procedures, such as formalized procedures to plan for
foreseeable risks. State/F has taken steps to address these challenges;
however, as of October 2008, State had not fully implemented
improvements to the systems' risk management. State officials noted
that they plan to complete these improvements by December 2008.
State's Management of FACTS and FACTS Info Is Beginning to Follow
Configuration Management Best Practices:
Both FACTS II and FACTS Info have recently implemented and formalized
processes for configuration management. The FACTS II contractor was
tasked with developing a configuration management and change control
plan to ensure that changes are made in a deliberate and coordinated
manner. However, a June 2008 independent verification and validation
report[Footnote 13] contracted by State found that a plan had not been
developed for FACTS II and that change control of FACTS Info had not
been formalized and was insufficient. We also found that a plan for
FACTS Info had not been developed and the change control status had not
been formalized. State/F tracked the changes to FACTS Info in an
informal notes screen in the FACTS Info system. However, the
independent verification and validation report found that the tracking
system did not track effectively all necessary change elements, and
that the system therefore prevented future developers from fully
understanding the changes that have occurred. In November 2008, State
officials noted that they acquired and implemented commercial-off-the-
shelf software to effectively track changes and that they have updated
the configuration management section of their project management plan
to address these weaknesses.
In April 2008, State/F and USAID established a FACTS Change Control
Review Board to help manage configuration management by identifying,
evaluating, and approving changes to FACTS II. However, as of October
2008, the board had not yet convened because the schedule and cost
change thresholds required for triggering the board's involvement had
not been reached. State/F used weekly management meetings with the
system developers to address configuration management issues. State/F
officials also acknowledged that they had difficulties placing FACTS
Info under the same change control board procedures as FACTS II
because, as an in-house system that does not use an outside contractor,
FACTS Info faces different types of schedule or cost risks than does
FACTS. Consequently, State/F officials said that they had developed a
less formal change control structure for FACTS Info managed by the new
FACTS Info Executive Board, which will serve as the decision-making
body for FACTS Info changes.
State's Management of FACTS and FACTS Info Does Not Follow Risk
Management Best Practices, but State Has Made Recent Improvements:
Neither FACTS II nor FACTS Info has fully followed formal processes for
risk management. In December 2007, State/F drafted a risk management
plan that identified having a fixed price contract as, according to
State/F officials, the greatest risk to developing FACTS II, because it
does not allow State/F to adjust the amount of the contract in response
to new requirements or unforeseen developments that may increase cost.
State/F indicated that having a firm fixed price contract was also the
primary action taken to reduce risk since the contract transferred most
of the risk for cost and schedule from State to the system developer.
State also stated that it used the June 2008 independent verification
and validation report to help it identify risks and took steps to
reduce schedule slippage as a result. However, State/F did not mitigate
this risk, resulting in State/F's assuming some duties from the
contractor. State/F's failure to mitigate this risk also led to a delay
in the release of FACTS II. As noted by the independent verification
and validation report and in our meetings with State/F officials, the
contractor responsible for maintaining FACTS I as well as developing
FACTS II found that maintaining FACTS I required far more time and
resources than expected. As a result, time and resources planned for
FACTS II were used for FACTS I, delaying the development of FACTS II.
In July 2008, 2 months prior to the expected deployment of FACTS II,
the contractor revised the draft risk management plan. This version of
the plan has a risk register--a log used to track projected risks--that
identifies six risks and provides a minimal mitigation strategy.
However, the risk register is designed only for FACTS II and does not
apply to FACTS Info.
State/F acknowledged that, as of October 2008, it had not fully
implemented a risk management plan. State officials noted they have
developed a revised risk management plan that incorporates FACTS Info
risks into the risk register; however, as of October 2008, the new plan
had not yet been fully implemented. State officials stated that they
expect of have a fully approved risk management plan in place by
December 2008.
Conclusion:
State/F is in charge of managing the development of the core
information systems for planning, budgeting, and reporting on U.S.
foreign assistance programs. State and USAID operating units upload
their operational budget plans into FACTS, and State/F is able to
aggregate this data in FACTS Info. However, these systems face
potential challenges to their continued development, because State/F
has not fully developed the processes needed for managing risks.
Because neither FACTS II nor FACTS Info has fully followed formal
processes for risk management, State/F cannot ensure that risks are
identified, analyzed, tracked, and mitigated, increasing the likelihood
that potential problems become actual problems. State has taken steps
to enhance its compliance with risk management. However, these actions
do not reduce State's responsibility to carry out oversight of risk
management to ensure success of the project. Moreover, State/F did not
mitigate a key risk that led to problems with the development of FACTS
II. Without improved risk management processes, risks may not be
effectively managed.
Recommendations for Executive Action:
To help ensure that FACTS II and FACTS Info are implemented
successfully and perform as designed, we recommend that the Secretary
of State direct the DFA to better utilize best practices for risk
management procedures to both systems. In particular, we recommend that
the DFA:
* identify and develop a comprehensive list of system development risks
for FACTS and FACTS Info, and:
* fully develop and implement formalized risk mitigation plans for
FACTS and FACTS Info.
Agency Comments and Our Evaluation:
State provided written comments on a draft of this report, which we
have reprinted in enclosure II. State noted the steps it has taken to
begin implementing the recommendations and expects to complete those
steps in December 2008.
In its technical comments on our draft, State officials reported that
they had acquired and implemented commercial-off-the-shelf software to
track changes to FACTS and FACTs info and updated their project
management plan to address prior weaknesses in the configuration
management. We subsequently received supporting documentation of these
activities, incorporated this information into our report, and removed
a draft recommendation concerning configuration management.
In its written comments, State also noted that FACTS does not collect
data from agencies other than State and USAID and that State does not
have agreements or procedures in place to collect detailed planning
information from other agencies. Our report acknowledges that FACTS
does not include this information. However, contractual documents for
FACTS I and FACTS II state that one of the purposes of FACTS is to
combine all U.S. government agency planning and reporting on foreign
assistance activities into one central data system. This purpose is in
accordance with one of the stated missions of the DFA to provide
leadership, coordination, and strategic direction within the U.S.
government as a whole to enhance foreign assistance effectiveness.
We are sending copies of this report to the Secretary of State, the
Administrator of USAID, and the Director of the Office of Management
and Budget. We will also make copies available at no charge on the GAO
Web site at [hyperlink, http://www.gao.gov].
If you or your staff have any questions about this report, please
contact me at (202) 512-3149 or gootnickd@gao.gov. Other key
contributors to this report were Audrey Solis, Thomas Costa, David
Powner, Karl Seifert, and George Taylor. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report.
Signed by:
David Gootnick:
Director, International Affairs and Trade:
Enclosures:
Enclosure I: List of Function 150 International Affairs Accounts under
DFA's Authority:
Congress appropriates to function 150 through the annual State
Department, Foreign Operations, and Related Programs appropriations
bill. It excludes programs within the defense budget or within the
budgets of domestic agencies.
Table 1: List of Function 150 International Affairs Accounts under
DFA's Authority:
Account acronym: ACI;
Full account name: Andean Counterdrug Initiative.
Account acronym: AEEB (formerly SEED);
Full account name: Assistance for Eastern Europe and the Baltic States
(formerly Support for Eastern European Democracy or SEED).
Account acronym: CRF (requested);
Full account name: Conflict Response Fund.
Account acronym: CSH;
Full account name: Child Survival and Health Programs.
Account acronym: DA;
Full account name: Development Assistance.
Account acronym: DCA;
Full account name: Development Credit Authority.
Account acronym: DF;
Full account name: Democracy Fund.
Account acronym: ERMA;
Full account name: U.S. Emergency Refugee and Migration Assistance.
Account acronym: ESF;
Full account name: Economic Support Fund.
Account acronym: FMF;
Full account name: Foreign Military Financing.
Account acronym: FSA;
Full account name: Assistance for the Independent States of the Former
Soviet Union.
Account acronym: GHAI[A];
Full account name: Global HIV/AIDS Initiative.
Account acronym: IDFA;
Full account name: International Disaster/Famine Assistance.
Account acronym: IMET;
Full account name: International Military Education and Training.
Account acronym: INCLE;
Full account name: International Narcotics Control and Law Enforcement.
Account acronym: IO&P;
Full account name: International Organizations and Programs.
Account acronym: MCC[A];
Full account name: Millennium Challenge Corporation.
Account acronym: MRA;
Full account name: Migration and Refugee Assistance.
Account acronym: NADR;
Full account name: Nonproliferation, Anti-Terrorism, Demining and
Related Programs.
Account acronym: PKO;
Full account name: Peacekeeping Operations.
Account acronym: OTI;
Full account name: USAID Office of Transition Initiatives.
Account acronym: PL-480 Title II;
Full account name: Public Law 480 Title II Food for Peace Program.
Source: Department of State.
[A] DFA has authority over foreign assistance programs that fall under
function 150 spending with several exceptions. Of the accounts listed
above, DFA only coordinates with GHAI or MCC and does not have approval
authority over those accounts. PL-480 programs are under the authority
of the USAID Administrator rather than the Director of Foreign
Assistance, although these positions are currently held by the same
individual.
[End of table]
[End of enclosure]
Enclosure II: Comments from the Department of State and GAO's
Evaluation:
Note: GAO comments supplementing those in the report text appear at the
end of this enclosure.
United States Department of State:
Assistant Secretary for Resource Management and Chief Financial
Officer:
Washington, D.C. 20520:
November 7, 2008:
Ms. Jacquelyn Williams-Bridgers:
Managing Director:
International Affairs and Trade:
Government Accountability Office:
441 G Street, N.W.
Washington, D.C. 20548-0001:
Dear Ms. Williams-Bridgers:
We appreciate the opportunity to review your draft report, "Foreign
Assistance: State Department Foreign Aid Information Systems Do Not
Follow Change and Risk Management Best Practices," GAO Job Code 320569.
[See comment 1]
The enclosed Department of State comments are provided for
incorporation with this letter as an appendix to the final report.
If you have any questions concerning this response, please contact
Michael Austin, Strategic Information Director, Office of the Director
of Foreign Assistance at (202) 647-2085.
Sincerely,
Signed by:
Bradford R. Higgins:
cc: GAO - Audrey Solis:
F - Henrietta H. Fore:
State/OIG - Mark Duda:
Department of State Comments on GAO Draft Report:
Foreign Assistance: State Department Foreign Aid Information Systems Do
Not Follow Change and Risk Management Best Practices:
(GAO-09-52R, GAO Code 320569):
The Department of State appreciates the opportunity to respond to the
GAO draft report "Foreign Assistance: State Department Foreign Aid
Information Systems Do Not Follow Change and Risk Management Best
Practices." We offer the following response to the GAO recommendations,
as well as updates/corrections to the text.
Recommendations:
Identify and develop a comprehensive list of system development risk
for FACTS and FACTS Info.
Fully develop and implement risk management plans for FACTS and FACTS
Info.
State Department Response:
The State Department is committed to the appropriate and effective risk
management of its information technology development projects, and has
identified and taken actions to mitigate risks throughout the
development of both FACTS and FACTS Info. Actions include the
completion of a risk assessment on FACTS I in November 2006;
preparation of a preliminary Risk Management Plan for FACTS II in
December 2007; and approval of a FACTS II risk management plan which
includes a risk register list of key FACTS II development risks in July
2008. In addition, State/F employed a professional independent
verification and validation (IV&V) contractor in June 2008 whose
responsibility was to identify and report risks to FACTS II and FACTS
Info's management, development, deployment, and preparation for steady
state operations. State/F carefully reviewed the IV&V findings and
recommendations, and took appropriate action to mitigate development
risk identified in both the IV&V report and FACTS II risk registry.
State/F is now finalizing a Program-level Risk Management Plan that
includes a risk registry listing of all risks for both FACTS and FACTS
Info identified during system development and by the IV&V. The program-
level risk management plan will include current status, impact
assessments, and mitigation strategies for risks identified in the risk
registry. The program-level risk management plan is scheduled for
completion in December 2008.
Other Comments:
Page 3, section "Results in Brief", Page 9 (sixth line and footnote):
please remove reference to "and the over 25 other USG departments and
agencies..." Reason: Our information systems do not collect data from
those agencies. The desire is for a comprehensive picture of foreign
assistance programs across the U.S. government, but we do not have
agreements or procedures in place to collect detailed planning
information on the use of appropriated funds from agencies beyond State
and USAID. USAID and State offices are encouraged to include
information on other USG agency programs in their operational plans in
a narrative section entitled "Work of USG players", but this is not
known to be comprehensive as we cannot assume that all posts have
complete information on the non-presence USG agencies. [See comment 2]
The following are GAO's comments on State's letter dated November 7,
2008.
GAO Comments:
1. During the formal comment period, State/F supplied technical
comments and additional evidence detailing improvements made to the
configuration management processes of FACTS and FACTS Info. We
incorporated this information into our report and removed a draft
recommendation concerning configuration management. Our revised title
reflects these changes.
2. Our report acknowledges that FACTS does not include information from
agencies other than State and USAID. However, contractual documents for
FACTS I and FACTS II state that one of the purposes of FACTS is to
combine all U.S. government agency planning and reporting on foreign
assistance activities into one central data system. This purpose is in
accordance with one of the stated missions of the DFA to provide
leadership, coordination, and strategic direction within the U.S.
government to enhance foreign assistance effectiveness.
[End of enclosure]
Footnotes:
[1] Neither FACTS nor FACTS Info is intended to serve as an accounting
or financial tracking system. Both State and USAID have separate
systems for these purposes.
[2] State and USAID separately maintain their respective operating
expense accounts--the Diplomatic and Consular Program account and the
Operating Expense account--which are not under the authority of DFA.
[3] These six countries represent a judgmental sample selected because
of the wide variety and number of foreign assistance programs at each
post and as part of our ongoing work addressing the management of
State/F.
[4] Strategic Plan Fiscal Years 2007-2012: Transformational Diplomacy,
U.S. Department of State and U.S. Agency for International Development,
revised May 7, 2007.
[5] Carnegie Mellon University's Software Engineering Institute is
recognized for its expertise in software and system processes and has
developed the Capability Maturity Model R Integration (CMMI SM) model
and a CMMI appraisal methodology to evaluate, improve, and manage
system and software development and engineering processes.
[6] These activities are derived from the Software Engineering
Institute's CMMI.
[7] These activities are derived from the Software Engineering
Institute's CMMI.
[8] In 2008, State/F expanded the number of operating units inputting
data into FACTS to 187.
[9] The operational plans are intended to provide a comprehensive
picture of all foreign assistance resources--even those that do not
fall under the authority of DFA and not currently captured in FACTS--
planned for implementation in a country by providing detail about the
specific use of appropriated funds for the current fiscal year.
[10] State/F also spent $210,000 to have a separate contractor develop
the technical requirements of both FACTS and FACTS Info.
[11] As an in-house system, the cost of FACTS Info development is
included within staff and other operational expenses, but State
estimated the lifecycle cost is $2.5 million.
[12] The foreign assistance budget formulation process is used to
develop the annual President's budget request as well as supplemental
information and details for certain foreign assistance programs.
Foreign assistance program dollars for both State and USAID, as well as
USAID's operating expenses, are contained in the DFA's foreign
operations congressional budget justification; State's operating
expenses--including those that support State/F--are instead contained
in the State Department congressional budget justification.
[13] The FACTS II and FACTS Info IV&V Initial Assessment, an
independent verification and validation report dated June 25, 2008, had
three objectives: (1) to verify and validate that FACTS II was being
designed successfully, (2) to determine the probability of a successful
implementation of FACTS II, and (3) to determine whether FACTS II would
meet State's objectives. We found the assessment's methodology sound.
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
