Department of State
Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel Gao ID: GAO-11-438T March 8, 2011This testimony discusses the U.S. Department of State's (State) efforts to train its personnel. It is based on our report, which is being released today. Because State is the lead U.S. foreign affairs agency, its personnel require certain knowledge, skills, and abilities to equip them to address the global security threats and challenges facing the United States--including the threat of Al-Qaeda and other terrorist organizations, HIV/AIDS and other pandemics, environmental degradation, nuclear proliferation, and failed states. In fiscal years 2006 through 2010, State's funding for training personnel grew by about 62 percent, and the department requested more than $266 million in fiscal year 2011 for programs providing training in professional skills such as foreign language proficiency, area studies, information technology, consular duties, and others needed for the conduct of foreign relations. State's Foreign Service Institute (FSI) is the primary training provider for the department's more than 66,000 Foreign Service, civil service, and locally employed staff worldwide. Our prior work has identified staffing and foreign language shortfalls at State, including challenges the department has faced in filling positions at the mid-level in particular, and in attracting qualified personnel for some hardship posts. The department is currently in the midst of what it has called the most challenging military-to-civilian transition in U.S. history in Iraq, one of the posts of greatest hardship where State personnel serve. Recent departmental initiatives--in particular, "Diplomacy 3.0," a multiyear effort launched in March 2009 with a primary aim of increasing the size of State's Foreign Service by 25 percent and the civil service by 13 percent--have underscored the importance of training to equip personnel to fulfill State's leadership role in world affairs and to advance and defend U.S. interests abroad. Today's testimony will discuss State's purpose and structure for training personnel, including leadership, management, professional, and area studies training, contributing to diplomatic readiness of State's Foreign Service and civil service personnel and locally employed staff overseas. It will also discuss the extent to which State's personnel training incorporates elements of effective federal training programs.
We found that State has taken many steps to incorporate the interrelated elements of an effective training program--planning, design, implementation, and evaluation--into its training for personnel, but the department's strategic approach to workforce training could be improved in several key areas. Specifically, we identified five areas where State can improve its training. First, State lacks a comprehensive training needs assessment process incorporating all bureaus and posts. Second, State developed guidance for employees about training opportunities, career paths, and how training can help employees attain career goals, but the guidance does not provide complete and accurate information. Third, State lacks a data collection and analysis plan for evaluating training, and thus cannot be assured that proper practices and procedures are systematically and comprehensively applied. Fourth, State could not sufficiently demonstrate consistent and appropriate support for training, because the department does not track detailed information on training cost and delivery that would allow for an analysis and comparison of employees in different groups, bureaus, regions, or posts. Lastly, State performance measures for training generally do not fully address training goals, and are generally output-rather than outcome-oriented.
GAO-11-438T, Department of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel
This is the accessible text file for GAO report number GAO-11-438T
entitled 'Department Of State: Additional Steps Are Needed to Improve
Strategic Planning and Evaluation of Training for State Personnel'
which was released on March 8, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Testimony:
Before the Subcommittee on Oversight of Government Management, the
Federal Workforce, and the District of Columbia, Committee on Homeland
Security and Governmental Affairs, U.S. Senate:
For Release on Delivery:
Expected at 10:00 a.m. EST:
Tuesday, March 8, 2011:
Department Of State:
Additional Steps Are Needed to Improve Strategic Planning and
Evaluation of Training for State Personnel:
Statement of Jess T. Ford, Director:
International Affairs and Trade:
GAO-11-438T:
Mr. Chairman and Members of the Subcommittee:
I am pleased to be here today to discuss the U.S. Department of
State's (State) efforts to train its personnel. My testimony is based
on our report, which is being released today.[Footnote 1] Because
State is the lead U.S. foreign affairs agency, its personnel require
certain knowledge, skills, and abilities to equip them to address the
global security threats and challenges facing the United States--
including the threat of Al-Qaeda and other terrorist organizations,
HIV/AIDS and other pandemics, environmental degradation, nuclear
proliferation, and failed states. In fiscal years 2006 through 2010,
State's funding for training personnel grew by about 62 percent, and
the department requested more than $266 million in fiscal year 2011
for programs providing training in professional skills such as foreign
language proficiency, area studies, information technology, consular
duties, and others needed for the conduct of foreign relations.
[Footnote 2] State's Foreign Service Institute (FSI) is the primary
training provider for the department's more than 66,000 Foreign
Service, civil service, and locally employed staff worldwide.[Footnote
3]
Our prior work has identified staffing and foreign language shortfalls
at State, including challenges the department has faced in filling
positions at the mid-level in particular, and in attracting qualified
personnel for some hardship posts.[Footnote 4] The department is
currently in the midst of what it has called the most challenging
military-to-civilian transition in U.S. history in Iraq, one of the
posts of greatest hardship where State personnel serve. Recent
departmental initiatives--in particular, "Diplomacy 3.0," a multiyear
effort launched in March 2009 with a primary aim of increasing the
size of State's Foreign Service by 25 percent and the civil service by
13 percent--have underscored the importance of training to equip
personnel to fulfill State's leadership role in world affairs and to
advance and defend U.S. interests abroad.
Today I will discuss State's purpose and structure for training
personnel, including leadership, management, professional, and area
studies training, contributing to diplomatic readiness of State's
Foreign Service and civil service personnel and locally employed staff
overseas. I will also discuss the extent to which State's personnel
training incorporates elements of effective federal training programs.
[Footnote 5]
Over the course of our work on this issue, we reviewed and analyzed
data and documentation related to State's training efforts, such as
strategic and workforce planning documents, funding data, and data on
personnel participation in training, as well as legislative,
regulatory, and State policy and procedural criteria relevant to
training. In addition, we reviewed training evaluation mechanisms used
by FSI. We interviewed key officials from 26 State bureaus and offices
in Washington, D.C., including FSI, the Bureau of Human Resources, and
the six geographic bureaus. We also conducted semistructured telephone
interviews with State officials with training-related responsibilities
at 12 overseas missions, and interviewed officials from State's
regional training centers located in Bangkok, Thailand; Ft.
Lauderdale, Florida; and Frankfurt, Germany. With input from State, we
completed a training assessment to determine the extent to which the
department's personnel training incorporates elements of effective
training programs. We used the results of this assessment to identify
any gaps in State's training based on criteria identified in GAO, the
Office of Personnel Management (OPM), State, and other legislative and
regulatory guidance and policy. In addition, in light of work that we
recently published on shortfalls in State personnel's foreign language
skills[Footnote 6], we did not focus on language training. We also did
not include within our scope an assessment of "hard skills" (e.g.,
security and law enforcement) training provided by State's Bureau of
Diplomatic Security. We conducted this performance audit from July
2009 to January 2011 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform
the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a
reasonable basis for our findings and conclusions based on our audit
objectives. More information on our scope and methodology and detailed
findings is available in the full report.[Footnote 7]
In brief, Mr. Chairman, we found that State has taken many steps to
incorporate the interrelated elements of an effective training
program--planning, design, implementation, and evaluation--into its
training for personnel,[Footnote 8] but the department's strategic
approach to workforce training could be improved in several key areas.
Specifically, we identified five areas where State can improve its
training. First, State lacks a comprehensive training needs assessment
process incorporating all bureaus and posts. Second, State developed
guidance for employees about training opportunities, career paths, and
how training can help employees attain career goals, but the guidance
does not provide complete and accurate information. Third, State lacks
a data collection and analysis plan for evaluating training, and thus
cannot be assured that proper practices and procedures are
systematically and comprehensively applied. Fourth, State could not
sufficiently demonstrate consistent and appropriate support for
training, because the department does not track detailed information
on training cost and delivery that would allow for an analysis and
comparison of employees in different groups, bureaus, regions, or
posts. Lastly, State performance measures for training generally do
not fully address training goals, and are generally output-rather than
outcome-oriented.
State's Annual Training Plan states that "the purpose of the
department's training program is to develop the men and women our
nation requires to fulfill our leadership role in world affairs and to
advance and defend U.S. interests." FSI is State's primary training
provider, offering entry-, mid-, and senior-level training for
employees as they progress through their careers. State guidance
outlines key training roles, including FSI's primary role in
developing training policies and facilitating necessary training, and
the Bureau of Human Resource's role in assigning employees to training
and working with FSI to help ensure it meets their needs. Other
bureaus, offices, and posts also share responsibilities for training.
FSI offers over 700 classroom courses, and has recently increased its
focus on distance learning. We found that overall, about 40 percent of
personnel training over the last 5 fiscal years, on average, was in
foreign language skills. Other training for personnel generally
focused on developing leadership, management, and other professional
and technical skills and knowledge.
State's personnel training reflects numerous aspects of effective
training programs, based on our assessment using the criteria GAO
previously identified. For example, State maintains a workforce
training plan, as required by federal regulations.[Footnote 9] FSI
leads efforts to prepare the plan annually; the plan is linked to
State's overall strategic plan, and presents a business case for
proposed training investments. FSI also publishes an annual schedule
of courses, which provides information for employees on FSI classroom
and distance learning course offerings. The schedule of courses
generally includes information for each course such as a brief
description, any prerequisites, course objectives, and relevant
competencies. As another example of a positive practice, State has
made an effort to use advances in technologies to enhance training
efforts. The number of distance learning offerings, as well as
employee participation in distance learning, has increased in recent
years. For example, State's latest annual training plan reported that
FSI developed 20 new custom distance learning courses during the prior
year, and data showed time spent by personnel completing distance
learning courses more than doubled from fiscal years 2006 through
2009--from about 113,000 hours in fiscal year 2006 to about 254,000 in
fiscal year 2009. In addition, we found that State has a range of
training evaluation mechanisms in place, including mid-and post-
training course evaluations. Since 2006, FSI has conducted an annual
training survey. FSI reported most respondents to the 2010 survey
were, in general, satisfied or very satisfied with training.[Footnote
10]
However, although State has developed an extensive program for
training personnel, our analysis found several gaps in the
department's efforts to strategically plan and prioritize training,
ensure efficient and effective training design and delivery, and
determine whether or how training and development efforts contribute
to improved performance and desired results. Each of the issue areas
we identified broadly relates to multiple elements, attributes, and
indicators throughout the interrelated training and development
process. While an agency's training program is not necessarily
expected to address every indicator identified in the GAO guidance,
based on our assessment, we identified strategic weaknesses related to
these areas as particularly important to ensuring effective planning,
design, implementation, and evaluation of personnel training.
For example, we found that:
* State lacks a systematic, comprehensive training needs assessment
process incorporating all bureaus and overseas posts. Since 2007,
State human resource reports noted that bureaus have not formally
conducted annual training needs assessments, and identified this as an
issue that should be addressed to help provide a realistic basis for
planning, budgeting, and directing training. According to the reports,
the Bureau of Human Resources intended to form an interoffice working
group to develop a comprehensive plan and implementation guidance to
support a department-wide effort for assessing training needs.
However, State had not yet formed an interoffice working group as of
November 2010.
* State developed guidance--known as training continuums--to provide
information for employees about training opportunities, career ladders
and paths, and how training can help employees attain career goals,
but the guidance documents do not provide complete and accurate
information for employees. While the documents state that they were
designed to provide a broad overview of appropriate training that
should be considered as employees plan their careers in the
department, including information on mandatory, recommended, and
suggested courses, we found issues that raised questions about their
usefulness and reliability as employee resources. For example, we
found that specific training requirements designated by bureaus and
posts for certain groups of employees are not always identified in the
training guidance. A key official from FSI's executive office
acknowledged that the guidance documents do not include complete and
accurate information for employees on training, and noted that the
documents have not been reviewed to ensure they uniformly reflect
departmental policies or standards.
* State has not developed a data collection and analysis plan for
evaluating training, which could help ensure that appropriate
procedures and criteria for evaluating training are systematically
applied across the board. As a result, it is not clear whether or how
State systematically makes decisions regarding how training programs
will be evaluated using different methods or tools, or how results
will be used. Our prior work highlights the importance of planning and
conducting evaluations of the effectiveness of training and
development efforts and notes that a data collection and analysis plan
can set priorities for evaluations and systematically cover the
methods, timing, and responsibilities for an agency's data collection.
[Footnote 11] While State has implemented mechanisms to evaluate
training, including course evaluations and an annual training survey,
these mechanisms do not fully incorporate locally employed staff, and
primarily focus on participant satisfaction or reaction to training,
rather than desired results, such as improved quality or efficiency of
work.
* State could not sufficiently demonstrate consistent and appropriate
support for training, because it does not track detailed data and
information on training cost and delivery that would allow for an
analysis and comparison of employees in different employee groups,
bureaus, regions, and posts. For example, State could not provide data
on the percentage of foreign affairs or political officers that had
completed required, recommended, or suggested training for their areas
of work. Although State tracks some data related to training funding
and delivery, the department does not have sufficient information that
could be used to ensure consistent and appropriate support for
training, or to help determine whether managers and employees have
needed training tools and resources. This is especially important
given evidence of variances in training across the department. For
example, while training officials we interviewed at some bureaus and
posts indicated they had sufficient funding and support for training,
others noted they faced significant resource challenges that impacted
the ability of employees to get necessary training.
* State has developed several training-related goals and measures, but
the measures do not fully address the goals, and are generally output
rather than outcome oriented. As a result, they do not provide a clear
means of determining whether State's training efforts achieve desired
results. For example, one training goal listed in FSI's fiscal year
2012 strategic resource plan, "workforce meets priority diplomatic and
operational requirements as a result of FSI training," includes
priorities and objectives to expand and enhance language training,
support training in stability operations, support new hire training,
and enhance public diplomacy training. However, the goal's two
measures, "language training success rate at FSI," and "development of
training continuum to support State's Office of Reconstruction and
Stabilization," are both output measures and do not fully address the
priority areas for the goal, such as support for new hire training or
public diplomacy training.
State's budget and focus on training have increased in recent years,
but the department has also faced, and will likely continue to face,
fluctuating and constrained resources and competing priorities when
determining what training is critical to its mission. Without
concerted efforts to further incorporate effective practices, State
cannot ensure training resources are targeted strategically, are not
wasted, and achieve cost-effective and timely results desired, and
thus cannot be assured that its employees are trained and equipped to
meet the challenges of their mission.
Our report being issued today includes several recommendations for the
Secretary of State to improve strategic planning and evaluation of the
department's efforts to train personnel, including for improvements to
State's efforts to assess training needs and efforts to ensure
training achieves desired results. State generally agreed with our
findings and recommendations.
Mr. Chairman, this concludes my prepared statement. I would be pleased
to respond to any questions you or other Members of the Subcommittee
may have at this time.
GAO Contact and Staff Acknowledgments:
For questions regarding this testimony, please contact Jess T. Ford,
(202)512-4268 or fordj@gao.gov. In addition to the contact named
above, Anthony Moran, Assistant Director; Lisa Helmer; Shirley Min;
Joe Carney; Virginia Chanley; Kieran Cusack; David Dayton; Patrick
Lockett; Reid Lowe; and Mary Moutsos provided significant
contributions to the work. Contact points for our offices of
Congressional Relations and Public Affairs may be found on the last
page of this testimony.
[End of section]
Footnotes:
[1] GAO, State Department: Additional Steps Are Needed to Improve
Strategic Planning and Evaluation of Training for State Personnel,
[hyperlink, http://www.gao.gov/products/GAO-11-241] (Washington, D.C.:
January 2011).
[2] According to State, the total number of Foreign Service, civil
service, and locally employed personnel increased from about 57,000 in
September 2006 to more than 66,000 as of September 2010, an increase
of about 17 percent.
[3] State's locally employed staff include foreign nationals and U.S.
citizen residents employed via direct-hire appointments, personal
services agreements, or personal services contracts.
[4] See GAO, State Department: Persistent Staffing and Foreign
Language Gaps Compromise Diplomatic Readiness, [hyperlink,
http://www.gao.gov/products/GAO-09-1046T] (Washington, D.C.: September
24, 2009); and State Department: Staffing and Foreign Language
Shortfalls Persist Despite Initiatives to Address Gaps, [hyperlink,
http://www.gao.gov/products/GAO-07-1154T] (Washington, D.C.: August 1,
2007).
[5] We previously developed guidance for assessing federal strategic
training and development efforts, including identifying four essential
and interrelated elements of the training and development process: (1)
planning, (2) design, (3) implementation, and (4) evaluation. The
guidance includes key attributes of effective federal training
programs to consider when assessing each of the four elements, along
with indicators related to each attribute. This guidance can be used
to identify potential gaps or areas where improvements may be made to
help ensure that training and development investments are targeted
strategically and not wasted on efforts that are irrelevant,
duplicative, or ineffective. GAO, Human Capital: A Guide for Assessing
Strategic Training and Development Efforts in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington,
D.C.: Mar. 1, 2004).
[6] GAO, Department of State: Comprehensive Plan Needed to Address
Persistent Foreign Language Shortfalls, [hyperlink,
http://www.gao.gov/products/GAO-09-955] (Washington, D.C.: Sept. 17,
2009). Our latest report notes that according to State, the department
has taken several steps to address prior GAO recommendations related
to language training needs and challenges, such as developing an
analytical model to better assess resource needs, including training,
to meet language requirements, and implementing mechanisms to ensure a
strategic approach to addressing foreign language needs.
[7] [hyperlink, http://www.gao.gov/products/GAO-11-241].
[8] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[9] 5 C.F.R. § 410.201.
[10] According to State, the 2010 annual training survey was sent to a
random sample of 5,105 Foreign Service and civil service employees, as
well as eligible family members. Among other things, the survey asked
respondents to rate FSI's training delivery methods, training
programs, and customer service. We determined that the results of this
survey were sufficiently reliable to provide a general indication of
employee satisfaction with training.
[11] Such a plan could also include guidelines to help ensure the
agency makes an ongoing effort to improve the quality and breadth of
data gathered. Our prior work also noted that developing and using
such a plan can guide an agency in a systematic approach to assessing
the effectiveness and efficiency of both specific training and
development programs and more comprehensively assessing its entire
training and development effort. [hyperlink,
http://www.gao.gov/products/GAO-04-546G].
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
