Department of State
Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel Gao ID: GAO-11-241 January 25, 2011Because the U.S. Department of State (State) is the lead U.S. foreign affairs agency, its personnel require certain knowledge, skills, and abilities to address the global challenges and security threats facing the United States. State devoted about $255 million to personnel training in fiscal year 2010; the department's Foreign Service Institute (FSI) is the primary training provider for State's more than 66,000 Foreign Service, civil service, and locally employed staff (LE staff) worldwide. GAO was asked to examine (1) State's purpose and structure for training personnel and (2) the extent to which State's training incorporates elements for effective training programs. GAO reviewed and analyzed data and documentation related to the agency's training efforts; completed a training assessment using a tool developed based on prior GAO guidance; and interviewed officials in Washington, D.C., and at 12 overseas posts.
State's purpose for training personnel is to develop the men and women the United States requires to fulfill its leadership role in world affairs and to advance and defend U.S. interests. State guidance outlines key training roles, including FSI's primary role in developing training policies and facilitating necessary training, and the Bureau of Human Resources' role in assigning employees to training and working with FSI to help ensure it meets their needs. Other bureaus, offices, and posts also share responsibilities for training. FSI currently offers more than 700 classroom courses, and has recently increased its focus on distance learning. Overall, about 40 percent of personnel training over the last 5 fiscal years, on average, was in foreign language skills. Other training for personnel generally focused on developing leadership, management, and other professional and technical skills and knowledge. State has taken many steps to incorporate the interrelated elements of an effective training program--planning, design, implementation, and evaluation--into its extensive training for personnel; however, the department's strategic approach to workforce training has several key weaknesses. The department demonstrated a variety of ways in which it has endeavored to develop an effective training program, such as by compiling an annual training plan, and implementing a range of training evaluation mechanisms and a learning management system that can be used to track training delivery. However, GAO's analysis found several gaps in the department's efforts to strategically plan and prioritize training, ensure efficient and effective training design and delivery, and determine whether or how training and development efforts contribute to improved performance and desired results. For example: (1) State lacks a systematic, comprehensive training needs assessment process incorporating all bureaus and overseas posts. (2) State developed training continuums to provide information for employees about training opportunities, career paths, and how training can help employees attain career goals, but the continuums do not provide complete and accurate information, and other guidance does not cover all personnel. (3) State lacks formal guidance for curriculum design and for data collection and analysis, and thus cannot be assured that proper practices and procedures are systematically and comprehensively applied. (4) State could not sufficiently demonstrate consistent and appropriate support for training, because the department does not track detailed information on training cost and delivery that would allow for an analysis and comparison of employees in different groups, bureaus, regions, or posts. (5) State's performance measures for training generally do not fully address training goals, and are generally output- rather than outcome-oriented. GAO is making several recommendations for State to improve strategic planning and evaluation of the department's efforts to train personnel, including for improvements to State's efforts to assess training needs and efforts to ensure training achieves desired results. State reviewed a draft of this report and generally agreed with our recommendations.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Jess T. Ford Team: Government Accountability Office: International Affairs and Trade Phone: (202) 512-4268GAO-11-241, Department of State: Additional Steps Are Needed to Improve Strategic Planning and Evaluation of Training for State Personnel
This is the accessible text file for GAO report number GAO-11-241
entitled 'Department Of State: Additional Steps Are Needed to Improve
Strategic Planning and Evaluation of Training for State Personnel'
which was released on March 8, 2011.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
United States Government Accountability Office:
GAO:
Report to the Honorable Daniel K. Akaka, U.S. Senate:
January 2011:
Department Of State:
Additional Steps Are Needed to Improve Strategic Planning and
Evaluation of Training for State Personnel:
GAO-11-241:
GAO Highlights:
Highlights of GAO-11-241, a report to the Honorable Daniel K. Akaka,
U.S. Senate.
Why GAO Did This Study:
Because the U.S. Department of State (State) is the lead U.S. foreign
affairs agency, its personnel require certain knowledge, skills, and
abilities to address the global challenges and security threats facing
the United States. State devoted about $255 million to personnel
training in fiscal year 2010; the department‘s Foreign Service
Institute (FSI) is the primary training provider for State‘s more than
66,000 Foreign Service, civil service, and locally employed staff (LE
staff) worldwide. GAO was asked to examine (1) State‘s purpose and
structure for training personnel and (2) the extent to which State‘s
training incorporates elements for effective training programs. GAO
reviewed and analyzed data and documentation related to the agency‘s
training efforts; completed a training assessment using a tool
developed based on prior GAO guidance; and interviewed officials in
Washington, D.C., and at 12 overseas posts.
What GAO Found:
State‘s purpose for training personnel is to develop the men and women
the United States requires to fulfill its leadership role in world
affairs and to advance and defend U.S. interests. State guidance
outlines key training roles, including FSI‘s primary role in
developing training policies and facilitating necessary training, and
the Bureau of Human Resources‘ role in assigning employees to training
and working with FSI to help ensure it meets their needs. Other
bureaus, offices, and posts also share responsibilities for training.
FSI currently offers more than 700 classroom courses, and has recently
increased its focus on distance learning. Overall, about 40 percent of
personnel training over the last 5 fiscal years, on average, was in
foreign language skills. Other training for personnel generally
focused on developing leadership, management, and other professional
and technical skills and knowledge.
State has taken many steps to incorporate the interrelated elements of
an effective training program”planning, design, implementation, and
evaluation”into its extensive training for personnel; however, the
department‘s strategic approach to workforce training has several key
weaknesses. The department demonstrated a variety of ways in which it
has endeavored to develop an effective training program, such as by
compiling an annual training plan, and implementing a range of
training evaluation mechanisms and a learning management system that
can be used to track training delivery. However, GAO‘s analysis found
several gaps in the department‘s efforts to strategically plan and
prioritize training, ensure efficient and effective training design
and delivery, and determine whether or how training and development
efforts contribute to improved performance and desired results. For
example:
* State lacks a systematic, comprehensive training needs assessment
process incorporating all bureaus and overseas posts.
* State developed training continuums to provide information for
employees about training opportunities, career paths, and how training
can help employees attain career goals, but the continuums do not
provide complete and accurate information, and other guidance does not
cover all personnel.
* State lacks formal guidance for curriculum design and for data
collection and analysis, and thus cannot be assured that proper
practices and procedures are systematically and comprehensively
applied.
* State could not sufficiently demonstrate consistent and appropriate
support for training, because the department does not track detailed
information on training cost and delivery that would allow for an
analysis and comparison of employees in different groups, bureaus,
regions, or posts.
* State‘s performance measures for training generally do not fully
address training goals, and are generally output- rather than outcome-
oriented.
What GAO Recommends:
GAO is making several recommendations for State to improve strategic
planning and evaluation of the department‘s efforts to train
personnel, including for improvements to State‘s efforts to assess
training needs and efforts to ensure training achieves desired
results. State reviewed a draft of this report and generally agreed
with our recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-11-241[ or key
components. For more information, contact Jess T. Ford at (202) 512-
4268 or fordj@gao.gov.
[End of section]
Contents:
Letter:
Background:
State Has Developed an Extensive Training Program in Support of Its
Mission, Primarily through the Foreign Service Institute:
State Workforce Training Incorporates Many Aspects of Effective
Training Programs, but Strategic Weaknesses Exist:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Objectives, Scope, and Methodology:
Appendix II: Attributes for Review of an Agency Training and
Development Program:
Appendix III: State Organization:
Appendix IV: Comments from the Department of State:
Appendix V: GAO Contact and Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: Selected Attributes and Supporting Indicators for Elements of
Effective Federal Training Efforts:
Table 2: Key Responsibilities for Training State Personnel According
to State‘s Foreign Affairs Manual:
Table 3: Primary Functions of FSI Schools:
Table 4: Key Strategic Weaknesses in State‘s Efforts to Train
Personnel:
Table 5: Funding for State Personnel Training, Fiscal Years 2006-2011,
Not Adjusted for Inflation:
Table 6: Completion of Required Leadership Skills Training, as of May
2010:
Table 7: FSI and Bureau of Human Resources Training-related Goals and
Measures, with GAO Assessments:
Figures:
Figure 1: Approximate Distribution of State‘s Workforce by Employment
Category, as of September 30, 2010:
Figure 2: Distribution of Overseas v. Domestic Locations of State
Foreign Service and Civil Service Employees:
Figure 3: State‘s Actual and Projected Foreign and Civil Service
Personnel Levels, Fiscal Years 2006-2014:
Figure 4: General Relationships between Elements of Federal Training
and Development Efforts:
Figure 5: FSI‘s Organization:
Figure 6: FSI Non-Language Classroom Course Offerings by Location,
Fiscal Year 2010:
Figure 7: Participation in Classroom, Distance Learning, and External
Language and Non-language Training by All State Employees, Fiscal
Years 2006-2010:
Figure 8: Participation in Classroom, Distance Learning, and External
Training by Employee Category, Fiscal Years 2006-2010:
Abbreviations:
CDO: Career Development Officer:
FSI: Foreign Service Institute:
IDP: individual development plan:
LE staff: locally employed staff:
OIG: Office of Inspector General:
OPM: Office of Personnel Management:
State: Department of State:
WDP: work development plan:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
January 25, 2011:
The Honorable Daniel K. Akaka:
United States Senate:
Dear Senator Akaka,
As the lead department for U.S. foreign affairs, the Department of
State (State) plays the primary role in developing and implementing
U.S. foreign policy. In support of that role, State's personnel
require certain knowledge, skills, and abilities to equip them to
address the complex security threats and challenges of global
interdependence that accompany 21st century diplomacy. Recent
departmental initiatives--in particular, "Diplomacy 3.0," a multiyear
effort launched in March 2009 with a primary aim of increasing the
size of State's Foreign Service by 25 percent and the civil service by
13 percent--have underscored the importance of training to equip
personnel to fulfill State's leadership role in world affairs and to
advance and defend U.S. interests abroad. Nongovernmental
organizations such as the American Academy of Diplomacy have confirmed
that challenges facing the United States--including the threat of Al-
Qaeda and other terrorist organizations, HIV/AIDS and other pandemics,
environmental degradation, nuclear proliferation, and failed states--
require a significantly more robust foreign affairs capacity featuring
skilled professionals. In fiscal years 2006 through 2010, State's
funding for training personnel grew by about 62 percent, and the
department requested more than $266 million in fiscal year 2011 for
programs providing training in professional skills such as foreign
language proficiency, area studies, information technology, consular
duties, and others needed for the conduct of foreign relations.
[Footnote 1] State's Foreign Service Institute (FSI) is the primary
training provider for the department's more than 66,000 Foreign
Service, civil service, and locally employed staff (LE staff)
worldwide[Footnote 2]
In response to your request for information about State's training of
its personnel, we examined (1) State's purpose and structure for
training personnel, and (2) the extent to which State's personnel
training incorporates elements of effective federal training programs.
[Footnote 3]
Our analysis focuses primarily on the training that FSI provides,
including leadership, management, professional, and area studies
training, contributing to diplomatic readiness of State's Foreign
Service and civil service personnel and LE staff overseas. In
addition, in light of work that we recently published on shortfalls in
State personnel's foreign language skills,[Footnote 4] this report
does not focus on language training. This report does not include
within its scope an assessment of "hard skills" (e.g., security and
law enforcement) training provided by State's Bureau of Diplomatic
Security.
To address our objectives, we reviewed and analyzed data and
documentation related to State's training efforts, such as strategic
and workforce planning documents, information and data on recent FSI
course offerings, data on personnel participation in training for
fiscal years 2006 to 2010, and overall funding for training during
that time period. We also reviewed legislative, regulatory, and State
policy and procedural criteria relevant to training. In addition, we
reviewed training evaluation mechanisms used by each of the four FSI
schools--the Leadership and Management School, School of Applied
Information Technology, School of Language Studies, and School of
Professional and Area Studies--as well as within each of 10 divisions
of the School of Professional and Area Studies[Footnote 5]. We
analyzed responses to training-related questions included in the
American Foreign Service Association's 2009 survey of State Foreign
Service members, including more than 1,000 responses to an open-ended
question regarding whether and how State training could be improved.
[Footnote 6] While the results of this survey are not generalizeable,
they provided valuable insights into potential areas for improvement.
We interviewed key officials from nongovernmental organizations
including the Center for Strategic and International Studies and the
Stimson Center, as well as from 26 State bureaus and offices in
Washington, D.C., including FSI, the Bureau of Human Resources, and
the six geographic bureaus. We conducted semistructured telephone
interviews with State officials with training-related responsibilities
at 12 overseas missions. We selected a nongeneralizeable sample of
countries designed to ensure geographic diversity; our criteria for
selection also included factors such as the size of the post and
hardship differential. We also interviewed officials from State's
regional training centers located in Bangkok, Thailand; Ft.
Lauderdale, Florida; and Frankfurt, Germany. With input from State, we
completed a training assessment to determine the extent to which the
department's personnel training incorporates elements of effective
training programs--planning, design, implementation, and
evaluation.[Footnote 7] We used the results of this assessment to
identify any gaps in State's training based on criteria identified in
GAO, the Office of Personnel Management (OPM), State, and other
legislative and regulatory guidance and policy. Appendix I contains
additional details about our scope and methodology.
We conducted this performance audit from July 2009 to January 2011 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
Background:
State defines its mission as advancing freedom by helping build and
sustain a more democratic, secure, and prosperous world composed of
well-governed states that respond to the needs of their people, reduce
widespread poverty, and act responsibly within the international
system. To achieve this mission, State relies on more than 66,000
Foreign Service, civil service, and LE staff at its headquarters in
Washington, D.C., and serving at 271 missions[Footnote 8] in 189
countries worldwide. State's training and development program supports
its strategic goal of strengthening consular and management
capabilities and attempts to develop and maintain a workforce
qualified to achieve its mission.
Distribution and Numbers of State Employees:
Nearly a third of State's workforce are Foreign Service and civil
service direct hires, and over half of State's workforce are LE staff.
The remaining workforce consists of domestic contractors and temporary
personnel. See figure 1.
Figure 1: Approximate Distribution of State's Workforce by Employment
Category, as of September 30, 2010:
[Refer to PDF for image: pie-chart]
Locally employed staff: 56%;
Foreign Service: 17%;
Civil service: 13%;
Domestic contractors: 9%;
Government temporary: 5%.
Source: GAO analysis of State data.
Note: Figures for domestic contractors and government temporary
workers are estimates.
[End of figure]
According to State's Five Year Workforce and Leadership Succession
Plan, about two-thirds of State's Foreign Service employees are
assigned to overseas posts and the remaining one-third are employed
domestically. Almost all of State's civil service employees are
assigned at domestic locations.[Footnote 9] See figure 2.
Figure 2: Distribution of Overseas v. Domestic Locations of State
Foreign Service and Civil Service Employees:
[Refer to PDF for image: stacked vertical bar graph]
Foreign service:
Domestic: 36%;
Overseas: 64%.
Civil service:
Domestic: 100%.
Total:
Domestic: 63%;
Overseas: 37%.
Source: State‘s Five Year Workforce and Leadership Succession Plan for
Fiscal Years 2010-2014.
[End of figure]
During fiscal years 2006 through 2010, State has increased its Foreign
Service and civil service workforce by about 17 percent, setting
priority on filling personnel shortages created in part by demands in
Iraq and Afghanistan. In March 2009, State announced plans to increase
its Foreign Service workforce by 25 percent and the civil service
workforce by 13 percent by fiscal year 2014 as part of the Diplomacy
3.0 initiative. By September 30, 2010, State had hired more than 1,900
Foreign Service and civil service employees in new positions[Footnote
10] and planned to fill approximately 2,000 additional new positions
through 2014 to address projected needs.
Figure 3 shows State's actual and projected numbers of Foreign Service
and civil service personnel in fiscal years 2006 through 2014.
Figure 3: State's Actual and Projected Foreign and Civil Service
Personnel Levels, Fiscal Years 2006-2014:
[Refer to PDF for image: stacked vertical bar graph]
Fiscal year: 2006;
Foreign service base level: 11,397;
Civil service base level: 8,270;
Foreign service new hires: 0;
Civil service new hires: 0.
Fiscal year: 2007;
Foreign service base level: 11,467;
Civil service base level: 8,784;
Foreign service new hires: 0;
Civil service new hires: 0.
Fiscal year: 2008;
Foreign service base level: 11,656;
Civil service base level: 9,328;
Foreign service new hires: 0;
Civil service new hires: 0.
Fiscal year: 2009;
Foreign service base level: 12,257;
Civil service base level: 9,614;
Foreign service new hires: 0;
Civil service new hires: 0.
Fiscal year: 2010;
Foreign service base level: 13,008;
Civil service base level: 10,039;
Foreign service new hires: 0;
Civil service new hires: 0.
Fiscal year: 2011;
Foreign service base level: 13,008;
Civil service base level: 10,039;
Foreign service new hires: 430;
Civil service new hires: 189
Fiscal year: 2012;
Foreign service base level: 13,438;
Civil service base level: 10,228;
Foreign service new hires: 410;
Civil service new hires: 189.
Fiscal year: 2013;
Foreign service base level: 13,848;
Civil service base level: 10,417;
Foreign service new hires: 402;
Civil service new hires: 189.
Fiscal year: 2014;
Foreign service base level: 14,250;
Civil service base level: 10,606;
Foreign service new hires: 82;
Civil service new hires: 40.
Source: GAO analysis of State personnel data, including State‘s
projections for fiscal years 2011-2014.
Notes:
State's projections for hiring do not include estimates for LE staff.
State's projections for new hires are as follows: FY2011: 430 Foreign
Service (FS) and 189 civil service (CS); FY2012: 410 FS and 189 CS;
FY2013: 402 FS and 189 CS; FY2014: 82 FS and 40 CS.
[End of figure]
State's Organization:
Each of State's six geographic bureaus--the Bureaus of African
Affairs, East Asian and Pacific Affairs, European and Eurasian
Affairs, Near Eastern Affairs, South and Central Asian Affairs, and
Western Hemisphere Affairs--coordinates the conduct of U.S. foreign
relations concerning a specific region of the world. Functional
bureaus, such as the Bureau of Economic, Energy, and Business Affairs
and Bureau of Political-Military Affairs, generally manage and
coordinate specific issues and activities within the department. In
addition, various offices report to the Secretary's office, including
the Office of the Inspector General and Office of Intelligence and
Research. See appendix III for State's organizational chart.
State's Workforce Training Policy:
State outlines its policies for employee training and career
development in two publications. The Foreign Affairs Manual describes
the functional statements and organization responsibilities and
authorities assigned to each of State's major components. The Foreign
Affairs Handbooks provide detailed procedural implementation of
policies and guidance outlined in the Foreign Affairs Manual. In
certain cases, the policies outlined in the manual and handbooks
reflect legislative criteria for training Foreign Service, civil
service, and LE staff.
* Foreign Service and LE staff. The objective of the Foreign Service
Act,[Footnote 11] as amended, is to strengthen and improve the Foreign
Service of the United States. The Act requires the Secretary of State
to maintain and operate an institute for training to promote career
development within the Foreign Service. This institute is to provide
necessary training and instruction in the field of foreign relations
to the members of the Foreign Service, including foreign national
employees--that is, LE staff who are not U.S. citizens--who provide
clerical, administrative, technical, fiscal, and other support at
foreign service posts abroad, and to employees of the department and
other U.S. departments and agencies. The Foreign Service Act also
requires the Secretary of State to establish a professional
development program to assure that members of the Foreign Service
obtain skills and knowledge required at various stages of their
careers, with primary attention to training for career candidate
officers and mid-career officers. In addition to department policies,
local labor laws, which vary from country to country, also apply to
all LE staff.
* Civil service. State's civil service workforce is regulated by OPM
guidelines and federal laws. In particular, according to OPM, the
Government Employees Training Act, as amended,[Footnote 12] created a
framework for agencies to plan, establish, implement, evaluate, and
fund training and development programs designed to improve the quality
and performance of the workforce. In addition, Executive Orders have
provided the Secretary of State with additional presidential direction
on implementation of the government Employees Training Act by
directing that agencies (1) develop training programs to address both
short- and long-range program needs specific to occupations or
organizational groups and (2) conduct periodic training needs
assessments.[Footnote 13]
Principles of Effective Federal Training Programs:
GAO's previously issued guide for assessing federal strategic training
and development efforts identifies 32 attributes, as well as
corresponding indicators for each attribute, relating to four broad,
interrelated elements of an effective training program: (1) planning,
(2) design, (3) implementation, and (4) evaluation.[Footnote 14]
Figure 4 depicts general relationships between these four elements
that help to produce a strategic approach to federal training and
development efforts.
Figure 4: General Relationships between Elements of Federal Training
and Development Efforts:
[Refer to PDF for image: illustration]
Planning/Front-end Analysis:
* Develop a strategic approach that establishes priorities and
leverages investments in training and development to achieve agency
results.
Design/Development:
* Identify specific training and development initiatives that, in
conjunction with other strategies, improve individual and agency
performance.
Implementation:
* Ensure effective and efficient delivery of training and development
opportunities in an environment that supports learning and change.
Evaluation:
Demonstrate how training and development efforts contribute to
improved performance and results.
Source: GAO.
[End of figure]
The guide serves as a flexible framework for assessing how agencies
plan, design, implement, and evaluate training and development
programs that contribute to improved organizational performance and
enhanced employee skills and competencies. For each of the four
elements of the training and development process, the guide provides a
set of attributes or key questions to consider when assessing a
training program, as well as a list of indicators to look for related
to each key question (see appendix II for a detailed listing of the
elements and their associated attributes). Because the guide is meant
to serve as a flexible framework, an agency's training and development
program is not necessarily expected to address every indicator.
However, the guide can be used to identify potential gaps or strategic
weaknesses in an agency's training program.
Table 1 lists examples of attributes, as well as supporting
indicators, for assessing each of the four elements outlined in the
guide.
Table 1: Selected Attributes and Supporting Indicators for Elements of
Effective Federal Training Efforts:
Training element: Planning;
Attribute: How does the agency identify the appropriate level of
investment to provide for training and development efforts and
prioritize funding so that the most important training needs are
addressed first?
Selected indicators: Goals and expectations for training and
development investments that are transparent and clearly defined and
whose rationale is consistent across the range of human capital
programs at the agency. A training plan or other document that
presents a business case for proposed training and development
investments, including the identified problem or opportunity, the
concept for an improved situation or condition, linkages with the
agency's strategic objectives, anticipated benefits and projected
costs, and ways to mitigate associated risks.
Training element: Design;
Attribute: How does the agency compare the merits of different
delivery mechanisms (such as classroom or computer-based training) and
determine what mix of mechanisms to use to ensure efficient and cost-
effective delivery?
Selected indicators: Analysis of cost data on different delivery
mechanisms. Strategies to continually update training and development
opportunities, such as making use of advances in technologies.
Training element: Implementation;
Attribute: Does the agency take actions to foster an environment
conducive to effective training and development (such as employing
qualified instructors; providing training space, facilities, and
equipment; and establishing appropriate systems and databases to
enable proper management and support of training)?
Selected indicators: Evidence that the agency has properly trained
managers to coach, evaluate, and conduct employee career discussions.
Space, facilities, and equipment that meet the developmental needs of
participants without creating unplanned excess capacity.
Training element: Evaluation;
Attribute: How does the agency incorporate evaluation feedback into
the planning, design, and implementation of its training and
development efforts?
Selected indicators: Systematic monitoring and feedback processes;
Informal feedback mechanisms.
Source: GAO-04-546G.
[End of table]
State Has Developed an Extensive Training Program in Support of Its
Mission, Primarily through the Foreign Service Institute:
Purpose and Key Responsibilities for Training:
State has articulated its training and professional development
mission in various agency plans and guidance and implements this
mission mainly through FSI. According to the organizational directive
outlined in the Foreign Affairs Manual, State "is fully committed to
the career development of all its employees, consistent with
organizational needs, in order to improve service, increase efficiency
and economy, and build and maintain a force of skilled and efficient
employees." The department's Annual Training Plan states that "the
purpose of the department's training program is to develop the men and
women our nation requires to fulfill our leadership role in world
affairs and to advance and defend U.S. interests."
Located at the George P. Shultz National Foreign Affairs Training
Center in Arlington, Virginia, FSI was established in 1947 to promote
career development within the Foreign Service and to provide necessary
training and instruction in the field of foreign relations to members
of the Foreign Service and to employees of the department and of other
agencies. It is State's primary training provider for personnel,
offering entry-, mid-, and senior-level training for employees as they
progress through their careers, maintaining personnel training
records, and overseeing personnel requests for external training.
[Footnote 15]
The Foreign Affairs Manual identifies training oversight authorities
for State officials and implementation responsibilities for FSI, the
Bureau of Human Resources, principal officers at post, bureau
officials, managers, supervisors, and employees. Table 2 highlights
key authorities and responsibilities.
Table 2: Key Responsibilities for Training State Personnel According
to State's Foreign Affairs Manual:
Responsible party: Secretary of State;
Key responsibilities: Authorized to establish and implement needed
training programs and provide required resources necessary to
establish and maintain such programs.
Responsible party: Director of FSI;
Key responsibilities: State's chief training official. Responsible for
establishing, administering, evaluating, and maintaining training
which meets the needs of State.
Responsible party: Director General of the Foreign Service and
Director of Human Resources;
Key responsibilities: Assign Foreign Service and civil service
employees to training, and work with the Director of FSI to help
ensure that training programs meet Foreign Service and civil service
needs.
Responsible party: Bureau Training Officials;
Key responsibilities: Identify bureau training needs. Develop a bureau
career guide that outlines the profiles of major occupations,
including knowledge, skills, and abilities necessary for each
occupation and grade.
Responsible party: Principal Officer at overseas post;
Key responsibilities: Supervise and coordinate all post-based training
activities. Provide formal and informal training for LE staff as
needed.
Source: State Foreign Affairs Manual.
[End of table]
The Foreign Affairs Manual also states that managers, supervisors, and
employees all have responsibilities in regard to training. Managers
and supervisors are responsible for determining specific employee
training needs and ensuring that employees receive training for
effective job performance. Employees are responsible for considering
training that will enable them to improve their performance and
prepare them for greater responsibilities at the department.
FSI Offers Wide Range of Training Opportunities to State Personnel:
FSI is organized into four schools, each with multiple divisions: the
School of Language Studies, with seven divisions; the School of
Applied Information Technology, with three divisions; the Leadership
and Management School, with four divisions; and the School of
Professional and Area Studies, with 10 divisions. Each school is
headed by a dean. FSI's Executive Director's office provides general
oversight and management for FSI's operations, and the Transition
Center provides training and other resources for personnel on topics
such as career transitions and preparing for life overseas. See figure
5.
Figure 5: FSI's Organization:
[Refer to PDF for image: organization chart]
Top level:
* Office of the Director.
Second level, reporting to Office of the Director:
* Deputy Director.
Third level, reporting to Deputy Director:
* School of Language Studies FSI/SLS;
* School of Applied Information Technology FSI/SAIT;
* Leadership and Management School FSI/LMS;
* School of Professional and Area Studies FSI/SPAS;
* Transition Center FSI/TC;
* Executive Director FSI/EX.
Fourth level, reporting to School of Language Studies FSI/SLS:
* Continuing Training and Testing FSI/SLS/CTT;
* East Asian and Pacific Languages FSI/SLS/EAP;
* European and African Languages FSI/SLS/EUA;
* Near East, Central and South Asian Languages FSI/SLS/NEA;
* Research, Evaluation and Development FSI/SLS/RE&D;
* Romance Languages FSI/SLS/ROM;
* Slavic, Pashto and Persian Languages FSI/SLS/SPP.
Fourth level, reporting to School of Applied Information Technology
FSI/SAIT:
* Business Applications FSI/SAIT/BA;
* Enterprise Technology FSI/SAIT/ET;
* Research, Learning and Development FSI/SAIT/RLD.
Fourth level, reporting to Leadership and Management School FSI/LMS:
* Crisis Management Training FSI/LMS/CMT;
* Executive Development Division FSI/LMS/EDD;
* Leadership Training Division FSI/LMS/LTD;
* Policy Leadership Division FSI/LMS/PLD.
Fourth level, reporting to School of Professional and Area Studies
FSI/SPAS:
* Area Studies FSI/SPAS/AS;
* Consular Training FSI/SPAS/CON;
* Curriculum and Staff Development FSI/SPAS/CSD;
* Economic and Commercial Studies FSI/SPAS/ECON;
* Management Tradecraft Training FSI/SPAS/MTT;
* Office Management Training FSI/SPAS/OMT;
* Orientation FSI/SPAS/OR;
* Political Training FSI/SPAS/POL;
* Public Diplomacy FSI/SPAS/PD;
* Stability Operations FSI/SPAS/SO.
Fourth level, reporting to Transition Center FSI/TC:
* Career Transition Center FSI/TC/CTC;
* Overseas Briefing Center FSI/TC/OBC;
* Transition Center Training FSI/TC/T.
Fourth level, reporting to Executive Director FSI/EX:
* Acquisitions FSI/EX/ACQ;
* Budget FSI/EX/BUD;
* General Services FSI/EX/GSO;
* Human Resources FSI/EX/HR;
* Registrar FSI/EX/REG;
* Management Analysts FSI/EX;
* Information Resource Management FSI/EX;
* Audio Visual FSI/EX/AVF:
- Instructional Support Division FSI/EX/ISD;
- Office of Information Management FSI/EX/OMIS Info Systems;
- Corporate Systems.
Source: State.
[End of figure]
Each school specializes in providing training in specific areas of
study. See table 3.
Table 3: Primary Functions of FSI Schools:
School: School of Language Studies;
Primary training functions: Offers training to Foreign Service
Officers, Foreign Service Specialists, and eligible family members in
over 60 languages from beginner to advanced levels at FSI's Arlington
campus, overseas locations, and through Distance Learning. Offers full-
time advanced level training programs for "superhard" languages[A] at
field schools in Seoul, Taipei, Tunis,[B] and Yokohama.
School: School of Applied Information Technology;
Primary training functions: Provides training to improve business
applications skills of all employees. Provides training in the
technologies employed across State for information technology (IT)
professionals. Offers Information Resources Management Tradecraft
courses that provide IT managers with broad IT management skills.
School: Leadership and Management School;
Primary training functions: Offers courses in crisis management
training, management and executive development, senior policy
seminars, and ambassadorial seminars.
School: School of Professional and Area Studies;
Primary training functions: Provides a variety of training programs
intended to instill and improve professional and tradecraft skills and
knowledge. Serves as locus for LE staff training.
Source: GAO analysis of State documentary evidence.
[A] State categorizes Korean, Chinese, Arabic, and Japanese--languages
that are exceptionally difficult for native English speakers to learn--
as superhard languages.
[B] FSI and the Near Eastern Affairs Bureau are working on
decentralizing the advanced Arabic program in the region, and toward
closing the FSI Tunis Field School in fiscal year 2012.
[End of table]
FSI offers over 700 different classroom courses at its Arlington
campus, regional centers, and overseas posts. FSI publishes a course
catalog listing the times and dates of upcoming courses, and officials
told us that FSI advertises new courses at quarterly meetings with
bureau training officers and through department cables. The duration
of classroom courses can range from a half day to 2 years for language
training. In fiscal year 2010, FSI had over 2,100 offerings of non-
language classroom courses, which include courses that focus on job-
related professional and technical skills, as well as leadership and
management skills, at its domestic and overseas locations. In addition
to providing training at the main campus, FSI offers courses at other
locations in the United States, regional centers, and overseas
locations. About 86 percent of these courses were offered at domestic
locations and 14 percent at overseas locations.[Footnote 16]
Approximately 70 percent of all FSI non-language courses in 2010 were
offered at the FSI campus in Arlington, and roughly half of the
remaining courses at domestic locations were offered at State
headquarters in Washington, D.C. See figure 6.
Figure 6: FSI Non-Language Classroom Course Offerings by Location,
Fiscal Year 2010:
[Refer to PDF for image: pie-chart]
FSI: 69%;
All other overseas posts: 10%;
All other domestic sites: 9%;
Washington, DC: 8%;
Regional centers: 4%.
Source: GAO analysis of State data.
Notes:
The duration of each course varies; however, according to State
officials, State does not track training hours by location.
The data depicted omit 1,200 domestic and overseas offerings of a 1-
hour course in fiscal year 2010 to train end users in State's new
departmentwide messaging system, SMART (State Messaging and Archive
Retrieval Tool). With the inclusion of SMART courses, State data for
fiscal year 2010 show that about 64 percent of FSI non-language
classroom courses were offered at domestic locations and 36 percent at
overseas locations.
Courses offered at the regional center in Ft. Lauderdale were counted
as overseas locations because the regional center provides services to
posts in the Western Hemisphere Affairs region.
[End of figure]
FSI has also increased its focus on distance learning in recent years.
According to State officials, in addition to offering classroom
training, FSI began offering distance learning courses in 2002. In
fiscal year 2010, FSI offered 190 customized courses, including
courses with little or no interaction with an instructor that allow
participants to complete the course at their own pace; real-time
courses in which students and instructors participate simultaneously
via various technologies; and "blended courses," which combine various
delivery methods such as time spent in the classroom and online. In
addition, FSI offers about 3,000 commercially developed courses that
are available at all times in a wide variety of topics, including
Microsoft applications and various business topics. According to FSI
officials, distance learning allows FSI to provide training for LE
staff overseas who otherwise would have been unable to access training.
State Provides Additional Training and Career Development
Opportunities:
While FSI is State's primary training provider, other bureaus in the
department also offer training for State and other federal government
employees. For example, the Office of Training and Performance Support
within the Bureau of Diplomatic Security offers security and law
enforcement training to personnel, including Special Agents, Regional
Security Officers, and others. The Diplomatic Security Training Center
provides about 85 instructor-led courses, including the Foreign
Affairs Counter Threat course for personnel who are deploying to
critical threat environments such as Iraq, Afghanistan, and Pakistan.
Other departmentwide bureau-sponsored training includes ethics
training by the Office of the Legal Advisor; Equal Employment
Opportunity laws and regulations training by the Office of Civil
Rights; and emergency preparedness, property record keeping,
procurement, logistics management, federal assistance, and safety
awareness training by the Bureau of Administration.
State also offers several career development programs for employees.
For example, in 2005 State introduced the Career Development Program
for Foreign Service employees, which sets requirements for advancement
into the senior ranks and focuses on developing appropriate
professional, leadership, language, and technical skills at each
level. State's Civil Service Mid-Level Rotational Program provides
opportunities for mid-level civil service employees to rotate to other
bureaus to broaden their skills, increase their knowledge, and enhance
their personnel and professional growth. In addition, State has
mentoring programs for entry-level, as well as more experienced,
employees.
According to State officials, bureaus, offices, and posts may also
develop and offer their own training and professional development
opportunities when the bureau, office, or post has specific needs that
make it more efficient for it to develop the training itself or seek
training outside the agency. Officials from several bureaus, offices,
and posts told us they have designed various training programs that
are tailored to the specific needs of their employees, which may
include orientation, on-the-job training, mentorship opportunities,
and annual conferences, workshops, or seminars. For example:
* The Bureau of Economic, Energy, and Business Affairs organizes a
half-day orientation session every 6 months for employees who are new
to the bureau.
* The Bureaus of European and Eurasian Affairs; International
Organizations; Consular Affairs; and Democracy, Human Rights and Labor
host annual conferences and workshops for various employees in their
bureaus.
* The Bureau of Intelligence and Research coordinates with training
programs at other federal agencies in the intelligence community to
arrange for its employees to attend specialized training in subject
matter that is not available through FSI.
* The Bureau of International Narcotics and Law Enforcement provides
training sessions that help employees gain familiarization with
various areas of law enforcement.
* Training officials at 11 of the 12 overseas posts we interviewed
said their post sponsors a language program, which may include classes
in the native language for Americans and English language courses for
LE staff.
* Officials at 8 of the 12 overseas posts we interviewed said they
host some type of orientation for employees new to the post.
In addition, officials told us that three of State's regional bureaus--
Western Hemisphere Affairs, European and Eurasian Affairs, and East
Asian and Pacific Affairs--operate regional centers that service
various needs of the posts in their respective regions in Fort
Lauderdale, Florida; Frankfurt, Germany; and Bangkok, Thailand.
[Footnote 17] Officials at all of the regional centers told us they
have a model for providing some training for employees in their
respective region, including LE staff. These officials also noted that
some of these courses were developed by FSI, while other courses have
been designed and developed by the regional center.
State employees may also participate in various external training
opportunities. For example, officials noted that certain civil service
employees in particular may take advantage of human resource or
technical courses offered through outside vendors such as the U.S.
Department of Agriculture Graduate School, that are not available
through FSI.[Footnote 18] In addition, employees may participate in
several long-term external career development opportunities, which are
generally available to tenured Foreign Service and civil service
employees who are at the mid-and senior-grade levels. These programs
normally last from 9 to 12 months and include the military schools
(War Colleges and Commands) and nondegree fellowships, as well as
bachelors and masters degree programs at various colleges and
universities. For example, State data showed that a total of about 130
Foreign Service and civil service employees participated in long-term
academic training in the 2009-2010 academic year. Further, in 2009,
almost 80 State employees participated in long-term development
programs related to interdepartmental collaboration on national
security and were assigned to long-term rotational positions at
agencies such as the Department of Defense and the Office of the
Director of National Intelligence.
Types and Amount of Training Vary for Foreign Service, Civil Service,
and LE Staff:
State personnel receive varying types and amounts of training,
depending on their position or category, career stage, specific
training needs, and available resources. State data on employee
training hours in fiscal years 2006 through 2010 show the following:
* Almost 90 percent of all training hours took place in the classroom,
with about half of these hours for language training and the other
half for non-language training.
* Time spent in distance learning comprised about 7 percent of all
training hours.
* Training by external providers comprised 5 percent of training hours.
* About 85 percent of employee hours in classroom training was for
Foreign Service employees, with about half of this time for language
training; about 15 percent of FSI classroom training was for civil
service and LE staff.
* Civil service employees spent about 70 percent of total hours in
external training.
* LE staff spent the largest amount of time in distance learning--
about 47 percent of total hours in this type of training.
Figures 7 and 8 illustrate the types and amounts of training that
State employees received in fiscal years 2006 through 2010.
Figure 7: Participation in Classroom, Distance Learning, and External
Language and Non-language Training by All State Employees, Fiscal
Years 2006-2010:
[Refer to PDF for image: pie-chart]
Classroom non-language: 48%;
Classroom language: 40%;
Distance learning non-language: 6%;
Distance learning language: 1%;
External training: 5%.
Source: GAO analysis of State data.
Note: Data depicted for fiscal year 2010 training participation
through March 31, 2010.
[End of figure]
Figure 8: Participation in Classroom, Distance Learning, and External
Training by Employee Category, Fiscal Years 2006-2010:
[Refer to PDF for image: 3 pie-charts]
Classroom training participation:
Foreign service: 85%;
Civil service: 9%;
Locally employed staff: 6%.
Distance learning training participation:
Foreign service: 36%;
Civil service: 17%;
Locally employed staff: 47%.
External training participation:
Foreign service: 29%;
Civil service: 70%;
Locally employed staff: 1%.
Source: GAO analysis of State data.
Note: Data depicted for fiscal year 2010 training participation
through March 31, 2010.
[End of figure]
State Workforce Training Incorporates Many Aspects of Effective
Training Programs, but Strategic Weaknesses Exist:
State has taken many steps to incorporate the interrelated elements of
an effective training program--planning, design, implementation, and
evaluation--into its training for personnel,[Footnote 19] but the
department's strategic approach to workforce training has several key
weaknesses. State demonstrated a variety of examples of ways in which
the department has endeavored to develop an effective training
program, such as by compiling an annual training plan and implementing
a range of training evaluation mechanisms. However, in our analysis of
the extent to which State's training program reflects key attributes
identified in prior GAO guidance, we found several key gaps in the
department's efforts to strategically plan and prioritize training,
ensure efficient and effective training design and delivery, and
determine whether or how training and development efforts contribute
to improved performance and desired results. These issue areas are
generally connected with various attributes and indicators associated
with more than one element of the training and development process, as
planning, design, implementation, and evaluation efforts are often
interrelated.[Footnote 20]
State's Training Reflects Aspects of Effective Training Programs:
State's personnel training reflects numerous aspects of effective
training programs, based on our assessment using the criteria GAO
previously identified. (See appendix II for a detailed listing of the
elements with their associated attributes.) For example, we identified
the following positive practices relevant to the major, interrelated
elements of the training and development process.
* Planning. State maintains a workforce training plan, as required by
federal regulations.[Footnote 21] FSI leads efforts to prepare the
training plan annually with input from other bureaus and offices; the
plan is linked to State's overall strategic plan, and presents a
business case for proposed training investments, including training
provided by FSI, the Bureau of Diplomatic Security, and other
personnel training programs, such as external and long-term training
opportunities.[Footnote 22] FSI's director--the chief training officer
for the department--and the Director General of the Foreign Service
and Director of Human Resources are members of the department's senior
management, reporting to State's Under Secretary for Management.
According to State, in demonstration of its commitment to training,
the department dedicates almost 1.5 percent--approximately $214
million in fiscal year 2010--of its program budget to personnel
training.[Footnote 23] State's Foreign Affairs Manual and Foreign
Affairs Handbooks include information on the department's legislative
authorities and policies that may relate to, or require training and
development for, Foreign Service, civil service, and LE staff. In
addition, FSI publishes an annual schedule of courses, which provides
information for employees on the more than 700 classroom courses FSI
offers, as well as approximately 190 custom-developed distance
learning courses.
* Design. FSI highlighted its involvement of curricula and education
experts as well as subject matter or technical experts in the
development of courses as a positive practice. FSI officials stated
all courses are designed to meet specific learning objectives. The
schedule of courses generally includes information for each course
such as a brief description, any prerequisites, course objectives, and
relevant competencies and precepts.[Footnote 24] State has also
designed programs that incorporate various cross-training
opportunities and targeted experiences. For example, Foreign Service
and civil service personnel may participate in rotational assignments,
congressional fellowships, OPM leadership seminars, or programs
through the National Defense University, among other opportunities;
and LE staff may participate in short-term details to other posts, or
opportunities to "shadow" a more experienced employee, according to
State officials. Additionally, State has made an effort to design and
develop courses that use advances in technologies to enhance its
training efforts, another example of a positive practice. The number
of distance learning offerings, as well as employee participation in
distance learning, has increased in recent years. For example, State's
latest annual training plan reported that FSI developed 20 new custom
distance learning courses during the prior year[Footnote 25]. FSI has
also piloted "blended" learning opportunities at some overseas posts,
which may include group instruction via videoconference along with
online course content.
* Implementation. State has taken steps to provide employees with a
work environment that promotes learning, such as by publishing links
to the schedule of courses and information on upcoming learning
opportunities, and providing access to distance learning courses from
FSI's intranet home page. To enhance accessibility, the schedule of
courses is also available in hard copy or CD-ROM. Moreover, State has
implemented oversight mechanisms to ensure that employees take
advantage of legitimate training opportunities and that the agency
does not pay for fraudulent training. State has also put in place
training or continued service agreements for certain training to help
ensure accountability as well as to encourage employees to accept the
goals of training efforts and apply knowledge gained through training
on the job. For example, FSI has an attendance policy requiring
students to participate in 80 to 100 percent of a classroom course--
depending upon duration and subject matter--and pass any applicable
testing requirement in order to receive official course credit. The
employee's home bureau must generally reimburse FSI in the event that
an employee does not show up or complete a course. In addition, FSI
generally reviews and approves all applications for external training.
Employees generally must sign an agreement to reimburse State if they
do not complete an external training course they registered for, and
employees must complete a post-training evaluation for any external
training. Several posts we interviewed noted they have training
agreements in place for training provided to LE staff, which may
require personnel to submit a report to supervisors on the outcome of
training or reimburse State for costs such as travel in the event they
do not successfully complete a training course.[Footnote 26]
* Evaluation. State has a range of training evaluation mechanisms in
place, including mid-and post-training course evaluations, some
incorporating supervisors as well as employees. Efforts to evaluate
training in order to assess the extent to which it contributes to
improved results is considered an essential component of an effective
training program. The department also has a learning management system
that can track delivery of training. For example, as noted previously,
State provided data to us on training hours and courses completed by
Foreign Service, civil service, and LE staff over the past 5 fiscal
years, including data on language and non-language classroom and
distance learning, as well as external training. Additionally, since
2006 FSI has conducted an annual training survey; for example, FSI
reported most respondents to the 2010 survey were, in general,
satisfied or very satisfied with training[Footnote 27]. Further, to
comply with an OPM mandate, according to State, in 2007 the department
conducted the biennial Quality of Work Life Survey, which addresses
human capital issues and trends.[Footnote 28] State reported that the
2007 survey found a majority of respondents agreed with the statement,
"I receive adequate training to do my job."
Weaknesses Exist in State's Strategic Approach to Personnel Training:
Although State's training practices and procedures reflect numerous
attributes and indicators of an effective training program, we found
gaps in six key areas (see table 4). Each of these issue areas broadly
relates to multiple elements, attributes, and indicators throughout
the interrelated training and development process. While an agency's
training program is not necessarily expected to address every
indicator identified in the GAO guidance, based on our assessment, we
identified strategic weaknesses related to these six issue areas as
particularly important to ensuring effective planning, design,
implementation, and evaluation of personnel training.
Table 4: Key Strategic Weaknesses in State's Efforts to Train
Personnel:
* State lacks a systematic, comprehensive training needs assessment
process incorporating all bureaus and overseas posts.
* State developed training continuums to provide information for
employees about training opportunities, career ladders and paths, and
how training can help employees attain career goals, but the
continuums do not provide complete and accurate information for
employees.
* State has not developed adequate curriculum design guidance or a
data collection and analysis plan, which could help ensure that
appropriate procedures and criteria for designing and evaluating
training are systematically applied across the board.
* State could not sufficiently demonstrate consistent and appropriate
support for training, because it does not track detailed data and
information on training cost and delivery that would allow for an
analysis and comparison of employees in different employee groups,
bureaus, regions, and posts.
* State has developed several training-related goals and measures, but
the measures do not fully address the goals, and are generally output
rather than outcome oriented.
Source: GAO analysis.
[End of table]
State Lacks Systematic, Comprehensive Training Needs Assessment:
Although State has several practices in place to identify training
needs, the department lacks a systematic, comprehensive training needs
assessment process clearly incorporating all bureaus and posts,
particularly at the occupational and individual levels. Our previous
work identifying elements of effective training states that an agency
can use organizational, occupational, and individual training needs
assessments to help ensure training is connected to improving
individual and agency performance in achieving results.[Footnote 29]
OPM guidance refers to this as a multilevel training needs assessment
process and states that to be successful, an assessment process should
be ongoing, involve management at all levels, and be integrated into
the agency's program planning and budgeting process.[Footnote 30]
Further, under Executive Order, agencies are to review, not less than
annually, programs to identify training needs, establish priorities
for training, and provide resources in accordance with those
priorities.[Footnote 31] In addition, State's Foreign Affairs Manual
states that bureau executive directors are responsible for identifying
bureau training needs. State guidance also notes that not conducting
training needs assessments may cause time and resources to be wasted
in developing and teaching skills that employees do not need, while
true training needs and barriers may go unaddressed.
We found that State has some processes in place to identify broad or
cross-cutting training needs at the organizational level. For example,
FSI officials told us the institute identifies training needs and
strategy in part through FSI officials' participation in various
departmental committees or working groups that may address training
issues or discuss training needs. They noted that FSI's Director, who
acts as the Chief Training Officer for the department, attends the
Secretary's weekly staff meetings and holds monthly meetings with the
Director General of the Foreign Service and Director of Human
Resources. The Deputy Director participates in periodic meetings with
the Bureau of Human Resources Office of Career Development and
Assignments. FSI officials added they were holding weekly meetings
with officials from the bureau on how to address training needs
related to the recent increase in hiring. Further, they said the Dean
of the School of Applied Information Technology participates in
periodic meetings with State's Chief Information Officer to stay
abreast of information technology training needs. According to State's
Bureau of Human Resources, State aims to identify training needs
related in particular to the department's foreign policy mission and
emerging issues.
However, State's processes for identifying both organizational, cross-
cutting, and more specific occupational training needs do not clearly
incorporate all bureaus and posts. For example, officials from a
number of bureaus told us that their bureaus had not conducted formal
training needs assessments. Only one bureau indicated it had recently
conducted an assessment relevant to training needs. To help identify
cross-cutting as well as more specific occupational training needs,
FSI officials said the institute holds quarterly meetings with bureau
training officers, providing opportunities to discuss any bureau
training needs and to share information, for example, about upcoming
course offerings. However, several bureau training officials we met
with noted that a bureau with a specific training need would generally
reach out to FSI on a case-by-case or ad hoc basis. They said that FSI
is generally responsive to ad hoc requests regarding specific training
needs, but discussions at the quarterly meetings typically address
topics such as training-related administrative processes, rather than
addressing bureau training needs.
State's Office of Inspector General (OIG) previously recommended that
State conduct annual training needs assessments and implement a more
comprehensive and systematic planning process for training
incorporating all bureaus and posts,[Footnote 32] such as by requiring
a training section in the department's annual bureau and mission
strategic plans,[Footnote 33] to help identify and address long-term
training needs across locations and all categories of employees in the
department. To address prior OIG findings, State reported that the
department had added a training "annex" to all bureau and mission
plans. However, State officials indicated the plans no longer include
a training annex, owing to the department's decision to streamline the
strategic planning process in order to make it less burdensome for
bureaus and posts. As a result, officials noted some bureau and
mission plans include information on training goals, needs, or
priorities, while others include no reference to training. None of the
12 posts we interviewed reported receiving guidance from headquarters
that could help the post prioritize, plan, and budget for training.
Some bureau and post officials stated they could benefit from
additional guidance, and indicated that a more systematic training
planning process across bureaus and posts could be helpful. However,
several officials noted that such a process could become a "paperwork
exercise" unless there were also a commitment to follow-through with
implementation.
In addition, relevant to occupational training needs, the Foreign
Affairs Manual states that training officers in each bureau are
responsible for developing a bureau career guide outlining profiles of
major occupations in the bureau including descriptions of the
knowledge, skills, and abilities needed for each occupation and grade.
However, most of the bureaus we met with had not developed a bureau
career guide.[Footnote 34] Officials indicated they would generally
rely instead on broader departmental guidance to help identify
relevant training. For example, State guidance states that civil
service employees must demonstrate competencies established by OPM. In
addition, State has broadly identified "core precepts" for Foreign
Service entry-, mid-, and senior-level employees, which are updated
periodically. Technical competencies have also been identified for
certain groups of employees, such as administrative or post management
officers. Officials indicated that more specific roles and
responsibilities, as well as any skills or abilities--though not
necessarily training--required for different Foreign Service, civil
service, and LE staff positions, are generally identified in job
announcements. They noted they also rely on FSI guidance on training
for employees in different career paths.[Footnote 35]
Also, to help identify individual training needs, bureau and post
officials noted that Foreign Service employees generally rely on
career development officers, in addition to supervisors at post, to
provide guidance and counseling on training and development, including
identifying necessary training. However, some post officials noted
this support may vary. A number of Foreign Service employees
responding to the open-ended question on training in the 2009 American
Foreign Service Association survey also cited concerns about career
development officer support. For example, some said their career
development officers generally had too many people assigned to them
and were ineffective at disseminating relevant information; were not
responsive to e-mails or phone calls; and were more focused on filling
positions than on supporting Foreign Service Officers' career
development. According to State's Bureau of Human Resources, as of
October 2010, State had 41 career development officers (CDO), each
supporting at least 250 Foreign Service generalists and specialists.
They noted that serving specialists and mid-level generalists in
particular, some CDOs support as many as 450 employees. The bureau
also has two dedicated staff--an assignments officer and a training
officer--who help ensure that entry-level personnel are enrolled in
required training. Human resources officials said that recent
increases in State personnel had presented challenges to the provision
of career development counseling, and that, although State had created
new entry-level CDO positions as a result, the need for additional mid-
level CDOs was under review. They added that they have also been
working to streamline administrative and technical processes to allow
officers to spend more of their time providing individual guidance and
counseling on training.
Further, although State encourages all employees to complete an
individual development plan (IDP), which can be a component of an
individual training needs assessment process, bureau and post
officials indicated that few Foreign Service or civil service
employees have completed an IDP. As a result, it is not clear whether
Foreign Service and civil service employees have adequate opportunity
to discuss with their supervisors any training they may need to
improve individual performance or prepare for future assignments. We
previously reported that the use of IDPs to identify both short-and
long-term developmental needs of each employee can help an agency
incorporate employees' developmental goals, and integrate the need for
continuous and career-long learning, into its planning processes.
[Footnote 36] State requires only certain entry-level civil service
employees to complete an IDP. Some bureau officials stated they
thought the IDP should be a broader requirement. Officials from
several of the bureaus we met with noted that while the IDP is
encouraged, it is not widely used by domestic bureau personnel,
although training needs may be identified for civil service personnel
as part of the annual performance appraisal process. Bureau and post
officials indicated that, although training needs could be covered as
part of the Foreign Service appraisal process, in contrast to the
civil service appraisal, the Foreign Service appraisal form does not
explicitly address training, and officials generally have not seen it
in the appraisals.
Post officials from 11 of the 12 posts we interviewed said their posts
use a work development plan (WDP), similar to an IDP, for LE staff.
Officials noted that posts are generally responsible for coordinating
and overseeing LE staff training.[Footnote 37] Some of these posts
said it is a general practice or requirement for supervisors to work
with LE staff at post to complete a WDP, including identifying any
training needs, as part of the annual appraisal process. Post
officials noted that they may review the employee development plans or
solicit broader input from supervisors or other relevant personnel at
the post, to help identify and prioritize training as part of the
post's annual budget and planning process. For example, officials at
one post noted they annually solicit training needs from all sections
at the post for the upcoming year and review the WDPs to compile a
prioritized list of training. However, the senior human resources
officer we spoke with at another post said that she primarily
identifies training needs and puts together an annual budget request
for training based on her own observations and oral discussions with
relevant personnel at the post. She noted that although she had
attempted to solicit broader, formal input from personnel at post on
individual training needs, she had received minimal response. She
added that she will review WDPs to help identify individual training
needs, but because the WDP is not a requirement at post she must rely
largely on her own observations and oral discussions with supervisors
of LE staff.
While State does not have a systematic, comprehensive training needs
assessment process, training officials we interviewed from several
bureaus and posts cited various areas of potential training need for
Foreign Service, civil service, and LE staff. For example, some bureau
officials cited a greater general need for training personnel in areas
such as project management, strategic planning, human resources, and
budgeting, as well as in English and computer skills for LE staff in
particular. Post officials also cited a general need for additional
training in areas such as project and contract management and for
strategic planning, public-private partnerships, NGO engagement, and
enhanced coverage of certain countries in geographic area studies,
among other areas.[Footnote 38] Further, a number of Foreign Service
employees responding to the open-ended question on training in the
2009 American Foreign Service Association annual survey cited a range
of potential training needs. For example, respondents cited needs for
additional training for generalists relevant to their particular areas
of work; additional training for specialists, including for Office
Management Specialists; additional and enhanced leadership training;
and improved and expanded language training, including expansion of
language training overseas. Respondents also cited the need for a
greater focus on training through their careers, such as by
establishing and enforcing minimum requirements at various career
stages beyond the current mandatory requirements.[Footnote 39] State
officials noted that State has increased attention to LE staff
training and development in recent years, although some officials
noted in the past, LE staff often received little or no training
despite significant need for training at some posts. Because Foreign
Service employees generally rotate to a new post every 2 years, LE
staff often provide the longevity, continued presence, and support
that are critical to the successful day-to-day operations of overseas
missions.[Footnote 40] In June 2009, FSI conducted a survey of post
management to identify LE staff training needs. Several bureau and
post officials told us they considered the results of the survey in
planning LE staff training. For example, officials representing the
Bureaus of Near Eastern Affairs and South and Central Asian Affairs
said that as a result of the survey, they identified a particular need
for supervisory skills training for LE staff working at certain posts
in those regions, and offered two relevant workshops in the region in
fiscal year 2010. FSI officials noted that as a result of the 2009
survey, they have been working to increase training offerings at posts
by expanding the number and reach of adjunct faculty. They added that
FSI plans to repeat the needs assessment periodically, though they
have not set any specific time frames as of November 2010.
Since 2007, State human resource reports also noted that bureaus have
not formally conducted annual training needs assessments, and
identified this as an issue that should be addressed to help provide a
realistic basis for planning, budgeting, and directing training.
According to the reports, the Bureau of Human Resources intended to
form an interoffice working group to develop a comprehensive plan and
implementation guidance to support a department-wide effort for
assessing training needs. State officials said they had recently begun
developing a plan to address the reports' findings, though they had
not yet formed an interoffice working group as of November 2010.
FSI Training Continuums Include Some Inaccurate and Incomplete
Information:
We found that documents on training--known as training continuums--
that FSI developed for employees do not include accurate and complete
information on training. The continuums generally identify training
and development for entry-, mid-, and senior-level employees, and
contain other information related to career planning. We previously
reported that providing transparent information for employees about
career maps and paths, how training opportunities could help employees
attain career goals, competency models, and training or professional
requirements, can help agencies communicate the importance of training
and their expectations for training programs to achieve results.
[Footnote 41] FSI has developed and published various training
continuums for State's Foreign Service, civil service, and LE staff,
including the Training Continuum for Civil Service Employees, the
Training Continuum for Foreign Service Generalists, the Training
Resource Guide for Foreign Service National Employees and Locally
Employed Staff, and the Leadership and Management Training Continuum,
among others.[Footnote 42]
State's OIG reported that FSI began developing these training
continuums in part to provide a framework for a departmental policy of
mandatory training, as well as to provide road maps tailored to the
particular needs of employees in various positions throughout their
careers.[Footnote 43] In addition, the continuums state they were
designed to provide a broad overview of appropriate training that
should be considered as employees plan their careers in the
department. For example, the training continuums provide information
on leadership skills training requirements that State established
under former Secretary of State Powell for mid-level and higher-ranked
Foreign Service and civil service employees.[Footnote 44] The
continuums also generally include information on other mandatory
personnel training requirements, such as cyber security training, as
well as information on recommended and suggested courses for
employees. Specifically, the continuums state that required courses
are defined as mandatory; recommended courses should be taken to
provide knowledge and skills for successful job performance; and
suggested courses are more specialized and should be taken depending
upon job duties. In addition, the continuums generally include
information on career development opportunities. For example, they
note that State's civil service, mid-level rotational program provides
opportunities for civil service mid-level employees to rotate to other
bureaus to broaden their skills, increase their knowledge, and enhance
their personnel and professional growth. They also provide information
on State's Career Development Program for Foreign Service employees,
which established certain mandatory and elective professional,
leadership, language, and technical skills requirements that personnel
must meet to be eligible for promotion to the Senior Foreign Service.
However, we found several issues that raise questions about the
usefulness and reliability of the continuums as resources for
employees in planning their training and development. For example,
although we found that the Training Continuum for Civil Service
Employees lists diversity awareness, ethics, and orientation training
as "required" for certain groups of employees, and as "recommended"
for other groups, a key official from FSI's executive office stated
that these courses are in fact mandatory requirements for all civil
service employees. The official added that these mandatory
requirements would also apply to Foreign Service personnel and are
among the few across-the-board training requirements that State
considers mandatory, along with cyber security and "No FEAR Act"
training.[Footnote 45] In addition, LE staff have minimal mandatory
training requirements other than cyber security and ethics training.
The FSI official clarified that even though we found that some of the
continuums list other, more specialized courses as required for
certain employees, the department officially considers training
mandatory, and tracks completion, only if the training is listed as
such by statute. The FSI official said that in some cases, decisions
regarding what courses would be listed in continuums as required,
recommended, or suggested were not fully vetted throughout the agency,
and that, as a result, the information may not be entirely accurate or
complete.
In addition, while officials noted that bureaus and posts may
designate specific courses as required for certain groups of
employees, we found that these requirements are not always identified
in the training continuums or other guidance for employees. Several
bureau and post officials noted that personnel would generally rely on
the training continuums to identify necessary training, including any
specialized training that may be required or helpful to them in
performing their jobs. However, officials from the Bureau of
International Organizations said that although FSI's course on
mulitilateral diplomacy is required of all International Organizations
personnel, it is not listed in the continuums. Training officials from
the Bureau of Democracy, Human Rights, and Labor said Foreign Service
labor officers must take the FSI labor officer skills course, though
it is not listed as a requirement in the continuum. Some bureau
officials said that employee supervisors are generally responsible for
ensuring that employees are aware of and take the required training.
Although some bureaus and posts had developed their own training
guidance--for example, on available or suggested training
opportunities, or on procedures required to register for training--the
guidance generally did not include information on any specific
training designated as required, recommended, or suggested by the
bureau or post for different employee groups.
The official from FSI's executive office acknowledged that the
continuums do not include complete and accurate information for
employees on training, and noted they have not been reviewed to ensure
they uniformly reflect departmental policies or standards. The
official added that some of the continuums have not been updated in
several years, there are not continuums for every employee group, and
because the continuums include some discrepancies or inaccuracies,
they should not be viewed as formal or official guidance. The official
also noted that FSI was considering shifting to a new, standardized
format for the training continuums going forward that would hopefully
improve the process and make the continuums easier to update in an
ongoing basis in the future. However, greater involvement and
collaboration from other bureaus would be required in order for FSI to
facilitate a comprehensive process to develop official continuums for
each job series, and the official noted that some bureaus had not been
very responsive to FSI's requests for input in the past.
FSI officials noted that information included in the continuums
regarding State's Career Development Program is official, as these
program requirements have been formally approved by the department.
The officials said that State has also developed separate guidance,
called playbooks, for certain groups of Foreign Service employees
regarding Career Development Program requirements.[Footnote 46] State
officials noted that both the continuums and the playbooks are
resources that can be used by employees to identify training and help
plan career development, although the continuums were developed for
general informational purposes and were not approved by agency
management, unlike the playbooks. However, the Career Development
Program only applies to State's Foreign Service employees. In
addition, although the playbooks generally provide information on
mandatory or elective leadership or technical skills training required
for promotion, they do not provide broader information that is
included in the continuums, such as information on recommended
training that may not be relevant to promotion under the Career
Development Program.
Although State has not established mandatory training for all
employees at every career stage, some bureau and post officials said
that specific training requirements and additional guidance for
different employee groups could be helpful. In response to the State
OIG's prior recommendation that the department establish specific
mandatory training for all employees at every career stage, State
reported it intended to eventually mandate specific training for all
employees throughout their careers; however, the mandatory training
the department subsequently established initially focused on the
leadership skills training. State also noted that although the
department viewed training as critical to assuring that employees have
the necessary skills to support State's mission, implementing broader
mandatory requirements would have serious resource implications.
Nevertheless, some bureau and post officials we interviewed indicated
that improved guidance on training priorities by career track, as well
as possibly establishing additional specialized mandatory training
requirements, could heighten the focus on training and help ensure
that employees get the training they need.
State Has Not Developed Formal, Comprehensive Curriculum Design
Guidance:
Although State provided some examples of ways in which it aims to
incorporate effective design practices for training and development
efforts, we found that the department has not developed comprehensive
written guidelines and criteria to be followed throughout the course
design process. Our prior work identified several key indicators of
effective training programs related to the training design process,
such as mechanisms, procedures, or explicit criteria for:
* determining whether to design training programs in-house or obtain
services from a contractor or external source;
* comparing merits of different delivery mechanisms (e.g., classroom
or computer-based training) and for determining the appropriate mix of
mechanisms to ensure efficient and cost-effective delivery;
* ensuring an environment conducive to effective training through
employment of quality instructors, facilitators, mentors, and coaches;
* involvement of line managers, technical experts, human capital
professionals, and others to develop an integrated way to address
specific performance gaps; and:
* incorporating measures of effectiveness into course designs.
[Footnote 47]
State has made an effort to incorporate effective design practices
into its training for Foreign Service, civil service, and LE staff.
For example, in demonstration of its efforts to identify cost-
effective and robust options for designing training and development,
State shared sample letters of obligation, such as a letter regarding
its decision to provide in-house training services for an FSI language
course following a streamlined cost competition. State also noted that
its training includes a mix of delivery mechanisms, including
classroom, distance learning, and on-the-job training, as delineated
in its training continuums and schedule of courses. According to FSI
officials, FSI employs civil service training experts and Foreign
Service subject matter experts or technical experts to teach courses,
and FSI's Curriculum and Staff Development Division in the School of
Professional and Area Studies offers courses for staff on training
design and delivery. They noted that all FSI instructors and course
managers are required to take training in theory and best practices in
adult learning.[Footnote 48] In April 2007, FSI launched an adjunct
faculty program, which allows State personnel who demonstrate
appropriate qualifications to provide training domestically or
overseas for FSI course credit. In addition, to help ensure the
integrated development of training to help address specific
performance gaps, according to State, FSI utilizes working groups
comprised of curricula and education experts, as well as technical
experts and practitioners of the subject matter in the course design
process. As a specific example, State developed and implemented a plan
to provide a combination of classroom and distance learning,
auditorium sessions, user guides, and deskside assistance to domestic
and overseas personnel to ensure the successful launch of a new
technology platform for combining cables and e-mails.[Footnote 49]
Also, officials provided some examples in which results of course
evaluations were incorporated into subsequent course offerings.
Nevertheless, although FSI officials said that FSI generally follows
accepted industry standards in developing courses, they stated that
FSI does not have comprehensive, documented guidance or standards to
be followed throughout the curriculum design process for personnel
training. While FSI has developed some instructional and curriculum
guidance that is included in its Administrative Procedures Handbook,
the sections relevant to course development are specific to distance
learning and not classroom training. In addition, the handbook notes
that the sections containing course development process and style
guides for distance learning are under development and not available.
Further, the handbook does not include clear guidance for determining
appropriate training delivery mechanisms or whether to design training
in-house or obtain outside services, or for incorporating training
evaluation results into course designs. Because of the lack of formal
curriculum design guidance, FSI's permanent staff, as well as those
who may be on temporary assignment or working under contract to
develop training, may be unaware of and not applying consistent and
appropriate practices and standards across the board throughout the
training design process.[Footnote 50]
State Lacks Data Needed to Assure That Support for Training Is
Consistent and Appropriate:
Although State collects some information on the cost and delivery of
training, the department does not collect data needed for an analysis
and comparison of training provided to employees in different groups,
bureaus, regions, or posts. As a result, State cannot be assured that
it is providing consistent and appropriate support and funding for
training and development. Our prior work highlights the importance of
quality data to evaluating the quality and effectiveness of training
and development efforts, and found that tracking the cost and delivery
of training is a key attribute of an effective training program.
[Footnote 51] We also found that the ability of an agency to
demonstrate consistent and appropriate support and funding for
training, and evidence that the agency provides needed tools and
resources to managers and employees for training, are indicators of
effective training programs.
While State provided data on overall funding for training personnel
that showed some trends, the department does not track data that would
allow a more detailed analysis and comparison across various employee
groups and locations. State provided data on overall funding for
training State personnel from fiscal years 2006 to 2011, which showed
the training budget increased from about $157 million in fiscal year
2006 (adjusted for inflation, expressed in 2010 dollars) to around
$255 million in fiscal year 2010 (adjusted for inflation, expressed in
2010 dollars).[Footnote 52] (See table 5.)
Table 5: Funding for State Personnel Training, Fiscal Years 2006-2011,
Not Adjusted for Inflation:
FSI[A]:
2006 training budget: $117,564,000;
2007 training budget: $122,178,000;
2008 training budget: $121,172,000;
2009 training budget: $129,060,000;
2010 training budget: $179,044,000;
2011 request: $182,279,000.
DS security training[B]:
2006 training budget: $15,273,000;
2007 training budget: $12,933,000;
2008 training budget: $31,270,000;
2009 training budget: $46,730,000;
2010 training budget: $64,021,000;
2011 request: $71,580,000.
IT central fund[C]:
2006 training budget: $6,806,000;
2007 training budget: $6,845,000;
2008 training budget: $8,559,000;
2009 training budget: $7,161,000;
2010 training budget: $6,080,000;
2011 request: $6,787,000.
MRV fund[D]:
2006 training budget: $6,692,000;
2007 training budget: $9,111,000;
2008 training budget: $6,300,000;
2009 training budget: $6,170,000;
2010 training budget: $6,170,000;
2011 request: $6,170,000.
Total[E]:
2006 training budget: $146,335,000;
2007 training budget: $151,067,000;
2008 training budget: $167,301,000;
2009 training budget: $189,121,000;
2010 training budget: $255,315,000;
2011 request: $266,816,000.
Source: GAO analysis of State data.
Notes:
FSI = Foreign Service Institute.
DS = Bureau of Diplomatic Security.
IT = information technology.
MRV = Machine Readable Visa.
[A] FSI's budget includes funding for FSI classroom and technology-
based (e.g. distance or online learning) training programs for
domestic and overseas Foreign Service, civil service, and LE staff.
[B] DS security training funding includes funding to support DS
training to provide personnel with necessary security, investigative,
and law enforcement skills. These amounts also include funding for
tactical and counter-threat training for State personnel deploying to
Iraq.
[C] Training-related funding under the IT Central Fund includes
funding for FSI's Student Training Management System and for FSI IT
learning infrastructure, as well as to train and develop the skills of
State's IT staff to support agency operations.
[D] Training-related funding from Machine Readable Visa (MRV) fees is
used for FSI consular training activities including classes,
conferences, workshops, and online courses for consular officers,
Information Management Specialists, FSNs, and consular agents. MRV
fees are paid by visa applicants.
[E] While information presented in this table on the training budget
represents the majority of funding for training State personnel, it
does not fully reflect State's efforts to train personnel, because the
department does not maintain complete information on funding for all
training-related efforts.
[End of table]
State officials noted that FSI's budget makes up the majority of
funding for training State personnel and includes resources for
training space, technology, instructors, curriculum development,
salaries of full-time students at FSI, and FSI-funded external
training costs. Bureaus also allot some money for external training
not covered by FSI--bureau allotments ranged from a total of about $3
million in fiscal year 2006 to about $4 million in fiscal year 2009.
However, according to State officials, the estimated funding for 2006
through 2009 does not fully reflect State's efforts to train
personnel, due to the way State tracks funding. Specifically, the
amounts do not include:
* funding spent by posts for training offered at post or for travel
per diem for Foreign Service, civil service, and LE staff to
participate in training regionally or in Washington, D.C.;
* funding spent by bureaus other than FSI for internal training-
related efforts; or:
* some additional training-related costs that may not be detailed as
training-related in State's operating budget.
In addition, State officials said that State does not collect detailed
data showing funding spent for internal and external training; funding
for training Foreign Service, civil service, and LE staff, and for
employees in certain specific employee groups; and funding by bureau,
region, and post.
State officials noted that although they lack a detailed breakdown of
training funding, they track some data on employee participation in
training. However, these data also do not allow for a complete or
detailed analysis and comparison of training for employees in
different groups or locations. Although State provided data on
training hours and courses completed by Foreign Service, civil
service, and LE staff for FSI classroom, distance learning, language,
and external training from fiscal years 2006 through 2010, State
officials said that a more detailed breakdown of this information by
bureau, region, post, or employee group is not available. Further,
although FSI's training continuums provide some guidance for personnel
on required, recommended, and suggested training for employees in
different groups, officials said that State generally only centrally
tracks employee completion of certain specific training mandated by
statute, such as cyber security training, as well for the required
leadership skills courses. State does not generally track
participation in other required, recommended, or suggested training.
As a result, for example, State could not provide data regarding the
percentage of foreign affairs or political officers that had completed
required, recommended, or suggested training for their areas of work.
This type of data could provide information on potential variations or
inconsistencies in training and help ensure employees are given
sufficient and appropriate training and development opportunities. For
example, data that State provided on the percentage of mid-and senior-
level Foreign Service and civil service personnel who had completed
the respective required leadership courses as of May 2010 showed some
differences among employee groups (see table 6). The data showed that
100 percent of Ambassadors had completed the Ambassadorial Seminar. In
addition, the data showed that a majority of Senior Foreign Service
and Senior Executive Service, as well as Foreign Service and civil
service GS-13 to GS-15 equivalents (FS-03 to FS-01), had taken the
required leadership training, however, a greater percentage of civil
service at each level were unenrolled or untrained. State reported
that Foreign Service employees who are recommended for promotion to
the next level must complete the respective requirement within a year
or the promotion will not become effective; in comparison, civil
service personnel are informed of the requirement but monitor their
own compliance.
Table 6: Completion of Required Leadership Skills Training, as of May
2010:
Percentage trained:
Ambassadors: 100%;
SES: 68.7%;
SFS: 94.6%;
GS-15: 61%;
FS-01: 79.5%;
GS-14: 69%;
FS-02: 86.5%;
GS-13: 72%;
FS-03: 87.5%.
Percentage enrolled:
Ambassadors: 0%;
SES: 7.2%;
SFS: 2.3%;
GS-15: 2.7%;
FS-01: 2%;
GS-14: 4%;
FS-02: 2.5%;
GS-13: 4%;
FS-03: 3.5%.
Percentage unenrolled/untrained:
Ambassadors: 0%;
SES: 24.1%;
SFS: 3.1%;
GS-15: 36.3%;
FS-01: 18.5%;
GS-14: 27%;
FS-02: 11%;
GS-13: 24%;
FS-03: 9%.
Source: GAO analysis of State data.
Note:
GS = General Schedule.
FS = Foreign Service.
SES = Senior Executive Service.
SFS = Senior Foreign Service.
[End of table]
Although State tracks some data related to training funding and
delivery, the department does not have sufficient information that
could be used to ensure consistent and appropriate support for
training, or to help determine whether managers and employees have
needed training tools and resources. This is especially important
given evidence of variances in training across the department.
Training officials we interviewed at a number of bureaus and posts
cited varying support for employee training and development. While
some bureaus and posts indicated they had sufficient funding and
support for training, others noted they faced significant resource
challenges that impacted the ability of employees to get necessary
training. For example:
* Bureaus. Officials from several bureaus said that finding time or
resources for training generally was not a challenge. However, this
was not the case for other bureaus. For example, officials
representing two of State's regional bureaus noted that a lack of time
was a main constraint preventing employees from taking needed
training. A training official representing two other bureaus said that
taking employees away from their day-to-day work for training poses
significant challenges, given that some employees have heavy
workloads. The official added that in her experience, although a
majority of bureau civil service supervisors have fulfilled the
mandatory leadership training requirements, supervisors sometimes
enroll but cancel owing to the "crisis of the day." In addition,
although officials from two bureaus said they provide individual
guidance or counseling to each Foreign Service and civil service
employee in the bureau, officials representing some other bureaus
indicated their competing responsibilities do not allow them to spend
much time providing guidance or counseling, and that their primary
role was to process training requests.
* Posts. Training officials we interviewed at 12 posts cited varied
support and resources for training. While several posts said they had
enough funding for training, other posts said funding was insufficient
or had fluctuated in recent years. Further, officials at most of the
posts we interviewed said finding time for training can be a challenge
at post. Only one post said time was not an issue, because the post
has a policy that allows each employee to set aside up to 2 to 3 hours
a week for training. Officials from several posts, as well as a number
of respondents to the 2009 American Foreign Service Association survey
noted although Foreign Service personnel are generally expected to
obtain needed training in between tours, personnel may not have time
for the training due to the departing post's holding onto staff as
long as possible and the subsequent post's requiring staff to arrive
as soon as possible. In addition, while two posts noted they benefited
from having personnel dedicated to training full time, training
officials from the majority of posts we interviewed said their posts
had no one dedicated to training full time.[Footnote 53] For example,
the mission training officer at one large post said she is only able
to spend about 10 percent of her time on training-related
responsibilities.[Footnote 54]
* Regional training. State also has regional centers that provide
training, among other services, in partnership with FSI in three
locations: the Western Hemisphere Affairs Training Division at the
Florida Regional Center (Florida center) in Ft. Lauderdale, Florida;
the Frankfurt Regional Support Center (Frankfurt center) Training and
Development Center in Frankfurt, Germany; and the Regional Employee
Development Center (Bangkok center) in Bangkok, Thailand. Officials
noted these centers primarily support training for LE staff in their
respective regions, and they offer training on-site and at other posts
in their regions. However, support for the centers varies, and the
centers do not formally serve employees working in each of State's
regions worldwide. For example, the Florida center is the only center
with a formal agreement with FSI; under the agreement, the Bureau of
Western Hemisphere Affairs funds an FSI staff member to direct and
provide training through the Florida center. Officials from the
Frankfurt center said they did not need a formal agreement in part
because they already have personnel with adjunct faculty status. In
contrast, officials from the Bangkok center said that they hoped to
establish a more formal agreement that would allow for an FSI
representative to be assigned to the center to help them further
develop their program and obtain adjunct faculty status for their two
full-time LE staff trainers. Officials from the Bankgok center noted
these two trainers currently cannot teach any FSI courses and can only
teach courses developed by the center that would not be eligible for
FSI course credit.[Footnote 55] In addition, officials from the
regional centers indicated that regional training services for the
Bureaus of African Affairs, Near Eastern Affairs, and to some extent
South and Central Asian Affairs are currently ad hoc, given that they
are not formally served by any of the above centers. Officials from
the Frankfurt center said that demand for their training from regions
not formally covered by other centers, including from employees who
work at posts in the African Affairs and Near Eastern Affairs regions
in particular, often exceeds what the Frankfurt center can provide.
For some general services operations courses, the officials said they
have had 120 applicants for 10 to 25 spots.
State Has Not Developed a Formal Plan for Evaluating the Effectiveness
of Training:
Although State has various training evaluation mechanisms in place,
the department lacks a training data collection and analysis plan. As
a result, it is not clear whether or how State systematically makes
decisions regarding how training programs will be evaluated using
different methods or tools, or how results will be used. Our prior
work highlights the importance of planning and conducting evaluations
of the effectiveness of training and development efforts and notes
that a data collection and analysis plan can set priorities for
evaluations and systematically cover the methods, timing, and
responsibilities for an agency's data collection.[Footnote 56] While
State has implemented mechanisms to evaluate training, including
course evaluations and an annual training survey, these mechanisms do
not fully incorporate LE staff, and primarily focus on participant
satisfaction or reaction to training, rather than desired results,
such as improved quality or efficiency of work.
State has a range of training evaluation mechanisms and procedures to
obtain feedback from supervisors and employees on training, such as
FSI's annual training survey, and mid-and-post-training course
evaluations. FSI has developed and updated lists of evaluations used
for different courses, which generally include information such as the
intended recipients (e.g., students or supervisors), frequency, and
timing of the course evaluation, as well as responsibilities for
compiling the evaluations and for reviewing responses received. FSI
also provided several examples of adjustments to training efforts in
response to feedback received through its course evaluation efforts.
For example, the course manager for Office Management Specialists
training provided a November 2009 memorandum summarizing a number of
changes made to the entry-level training as a result of course
evaluations, among other factors. In addition, FSI provided a summary
of results of an October 2009 offering of the Iraq Provincial
Reconstruction Team course. The summary highlighted changes that had
been made prior to the October offering as well as recommendations for
modifications to be made in the subsequent offering of the course.
However, while FSI has compiled lists of evaluation mechanisms used
for different courses, these lists do not include guidance on setting
priorities for training evaluations, or how to determine appropriate
methods, timing, or responsibilities for evaluating training and
development efforts. They also do not include guidelines to help
ensure State makes an ongoing effort to improve the quality and
breadth of data gathered. We found several potential gaps and areas
for improvement in State's efforts to evaluate personnel training. For
example, FSI's annual training survey is sent to a random sample of
State's Foreign Service and civil service employees and eligible
family members but not LE staff. Likewise, several training-related
questions are also included in the department's biannual Quality of
Work Life Survey, which is sent to a random sample of Foreign Service
and civil service employees but not to LE staff.[Footnote 57] In
addition, although FSI conducted a June 2009 survey of LE staff
training needs, the survey was sent to post management, not to LE
staff, and FSI had not set specific time frames for repeating the
survey as of November 2010. Several officials noted that LE staff
often identify training needs or issues through the process of
creating a Work Development Plan, but not all of the posts we
interviewed required these plans.[Footnote 58]
In addition, respondents to the 2010 Annual Training Survey indicated
they were generally satisfied overall with FSI training delivery,
programs offered, and customer service.[Footnote 59] However,
relatively fewer respondents indicated they were satisfied with the
number of courses offered and their ability to attend courses in their
geographic region. Additionally, some respondents said that they did
not receive training that would have been helpful, such as tradecraft
(e.g., consular, political, or public diplomacy), leadership and
management, language, or information technology end-user training.
However, despite the diverse training needs and requirements of State
personnel, FSI officials told us they were not able to break out
responses to determine whether there were any differences, for
example, between responses of Foreign Service and civil service
employees, or those serving at headquarters or in overseas regions.
Course evaluations used by FSI's schools vary and generally focus on
participant satisfaction or reaction to training, rather than
knowledge gained, behavior change, or impact of the training.[Footnote
60] We reviewed examples of course evaluations used by each of FSI's
schools--the Leadership and Management School, School of Applied
Information Technology, School of Language Studies, and School of
Professional and Area Studies--as well as within each of the 10
divisions under the School of Professional and Area Studies.[Footnote
61] We found that courses generally incorporated mid-course and end-of-
course student surveys, while some also included post-training surveys
(e.g., typically several months after the end of the course), and a
few included supervisors. The surveys included various numbers of
multiple choice and open-ended questions and generally addressed
participant satisfaction or reaction to training but less frequently
included the types of questions that could be used to help determine
on-the-job behavior change or impact of training. For example, for the
Stability Operations division under FSI's School of Professional Area
Studies, we reviewed several end-of-course surveys and post-training
surveys. We found one end-of-course survey (Iraq Familiarization)
included only questions regarding participant satisfaction with course
content; one end-of-course survey (Foundations of Reconstruction &
Stabilization Operations) included questions regarding participant
satisfaction with training as well as self-assessments of knowledge
gained; and four others (Afghanistan Familiarization, Afghanistan and
Iraq Provincial Reconstruction Team end-of-course, and Afghanistan
Provincial Reconstruction Team post-training) included the above types
of questions as well as a question regarding on-the-job behavior
change resulting from training.
While some of FSI's training evaluation mechanisms include questions
that relate to on-the-job behavior change or the impact of training,
FSI's Director of Curriculum and Staff Development said that the
institute's course evaluations tend to focus primarily on participant
reaction to training. The official noted that the evaluations focus to
some extent on knowledge or skills gained, and less on behavior change
or impact of training. FSI officials noted that determining the direct
impact of training can be difficult, particularly in terms of any
resulting cost savings.
Performance Measures Do Not Fully Address Training Goals:
State's performance measures for training generally do not fully
address training goals. We previously reported that an agency's use of
measures to assess training and development efforts is an important
aspect of effective training programs.[Footnote 62] Our prior work
also noted that training and development performance measures, aligned
with the agency's mission and goals, can be used to help ensure
accountability and assess progress toward achieving results.
Indicators of effective training programs include the use of both
quantitative and qualitative measures that provide meaningful data on
training policies and practices, and show how specific efforts promote
mission accomplishment.
State and the U.S. Agency for International Development's Strategic
Plan for Fiscal Years 2007-2012 includes one overarching strategic
goal related to training: "strengthening consular and management
capabilities." However, the goal is not focused exclusively on
training; the plan states that the "overarching goal—in this area is
to provide the best visa and American Citizen services possible—and to
ensure a high quality workforce, including locally employed staff."
Among other priorities listed for the goal, one relates to training
and notes that State and FSI will "integrate systems and coordinate
strategies to improve the skill base, diversity, and performance of
our workforce." The plan does not include any targets or measures
specific to training.
While State's broader strategic plan does not include any training
measures, the bureau strategic resource plans for FSI and for the
Bureau of Human Resources include several training-related goals and
measures, linked to the broader departmental strategic goal on
training. Specifically, the plans include a total of five training-
related goals, with two or three indicators or measures for each goal.
However, the measures generally do not fully address the goals and are
output-rather than outcome-related.[Footnote 63] As a result, they do
not provide a clear means of determining whether State's training
efforts achieve desired results. Table 7 provides further details
about each of the training goals and measures.
Table 7: FSI and Bureau of Human Resources Training-related Goals and
Measures, with GAO Assessments:
FSI's fiscal year 2012 strategic resource plan includes four training
goals, with two indicators, or measures, listed for each goal:
Goal: Workforce meets priority diplomatic and operational requirements
as a result of FSI training;
Goal description: Goal description includes priorities and objectives
such as:
* Expand and enhance language training;
* Support training in stability operations;
* Support for new hire training;
* Increase the number of leadership training offerings to match
increased hiring and stream of mid-level employees progressing to
management levels; and;
* Enhance public diplomacy training, including social media and
technology training courses;
Goal measures and GAO assessment: The goal's two measures, "language
training success rate at FSI," and "development of training continuum
to support State's Office of Reconstruction and Stabilization," are
both output measures and do not fully address the adjacent priority
areas for the goal. For example, the measures do not address support
for new hire training or public diplomacy training.
Goal: Global workforce can more widely access training through
distance learning technologies;
Goal description: Goal description includes priorities and objectives
such as:
* Enhance support for distance learning and update technological
training platforms for distance and classroom learning, such as
through use of Smart boards and a more robust video distribution
platform with expanded reach;
* Undertake major effort to update and integrate employee, training,
and learning management systems to ensure ability to meet training
data reporting requirements and better support regions and posts; and;
* Expand distance learning language training and improve its efficacy,
for example, through creating learning "portals" between students and
teachers and leveraging social media;
Goal measures and GAO assessment: The goal's two measures, "distance
learning growth: increased use of FSI's learning management system and
distance learning," and "increase in foreign affairs distance learning
products available," are both output measures and do not fully address
goal priorities or objectives, such as the effort to update and
integrate data systems for training purposes.
Goal: Management practices promote efficiency and effectiveness;
Goal description: Goal description includes priorities and objectives
such as:
* Develop additional LE staff adjunct faculty;
* Continue to support regional training centers; and;
* Modify curricula to address change to new governmentwide framework
for management of sensitive but unclassified information (to be
labeled as controlled unclassified information);
Goal measures and GAO assessment: The goal's two measures, "overall
satisfaction with training at FSI," and "FSI cost per student
trained/enrollment," do not fully address all priorities or objectives
included in the goal description, such as adjunct faculty and regional
training. In addition, the first measure is not a clearly adequate
indicator of effectiveness. Also, because LE staff are not included in
FSI's annual training survey of employee satisfaction, the measure
does not encompass all State employees.
Goal: Core training continues to fulfill baseline requirements and
meet new challenges/new skills;
Goal description: Goal description includes priorities and objectives
such as:
* Expand training for mid-level supervisors;
* Enhance area studies training, as well as higher-level proficiency
in priority languages like Arabic and Chinese; and;
* Continue providing entry and mid-level training in basic and
priority areas including information technology, political,
economic/commercial, consular, public diplomacy, and counterterrorism;
Goal measures and GAO assessment: The two measures for this goal, "FSI
language training satisfaction rate," and "FSI information technology
training satisfaction rate," are both output measures and do not fully
address the goal priorities or objectives, such as area studies and
entry-and mid-level training in political, economic/commercial,
consular, public diplomacy, and counterterrorism.
The Bureau of Human Resource's fiscal year 2012 strategic resource
plan includes one training-related goal, with three indicators, or
measures for the goal:
Goal: Strengthen diplomacy by effectively recruiting, hiring,
developing, and assigning employees;
Goal description: Training-related priorities or objectives for the
goal address the bureau's performance management responsibilities,
including responsibilities to ensure employees have needed diplomacy,
language, leadership, and other skills to understand and operate in a
changing environment;
Goal measures and GAO assessment: The three measures, "percent of
overseas positions vacant," "percent of language designated positions
at overseas missions filled by employees who fully meet or exceed the
language requirement," and "recruiting, examining, and evaluation
success as measured against annual Foreign Service hiring goals," do
not fully address the training priorities or objectives, as they
primarily address Foreign Service vacancies and recruitment. The
second measure may indirectly relate to language training, as it could
be achieved by hiring personnel who already have the requisite skills,
as well as through language training.
Source: GAO analysis of State information.
[End of table]
Conclusions:
State has developed an extensive program to equip its personnel to
fulfill State's leadership role in world affairs and to advance and
defend U.S. interests. In recent years in particular, State officials
noted the department has enhanced its focus on training and
development of Foreign Service, civil service, and LE staff, such as
by establishing leadership skills training requirements and increasing
distance learning. State has many training practices and processes in
place reflecting the elements of effective training programs,
including the development of an annual workforce training plan and
hundreds of classroom and distance learning offerings, as well as
cross-training and other career development opportunities for
personnel. State has incorporated the use of technology to enhance
training efforts, and implemented oversight mechanisms such as
training agreements to help ensure employees take advantage of
legitimate training. In addition, State has various mechanisms in
place to evaluate training, such as an annual training survey and
course evaluations.
However, we found strategic weaknesses and room for improvement in
several key areas. First, without a systematic, comprehensive
assessment of training needs, State cannot be assured training is
connected to true needs and priorities. Second, without transparent,
complete, and accurate information for all employees on training
opportunities, including on any required or recommended training for
specific positions, State employees may be hampered in their efforts
to plan for training and development throughout their careers. In
addition, without taking several steps to address weaknesses in
State's efforts to evaluate personnel training, it is not clear that
State is sufficiently able to assess whether employees have the
information, competencies, and skills they need to work effectively.
For example, it is not clear that State's training performance
measures provide meaningful information to help the department
determine the quality and effectiveness of training efforts. State
also lacks a plan for training data collection and analysis, as well
as adequate mechanisms for determining whether the department is
providing consistent and appropriate support and funding for training
all employees--including LE staff.
State's budget and focus on training have increased in recent years,
but the department has also faced, and will likely continue to face,
fluctuating and constrained resources and competing priorities when
determining what training is critical to its mission. Without
concerted efforts to further incorporate effective practices, State
cannot ensure training resources are targeted strategically, are not
wasted, and achieve cost-effective and timely results desired, and
thus cannot be assured that its employees are trained and equipped to
meet the challenges of their mission.
Recommendations for Executive Action:
We recommend that the Secretary of State take the following five
actions:
To help ensure that State's personnel training is connected to
improving individual and agency performance and that department
resources are directed to actual training needs and priorities, direct
FSI and the Bureau of Human Resources, in collaboration with other
bureaus and offices, as appropriate, to develop and implement a plan
for a systematic, comprehensive training needs assessment process,
incorporating all bureaus and posts.
To enhance State's efforts to provide transparent, complete, and
accurate information to help employees plan training and development
throughout their careers, direct FSI and other bureaus and offices, as
appropriate, to collaborate in developing and updating information for
employees on training to ensure that employees have complete and
accurate guidance, including information on any mandatory, required,
and recommended training for specific employee groups.
To help ensure that State's performance measures for training provide
meaningful data and more fully address the department's training
goals, direct FSI and the Bureau of Human Resources to review the
performance measures and revise or enhance the measures, as
appropriate.
To enhance State's capacity to evaluate workforce training, direct FSI
and the Bureau of Human Resources to develop a data collection and
analysis plan for training, including guidance for determining the
methods, timing, and responsibilities for training data collection, as
well as how results will be used.
To improve State's ability to determine whether it is providing
consistent and appropriate support and funding for employee training
and development across employee groups and locations, direct FSI and
the Bureau of Human Resources to identify ways to improve the
collection and analysis of training data and results, such as by
enhancing the level of detailed information gathered to determine
whether employees across groups and locations are getting needed
training, and enhancing efforts to determine the impact of training.
These efforts should also include steps to further incorporate LE
staff into State's training evaluation mechanisms.
Agency Comments and Our Evaluation:
We provided a draft of this report to the Department of State. State
provided written comments, which are reproduced in appendix IV.
State generally agreed with our recommendations, and expressed
appreciation for the report's recognition of the wide variety of
training State offers for the department's Foreign Service, civil
service, and locally employed staff worldwide. State highlighted the
importance of assessing training needs, and noted that regular review
of training performance measures is appropriate. The department also
stated that it strives to improve the accuracy, completeness, and
usefulness of training data, and would look for ways to enhance its
ability to assess the effectiveness of training and development
efforts across employee groups and locations.
In our draft report, we recommended that the Secretary of State direct
FSI to develop formal curriculum design guidance identifying
guidelines and criteria to be followed throughout the course
development process, to help ensure that clear and consistent criteria
and procedures are followed throughout the curriculum design process.
After we provided the draft report to State, the department provided
documentation developed by FSI to address this recommendation.
Specifically, State provided an official document developed in
December 2010 that describes and lays out FSI's course design
procedures and guidance, including for determining course objectives,
identifying quality instructors, and evaluating course design. As a
result of these actions, we have removed the relevant recommendation
from the final report.
State also indicated that senior leadership from the department's
Bureau of Human Resources and FSI were not interviewed as part of this
review. However, we disagree with this characterization of our review.
During the course of our review we conducted numerous interviews with
relevant officials representing 26 State bureaus and offices in
Washington, D.C., including senior officials from FSI and the Bureau
of Human Resources--in some cases on multiple occasions. We also
conducted semi-structured telephone interviews with State officials
with training-related responsibilities at 12 overseas missions, and
from State's regional training centers located in Bangkok, Thailand;
Ft. Lauderdale, Florida; and Frankfurt, Germany. Additionally, we had
multiple discussions and e-mail communications regarding training-
related issues and questions with officials from FSI and the Bureau of
Human Resources who were designated by State as our primary contacts
throughout the audit. These officials provided up-to-date and accurate
information to us on State's training efforts in response to our
inquiries and represented the views of senior management, where
applicable (see appendix I for additional information on our
objectives, scope, and methodology).
State also provided technical comments, which we have incorporated
throughout the report, as appropriate.
As agreed with your office, unless you publicly announce the contents
of the report earlier, we plan no further distribution until 30 days
after the report date. At that time, we will send copies of the report
to interested congressional committees and to the Secretary of State.
We will also make copies available to others upon request. In
addition, the report will be available at no charge on the GAO web
site at [hyperlink, http://www.gao.gov].
If you or your staff members have any questions about this report,
please contact me at (202) 512-4268 or fordj@gao.gov. Contact points
for our Offices of Congressional Relations and Public Affairs may be
found on the last page of this report. Other GAO contacts and staff
acknowledgments are listed in appendix V.
Sincerely yours,
Signed by:
Jess T. Ford:
Director, International Affairs and Trade:
[End of section]
Appendix I: Objectives, Scope, and Methodology:
To address our objectives, our analysis focused primarily on the
Foreign Service Institute's (FSI) training, including leadership,
management, professional, and area studies training contributing to
diplomatic readiness of the Department of State's (State) Foreign and
civil service personnel, as well as locally employed staff (LE staff)
overseas. In light of recent GAO work addressing shortfalls in State
personnel's foreign language skills, this report does not focus on
language training.[Footnote 64] This report does not include an
assessment of "hard skills" (e.g. security and law enforcement)
training provided by State's Bureau of Diplomatic Security within its
scope. Specifically, this report addresses the following questions:
(1) What is State's purpose and structure for training personnel? (2)
To what extent does State's training for personnel incorporate the
elements of effective federal training programs? [Footnote 65]
To identify State's purpose and structure for training personnel, we
reviewed and analyzed legislative, regulatory, and State policy and
procedural criteria relevant to training, including information
contained in State's Foreign Affairs Manual on key training roles and
responsibilities. We also collected, reviewed, and analyzed documents
and data including State's annual training plan; departmental and
human capital strategic and workforce planning documents;
organizational charts; information available on State's Internet and
intranet Web pages; information and data on recent FSI course
offerings; and data on personnel participation in training for fiscal
years 2006 to 2010. To assess the reliability of data on FSI course
offerings and on personnel participation in training, we reviewed
relevant documentation and interviewed knowledgeable agency officials.
We determined data on FSI course offerings and on personnel
participation in training were sufficiently reliable to permit an
analysis of where courses are being offered and of the relative amount
of time Foreign Service, civil service, and LE staff spent in
training, including for FSI language and non-language classroom and
distance learning, and external training. To further our understanding
of State personnel training, we interviewed key officials at FSI, the
Bureau of Human Resources, and other bureaus and offices in
Washington, D.C.
To determine the extent to which State's training for personnel
incorporates the elements of effective federal training programs, in
addition to the above, we completed a training assessment with input
from State, using the prior GAO guidance.[Footnote 66] Specifically,
we examined State documents and data relevant to each of the 32
attributes identified in the GAO guidance, and the indicators
associated with each attribute. We used the results of this assessment
to identify strategic weaknesses in State's training based on criteria
contained in GAO and the Office of Personal Management (OPM) guidance,
as well State and other legislative and regulatory guidance and
policy. We reviewed training evaluation mechanisms used by each of the
four FSI schools--the Leadership and Management School, School of
Applied Information Technology, School of Language Studies, and School
of Professional and Area Studies--as well as within each of the 10
divisions under the School of Professional and Area Studies.[Footnote
67] We examined mid- , end-of-course, and post-training course
evaluation results received for a nonrandom sample of about 37
offerings of various courses by the different schools and divisions.
We also reviewed results of broader evaluations of FSI's consular and
language training. Although the results of these evaluations cannot be
generalized to all courses and locations, they represent results for a
broad range of courses across all schools and divisions. We examined
State data on overall funding for training from fiscal years 2006 to
2010, as well as on the percentage of personnel that had completed
State's mandatory leadership skills training requirements as of May
2010. We reviewed results of FSI's annual training survey of Foreign
and civil service employees, as well as responses to training-related
questions included in State's biannual Quality of Work Life Survey and
State's responses to OPM's biannual Federal Human Capital Survey. To
assess the reliability of these data, we reviewed relevant
documentation, including information regarding survey methodology and
response rates. We determined that these data were sufficiently
reliable to provide a general indication of employee satisfaction with
training. We also analyzed responses to training-related questions
included in the American Foreign Service Association's 2009 survey of
State Foreign Service members, including more than 1,000 responses to
an open-ended question regarding whether and how State training could
be improved.[Footnote 68] While the results of this survey are not
generalizeable, they provided us with valuable insights into potential
areas for improvement.
In addition, to enhance our understanding of State's training
practices and potential issues or challenges, we interviewed relevant
officials from 26 State bureaus and offices in Washington, D.C.,
including FSI, the Bureau of Human Resources, and the six geographic
bureaus. For example, we interviewed senior officials from FSI
including FSI's Director, Deputy Director, and the Executive Director
for Management, as well as senior leadership representing each of the
four FSI schools--the Leadership and Management School, School of
Applied Information Technology, School of Language Studies, and School
of Professional and Area Studies. Within the Bureau of Human
Resources, we met with senior officials including the Chief of Career
Development, Division Director of Career Development and Assignments,
Director and Deputy Director of Overseas Employment, and Director of
Policy Coordination. We also conducted semi-structured telephone
interviews with State officials with training-related responsibilities
at 12 overseas missions. We selected a nongeneralizeable sample of
countries designed to ensure geographic diversity; our criteria for
selection also included factors such as a range of post sizes and
hardship differentials. We selected two missions located in each
region--Africa, the Americas (Western Hemisphere), East Asia and the
Pacific, Europe and Eurasia, Middle East and North Africa, and South
and Central Asia.[Footnote 69] Since we did not conduct interviews
with a generalizeable sample of overseas missions, our observations
from these interviews are illustrative but may not be representative
of training at all overseas locations. We also interviewed officials
from State's regional training centers located in Bangkok, Thailand;
Ft. Lauderdale, Florida; and Frankfurt, Germany. Finally, we
interviewed key officials from nongovernmental organizations including
the Center for Strategic and International Studies and the Stimson
Center.[Footnote 70]
We conducted this performance audit from July 2009 to January 2011 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.
[End of section]
Appendix II: Attributes for Review of an Agency Training and
Development Program:
Following is a compilation of the 32 questions or attributes explored
in detail throughout GAO guidance on assessing an agency's strategic
training and development efforts.[Footnote 71] These attributes are
designed for federal agencies to consider in ensuring that training
and development investments are targeted strategically and not wasted
on efforts that are irrelevant, duplicative, or ineffective. The
attributes are distributed across the four interrelated elements of
the training and development process--planning, design,
implementation, and evaluation. The guidance also includes numerous
corresponding indicators for each attribute.
Planning:
a. Does the agency have training goals and related performance
measures that are consistent with its overall mission, goals, and
culture?
b. To what extent do the agency's strategic and annual performance
planning processes incorporate human capital professionals in
partnership with agency leadership and other stakeholders in
addressing agency priorities, including training and development
efforts?
c. How does the agency determine the skills and competencies its
workforce needs to achieve current, emerging, and future agency goals
and missions and identify gaps, including those that training and
development strategies can help address?
d. How does the agency identify the appropriate level of investment to
provide for training and development efforts and prioritize funding so
that the most important training needs are addressed first?
e. What measures does the agency use in assessing the contributions
that training and development efforts make toward individual mastery
of learning and achieving agency goals?
f. How does the agency incorporate employees' developmental goals in
its planning processes?
g. How does the agency integrate the need for continuous and lifelong
learning into its planning processes?
h. Does the agency consider governmentwide reforms and other targeted
initiatives to improve management and performance when planning its
training and development programs?
i. Does the agency have a formal process to ensure that strategic and
tactical changes are promptly incorporated in training and development
efforts as well as other human capital strategies as needed?
Design:
a. What steps does the agency take to ensure that training is
connected to improving individual and agency performance in achieving
specific results?
b. How is the design of the training or development program integrated
with other strategies to improve performance and meet emerging
demands, such as changing work processes, measuring performance, and
providing performance incentives?
c. Does the agency use the most appropriate mix of centralized and
decentralized approaches for its training and development programs?
d. What criteria does the agency use in determining whether to design
training and development programs in-house or obtain these services
from a contractor or other external source?
e. How does the agency compare the merits of different delivery
mechanisms (such as classroom or computer-based training) and
determine what mix of mechanisms to use to ensure efficient and cost-
effective delivery?
f. Does the agency determine a targeted level of improved performance
in order to ensure that the cost of a training or development program
is appropriate to achieve the anticipated benefit?
g. How well does the agency incorporate measures of effectiveness into
courses it designs?
Implementation:
a. What steps do agency leaders take to communicate the importance of
training and developing employees, and their expectations for training
and development programs to achieve results?
b. Is there a training and performance organization that is held
accountable, along with the line executives, for the maximum
performance of the workforce?
c. Are agency managers responsible for reinforcing new behaviors,
providing useful tools, and identifying and removing barriers to help
employees implement learned behaviors on the job?
d. How does the agency select employees (or provide the opportunity
for employees to self-select) to participate in training and
development efforts?
e. What options has the agency considered in paying for employee
training and development and adjusting employee work schedules so that
employees can participate in these developmental activities?
f. Does the agency take actions to foster an environment conducive to
effective training and development?
g. What steps does the agency take to encourage employees to buy in to
the goals of training and development efforts, so that they
participate fully and apply new knowledge and skills when doing their
work?
h. Does the agency collect data during implementation to ensure
feedback on its training and development programs?
Evaluation:
a. To what extent does the agency systematically plan for and evaluate
the effectiveness of its training and development efforts?
b. Does the agency use the appropriate analytical approaches to assess
its training and development programs?
c. What performance data (including qualitative and quantitative
measures) does the agency use to assess the results achieved through
training and development efforts?
d. How does the agency incorporate evaluation feedback into the
planning, design, and implementation of its training and development
efforts?
e. Does the agency incorporate different perspectives (including those
of line managers and staff, customers, and experts in areas such as
financial, information, and human capital management) in assessing the
impact of training on performance?
f. How does the agency track the cost and delivery of its training and
development programs?
g. How does the agency assess the benefits achieved through training
and development programs?
h. Does the agency compare its training investments, methods, or
outcomes with those of other organizations to identify innovative
approaches or lessons learned?
[End of section]
Appendix III: State Organization:
[Refer to PDF for image: organization chart]
Top level:
Secretary of State (S);
* Counselor and Chief of Staff (S/COS);
* United States Agency for International Development (USAID)
Administrator;
* United States Permanent Representative to the United Nations (USUN).
Second level, reporting to the Secretary of State:
* Deputy Secretary of State D(L);
* Deputy Secretary of State D(S);
* Executive Secretariat (S/ES) Executive Secretary.
Third level, reporting to the Secretary of State:
* Under Secretary for Political Affairs (P):
- African Affairs (AF) Assistant Secretary;
- European and Eurasian Affairs (EUR) Assistant Secretary;
- East Asian and Pacific Affairs (EAP)Assistant Secretary;
- Near Eastern Affairs (NEA) Assistant Secretary;
- South and Central Affairs (SCA) Assistant Secretary;
- Western Hemisphere Affairs (WHA) Assistant Secretary;
- International Narcotics and Law Enforcement (INL) Assistant
Secretary;
- International Organizations (IO) Assistant Secretary.
* Under Secretary for Economic, Energy and Agricultural Affairs (E):
- Economic, Energy and Business Affairs (EEB) Assistant Secretary.
* Under Secretary for Arms Control and International Security Affairs
(T):
- International Security and Nonproliferation (ISN) Assistant
Secretary;
- Political-Military Affairs (PM) Assistant Secretary;
- Verification, Compliance and Implementation (VCI) Assistant
Secretary.
* Under Secretary for Public Diplomacy and Public Affairs (R):
- Education and Cultural Affairs (ECA) Assistant Secretary;
- International Information Programs (IIP) Coordinator;
- Public Affairs (PA) Assistant Secretary.
* Under Secretary for Management(M):
- Administration (A) Assistant Secretary;
- Consular Affairs (CA) Assistant Secretary;
- Diplomatic Security and Foreign Missions (DS)Assistant Secretary;
- Foreign Service Institute (FSI) Director;
- Human Resources (HR) Director General of the Foreign Service and
Director of Human Resources;
- Information Resource Management (IRM) Chief Information Officer;
- Medical Services (M/MED) Director;
- Overseas Buildings Operations (OBO) Director;
- Resource Management (RM) Chief Financial Officer.
* Under Secretary for Democracy and Global Affairs (G):
- Democracy, Human Rights and Labor (DRL) Assistant Secretary;
- Oceans and Int'l Environmental and Scientific Affairs (OES)
Assistant Secretary;
- Population, Refugees and Migration (PRM) Assistant Secretary;
- Office to Monitor and Combat Trafficking in Persons (G/TIP)
Ambassador at Large.
Fourth level, reporting to the Secretary of State:
* Intelligence and Research (INR)Assistant Secretary;
* Legislative Affairs (H) Assistant Secretary;
* Office of the Chief of Protocol (S/CPR) Ambassador;
* Office of Civil Rights (S/OCR) Director;
* Office for Counterterrorism (S/CT) Coordinator and Ambassador at
Large;
* Office of U.S. Foreign Assistance (F) Director;
* Office of Global AIDS Coordinator (S/GAC) Ambassador at Large;
* Office of Global Women's Issues (S/GWI) Ambassador at Large;
* Office of Inspector General (OIG) Inspector General;
* Office of International Energy Coordinator (S/IEC) Coordinator;
* Office of Legal Adviser (L) Legal Adviser;
* Office of Policy Planning (S/P) Director;
* Office of Reconstruction and Stabilization (S/CRS) Coordinator;
* Office of War Crimes Issues (S/WCI) Ambassador at Large.
Source: State.
[End of section]
Appendix IV: Comments from the Department of State:
Note: GAO comments supplementing those in the report text appear at
the end of this appendix.
United States Department of State
Chief Financial Officer:
Washington, D.C. 20520:
January 6, 2011:
Ms. Jacquelyn Williams-Bridgers:
Managing Director:
International Affairs and Trade:
Government Accountability Office:
441 G Street, N.W.
Washington, D.C. 20548-0001:
Dear Ms. Williams-Bridgers:
We appreciate the opportunity to review your draft report, "Department
of State: Additional Steps Are Needed to Improve Strategic Planning
and Evaluation of Training for State Personnel," GAO Job Code 320696.
The enclosed Department of State comments are provided for
incorporation with this letter as an appendix to the final report.
If you have any questions concerning this response, please contact
Bert Curtis, Policy Analyst, Bureau of Human Resources at (202) 647-
2655.
Sincerely,
Signed by:
James L. Millette:
cc: GAO ” Jess Ford:
DGHR ” Nancy J. Powell:
State/OIG ” Evelyn Klemstine:
[End of letter]
Department of State Comments on GAO Draft Report:
Additional Steps Are Needed to Improve Strategic Planning and
Evaluation of Training for State Personnel (GA0-11-241, GAO Code
320696):
The Department thanks GAO for its evaluation of the training of the
Department's three major work forces: Civil Service, Foreign Service
and Locally Employed staff. The Department appreciates GAO's
recognition of the wide variety of training at the Foreign Service
Institute (FSI), around the world and in the virtual realm as we seek
to put the right people into the right place at the right time with
the right skills.
The Department has long welcomed GAO's interest in the training and
development of our employees. Recent GAO studies have approached the
Department's training from a variety of perspectives. GAO's report
(GA0-11-108) on activities intended to improve inter-agency national
security collaboration describes the Department's leadership among
civilian agencies in promoting interagency understanding via a myriad
of FSI courses, rotations, and details. As a result of GAO's report
(GAO-09-955) on foreign language programs at the Department, we are
developing a complex model to predict the Department's resource needs
in language training, which will be the precursor of a second model
for training overall, if funding allows.
We were pleased to note that of the 32 attributes of our training
programs reviewed by GAO, the vast majority were found to be
satisfactory, with only six areas, by GAO's account, in need of
improvement. We believe that one, in particular, has already been
accomplished fully by the Department and we will further explore our
options on the others.
We would like to note that, unfortunately, senior leadership in the
Department's Bureau of Human Resources (HR) and FSI were not
interviewed as a part of the GAO review. We believe that such
interaction would have added substantially to an understanding of the
Department's training programs.
Recommendation 1: Develop and implement a plan for a systematic,
comprehensive training needs assessment process, incorporating all
bureaus and posts. We agree that the assessment of training needs is a
critical part of the human resource tool kit. HR contracted for two
comprehensive job analyses for Foreign Service generalists and
specialists that were completed, respectively in 2007 and 2009. The
job analyses asked questions about employee preparation for specific
tasks, and FSI used this data to modify its course offerings to
address training needs.
Due to the fact that the Civil Service does not have an up-or-out
system parallel to that of the Foreign Service, a career "roadmap" for
the Department's Civil Service workforce will not have the same level
of detail or complexity as the Foreign Service's Career Development
Program. The Office of Civil Service Human Resource Management (CSHRM)
already has well-defined training plans for entry level employees in
our Mission Critical Occupations, the majority of whom are hired
through highly structured federal internship and fellowship programs.
CSHRM is preparing an action plan to conduct a more formal training
needs assessment for the Department's Civil Service overall.
The Department's diverse Locally Employed staff workforce of over
43,000 hired at our 270 unique posts requires flexibility in assessing
needs and planning and administering training. The Department has
determined that decentralized training decisions are practical and
necessary. Often training plans need to be adjusted to accommodate the
changing budget outlook for a fiscal year. In addition, training needs
can change rapidly as priorities shift quickly due to unexpected
vacancies and subsequent reorganization. Post's HRO and/or Management
Officers working in close collaboration with individual supervisors
are in the best position to assess training needs.
Much of the training at our posts is accomplished via on-the-job
experience, a form of training not explicitly covered by this review.
Since other training overseas is achieved with post-specific funding,
the Director General considers it essential that all posts renew their
commitment to training -- keeping in mind the Quadrennial Diplomacy
and Development Review (QDDR) recommendations -- and establish
training budgets and plans early in the fiscal year. These training
budgets are difficult to "wall off" against priorities and protect
against routine encroachment, a challenge that may prove particularly
daunting in the coming year, when creative solutions, such as virtual
training opportunities, will be essential.
Recommendation 2: Collaborate in developing and updating information
for employees on training to ensure that employees have complete and
accurate guidance, including information on any mandatory, required,
and recommended training for specific employee groups. Our Foreign
Service has comprehensive guidance available in the form of the Career
Development Program. Mandatory training is outlined in the
Department's Foreign Affairs Manual which is official Department
guidance. We agree with GAO that we should seek to improve agency-wide
career development guidelines for the Civil Service workforce. Over
the next three years, we will focus on mid-level employees in our
Mission Critical Occupations via the training needs assessment plan
mentioned in Recommendation #1. We will develop or adjust Department
guidelines once we have completed the assessment and coordinate with
FSI to identify appropriate training. [See comment 1]
Recommendation 3: Develop formal curriculum design guidance
identifying guidelines and criteria to be followed throughout the
course development process. We do not agree that the Department lacks
such formal guidance. FSI has a rigorous process for course design and
approval, which includes consultation with FSI's many experts in adult
learning methodology, peer review, analysis of target audience,
learning objectives, and delivery mechanisms as well as those with
subject matter expertise. For easier reference, and to facilitate
understanding of our methodology, FSI has recently documented its
course design process within one document. During our review of the
draft report, we presented this document to GAO.
Recommendation 4: Review training performance measures and revise or
enhance the measures, as appropriate. FSI originally developed
performance measures in the Bureau Strategic and Resource Plan (BSRP)
as part of OMB's Performance Assessment Rating Tool. However, we agree
that regular review of performance measures is appropriate.
Recommendation 5: Develop a data collection and analysis plan for
training, including guidance for determining the methods, timing, and
responsibilities for training data collection, as well as how results
will be used. The Department currently collects training delivery data
for its workforces in a variety of custom designed systems and
regularly strives to improve the accuracy, completeness and usefulness
of this data. We agree that additional tools in the area of
effectiveness would be useful and agree to examine whether our
existing reporting systems might be modified, within existing
resources, toward this end.
Over the last 18 months, FSI has expanded its use of "Level 3"
evaluations-”measuring the workplace impact of training after the
student has returned to the job. FSI plans to continue this expansion,
and if technological, financial and human resources are available,
work with HR to implement a more sophisticated means of data
collection and analysis which could feed into Recommendation #6.
Recommendation 6: Identify ways tq improve the collection and analysis
of training data and results across employee groups and locations.
Bureau training officers have access to training information by skill
and location through the data warehouse known as the Knowledge Center.
FSI and HR agree to explore ways to ensure that bureau executive
directors are made aware of the potential uses of this data and--if
technological, financial and human resources are available-”to
identify potential improvements to existing training delivery and
effectiveness based upon the data collection and anal sis plan.
Special emphasis would be given to training effectiveness by location
of the employee and employee skill codes.
Following are GAO's comments on State's letter dated January 6, 2011.
GAO Comment:
1. We still believe that gaps exist in guidance available for the
Foreign Service, as well as for civil service and locally employed
staff, because key tools used to provide employees with information on
training are not comprehensive. Specifically, we found that documents
for employees on training--known as training continuums--that FSI
developed do not include complete and accurate information. While the
continuums state that they were designed to provide a broad overview
of appropriate training that should be considered as employees plan
their careers in the department, including information on mandatory,
recommended, and suggested courses, we found issues that raised
questions about the usefulness and reliability of the continuums as
employee resources. For example, we found that specific training
requirements designated by bureaus and posts for certain groups of
employees are not always identified in the training continuums. A key
official from FSI's executive office stated that in some cases,
decisions regarding what information would be listed in the continuums
were not fully vetted throughout the agency. The official acknowledged
that the continuums do not include complete and accurate information
for employees on training, and noted that the documents have not been
reviewed to ensure they uniformly reflect departmental policies or
standards.
[End of section]
Appendix V: GAO Contact and Staff Acknowledgments:
GAO Contact:
Jess T. Ford, (202)512-4268 or fordj@gao.gov:
Staff Acknowledgments:
In addition to the contact named above, Anthony Moran, Assistant
Director; Lisa Helmer; Shirley Min; Joe Carney; Virginia Chanley;
Kieran Cusack; David Dayton; Patrick Lockett; Reid Lowe; and Mary
Moutsos provided significant contributions to the work. Etana Finkler,
Farhanaz Kermalli, and Mona Sehgal provided technical assistance.
[End of section]
Related GAO Products:
Foreign Language Capabilities: Departments of Homeland Security,
Defense, and State Could Better Address Their Foreign Language Needs
and Capabilities and Address Shortfalls. [hyperlink,
http://www.gao.gov/products/GAO-10-715T]. Washington D.C.: July 29,
2010.
Aviation Security: Efforts to Validate TSA's Passenger Screening
Behavior Detection Program Underway, but Opportunities Exist to
Strengthen Validation and Address Operational Challenges. [hyperlink,
http://www.gao.gov/products/GAO-10-763]. Washington D.C.: May 20, 2010.
Department of State: Persistent Staffing and Foreign Language Gaps
Compromise Diplomatic Readiness. [hyperlink,
http://www.gao.gov/products/GAO-09-1046T]. Washington, D.C.: September
24, 2009.
Department of State: Additional Steps Needed to Address Continuing
Staffing and Experience Gaps at Hardship Posts. [hyperlink,
http://www.gao.gov/products/GAO-09-874]. Washington, D.C.: September
17, 2009.
Department of State: Comprehensive Plan Needed to Address Persistent
Foreign Language Shortfalls. [hyperlink,
http://www.gao.gov/products/GAO-09-955]. Washington, DC.: September
17, 2009.
Veterans' Benefits: Increased Focus on Evaluation and Accountability
Would Enhance Training and Performance Management for Claims
Processors. [hyperlink, http://www.gao.gov/products/GAO-08-561].
Washington, D.C.: May 27, 2008.
State Department: Staffing and Foreign Language Shortfalls Persist
Despite Initiatives to Address Gaps. [hyperlink,
http://www.gao.gov/products/GAO-07-1154T]. Washington D.C.: August 1,
2007.
Border Patrol: Costs and Challenges Related to Training New Agents.
[hyperlink, http://www.gao.gov/products/GAO-07-997T]. Washington,
D.C.: June 19, 2007.
Department of State: Staffing and Foreign Language Shortfalls Persist
Despite Initiatives to Address Gaps. [hyperlink,
http://www.gao.gov/products/GAO-06-894]. Washington D.C.: August 4,
2006.
Department of Homeland Security: Strategic Management of Training
Important for Successful Transformation. [hyperlink,
http://www.gao.gov/products/GAO-05-888]. Washington D.C.: September
23, 2005.
Aviation Security: Flight and Cabin Crew Member Security Training
Strengthened, but Better Planning and Internal Controls Needed.
[hyperlink, http://www.gao.gov/products/GAO-05-781]. Washington D.C.:
September 6, 2005.
Human Capital: A guide for Assessing Strategic Training and
Development Efforts in the Federal Government. [hyperlink,
http://www.gao.gov/products/GAO-04-546G]. Washington D.C.: March 1,
2004.
Human Capital: Selected Agencies' Experiences and Lessons Learned in
Designing Training and Development Programs. [hyperlink,
http://www.gao.gov/products/GAO-04-291]. Washington D.C.: January 30,
2004.
U.S. Public Diplomacy: State Department Expands Efforts but Faces
Significant Challenges. [hyperlink,
http://www.gao.gov/products/GAO-03-951]. Washington, D.C.: September
4, 2003.
[End of section]
Footnotes:
[1] According to State, the total number of Foreign Service, civil
service, and locally employed personnel increased from about 57,000 in
September 2006 to more than 66,000 as of September 2010, an increase
of about 17 percent.
[2] State's LE staff include foreign nationals and U.S. citizen
residents employed via direct-hire appointments, personal services
agreements, or personal services contracts.
[3] We previously developed guidance for assessing federal strategic
training and development efforts, including identifying four essential
and interrelated elements of the training and development process: (1)
planning, (2) design, (3) implementation, and (4) evaluation. The
guidance includes key attributes of effective federal training
programs to consider when assessing each of the four elements, along
with indicators related to each attribute. This guidance can be used
to identify potential gaps or areas where improvements may be made to
help ensure that training and development investments are targeted
strategically and not wasted on efforts that are irrelevant,
duplicative, or ineffective. GAO, Human Capital: A Guide for Assessing
Strategic Training and Development Efforts in the Federal Government,
[hyperlink, http://www.gao.gov/products/GAO-04-546G] (Washington,
D.C.: Mar. 1, 2004).
[4] GAO, Department of State: Comprehensive Plan Needed to Address
Persistent Foreign Language Shortfalls, [hyperlink,
http://www.gao.gov/products/GAO-09-955] (Washington, D.C.: Sept. 17,
2009).
[5] The 10 divisions under the School of Professional and Area Studies
are Management Tradecraft, Area Studies, Consular Training, Curriculum
and Staff Development, Economic and Commercial Studies, Office
Management Training, Orientation, Political Training, Public
Diplomacy, and Stability Operations.
[6] We provided input to the training-related questions that were
included in the American Foreign Service Association survey.
[7] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[8] Missions include embassies, consulates, and branch offices.
[9] State currently has approximately 150 civil service employees
serving in temporary Foreign Service appointments to provide support
in critical posts overseas.
[10] The new positions are in addition to positions vacated through
attrition.
[11] Public Law No. 96-465, 22 U.S.C. §3901 et. seq.
[12] Codified at Title 5 U.S.C. Chapter 41 (5 U.S.C. §4101 et. seq.).
This chapter, with a few exceptions, does not apply to the Foreign
Service of the United States.
[13] See Executive Order No. 11348, April 20, 1967, 32 F.R. 6335, as
amended by Executive Order No. 12107, December 28, 1978, 44 F.R. 1055.
[14] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[15] In addition to State employees, FSI offers training to employees
from approximately 50 other federal agencies.
[16] Our calculations of FSI's course offerings omitted 1,200
offerings of a 1-hour end-user course on State's new departmentwide
messaging system.
[17] According to State officials, the Regional Service Center in
Frankfurt also provides services, including training, to State
employees posted in the South and Central Asian Affairs region,
because this region was once part of the Bureau of European and
Eurasian Affairs. Employees at posts in bureaus that do not operate
regional service centers may take FSI courses at one of the other
centers on a space-available basis.
[18] FSI reviews and approves applications for external training and
maintains records of all external training it funds, in order to
prevent duplicative efforts. FSI may contribute up to $995 per
external training course per quarter for each Foreign Service and
civil service employee.
[19] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[20] While an agency is not necessarily expected to reflect all of the
indicators associated with each of the 32 attributes identified in the
GAO guidance, we used the guidance as a flexible framework for
identifying potential areas for improvement in State's workforce
training efforts.
[21] 5 C.F.R. § 410.201.
[22] We did not assess "hard skills" security and law enforcement
training provided by the Bureau of Diplomatic Security as part of this
review.
[23] While prior GAO guidance does not identify a specific target
percentage or amount of an agency's budget that should be dedicated to
training, State's workforce training plan includes information on the
amount of funding that State dedicates to training as overall evidence
of the department's commitment and support for training and
development.
[24] State uses a model of competencies established by OPM for civil
service executives, managers, and supervisors to perform effectively
in their positions. For junior-, mid-, and senior-level Foreign
Service professionals, State developed precepts specifying key skills
and expected levels of performance.
[25] In addition, during fiscal years 2006 through 2009, the total
number of hours that personnel spent completing distance learning
courses more than doubled--from about 113,000 hours in fiscal year
2006 to about 254,000 in fiscal year 2009. Data include hours spent
completing all distance learning course offerings, including custom-
developed courses, by Foreign Service, civil service, and LE staff
over the time period.
[26] Officials noted training policies and practices may vary by post,
depending in part upon the laws of the host country, which may
influence post decisions regarding the management of training at post.
[27] According to State, the 2010 annual training survey was sent to a
random sample of 5,105 Foreign Service and civil service employees, as
well as eligible family members. Among other things, the survey asked
respondents to rate FSI's training delivery methods, training
programs, and customer service. We determined that the results of this
survey were sufficiently reliable to provide a general indication of
employee satisfaction with training.
[28] According to State, the department conducts the Quality of Work
Life Survey in odd years, when OPM does not conduct the Human Capital
Survey. The 2007 survey was sent to a random sample of Foreign Service
and civil service employees. We determined that the results of this
survey were sufficiently reliable to provide a general indication of
employee satisfaction with training. State officials told us results
of the 2009 survey were not available as of November 2010.
[29] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[30] According to OPM's Training Needs Assessment Handbook: A Guide
for Conducting a Multi-level Needs Assessment (1994), an
organizational needs assessment generally occurs at the highest level
in the organization where broad, cross-cutting, cross-functional
guidance is established and budget or resource decisions for training
are made; an occupational needs assessment focuses on what
competencies and characteristics are required for critical occupation
groups; and an individual needs assessment focuses on specific
knowledge, skills, and abilities required by each employee, viewed
within the context of the agency's strategic goals.
[31] See Executive Order No. 11348, April 20, 1967, 32 F.R. 6335, as
amended by Executive Order No. 12107, December 28, 1978, 44 F.R. 1055.
[32] Office of Inspector General, Inspection of the Foreign Service
Institute, ISP/I-99-16 (Washington, D.C., Department of State, 1999);
and Compliance Follow-up Review of the Inspections of the Bureau of
Human Resources and the Foreign Service Institute, 01-HR-R-060
(Washington, D.C., Department of State, September 2001).
[33] Each of State's bureaus and missions develops an annual bureau or
mission strategic plan, which includes a description of planned
activities for the coming year, as well as goals and indicators or
measures for determining progress, and requested funding. Beginning
with fiscal year 2012, the plans are referred to as strategic resource
plans.
[34] While most of the bureaus we met with had not developed a bureau
career guide, one bureau developed guidance for employees that
includes the type of information called for in the Foreign Affairs
Manual; specifically, it includes professional development and
training goals and objectives for different groups and levels of
employees in the bureau, as well as detailed lists of required and
recommended bureau-sponsored, FSI, and external training for different
groups and levels of bureau employees.
[35] See pp. 32-36 of this report for further discussion regarding FSI
guidance on training for employees in different career paths.
[36] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[37] Posts that we interviewed indicated that because Foreign Service
employees generally receive any needed training either prior to
arriving at a post or between tours, training at the post is generally
focused on LE staff.
[38] FSI officials noted that FSI currently offers training in each of
these areas, and that in some cases, resource constraints may prevent
personnel from enrolling in relevant courses. They added that FSI
generally decides whether or not to continue, expand, or modify course
offerings based on enrollment data and on feedback received.
[39] In response to prior GAO recommendations related to language
training needs and challenges, State officials said State has taken
steps including developing an analytical model to better assess
resources needed, including training, to meet language requirements.
State has also designated a senior Foreign Service Officer as the
"Strategic Language Coordinator," who focuses on language issues and
works to ensure a strategic approach to addressing foreign language
needs. They added State also has a foreign language working group that
meets regularly, which recently tightened requirements for getting
language requirements waived. Officials also said State provided
additional guidance to help posts determine language requirements for
different positions, and is continuing to review the process.
[40] A large number of LE staff serve in roles such as local security
guards, drivers, or maintenance workers. Others may provide mission
support in office management, fraud investigations, or visa
assistance, among other areas. State officials noted although many LE
staff could benefit from training in support functions, such as
customer service and supervisory skills, a smaller number need
training in more substantive areas, such as public diplomacy.
[41] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[42] Other continuums include the Training Continuum for Foreign
Service Financial Management, General Services, and Human Resource
Officers; the Foreign Affairs Life Skills Training Continuum; the
Language Continuum; the Training Continuum for Office Management
Specialists; and the Training Continuum for Foreign Service IT
Professionals. According to State, FSI is in the process of finalizing
a continuum for Foreign Service Facility Managers. In addition, the
Bureau of Diplomatic Security has training continuums for Security
Engineering Officers, Intelligence Research Analysts, Security
Technical Specialists, and Special Agents/Criminal Investigators.
[43] State's OIG previously recommended the department develop
processes for identifying and projecting training needs and establish
specific mandatory training for all employees at every career stage.
OIG, ISP/I-99-16 and 01-HR-R-060.
[44] Specifically, the leadership training generally includes 1 to 2
weeks of basic, intermediate, and advanced leadership skills training
for Foreign Service and civil service personnel at the mid-level and
above; employees new to the Senior Executive Service and the Senior
Foreign Service, as well as new Ambassadors, must also enroll in
respective senior or ambassadorial seminars.
[45] The Notification and Federal Employee Antidiscrimination and
Retaliation Act (No FEAR Act), P.L. 107-174, requires federal agencies
to provide training to employees about their rights under
antidiscrimination and whistleblower laws.
[46] Specifically, the officials said that State has developed Career
Development Playbooks to provide formal guidance to certain groups of
Foreign Service employees regarding State's Career Development Program
requirements. After reviewing a draft of this report, State officials
noted that the new, standardized format FSI developed for the training
continuums no longer includes information on the Career Development
Program, because the new format streamlines the information that is
included in the continuums and provides links to further information
available on the Bureau of Human Resources' and FSI's internal Web
sites.
[47] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[48] FSI's training design workshop covers topics such as determining
training needs for a course, specifying training goals and objectives,
selecting appropriate training methodologies, and documenting a
training design for conducting the training.
[49] State Messaging and Archive Retrieval Toolset, SMART.
[50] In response to a draft of this report, State provided new
evidence of recent steps taken by the department to address our
finding regarding the lack of formal curriculum design guidance. See
p.53 of this report for more information.
[51] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[52] To identify trends in the training budget, we adjusted these
numbers for inflation; however, funding data presented in the table
represent the actual training budget.
[53] These posts did not necessarily indicate that they needed a full-
time training official; for example, one small post indicated it did
not need a full-time training official.
[54] This post does have a full-time foreign language training
coordinator, although the mission training officer is the primary
official responsible for non-language training at post.
[55] In November 2010, FSI officials stated that FSI intends to place
an FSI trainer at the Bangkok center by the end of 2010, with the aim
of expanding adjunct faculty in that location.
[56] Such a plan could also include guidelines to help ensure the
agency makes an ongoing effort to improve the quality and breadth of
data gathered. Our prior work also noted that developing and using
such a plan can guide an agency in a systematic approach to assessing
the effectiveness and efficiency of both specific training and
development programs and more comprehensively assessing its entire
training and development effort. [hyperlink,
http://www.gao.gov/products/GAO-04-546G].
[57] State officials noted that the Quality of Work Life Survey is
managed by OPM, and it is not within State's purview to change its
scope.
[58] FSI officials stated they rely primarily on supervisors of LE
staff to evaluate training for LE staff, as they believe the
supervisors are in the best position to assess any further staff
training needs.
[59] According to State, the 2010 annual training survey was sent to a
random sample of 5,105 Foreign Service and civil service employees, as
well as eligible family members. The survey asked respondents to rate
FSI's training delivery methods, training programs, and customer
service, among other things. We determined that the results of this
survey were sufficiently reliable to provide a general indication of
employee satisfaction with training.
[60] Training experts commonly refer to four progressive levels of
training evaluation: level one measures reaction, or how participants
feel about various aspects of the training; level two measures
learning, or knowledge acquired due to training; level three measures
behavior, or the extent to which participants change on-the-job
behavior as a result of training; and level four measures results or
impact, such as higher productivity, reduced cost, lower employee
turnover, or improved quality of work.
[61] The 10 divisions under the School of Professional and Area
Studies are Management Tradecraft, Area Studies, Consular Training,
Curriculum and Staff Development, Economic and Commercial Studies,
Office Management Training, Orientation, Political Training, Public
Diplomacy, and Stability Operations.
[62] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[63] Our prior work highlights the importance of outcome metrics for
training programs, to ensure accountability and assess progress toward
achieving results. [hyperlink,
http://www.gao.gov/products/GAO-04-546G].
[64] See GAO, Department of State: Comprehensive Plan Needed to
Address Persistent Foreign Language Shortfalls, [hyperlink,
http://www.gao.gov/products/GAO-09-955] (Washington, D.C.: Sept. 17,
2009). In response to prior GAO recommendations related to language
training needs and challenges, State officials said State has taken
steps including developing an analytical model to better assess
resources needed, including training, to meet language requirements.
State has also designated a senior Foreign Service Officer as the
"Strategic Language Coordinator," who focuses on language issues and
works to ensure a strategic approach to addressing foreign language
needs. They added State also has a foreign language working group that
meets regularly, which recently tightened requirements for getting
language requirements waived. Officials also said State provided
additional guidance to help posts determine language requirements for
different positions, and is continuing to review the process.
[65] Previous GAO work has developed guidance for assessing federal
strategic training and development efforts, including identifying four
essential and interrelated elements of the training and development
process: (1) planning, (2) design, (3) implementation, and (4)
evaluation. The guidance includes key attributes of effective federal
training programs to consider when assessing each of the four
elements, along with indicators related to each attribute. This
guidance can be used to identify potential gaps or areas where
improvements may be made to help ensure that training and development
investments are targeted strategically and not wasted on efforts that
are irrelevant, duplicative, or ineffective. GAO, Human Capital: A
Guide for Assessing Strategic Training and Development Efforts in the
Federal Government, [hyperlink,
http://www.gao.gov/products/GAO-04-546G] (Washington, DC: Mar. 1,
2004).
[66] [hyperlink, http://www.gao.gov/products/GAO-04-546G].
[67] The 10 divisions under the School of Professional and Area
Studies are Management Tradecraft, Area Studies, Consular Training,
Curriculum and Staff Development, Economic and Commercial Studies,
Office Management Training, Orientation, Political Training, Public
Diplomacy, and Stability Operations.
[68] We provided input to the training-related questions that were
included in the American Foreign Service Association's survey.
[69] Specifically, we conducted interviews with training officials
from missions in Kenya and Nigeria in Africa, Mexico and Suriname in
the Americas, China and the Marshall Islands in East Asia and the
Pacific, Switzerland and Ukraine in Europe and Eurasia, Egypt and
Algeria in the Middle East and North Africa, and India and Uzbekistan
in South and Central Asia.
[70] We conducted interviews with key officials from nongovernmental
organizations that have reported on issues related to training for
State personnel in recent years. See Center for Strategic and
International Studies, The Embassy of the Future (Washington, D.C.:
2007), and The American Academy of Diplomacy, A Foreign Affairs Budget
for the Future: Fixing the Crisis in Diplomatic Readiness (Washington,
D.C.: October 2008). The Stimson Center was a contributor to the
American Academy of Diplomacy report.
[71] GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, [hyperlink,
http://www.gao.gov/products/GAO-04-546G] (Washington, DC: Mar. 1,
2004).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
