Department of Homeland Security
Strategic Management of Training Important for Successful Transformation Gao ID: GAO-05-888 September 23, 2005Training can play a key role in helping the Department of Homeland Security (DHS) successfully address the challenge of transformation and cultural change and help ensure that its workforce possesses the knowledge and skills needed to effectively respond to current and future threats. This report discusses (1) how DHS is addressing or planning to address departmentwide training and the related challenges it is encountering; (2) examples of how DHS training practices, specifically those related to planning and evaluation, reflect strategic practices; and (3) examples of how DHS uses training to foster transformation and cultural change.
DHS has taken several positive steps toward establishing an effective departmentwide approach to training, yet significant challenges remain. Progress made in addressing departmentwide training issues, but efforts are still in the early stages and face several challenges. Actions taken by DHS include issuing its first training strategic plan in July 2005, establishing training councils and groups to increase communication across components, and directly providing training for specific departmentwide needs. However, several challenges may impede DHS from achieving its departmental training goals. First, the sharing of training information across components is made more difficult by the lack of common or compatible information management systems and a commonly understood training terminology. Second, authority and accountability relationships between the Office of the Chief Human Capital Officer and organizational components are not sufficiently clear. Third, DHS's planning may be insufficiently detailed to ensure effective and coordinated implementation of departmentwide training efforts. Finally, according to training officials, DHS lacks resources needed to implement its departmental training strategy. Examples of planning and evaluation of training demonstrate some elements of strategic practice. Specific training practices at both the component and departmental levels may provide useful models or insights to help others in DHS adopt a more strategic approach to training. We found that some components of DHS apply these practices, while others do not. For example, Customs and Border Protection (CBP) aligns training priorities with strategic goals through planning and budgeting processes. In the area of evaluation, the Federal Law Enforcement Training Center obtains feedback from both the trainee and the trainee's job supervisor to inform training program designers in order to make improvements to the program curriculum. Training has been used to help DHS's workforce as it undergoes transformation and cultural change. The creation of DHS from different legacy organizations, each with its own distinct culture, has resulted in significant cultural and transformation challenges for the department. At the departmental level, one of the ways DHS is addressing these challenges is by encouraging the transformation to a shared performance-based culture through the implementation of its new human capital management system, MAXHR. DHS considers training to be critical to effectively implementing this initiative and defining its culture. Toward that end, the department is providing a wide range of training, including programs targeted to executives, managers, and supervisors. For example, at the component level, CBP has developed cross-training to equip employees with the knowledge needed to integrate inspection functions once carried out by three different types of inspectors at three separate agencies.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-05-888, Department of Homeland Security: Strategic Management of Training Important for Successful Transformation
This is the accessible text file for GAO report number GAO-05-888
entitled 'Department of Homeland Security: Strategic Management of
Training Important for Successful Transformation' which was released on
September 23, 2005.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Oversight of Government
Management, the Federal Workforce, and the District of Columbia,
Committee on Homeland Security and Governmental Affairs, U.S Senate:
United States Government Accountability Office:
GAO:
September 2005:
Department of Homeland Security:
Strategic Management of Training Important for Successful
Transformation:
GAO-05-888:
GAO Highlights:
Highlights of GAO-05-888, a report to the Chairman, Subcommittee on
Oversight of Government Management, the Federal Workforce, and the
District of Columbia, U.S. Senate:
Why GAO Did This Study:
Training can play a key role in helping the Department of Homeland
Security (DHS) successfully address the challenge of transformation and
cultural change and help ensure that its workforce possesses the
knowledge and skills needed to effectively respond to current and
future threats. This report discusses (1) how DHS is addressing or
planning to address departmentwide training and the related challenges
it is encountering; (2) examples of how DHS training practices,
specifically those related to planning and evaluation, reflect
strategic practices; and (3) examples of how DHS uses training to
foster transformation and cultural change.
What GAO Found:
DHS has taken several positive steps toward establishing an effective
departmentwide approach to training, yet significant challenges remain.
Progress made in addressing departmentwide training issues, but efforts
are still in the early stages and face several challenges. Actions
taken by DHS include issuing its first training strategic plan in July
2005, establishing training councils and groups to increase
communication across components, and directly providing training for
specific departmentwide needs. However, several challenges may impede
DHS from achieving its departmental training goals. First, the sharing
of training information across components is made more difficult by the
lack of common or compatible information management systems and a
commonly understood training terminology. Second, authority and
accountability relationships between the Office of the Chief Human
Capital Officer and organizational components are not sufficiently
clear. Third, DHS‘s planning may be insufficiently detailed to ensure
effective and coordinated implementation of departmentwide training
efforts. Finally, according to training officials, DHS lacks resources
needed to implement its departmental training strategy.
Examples of planning and evaluation of training demonstrate some
elements of strategic practice. Specific training practices at both the
component and departmental levels may provide useful models or insights
to help others in DHS adopt a more strategic approach to training. We
found that some components of DHS apply these practices, while others
do not. For example, Customs and Border Protection (CBP) aligns
training priorities with strategic goals through planning and budgeting
processes. In the area of evaluation, the Federal Law Enforcement
Training Center obtains feedback from both the trainee and the
trainee‘s job supervisor to inform training program designers in order
to make improvements to the program curriculum.
Training has been used to help DHS‘s workforce as it undergoes
transformation and cultural change. The creation of DHS from different
legacy organizations, each with its own distinct culture, has resulted
in significant cultural and transformation challenges for the
department. At the departmental level, one of the ways DHS is
addressing these challenges is by encouraging the transformation to a
shared performance-based culture through the implementation of its new
human capital management system, MAXHR. DHS considers training to be
critical to effectively implementing this initiative and defining its
culture. Toward that end, the department is providing a wide range of
training, including programs targeted to executives, managers, and
supervisors. For example, at the component level, CBP has developed
cross-training to equip employees with the knowledge needed to
integrate inspection functions once carried out by three different
types of inspectors at three separate agencies.
What GAO Recommends:
We recommend that the Secretary of Homeland Security (1) adopt
additional good planning practices, specifically, creating a clearer
crosswalk between training goals and DHS‘s organizational and human
capital strategic goals and developing appropriate performance measures
and targets; (2) specify authority/accountability relationships
between CHCO and components on training; (3) ensure the department and
components develop detailed training implementation plans; and (4) when
setting funding priorities, give appropriate attention to supporting
training councils and groups. DHS generally agreed with the report‘s
recommendations.
www.gao.gov/cgi-bin/getrpt?GAO-05-888.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact George Stalcup at (202)
512-9490 or stalcupg@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
DHS Has Made Progress in Addressing Departmentwide Training Issues, but
Challenges May Impede Its Efforts:
DHS's Approach to Planning and Evaluation of Training Demonstrates Some
Elements of Strategic Practice, but Is Still Evolving:
DHS Has Used Training in an Effort to Help Its Workforce as the
Department Undergoes Transformation and Cultural Change:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Scope and Methodology:
Appendix II: "One Face at the Border" Training at U.S. Customs and
Border Protection:
Appendix III: Comments from the Department of Homeland Security:
Table:
Table 1: Responsibilities and Selected Training Characteristics of DHS
Components Included in Our Review:
Figures:
Figure 1: DHS Organizational Structure:
Figure 2: Departmental Training Councils and Subgroups at DHS:
Figure 3: Timeline for MAXHR Training Scheduled Through the End of
2005:
Figure 4: Levels of Training Evaluation:
Figure 5: DHS Components Included in Our Review:
Abbrevations:
ADL: Advanced Distributed Learning:
ADLG: Advanced Distributed Learning Group:
CBP: U.S. Customs and Border Protection:
CHCO: Chief Human Capital Officer:
CIS: U.S. Citizenship and Immigration Services:
DHS: Department of Homeland Security:
FLETC: Federal Law Enforcement Training Center:
FTE: full-time equivalent:
HPT: Human Performance Technology:
ICE: U.S. Immigration and Customs Enforcement:
LTDG: Leadership Training and Development Group:
NTEP: National Training Evaluation Program:
SES: Senior Executive Service:
TLC: Training Leaders Council:
TSA: Transportation Security Administration:
United States Government Accountability Office:
Washington, DC 20548:
September 23, 2005:
The Honorable George V. Voinovich:
Chairman:
Subcommittee on Oversight of Government Management, the Federal
Workforce, and the District of Columbia:
Committee on Homeland Security and Governmental Affairs:
United States Senate:
Dear Mr. Chairman:
Addressing an organization's culture--that is, its underlying
assumptions, beliefs, values, attitudes, and expectations--is at the
heart of any serious organizational transformation or change management
initiative. As the Department of Homeland Security (DHS) strives to
protect the nation from terrorism, it faces significant challenges in
transforming from 22 separate agencies and programs to a single
coordinated department, requiring the integration of approximately
180,000 employees as well as multiple management systems and processes.
In recognition of these challenges, we have designated the
implementation and transformation of the department as high
risk.[Footnote 1]
Training and development can play a key role in helping DHS
successfully address the challenge of transformation and cultural
change and help ensure that its workforce possesses the knowledge,
skills, and competencies needed to effectively respond to current and
future threats.[Footnote 2] In addition, a strategic approach to the
management of training can help to effectively target limited resources
and further the achievement of its organizational goals. To this end,
our framework for assessing training management provides a way for DHS
to recognize and develop such an approach.[Footnote 3]
You asked us to examine how DHS, as a federal agency undergoing
transformation, uses training to help achieve its organizational goals.
In response to your request, this report discusses (1) how DHS is
addressing or planning to address departmentwide training and the
related challenges it is encountering; (2) examples of how DHS training
practices, specifically those related to planning and evaluation,
reflect strategic practices; and (3) examples of how DHS uses training
to foster transformation and cultural change.
To achieve our objectives, we reviewed internal training at both the
departmental and component levels, focusing on the systems and
processes in place to manage, plan, and evaluate training for DHS's
workforce. To this end, we analyzed training, management, and planning
documents and interviewed numerous officials responsible for training
issues in the Office of the Chief Human Capital Officer (CHCO office)
and at six organizational components collectively responsible for
training 95 percent of the DHS workforce. To determine whether DHS used
a strategic approach in managing, planning, and evaluating its training
activities, we drew on our previous work regarding strategic planning
and effective management practices, as well as criteria contained in
our guide for assessing strategic training and development efforts in
the federal government. We recognize that DHS provides a significant
amount of training to external audiences, such as state and local
governments; however, given the nature of your request and interest in
examples of how DHS is using training to foster its organizational
transformation, we did not include training intended for audiences
external to DHS within the scope of our review. For more information on
our scope and methodology, see appendix I.
We conducted our work from November 2004 through July 2005 in
accordance with generally accepted government auditing standards.
Results in Brief:
DHS has made progress in addressing departmentwide training issues, but
these efforts are still in the early stages and challenges may impede
these efforts. Thus far, DHS has taken several steps toward
establishing a departmentwide approach to training, including the
following:
* DHS has issued its first training strategic plan providing a
strategic vision for departmental training. It used a collaborative and
inclusive process to develop the plan, consulting with component
training leaders and others.
* It also has established training councils and groups with the goal of
increasing communication across components and fostering greater
collaboration and coordination.
* Finally, it has provided training targeted to address specific
departmentwide needs. Examples of areas where DHS has directly provided
or supported training on the departmental level include (1)
implementation of MAXHR, DHS's new human capital management
system;[Footnote 4] (2) leadership development; and (3) civil rights
and liberties.
However, we identified four challenges that may impede the department
from achieving departmental training goals. First, the sharing of
training information across components is hampered by the lack of a
common or compatible information management infrastructure and the
absence of a commonly understood terminology. Officials told us that
the lack of compatible information technology systems complicated their
efforts to make the most efficient use of training resources across
components. Second, authority and accountability relationships between
the CHCO office and the organizational components are not sufficiently
clear. A clear and agreed-upon understanding of the specific
responsibilities and authorities of the key organizations involved in
training should significantly improve DHS's ability to effectively
implement its training strategies. The department recognizes this need
to clarify the responsibilities and authorities of the CHCO office and
the components and has addressed this need in its training strategic
plan. Third, DHS's planning may be insufficiently detailed to ensure
effective and coordinated implementation of departmentwide training
efforts. Because they share authority for training, the department and
the components need to develop detailed implementation plans to help
ensure that departmentwide training initiatives are coordinated and
effectively implemented. Fourth, according to training officials, DHS
lacks resources needed to implement its departmental training strategy.
While still evolving, some of DHS's training practices at both the
component and department levels demonstrate strategic elements in the
areas of planning and evaluation and may provide useful models or
insights to help others in DHS adopt a more strategic approach to
training. We have reported previously that as part of the planning
process, agencies demonstrating a strategic approach to training align
their training efforts with overall organizational strategic
priorities; some components of DHS apply these practices, while others
do not. U.S. Customs and Border Protection (CBP), for example, employs
practices intended to align training priorities with strategic goals
through planning and budgeting processes. The U.S. Coast Guard also
demonstrates a strategic approach by using a process for determining
whether training is the appropriate intervention to address a specific
performance problem.
With respect to evaluation, we have reported that agencies
demonstrating a strategic approach to training employ a variety of
practices, such as systematically evaluating training, actively
incorporating feedback during training design, and obtaining feedback
from multiple perspectives. Several components and programs we examined
at DHS demonstrate these practices, while others do not. For example,
the Federal Law Enforcement Training Center (FLETC) obtains feedback
from both the trainee and the trainee's job supervisor to inform
training program designers making improvements to the curriculum. CBP
and the CHCO office also use various strategies to evaluate training
programs and take appropriate actions.
DHS also has used training to help foster transformation and cultural
change as the department moves from multiple distinct organizational
cultures to a new culture that endeavors to be increasingly integrated
and performance focused. This is not an easy process, and the creation
of DHS from 22 different agencies and programs has resulted in
considerable cultural and transformation challenges. At the department
level, DHS has addressed these challenges by encouraging the
transformation to a shared performance-based culture through the
adoption of a new human capital management system, known as MAXHR. As
an essential part of implementing this initiative, DHS developed
targeted training for executives, managers, and supervisors, providing
these groups with the tools and information needed to champion the
benefits of a performance-based culture and successfully implement
MAXHR in their components. In another example, this time at the
component level, in order to improve coordination and communication
across inspection functions and enhance flexibility of the workforce,
CBP created the new positions of CBP officer and CBP agriculture
specialist. Cross-training of employees in these new positions helped
CBP to integrate the inspection functions of three former agencies. In
addition, CBP designed and piloted a training module that specifically
targeted how supervisors could more effectively understand the value
and perspective of staff coming from legacy organizational cultures.
To help DHS further establish and implement an effective and strategic
approach to departmental training, we recommend that the Secretary of
Homeland Security take the following actions: (1) adopt additional good
strategic planning and management practices to enhance the department's
training strategic plan by creating a clearer crosswalk between
specific training goals and objectives and DHS's organizational and
human capital strategic goals and developing appropriate performance
measures and targets; (2) clearly specify authority and accountability
relationships between the CHCO office and organizational components
regarding training, as a first step toward addressing issues DHS has
identified for fiscal year 2006; (3) ensure that the department and
component organizations develop detailed implementation plans and
related processes for training initiatives; and (4) when setting
funding priorities, give appropriate attention to providing resources
to support training councils and groups to further DHS's capacity to
achieve its departmentwide training goals.
We provided a draft of this report to the Secretary of Homeland
Security for comment. DHS generally agreed with our recommendations.
The department provided technical comments that we incorporated where
appropriate. DHS's written response is reprinted in appendix III.
Background:
The Homeland Security Act of 2002 created DHS, bringing together 22
agencies and programs responsible for important aspects of homeland
security.[Footnote 5] The intent behind the creation of a single
department was to improve coordination, communication, and information
sharing among these previously separate entities, thereby increasing
their effectiveness in protecting the nation's security. Each of these
organizations brought with it the capacity and expertise to provide
training for its particular aspect of homeland security. For example,
in several cases such as the Coast Guard and FLETC, this training
capacity, as well as the management systems supporting it was
transferred intact with the creation of the new department. In other
cases, such as CBP and U.S. Immigration and Customs Enforcement (ICE),
the training functions of legacy organizations were merged. Table 1
presents information on selected training characteristics of components
in our review, including the origin of each component's training
function. In addition, the Act led to the creation of the CHCO position
in DHS responsible for, among other human capital topics, oversight and
planning of the training of employees.[Footnote 6] The CHCO, who
reports directly to the department's Under Secretary for Management,
has primary responsibility for defining and developing the department's
role regarding training. Figure 1 depicts these positions as well as
the department's major components in the context of DHS's overall
organizational structure.
Training both new and current staff to fill new roles and work in
different ways will play a crucial part in the ability of federal
departments and agencies, such as DHS, as they work to successfully
transform their organizations. In 2004, we issued an assessment guide
that introduces a framework for evaluating the management of training
in the federal government.[Footnote 7] As presented in our guide, the
training process can be segmented into four broad, interrelated phases:
(1) planning/front-end analysis, (2) design/development, (3)
implementation, and (4) evaluation. For each of these phases, we
summarize key attributes of effective training programs and offer
related issues and questions. Using this framework, this report
identifies selected strategic training practices, with a focus on the
planning and evaluation phases, that may offer an opportunity for
others in DHS to build on experiences and practices discussed below.
Table 1: Responsibilities and Selected Training Characteristics of DHS
Components Included in Our Review:
U.S. Citizenship and Immigration Services (CIS); 10,207 FTEs;
Overview of component responsibilities:
* Administer immigration and naturalization adjudication functions,
including immigrant visa petitions, naturalization petitions, and
asylum and refugee applications;
* Establish and administer policies for immigrant services and
benefits;
Selected characteristics of training:
* Training function was transferred from former Immigration and
Naturalization Service with the exception of law enforcement and soft
skills training;
* Conducts extensive field training through district offices and
service centers;
* Operates CIS Academy at FLETC site in Glynco, GA, for basic training
of new employees;
* Provides refresher training for adjudicators at field sites;
* Supervisory training provided by ICE's Leadership Development Center
in Dallas, TX, through a shared services agreement between CIS, ICE,
and CBP.
U.S. Coast Guard; 46,847 FTEs;
Overview of component responsibilities:
* Protect the public, environment, and U.S. economic interests in
nation's ports, waterways, coasts, and international waters;
* Specific responsibilities include: Maritime safety (e.g., search and
rescue), maritime mobility (e.g., aids to navigation and waterways
management), protection of natural resources, maritime security (e.g.,
drug interdiction), and national defense;
Selected characteristics of training:
* Training function transferred as a whole along with the rest of the
Coast Guard from Department of Transportation;
* Operates multiple training programs for (1) indoctrination for new
employees, (2) apprenticeship after indoctrination course is completed,
and (3) specialized skills (e.g., law enforcement inspections);
* Operates special leadership training program at Leadership
Development Center, New London, CT;
* Uses some Department of Defense training courses for specialized
skills.
U.S. Customs and Border Protection (CBP); 40,761 FTEs;
Overview of component responsibilities:
* Protect U.S. borders from terrorism, at and between official ports of
entry, and foster legitimate trade and travel;
Selected characteristics of training:
* Training function structure and processes largely carried over from
former U.S. Customs organization;
* Basic CBP officer training provided at CBP Academy at FLETC site in
Glynco, GA, and basic CBP agricultural specialist training at CBP
Academy at U.S. Department of Agriculture-Professional Development
Center in Frederick, MD;
* Provides basic Border Patrol training at Border Patrol Academy in
Artesia, NM;
* Provides extensive field training at ports throughout the United
States;
* Supervisory training provided at ICE's Leadership Development Center
in Dallas, TX, through a shared services agreement between CIS, ICE,
and CBP.
Federal Law Enforcement Training Center (FLETC); 959 FTEs;
Overview of component responsibilities:
* Train and prepare law enforcement professionals across government;
Selected characteristics of training:
* Training function transferred as a whole from Department of the
Treasury with formation of DHS;
* Consolidates law enforcement training at five FLETC academies for
multiple DHS components;
* FLETC budget covers most of the basic training provided to DHS
components; auxiliary costs, such as room and board, are reimbursed by
components.
U.S. Immigration and Customs Enforcement (ICE); 14,950 FTEs;
Overview of component responsibilities:
* Prevent acts of terrorism by targeting people, money, and materials
that support terrorist and criminal activities focusing on the nation's
border, economic, transportation, and infrastructure security;
Selected characteristics of training:
* Training function partly carried over from legacy Immigration and
Naturalization Service and legacy Customs Inspections and includes a
separate training organization for the Federal Protective Service;
* Federal Protective Service trains its uniformed officers at FLETC
site in Glynco, GA, and ICE Academy is located at same FLETC site;
* ICE's Leadership Development Center in Dallas, TX, provides
supervisor and manager training for multiple DHS components.
U.S. Secret Service; 6,526 FTEs;
Overview of component responsibilities:
* Protect the President and other designated personnel;
* Protect the country's currency and financial infrastructure;
* Provide security for designated national events;
* Selected characteristics of training;
* Training function transferred as a whole along with the rest of the
Secret Service from Department of the Treasury;
* Trains both special agents and uniformed law enforcement officers
with most instruction taking place at Rowley Training Center in
Beltsville, MD;
* Basic training for new employees takes place at FLETC site in Glynco,
GA.
Transportation Security Administration (TSA); 52,467 FTEs;
Overview of component responsibilities:
* Protect the nation's transportation systems to ensure freedom of
movement for people and commerce;
Selected characteristics of training:
* Training function greatly expanded after transfer from Department of
Transportation with formation of DHS; separate training organization in
place for the Federal Air Marshal Service, which was recently
transferred as a result of DHS's Second Stage Review;
* Some training delivered through TSA headquarters office in Arlington,
VA, which also oversees contracted training activities through its
quality assurance unit;
* Most training delivered in field (airport) sites through contractors
and approved instructors;
* Operates an academy in Artesia, NM, and an academy at the FLETC site
in Glynco, GA;
* Federal Air Marshal Service trains new hires at an initial program in
Artesia, NM, and then they receive specialized training in Atlantic
City, NJ; Federal Air Marshal Service field offices also conduct
ongoing training.
Sources: GAO presentation of DHS information and the President's Fiscal
Year 2006 Budget.
Note: Figures showing full-time equivalents (FTE) for components
reflect FTEs listed under "current services" from the President's
Fiscal Year 2006 Budget.
[End of table]
The results of a governmentwide survey conducted by the Office of
Personnel Management in 2004 on human capital practices and employee
attitudes suggest that efforts to identify and build upon examples of
good training practice within DHS may be particularly
relevant.[Footnote 8] For each of the eight questions in the 2004
Federal Human Capital Survey that focused on training related topics,
the percentage of DHS respondents providing positive responses
(typically the top two options on a five-point scale) was lower than
the governmentwide average. In fact, the DHS response ranged from 5 to
20 percentage points lower than the governmentwide average for the same
questions. For example, 54 percent of respondents at DHS indicated that
they received the training they needed in order to perform their jobs,
compared to 60 percent governmentwide. Half (50 percent) of DHS
respondents said that they were either satisfied or very satisfied with
the training they received for their present jobs, as opposed to 55
percent that expressed these levels of satisfaction governmentwide. The
largest difference involved having electronic access to learning and
training programs, where 51 percent of DHS respondents responded
positively, compared to 71 percent governmentwide. A DHS official told
us that the department is aware of the challenges reflected in these
data and is currently exploring options with the Office of Personnel
Management to conduct further analysis. The aim of this work would be
to identify areas where DHS might target additional attention as well
as provide a baseline for future attitude measures.
Figure 1: DHS Organizational Structure:
[See PDF for image]
Note: This organization chart shows the expected end state resulting
from a reorganization announced by the department in July 2005 at the
conclusion of DHS's Second Stage Review process.
[End of figure]
DHS Has Made Progress in Addressing Departmentwide Training Issues, but
Challenges May Impede Its Efforts:
DHS has made progress in addressing departmentwide training issues and
these efforts reflect some of the elements of a strategic approach
toward training as described in our previous work.[Footnote 9] Most
training-related activities at DHS--such as planning, delivery, and
evaluation--primarily take place at the component level and relate to
mission issues. Therefore, any successful approach regarding
departmentwide training issues will require the concerted and
coordinated efforts of multiple components within DHS as well as the
ability of the CHCO to effectively lead a network of different training
organizations. The department's current efforts, although promising,
are still in the early stages and they face significant challenges.
Unless these challenges are successfully addressed they may impede
DHS's ability to achieve its departmentwide training goals.
DHS Recently Developed a Departmental Training Strategy:
DHS recently developed a coordinated departmental training strategy
that supports broader human capital and organizational goals and
objectives. We have previously reported that effective organizations
establish clear goals with an authority structure able to carry out
strategies and tactics, that is, the day-to-day activities needed to
support the organization's vision and mission. By so doing, a well-
designed training function can be directly linked to the organization's
strategic goals and help to ensure that the skills and competencies of
its workforce enable the organization to perform its mission
effectively.
DHS's department-level training strategy is presented in its human
capital and training strategic plans. Issued in October 2004, its human
capital strategic plan includes selected training strategies, such as
developing a leadership curriculum to ensure consistency of
organizational values across the department and using training to
support the implementation of the new DHS human capital management
system, MAXHR. In July 2005, DHS issued its first departmental training
plan, Department of Homeland Security Learning and Development
Strategic Plan, which provides a strategic vision for departmentwide
training. This plan is a significant and positive step toward
addressing departmentwide training challenges. The plan identifies four
short-term goals for fiscal year 2006 and one long-term goal for fiscal
years 2006 through 2010. Among the short-term goals are such tasks as
defining the scope of training activities and improving the governance
process between the CHCO office and individual organizational
components, supporting the rollout of MAXHR, identifying/implementing
best practices, and addressing specific concerns regarding DHS's
training facilities and advanced distributed learning studies. The plan
also articulates a long-term goal for DHS to "become a recognized world-
class learning organization where managers and supervisors effectively
lead people."
Each of these goals is followed by supporting strategies and tactics.
For example, to achieve its goal of ensuring the best use of training
resources through the identification and implementation of best
practices, the plan identifies specific strategies, one of which is to
improve the awareness of ongoing DHS training activities among
organizational components. This strategy is, in turn, supported by
still more specific tactics such as developing a site on the DHS
Interactive system to facilitate the sharing of information across the
training community.
More significant than the fact that DHS issued a training strategic
plan document is the fact that DHS followed an inclusive and
collaborative process while developing it. We have previously reported
that for high-performing, results-oriented organizations, a strategic
plan is not simply a paper-driven exercise or onetime event, but rather
the result of a dynamic and inclusive process wherein key stakeholders
are consulted and involved in the identification of priorities and the
formation of strategies.[Footnote 10] When creating its plan, DHS
consulted training leaders at components throughout the department, in
addition to others, to help develop and review its content. Several
training leaders we spoke with thought highly of this process and the
extent to which it provided them opportunities to contribute and
comment on the draft plan.
DHS has made considerable progress in addressing departmentwide
training issues through the development of its first training strategic
plan. However, there are areas where future efforts can be improved.
Linkage to DHS organizational and human capital strategic plans. Our
past work on strategic planning and management practices shows that
effective strategic plans describe the alignment between an agency's
long-term goals and objectives and the specific strategies planned to
achieve them. Clearly linking training tactics with particular
organizational objectives creates a direct line of sight that can both
facilitate the ability of staff to work toward mission goals and enable
stakeholders to provide meaningful oversight. In the introduction to
the DHS training strategic plan, the department's CHCO highlights the
value of this practice stating that "the key purpose of [the] plan is
to align our education, training and professional development efforts
with the President's Management Agenda and the Department's vision,
mission, core values and strategic plan."
The DHS training strategic plan contains examples of goals, strategies,
and tactics that align with and support goals found in the department's
human capital and organizational strategic plans; however, these
linkages are never actually identified or discussed in the plan itself.
For example, the DHS training strategic plan contains a goal and
several tactics related to MAXHR training. These, in turn, support a
MAXHR goal and strategy in the department's human capital strategic
plan as well as the "organizational excellence" goal of the DHS
strategic plan. However, the training strategic plan does not show
these linkages. Identifying such linkages, either in the training plan
itself or in an appendix, would more clearly communicate to both
internal and external stakeholders the connections and justifications
for specific training goals, strategies, and tactics.
DHS's own human capital strategic plan provides an illustration of one
way to communicate linkages between goals and strategies contained in
the plan and the broader organizational goals they are intended to
support. For example, in an appendix, the DHS human capital strategic
plan contains a matrix that directly links strategies, such as
developing a new Senior Executive Service (SES) performance management
system, with specific objectives contained in the DHS strategic plan as
well as the President's Management Agenda human capital standards for
success.
Usefulness of performance measures. We have previously reported several
key characteristics of effective strategic and management plans,
including the need for performance measures.[Footnote 11] Appropriate
performance measures along with accompanying targets are important
tools to enable internal and external stakeholders to effectively track
the progress the department is making toward achieving its training
goals and objectives. To this end, organizations may use a variety of
performance measures--output, efficiency, customer service, quality,
and outcome--each of which focuses on a different aspect of
performance.
The DHS training strategic plan contains few specific performance
measures for its goals or strategies and all of these are output
measures. For example, the plan makes use of output measures in its
requirement that certain actions, such as the development of a new
management directive or the chartering of a team, be completed by the
end of fiscal year 2006, and in establishing a deadline for when
reports need to be completed in order to be included in the 2007 plan.
In contrast to output measures like these, which gauge the level of
activity or effort by measuring whether a particular thing is produced
or service performed, other types of measures, such as measures of
customer satisfaction or program outcomes, focus on the impact or
results of activities. By appropriately broadening the mix of measures
it uses and more clearly identifying targets against which DHS can
assess its performance, DHS can improve the usefulness of its plan.
After we completed our audit work, DHS training officials informed us
that they decided to delay the development of performance measures
until the rollout of the plan, when they could be developed by
individual teams, as needed. They subsequently informed us that these
teams will be held accountable to establish further performance
measures that are outcome based and results oriented.
DHS's human capital strategic plan again provides an illustration of
how the department's training strategic plan might begin to work toward
the inclusion of different types of performance measures. For example,
accompanying the strategy that DHS assess the feasibility of
establishing a 21st Century Leadership Training and Development
Institute, the plan identifies two performance measures--customer
satisfaction and cost of delivery--along with specific targets for
each. For the customer satisfaction measure, the plan establishes a
target of 4.5 on a scale from 1 to 5. The plan also includes specific
tactics to achieve the strategy, such as developing and obtaining cross-
organizational support, developing measures and methodologies for
leadership training, and implementing a learning management system,
along with key milestone dates for completing them.
The department may benefit from considering the experiences of leading
organizations regarding the development of results-oriented performance
measurement. In general, results-oriented organizations we have studied
that were successful in measuring their performance developed measures
that were:
* tied to program goals and demonstrated the degree to which the
desired results were achieved,
* limited to the vital few that were considered essential to producing
data for decision making,
* responsive to multiple priorities, and:
* responsibility linked to establish accountability for
results.[Footnote 12]
Similar to the consultative process DHS followed when developing the
goals and strategies contained in its training strategic plan,
decisions concerning the selection of an appropriate set of performance
measures should also be based on input from key stakeholders to
determine what is important to them when assessing the department's
performance regarding training. Clear and appropriate performance
measures, developed in this way, can also provide DHS with valuable
information, especially significant in the current fiscal environment,
when it seeks to justify requests for resources from Congress.
Training Councils and Groups Created by DHS Increase Communication
across Components:
Under the overall direction of the CHCO office, DHS has established a
structure of training councils and groups that cover a wide range of
issues and include representatives from each organizational component
within DHS. The department is in the process of using these bodies to
facilitate communication and the sharing of information within its
diverse training community. In some instances, these councils and
groups foster greater collaboration and coordination on training
policies, programs, and the sharing of training opportunities. We have
previously reported that agencies with a strategic approach to training
recognize the importance of having training officials and other human
capital professionals work in partnership with other agency leaders and
stakeholders on training efforts.[Footnote 13]
The Training Leaders Council (TLC) plays a vital role in DHS's efforts
to foster communication and interchange among the department's various
training communities. This council consists of senior training leaders
from each of the department's components as well as representatives
from several department-level headquarters staff and support
organizations with an interest in training-related issues.[Footnote 14]
Started in October 2004 and formally chartered by the CHCO in March
2005, the TLC's mission is to establish and sustain a collaborative
community with the aim of promoting high-quality training, education,
and development throughout DHS. To this end, it functions as a convener
of training leaders from throughout the department and provides an
overarching framework for several preexisting training groups and
councils that were reestablished as standing committees of the TLC.
Membership of the TLC consists of senior training leaders from each DHS
component. In addition, most of these leaders as well as other training
staff serve on one or more of its subgroups. See figure 2 for
descriptions of the TLC and each of its subgroups.
Figure 2: Departmental Training Councils and Subgroups at DHS:
[See PDF for image]
[End of figure]
One key function of the TLC and these other training groups is to serve
as a "community of practice" wherein officials can discuss common
training challenges and share knowledge and best practices. For
example, the Training Evaluation and Quality Assurance Group, composed
of DHS training professionals responsible for evaluating and ensuring
the quality of DHS training programs, conducted an informal survey of
evaluation practices in various components with the intent of
identifying effective evaluation approaches. A training official
involved in the group told us that this survey was particularly
important for the department's newer organizations, such as the
Directorate for Information Analysis and Infrastructure Protection,
which need to establish new practices from scratch. According to this
official, his directorate and other organizations within DHS plan to
use the group as a way to tap into the experience of other components
within the department, such as CBP and FLETC, which have considerably
more experience with training evaluation.
In addition to sharing information about training practices, these
groups can also provide a forum for exchanging practical information
with the goal of making more efficient use of existing resources. For
example, one training official told us that as a result of information
obtained at TLC meetings, the official became aware of the existence of
free training space available at facilities of two other components
located in the Washington, D.C., metro area. Also, as a result of
participating in these meetings, the official's organization was able
to send an additional person to the Federal Executive Institute after
becoming aware that another component had surplus spaces and was
offering them at a reduced price to other components within
DHS.[Footnote 15]
Another role carried out by the TLC is to collaborate on the
formulation of training policies and advise the department's CHCO
accordingly. For example, the TLC, in cooperation with staff from the
CHCO office and an external contractor, conducted a survey of training
sites throughout the department in 2004. This study cataloged available
physical resources and site capacities with the aim of identifying
potential opportunities to share these resources more efficiently,
consolidate unneeded or duplicative sites, and identify other
opportunities to increase training collaboration and effectiveness.
This effort resulted in a series of recommendations that were
subsequently incorporated into the department's training strategic
plan.
The activities of the department's Advanced Distributed Learning Group
(ADLG) provides another example of how training officials from
different components have worked together to develop proposals for
solutions to departmentwide challenges. This group identified several
issues in the area of technology and learning, including the need for a
compatible IT infrastructure across components and the fact that some
components lacked established systems with which to coordinate and
manage training opportunities and attendance. Working with an outside
consultant, the ADLG's efforts resulted in a proposal that DHS create a
new Advanced Distributed Learning (ADL) Program Management Office to
oversee the process of setting common standards. This proposal was
subsequently included as part of the department's training strategic
plan. In addition, the ADLG's work also led to DHS entering into a
memorandum of understanding with the Office of Management and Budget
and the Office of Personnel Management to create a DHS headquarters
learning management system. Throughout this process, the ADLG
represented the interests of the DHS training community as it worked
with representatives from the Chief Information Officer's office and
other functions within the department, as well as outside consultants.
Despite these positive steps, DHS's effort to foster communication and
coordination through departmentwide training councils and groups is at
a relatively early stage and so far has produced varied results. Some
training organizations, such as the TLC and ADLG, have met regularly
leading to tangible results, while others such as the Training
Evaluation and Quality Assurance Group, have met a few times and have
only begun to set the groundwork for substantive coordination and
collaboration in these areas. In addition, a training official told us
that even active organizations like the TLC have encountered
difficulties related to the relative lack of staff support for these
efforts. As a result, additional burdens sometimes fall to the leaders
and members of these groups who, in addition to serving on one or more
departmental training groups or councils, must carry out full-time
training positions at their home components.
DHS Provides Training to Address Selected Departmentwide Needs:
Another way DHS addresses departmentwide training issues is to directly
provide training interventions or resources that address selected
departmentwide needs, goals, or objectives. Three examples of the areas
where DHS has worked to directly provide or support training on the
departmental level are the following: (1) training related to the
implementation of MAXHR, (2) DHS leadership development, and (3)
training related to civil rights and civil liberties.
Training for MAXHR implementation. DHS's new human capital management
system, known as MAXHR, represents a fundamental change in many of the
department's human capital policies and procedures that will affect a
large majority--approximately 110,000--of its civilian employees. MAXHR
covers many key human capital areas, such as pay, performance
management, classification, labor relations, adverse actions, and
employee appeals, and will be implemented in phases affecting
increasing numbers of employees over the next several years.
DHS correctly recognizes that a substantial investment in training is a
key aspect of effectively implementing MAXHR, and in particular, the
new performance management system it establishes. The need for in-depth
performance management and employee development training is further
supported by the department's results on the 2004 Federal Human Capital
Survey. In this survey, just over half of DHS respondents--51 percent-
-believe supervisors or team leaders in their work units encourage
their development at work, significantly less than the governmentwide
response of 64 percent. DHS officials said they plan to educate all
affected DHS employees on the details of the new system, how it will
affect them, and the purpose of the changes. To do this, the department
decided to develop, coordinate, and manage MAXHR training centrally
through the CHCO office and offered its first training in May 2005. DHS
plans to continue to provide its workforce with MAXHR training over the
next several years following a phased approach that takes into account
both when individual provisions of the new regulations take effect as
well as the different audiences that exist within the DHS community,
including human capital personnel, supervisors, and general employees.
See figure 3 for a depiction of planned training during 2005 and its
intended audiences.
Figure 3: Timeline for MAXHR Training Scheduled through the End of
2005:
[See PDF for image]
Note: This training schedule reflects the department's plans as of
early August 2005. However, we were told by a DHS official that these
dates are subject to change.
[End of figure]
The department has worked with contractors to develop training that
uses a variety of approaches, including classroom instruction, ADL,
handbooks, manuals, and quick reference guides, depending on specific
needs. For example, in May 2005, labor relations/employee relations
specialists and attorneys in the department received 2-½ days of
training on the provisions of the new regulations and the major
difference between them and previous programs. Structured as a "train
the trainer" type intervention intended to prepare participants to
conduct supervisor briefings in their own components, this was an
instructor-led course held at sites across the country. In addition to
educating individuals about the regulations, procedures, and systems
associated with MAXHR and the adoption of a new performance management
system, the department also plans to offer training specifically
targeted to developing the skills and behaviors that will be necessary
for its successful implementation. For example, in July 2005
supervisors began to receive training on techniques for providing
meaningful feedback to, coaching, and mentoring employees.
DHS leadership development training. Leadership development is another
area top management in DHS acknowledged as appropriate for
departmentwide training to supplement existing component-level
offerings. In 2004, the Secretary of Homeland Security announced the
"One DHS" policy that identified the need to establish a common
leadership competency framework for the department, as well as a
unified training curriculum for current and future leaders. The purpose
of this framework was to identify the skills, abilities, and attributes
necessary for success as a DHS leader and to establish measurable
standards for evaluation.
To this end, the CHCO established the DHS Leadership Training and
Development Group (LTDG), comprising training officials from each DHS
component who combined an expertise in leadership development with
personal knowledge of the missions and unique aspects of their
particular organizational components. The LTDG met regularly from late
2003 to mid-2004. During this time, the group developed a set of new
core leadership competencies for DHS supervisors, managers, and
executives, which it issued in April 2004. According to a DHS official,
since the development of these new competencies they have been used by
one component as part of its own leadership development plan and they
have also helped to guide and inform current MAXHR leadership
development efforts.
DHS has recently taken steps regarding another facet of its leadership
development initiative--its SES Candidate Development Program. In June
2005, DHS issued a management directive establishing the SES Candidate
Development Program, which included a rigorous selection process and
critical leadership development opportunities, such as mentoring,
developmental assignments, and action learning designed to give SES
candidates experience in different job roles. DHS initially announced
that it planned to implement the program in fiscal year 2005, but now
may delay doing so until fiscal year 2006.
Civil rights/civil liberties training. A third area in which DHS has
taken steps to provide or support departmental training involves civil
rights and civil liberties. FLETC's Behavioral Science Division and
Legal Division, working with the DHS Office of Civil Rights and Civil
Liberties, produced several training interventions, including Web-
based, CD-ROM, and in-person programs designed to increase sensitivity
and understanding in protecting human and constitutional rights. As
part of this effort, FLETC held diversity seminars that focused on
promoting understanding and respect of religious practices,
particularly involving those of the Arab and Muslim communities. In
another example of this effort, the Office of Civil Rights and Civil
Liberties produced Web-based training on current policies regarding
racial profiling.
Our interviews with DHS training leaders suggest that further
improvements can be made in communicating the availability of selected
departmentwide training programs and resources. Staff at the Office of
Civil Rights and Civil Liberties provided copies of its civil rights
and liberties programs to training offices at each component in the
department. While some senior training officials told us that their
components actively disseminated this material by placing it on the
component's training Web site or incorporating it into preexisting
courses, other senior training officials we spoke with were unaware of
any departmental training on these topics. In addition, other officials
told us that their component's training office had independently
developed its own material on Arabic sensitivity training, wholly apart
from similar efforts undertaken by others in the department. More
specifically, they told us that their development of certain training
modules predated the development of very similar modules later prepared
by DHS's Office of Civil Rights and Civil Liberties and FLETC, leading
these officials to conclude that they may have been able to assist
departmental efforts by sharing their work had they been aware of them.
Challenges May Impede Achievement of Departmentwide Training Goals:
As DHS moves forward, it faces challenges to achieving departmentwide
training goals. These challenges include lack of common management
information systems, the absence of commonly understood training
terminology across components, the lack of specificity in authority and
accountability relationships between the CHCO office and components,
insufficient planning for effective implementation, and insufficient
resources for ensuring effective implementation of training strategies.
Sharing of Training Information across Components Made More Difficult
by Lack of Compatible Infrastructure and Commonly Understood
Terminology:
The formation of DHS from 22 legacy agencies and programs has created
challenges to achieving departmentwide training goals. Of particular
concern to the training officials we spoke with are the lack of common
management information systems and the absence of commonly understood
training terminology across components. The training functions at DHS's
components largely operate as they did before the creation of the
department, with many of the same policies, practices, and
infrastructures of their former organizations, and within these
organizations are, for the most part, the same training leaders. It
will take time for these organizations to evolve into a coordinated,
integrated department. We have previously reported that successful
transformations of large organizations, even those faced with less
strenuous reorganizations and pressure for immediate results than DHS,
can take from 5 to 7 years to fully take hold.[Footnote 16]
One issue DHS officials raised was the lack of common or compatible
management information systems, such as information technology or
financial management, which can inform decision makers' efforts to make
efficient use of training resources across components. For example, DHS
officials stated that a key challenge they encountered involved the
difficulty of knowing what others were doing outside their particular
offices or components. DHS lacks any unified sourcebook that employees
could consult for the names, telephone numbers, and other relevant
information of key contact persons in areas such as acquisition.
Obtaining accurate information about resources and products available
in the marketplace as well as data on users, vendors, and kinds of work
has been a challenge to that effort. Another issue cited by officials
concerned the lack of compatibility between learning management systems
across components. In addition, some training officials expressed
concerns about the accuracy or timeliness of some training data, which
can limit or at least considerably delay their ability to track and
fully account for funds spent on training and training-related travel.
DHS has several efforts under way to address these issues, including
the development of an online training facilities inventory intended to
increase awareness of existing resources across the department and its
decision to begin developing common ADL policies and standards.
Officials also told us that there was little or no common understanding
among DHS organizational components regarding the meaning of such basic
terms as "subject matter expert," "orientation," and even "training."
The lack of commonly understood terminology has presented challenges
when officials from different components, including those participating
in departmental training councils and groups, try to share practices
with each other. These officials told us that the lack of commonly
understood terminology can also affect their interactions with outside
entities, such as contractors and state and local agencies. Besides
facilitating communications and enabling components to share practices,
a DHS official told us that a common nomenclature would increase the
transparency of training practices to external contractors as well as
the internal DHS training community. The department's training
strategic plan calls for the creation of a common training language and
glossary of terms in fiscal year 2006, and officials told us that they
are currently in the early stages of creating such a glossary.
Authority and Accountability Relationships between CHCO Office and
Components Are Not Sufficiently Specific:
An effective management control environment appropriately assigns
authority and delegates responsibility to the proper personnel to
achieve organizational goals and objectives.[Footnote 17] In such an
environment, staff members who are delegated responsibility are given
corresponding authority. In light of this, DHS's adoption of a "dual
accountability" governance structure in 2004 presents certain
challenges. Under this concept, heads of organizational components and
the CHCO share responsibility for effective training in DHS. With a
shared responsibility for DHS training, both the CHCO and component
heads should have appropriate authority for making decisions regarding
training.[Footnote 18] DHS does not specify how authority for training
matters will be shared between the CHCO office and components for
budgeting, staffing, and policy (e.g., determining which training
functions, if any, should remain with components or be performed by DHS
headquarters). The DHS management directive on training currently in
place is a high-level two-page document that provides very few
specifics on policies, procedures, and authorities for the CHCO office
and the components.
The department recognizes the need to clarify the responsibilities and
authorities of the CHCO office and the components, as indicated by its
inclusion in the DHS training strategic plan. Many of the tactics
included in the plan would be difficult to successfully implement
without first having a clear understanding of the responsibilities and
authorities of the key organizations involved. More specifically, in
the absence of clear authority relationships, decisions regarding how
particular component training goals and strategies are to be
incorporated in the DHS training strategic plan, or which training
facilities should be consolidated to achieve departmental efficiencies,
will be difficult to make. Without moving ahead with this effort in a
timely fashion and completing the process of specifying how the CHCO
office and components will share authority over training matters, it
will be difficult for DHS to make the progress necessary on its
departmentwide training agenda if it is to effectively implement the
many strategies and tactics planned for fiscal year 2006.
In addition, DHS's efforts at coordinating training across components
and clarifying roles and relationships between departmental functions
and organizational components may be further hampered by the fact that
the management directive governing the integration of the human capital
function claims that the Coast Guard[Footnote 19] and the Secret
Service[Footnote 20] are statutorily exempt from its application. We
found no reasonable basis to conclude that the directive could not be
made applicable to them and are not aware of any explicit statutory
exemption that would prevent the application of this directive.
Moreover, exempting the Coast Guard and the Secret Service from the
provisions of this directive casts doubt on the authority and
accountability relationships between these components and the CHCO,
potentially complicating the department's objective of clarifying the
responsibilities, accountability, and authorities of the CHCO office
and the components set forth in DHS's training strategic plan.
Planning May Be Insufficiently Detailed to Ensure Effective
Implementation in "Dual Accountability" Environment:
In and of itself, DHS's dual accountability authority structure is not
an obstacle to implementation of departmentwide training efforts.
However, without detailed implementation plans, it presents potentially
significant challenges. Because of this shared authority, DHS will need
to take great care when planning for departmentwide training
initiatives involving multiple organizational components to ensure that
resources are aligned with organizational units performing activities,
especially related to cross-organizational sharing of training and
delivery of common training. The lack of comprehensive and rigorous
planning can lead to confusion over responsibilities, lack of
coordination, and missed deadlines. Regular and rigorous use of
detailed implementation plans is necessary to implement decisions and
carry out activities in a coordinated manner.
After we completed our audit work, DHS informed us that it plans to
establish 31 tactic teams to take ownership of each of the tactics
presented in the DHS training strategic plan to be completed by the end
of fiscal year 2006. As of mid-August 2005, DHS provided us with
documentation indicating that 3 of these teams have been established to
date. These teams appear to have taken promising steps toward the
establishment of detailed plans for implementing their respective
training tactics by developing draft objectives, deliverables, and
closure criteria. But as fiscal year 2006 approaches, time is short for
the CHCO office and the components to establish the remaining teams and
then take the actions necessary to develop and put in place the
detailed plans that will be critical for effectively implementing DHS's
many training tactics by the end of the coming fiscal year. The TLC's
ADLG has made use of this type of detailed approach in a report
proposing a distance learning architecture for the department. Appended
to its report is a detailed plan outlining the major activities,
milestones, resources, and components needed to support the successful
implementation of the proposal.
Resources Needed to Implement Departmental Training Strategy:
Several training officials told us they were concerned about the lack
of dedicated resources and related capacity to carry out departmental
initiatives. At the time we started our review, the CHCO office had
only one full-time permanent employee dedicated to carrying out these
activities; consequently, both training leaders and staff from
organizational components were relied on to contribute to
departmentwide efforts. After we concluded our audit work, a DHS
official told us that the CHCO office had recently hired two additional
full-time training staff: an ADL program manager and a staffer to
oversee a recently approved SES candidate development program and
headquarters operational leadership development. Individual components
have also provided some assistance to departmentwide efforts through
the appointment of temporary personnel. In late 2004, CBP and FLETC
each detailed a staff member to the CHCO office to work on training-
related projects. In addition, DHS has contracted for services to
address selected departmentwide issues, such as setting common
standards for ADL and reviewing DHS training facilities.
DHS's departmental training councils and groups are almost exclusively
staffed by component training leaders who already have full-time
training commitments. The department's training strategic plan
identifies many tactics for fiscal year 2006--including creating a
common training language and glossary of training terms, establishing a
repository for course catalog information, and developing a DHS
training Web site--that will require considerable staff support to
implement. Successful and timely completion of these and other
initiatives will depend on sufficient resources being provided.
DHS's Approach to Planning and Evaluation of Training Demonstrates Some
Elements of Strategic Practice, but Is Still Evolving:
It is essential for federal agencies to ensure that their training
efforts are part of--and are driven by--their organizational strategic
and performance planning processes. We have reported that aligning
training with strategic priorities and systematically evaluating
training activities play key roles in helping agencies to ensure that
training is strategically focused on improving performance and meeting
overall organizational goals.[Footnote 21] Strategic training practices
in several DHS components or programs may provide models or insights to
others in the department regarding ways to improve training
practices.[Footnote 22] In areas where some components employed
strategic practices, other components did not.
Aligning Training with Organizational Priorities Is a Strategic
Practice:
We have previously reported that agencies demonstrating a strategic
approach to training align their training efforts with overall
strategic priorities. To do this, agencies can employ a variety of
practices, such as linking training activities to strategic planning
and budgeting and performing front-end analysis to ensure that training
activities are not initiated in an ad hoc, uncoordinated manner, but
rather are focused on improving performance toward the agency's
goals.[Footnote 23] Some components in DHS applied the strategic
practice of aligning training with organizational priorities, while
others did not.
CBP links its new and existing training activities to its strategic
priorities when planning for its strategic initiatives and
expenditures. Importantly, the head of training at CBP is at the
decision-making table with other CBP leaders to help establish training
priorities consistent with the priorities of the CBP Commissioner.
Relevant program managers are asked, "What training do you need to
achieve the goals in your strategic plan?" Such discussions took place
during planning for CBP's custom trade pact initiative.
During each budget cycle, CBP's central training office issues a "call
for training" to its mission and mission support customers to estimate
CBP's training needs for existing training activities and prioritize
these needs based upon the Commissioner's priorities. Prior to
establishing this process, training was mostly decided on a first come,
first serve basis without clear and transparent linkages to
organizational priorities. CBP's current process results in an annual
training plan in which training needs are identified by priority as
well as major occupational type, such as border patrol agent or CBP
officer. Training decisions are based on whether training requested is
critical, necessary, or "nice to have." During fiscal year training
plan implementation, CBP tracks actual training activity through a
central database to determine whether CBP is using its planned training
resources. By tracking plan usage through a centrally managed database,
CBP is able to reallocate unused training funds prior to the end of the
fiscal year for either training activities that were not included in
its original plan because of capacity constraints, or for emerging
training priorities.
The Coast Guard has adopted a strategic and analytic approach to
training through its use of the Human Performance Technology (HPT)
model--a front-end training assessment process to determine the cause
of performance problems. The process starts with the assumption that
many factors influence individual and unit performance and it is
important to determine what the factors are before concluding that
training is the solution. From its HPT analysis, the Coast Guard
determines whether training is needed or whether another type of
solution, such as a policy change, would be more appropriate. For
example, in addressing a problem in aviation maintenance, a Coast Guard
working group looked at likely causes of its performance problems and
concluded that focusing on making aviation maintenance training better
was not the only solution. More specifically, training officials
encountered problems with job dissatisfaction and subpar performance
from aviation chief warrant officers. In this case, training officials
used HPT to analyze the nature of work performed by those responsible
for aviation maintenance and concluded that there was not a good match
between job skills and responsibilities. Specifically, over the last 20
years, the scope and nature of the work performed by chief warrant
officers changed significantly from maintaining components to managing
aircraft systems. Performance problems were mainly caused by
significant changes in the job functions of these officers over the
years rather than by a lack of adequate training.
In cases where the HPT analysis concludes that training is warranted, a
training analysis is performed to determine the specific training
interventions. For example, in implementing activities related to the
Maritime Transportation Security Act, the Coast Guard analyzed its
training needs through the HPT process to determine training necessary
to help maritime inspectors reduce the exposure of ports and waterways
to terrorist activities. The analysis identified the skills and
knowledge necessary for new maritime inspector tasks and provided
training interventions, such as developing job aids and targeted
classes, to prepare inspectors for the tasks most relevant to support
their new role. New courses were piloted and then subjected to
multilevel evaluations to assess their effectiveness and potential
impact on employee performance.
Systematic Evaluation of Training Is a Strategic Practice:
Agencies demonstrating a strategic approach to training employ a
variety of practices, such as systematically evaluating training,
actively incorporating feedback during training design, and using
feedback from multiple perspectives.[Footnote 24] Several components
and programs we examined at DHS used these practices, while others did
not.
One commonly accepted model used for assessing and evaluating training
programs consists of five levels of assessment (see fig. 4).[Footnote
25] In our review, virtually all components captured Level I data
focusing on end-of-course reactions, while several also collected Level
II data focusing on changes in employee skill, knowledge, or abilities.
Several components evaluated, or were planning to evaluate, the impact
of selected training programs on individual behavior, represented by
Level III evaluations.
Figure 4: Levels of Training Evaluation:
[See PDF for image]
Note: GAO presentation based on information from Donald L. Kirkpatrick,
Evaluating Training Programs: The Four Levels (San Francisco: Berrett-
Koehler Publishers, Inc., 1998), and Jack J. Phillips, ed.,
Implementing Evaluation Systems and Processes (Alexandria, Va.:
American Society for Training and Development, 1998).
[End of figure]
To measure the real impact of training, however, agencies need to move
beyond data focused primarily on inputs and outputs and develop
additional indicators that help determine how training efforts
contribute to the accomplishment of agency goals and objectives. At a
couple of components, DHS officials told us they conducted Level IV
evaluations, which assess the effectiveness of training interventions.
We found no examples of the department or its components measuring the
return on investment of training activities (Level V).[Footnote 26]
Training effectiveness should be measured against organizational
performance; however, not all levels of training evaluation require or
are suitable for return on investment analysis. Determining whether
training programs merit the cost of using such an approach depends upon
the programs' significance and appropriateness.
CBP takes a systematic approach to evaluating its training activities
through its National Training Evaluation Program (NTEP) to help program
managers and trainers make more informed decisions on the effectiveness
of training courses and their delivery. Despite the fact that CBP is a
large and decentralized organization, NTEP has enabled it to collect
course evaluation information and make this information available to a
wide range of users in a timely manner. NTEP has also standardized
evaluation data to allow for comparison of training throughout various
field locations. Before the rollout of NTEP, CBP did not use a standard
mechanism for collecting evaluation data, which, according to a CBP
official, made it difficult to gather evaluation data nationally.
CBP focuses on collecting both end-of-course student reactions (Level
I) and supervisor assessments of student on-the-job performance after
attending the training (Level III). Electronic or paper-based
evaluations are entered into the NTEP information system. The "close to
real time" online data enables supervisors to perform trend analysis on
training quality and provides opportunities for them to troubleshoot
training deficiencies and identify high-performing courses. The NTEP
online system allows CBP employees access to evaluation data on a need
to know basis with four levels of access, while enabling them to locate
evaluation data for any training class by date. Evaluation reports are
aggregated for review by senior CBP officials.
A CBP official told us that collecting course evaluation data is labor
intensive, especially since many field operations still use paper
processes. In addition, CBP has experienced a relatively low submission
rate for Level I evaluation data for many of its training classes. The
official told us that this was especially true for end-of-course
reactions from staff in the field, where only about one-third of
officer-related course participants submit evaluation forms. Given cost
and labor challenges, CBP has targeted areas for evaluation that it
believes are important, such as training related to its "One Face at
the Border" initiative.
In addition, agencies with a strategic approach to training do not wait
until the conclusion of a training intervention to conduct evaluations.
Rather, they approach evaluation through an iterative process capable
of informing all stages of training.[Footnote 27] DHS's CHCO office
used multiple forms of feedback from employees to develop its training
strategy for MAXHR. From February through April 2005, the department
administered surveys and conducted focus groups to obtain information
on the needs, attitudes, and reactions of different communities
affected by MAXHR. Shortly after issuing its new human capital
regulations, the department provided basic information to all employees
on the nature and timeline of changes they could expect under MAXHR
through a Web broadcast. After the broadcast an online survey was used
to obtain feedback from employees regarding the broadcast itself and
their general feelings and concerns about the MAXHR initiative.
DHS followed this initial survey with a larger survey to gather
additional feedback on how information regarding MAXHR had been
communicated, as well as specific areas where employees wanted
additional information. Concerns about the need for training were
prominent among the more than 9,000 responses received, with
respondents ranking training as the second most serious challenge to
the successful implementation of MAXHR. According to a senior DHS
official, the survey results will inform subsequent training and
communication efforts.
DHS also collected evaluative feedback by conducting a series of focus
groups held in locations across the country. The aim of these sessions
was to validate the design of the performance management program
established under MAXHR and identify concerns that would inform the
development of additional training. Consistent with the strategic
training practice of seeking out different perspectives when
redesigning and assessing training efforts, DHS staff held separate
focus groups for bargaining unit employees, non-bargaining unit
employees, and supervisors and managers at all of the locations
visited. This enabled them to identify issues of particular concern to
each of these groups as well as issues common to all three. For
example, both the bargaining unit and non-bargaining unit employee
focus groups raised concerns about supervisors having inadequate skills
for fairly administering the new performance management system. This
concern was also shared by supervisors and managers themselves who
expressed the need for additional skills training in areas such as goal
setting and providing performance feedback. The sessions validated the
CHCO office's plans to offer performance management training to
supervisors and managers before the implementation of the new system
and assisted in refining issues for future training.
FLETC's methods for evaluating its major training programs include
feedback from multiple perspectives when examining the benefits of
training on actual employee job performance. FLETC's Level III
evaluations obtain feedback from both trainees and their supervisors to
inform future improvements to training curricula. Evaluation results
are compiled into a comprehensive report used during FLETC's periodic
curriculum reviews on its major training programs, such as the Natural
Resource Police Training Program. The report contains detailed feedback
from both the trainee and supervisor perspectives 6 months to 1 year
after the trainee has attended the training program. For example, for
the Natural Resource Police Training Program, FLETC analyzed how well
the program prepared trainees in all aspects of their jobs. In this
case, analysis identified those courses that had benefited program
trainees the least--including determining speed from skid marks and
death notification. Training designers can use report information to
improve program curricula and refocus training on knowledge and skill
areas most critical to performing the job. In addition to Level III
evaluation results, its training designers make program and individual
class changes by using other methods of evaluation, such as direct
student feedback after classes and trainee examinations, which
determine how well the trainees understood the course material
immediately following the program.
DHS Has Used Training in an Effort to Help Its Workforce as the
Department Undergoes Transformation and Cultural Change:
The creation of DHS resulted in significant cultural and
transformational challenges for the department. We have previously
reported that training is one way organizations successfully address
cultural issues while simultaneously facilitating new ways to work
toward the achievement of organizational goals.[Footnote 28] Among the
DHS components in our review, some merged cultures from different
legacy organizations (CBP, ICE), another component came as a small
organization that greatly expanded when joining DHS (Federal Air
Marshal Service), while others joined DHS intact (Secret Service, Coast
Guard, FLETC), and still another was previously a part of a larger
legacy organization (CIS). Each component faces the need to find a way
to identify itself as part of the larger DHS organization, that is,
with a sense of affiliation rather than as an outsider looking in. At
the same time, components must either maintain their existing cultures
or develop new cultures to adapt to changing missions and needs. The
key is to build upon positive aspects of the components' cultures as
the larger organization develops its own culture.
Agencies that undergo successful transformations change more than just
their organizational charts, they also make fundamental changes in
basic operations, such as how they approach strategic human capital
management. DHS understands this, and the MAXHR initiative is part of
an effort by the department to fundamentally change its approach to
human capital management by establishing a personnel system that is
flexible, performance oriented, and market based.
The Secretary of Homeland Security and other top officials have
actively supported the role of training in implementing these changes
by making it a leadership expectation that all DHS executives,
managers, and supervisors be personally involved as both participants
in and supporters of MAXHR training efforts. The CHCO office, working
with the assistance of outside contractors, has developed several
training interventions aimed at providing these groups with the tools
and information needed to champion the benefits of a performance-based
culture and successfully implement MAXHR in their components.
In August 2005, DHS sponsored a 2-½ day training program for 350 to 400
of the department's senior executives and flag officers. The program
covered a range of topics, including an update on current DHS
priorities; techniques and best practices for how senior leaders can
effectively support and implement these priorities; as well as specific
management, communication, and training approaches that can be used to
support the creation of a performance-based culture. The Secretary,
Deputy Secretary, and Under Secretary for Management all participated
in the program, which also featured presentations from human capital
and organizational change experts from outside the department. In
addition to its focus on MAXHR implementation, which included both
large and small group sessions wherein participants could discuss
performance management and share information on practices, the course
also provided a forum for the department's top leadership and senior
executives to review the then recently issued recommendations resulting
from the Secretary's Second Stage Review process.
Another training intervention sponsored by the department directly
targets managers and supervisors who will be responsible for carrying
out many of the key behaviors associated with the new system and whose
active support is viewed by DHS as critical for achieving the
transformation to a performance-based culture. The 2-½ day program
focuses on developing and improving interpersonal, managerial, and
other so-called soft skills. DHS expects to provide the training to
approximately 12,000 managers and supervisors throughout the
department.
On the component level, training has also played an important role in
CBP's effort to transform from the traditional, largely siloed approach
used by its legacy agencies when protecting our borders to a new
integrated concept that it believes is more in line with its current
needs. Officials noted that the merger into CBP led to some resistance
from employees who had not yet understood or accepted the reasons for
the merger. These same officials acknowledged that they must continue
to work at informing employees why changes were made and provide
vehicles for better integration through training. For example, in the
"One Face at the Border" initiative, supervisory training has
incorporated some elements of cultural integration by including a
session on bridging the culture gap. Officials at CBP designed and
piloted a training module to be added to the supervisory curriculum
specifically targeting how they can more effectively understand the
value and perspective of staff coming out of the legacy organizational
cultures.
In addition, training played a key role in facilitating the transition
of CBP's workforce from its three legacy organizations. Training for
the new CBP officer and CBP agriculture specialist positions aimed to
improve coordination and communication across inspection functions and
enhance the flexibility of CBP's workforce. Specifically, CBP created a
series of training courses to provide former Customs and former
Immigration and Naturalization Service officers with the knowledge and
skills necessary to carry out the responsibilities of this new
position. To develop this training, CBP-wide working groups identified
and validated critical tasks for the new frontline CBP officer to
perform. A mix of training delivery methods were used (i.e., e-learning
and instructor led), and classroom knowledge and skills were reinforced
with on-the-job training. CBP provided extensive train-the-trainer
courses so that trainers could return to their field sites and instruct
officers there. (See app. 2.)
Conclusions:
DHS must continue to make progress on three important aspects of
training as it moves forward: (1) forging an effective role for
training at the departmental level and implementing its departmentwide
training strategy; (2) taking a strategic approach to training
practices, in part by building upon examples of good practice to be
found among its former organizations, as well as considering other
examples of strategic practices; and (3) finding ways that training can
help to foster organizational transformation and cultural change within
the department. To date, DHS has taken positive steps in these areas,
yet significant challenges lie ahead.
The ability to make decisions from a departmentwide perspective and
then effectively implement them will help determine whether training in
DHS achieves its intended results. Strong leadership will play a
critical role in this process. To be successful, DHS will need to have
both a clear plan of action as well as the ability to anticipate and
overcome several implementation challenges. The creation of the TLC and
the development of the department's first training strategic plan both
represent a good start in this process. Better performance measures,
more specific milestones, and the inclusion of performance targets
would make DHS's strategic training plan a more useful tool for both
internal and external stakeholders to use in tracking the department's
progress toward achieving its training objectives. Clarifying authority
relationships between the CHCO and component heads, developing detailed
implementation plans, and giving appropriate attention to providing
resources to implement training initiatives when setting funding
priorities are also likely to be critical factors in building and
sustaining an effective role for department-level training at DHS.
A strategic approach toward training is also very important as DHS
seeks to build on its current efforts and strives to move forward. As
we have noted, some programs and components in DHS already use specific
strategic training practices, and other components within the
department can benefit from their example. As DHS implements new
training programs, such as the large-scale, multistage training being
developed to support the implementation of MAXHR, it has a valuable
opportunity to reflect the lessons learned from these experiences in
subsequent departmentwide training efforts. Finally, the transition to
a new department has brought with it cultural challenges, and training
can play a role in both defining and refining an effective DHS culture
without sacrificing the cultural history of its components.
Recommendations for Executive Action:
To help DHS establish and implement an effective and strategic approach
to departmentwide training, we recommend that the Secretary of Homeland
Security take the following actions:
* adopt additional good strategic planning and management practices to
enhance the department's training strategic plan by (1) creating a
clearer crosswalk between specific training goals and objectives and
DHS's organizational and human capital strategic goals and (2)
developing appropriate training performance measures and targets;
* clearly specify authority and accountability relationships between
the CHCO office and organizational components regarding training as a
first step to addressing issues DHS has identified for fiscal year 2006;
* ensure that the department and component organizations develop
detailed implementation plans and related processes for training
initiatives; and:
* when setting funding priorities, give appropriate attention to
providing resources to support training councils and groups to further
DHS's capacity to achieve its departmentwide training goals.
Agency Comments:
We provided a draft of this report to the Secretary of Homeland
Security for comment and received written comments from DHS that are
reprinted in appendix III. In addition, we received technical comments
and clarifications, which we incorporated where appropriate. DHS
generally agreed with our recommendations.
We will provide copies of this report to the Secretary of Homeland
Security and other interested parties. Copies will also be provided to
others upon request. In addition, this report is available at no charge
on the GAO Web site at http://www.gao.gov.
If you have any questions about this report, please contact me at (202)
512-9490 or stalcupg@gao.gov. Major contributors to this report were
Kimberly Gianopoulos, Assistant Director; Peter J. Del Toro; Robert
Yetvin; and Gerard Burke.
Sincerely yours,
Signed by:
George H. Stalcup:
Director, Strategic Issues:
[End of section]
Appendix I: Scope and Methodology:
To achieve our objectives, we reviewed training at the Department of
Homeland Security (DHS) at both the departmental and component levels.
When examining training at the departmental level, we collected,
reviewed, and analyzed the department's training rules, procedures,
policies, and organizational charts; departmental, human capital, and
training strategic plans; human capital and training management
directives; Internet and intranet Web pages; and other relevant
documents. To further our understanding of training at DHS and the
issues and challenges involved, we interviewed training and human
capital officials in the Office of the Chief Human Capital Officer and
the leaders and coleaders of DHS's training councils and groups. We
also observed the January 2005 meeting of the Training Leaders Council.
We supplemented our review of departmental training at DHS by examining
the department's effort to use training related to MAXHR to foster
transformation and cultural change in the department.
In addition, we reviewed training at major organizational components in
DHS and selected the six largest components based on staff size and
budget. Using these criteria, we reviewed training at Customs and
Border Protection (CBP), Citizenship and Immigration Services,
Immigration and Customs Enforcement (including the Federal Air Marshal
Service, the Federal Protective Service, and the Leadership Development
Center), the Coast Guard, the Secret Service, and the Transportation
Security Administration. See figure 5 for a depiction of the DHS
organizational structure in place during the time of our review. These
components collectively represent about 95 percent of the total staff
at DHS. We also included the Federal Law Enforcement Training Center
because of the special role it plays in training employees from other
DHS components.
Figure 5: DHS Components Included in Our Review:
[See PDF for image]
Note: This figure does not depict all components and entities in the
department, but is intended to provide a general framework within which
to present the components and centers included in our review. The
organizational structure depicted above was in place during our review,
and does not reflect actual or proposed changes related to DHS's Second
Stage Review.
[End of figure]
When examining training at selected components, we reviewed component-
level strategic, human capital, and training plans when available;
training budget requests and expenditure documents; training
procedures, policies, and organizational charts; rules and policies for
identifying and prioritizing training programs; Internet and intranet
Web pages; selected training course materials; and other relevant
documents produced by these components. To further our understanding of
training at the component level, we also interviewed training officials
at each of the selected components and identified these individuals
based on their knowledge, experience, and leadership roles. We
conducted our interviews at component headquarters or field offices
located in the Washington, D.C., area. In addition, as part of our
review of DHS's efforts to foster transformation and cultural change,
we observed training related to CBP's "One Face at the Border"
initiative in northern Virginia.
To help determine whether DHS used a strategic approach in planning and
evaluating its training activities at the departmental or component
levels, we referenced criteria contained in our guide for assessing
strategic training and development efforts in the federal
government.[Footnote 29] This guide outlines a framework for assessing
training efforts, consisting of a set of principles and key questions
that federal agencies can use to ensure that their training investments
are targeted strategically and not wasted on efforts that are
irrelevant, duplicative, or ineffective. We selected our case examples
based on their suitability for demonstrating specific strategic
training practices. Other components within DHS may, or may not, be
engaged in similar practices. To determine whether DHS followed leading
management practices in planning and implementing departmentwide
training, we also drew on our previous work on strategic planning and
effective management practices.
We did not include within our scope training intended for audiences
external to DHS, and we generally covered training and training
management in effect during the period in which we did our work. We
conducted our work from November 2004 through July 2005 in accordance
with generally accepted government auditing standards. We requested
comments on a draft of this report from DHS, which are reprinted in
appendix III. The comments are addressed in the Agency Comments section
of this report.
[End of section]
Appendix II: "One Face at the Border" Training at U.S. Customs and
Border Protection:
One of the initial goals for creating DHS was to better protect the
United States from terrorists entering the country, and ports of entry
are the means through which terrorists can enter. The creation of CBP
within DHS merged border inspection functions at U.S. ports of entry,
which had previously been performed by three separate agencies. Known
as "One Face at the Border," this initiative created the positions of
CBP officer and CBP agriculture specialist that combined aspects of
three former inspector functions. This initiative aimed to improve
coordination and communication of inspections to better protect the
nation's borders from terrorists as well as to improve entry for
legitimate travel and trade.
To successfully make the transition to these new positions, significant
training was needed. Specifically, CBP created a series of training
courses to provide former U.S. Customs and former Immigration and
Naturalization Service officers with the knowledge and skills necessary
to carry out the responsibilities of this new position. In addition,
CBP officers received training to meet CBP's new mission priority of
terrorism prevention. Although the emphasis was on cross-training
legacy officers, the new curriculum was also geared to new hires.
Because agricultural inspections are more specialized, CBP officers
receive training sufficient to enable them to identify potential
agricultural threats, make initial regulatory decisions, and determine
when to make referrals to CBP agriculture specialists. More detailed
agricultural inspections are performed by these specialists who have
substantial training and background in agricultural issues.
A variety of training delivery methods were used (e.g., e-learning and
classroom) and these training methods were reinforced with extensive on-
the-job training. In addition to traditional content areas (e.g., cross-
training for former U.S. Customs officers includes courses on
immigration fundamentals and immigration law), training courses also
covered CBP's new priority mission of preventing terrorism (e.g.,
training in detecting possible terrorists and fraudulent documents,
honing interviewing skills, and making appropriate referrals to staff
for additional inspection). CBP emphasizes on-the-job training in an
effort not to place inspectors on the job without direct supervisory
and tutorial backup. Training for new recruits has also been modified
to include a preacademy orientation program at the port location where
the recruit will eventually work before he or she receives academy
training. This is a 72-day course for CBP officers and a 46-day course
for CBP agriculture specialists.
CBP's main strategy to prepare for field delivery of training was to
provide extensive train-the-trainer courses so that trainers could
return to their field sites and instruct officers there. Training
priorities were established with the idea of spacing the training out
so that field offices would not be overwhelmed. For example, CBP rolled
out its primary cross-training to airports, while antiterrorism
training was rolled out to land borders.
Officials reported that cross-training benefited CBP officers since
they have gained more knowledge by learning both immigration and
customs laws and procedures. This increase in knowledge has the
potential benefit of providing more variety in job tasks as well as
increasing the opportunities for advancement since an officer can now
apply for supervisory-level positions that had previously been open
only to former U.S. Customs or Immigration and Naturalization Service
officers.
Change has not come about without challenges, however, as many officers
were reported to have resisted changes to their responsibilities,
mainly related to the difficulties in learning a new set of procedures
and laws. Officials noted that there has been an enormous amount of
required training for CBP officers, and it can sometimes be
overwhelming. For former officers, in addition to completing an
extensive cross-training schedule and new training related to terrorism
prevention, there are many other required courses related to their
mission. For example, training modules are required in areas such as
body scanning, hazardous materials, cargo inspection, and seized
assets.
Although staffing challenges may ultimately be relieved with trained
officers able to perform dual inspections, officials noted that it has
been extremely difficult to take staff off-line to complete the "One
Face at the Border" training. One official said that classes have been
very difficult to schedule because of the constant pressure to staff
operations. For example, in one case, a class was canceled right after
it began because the trainees were pulled out to staff their inspection
booths. This official also noted that trainers have had to be very
flexible to accommodate staff schedules to ensure that training occurs.
[End of section]
Appendix III: Comments from the Department of Homeland Security:
U.S. Department of Homeland Security:
Washington, DC 20528:
September 7, 2005:
Mr. George H. Stalcup:
Director:
Strategic Issues:
U.S. Government Accountability Office (GAO):
441 G Street, NW, 2440C:
Washington, DC 20548:
Dear Mr. Stalcup:
RE: GAO-05-888, Department of Homeland Security (DHS): Strategic
Management of Training Important for Successful Transformation (GAO Job
Code 450353).
Thank you for the opportunity to review the draft report. The report is
a positive statement of the initial actions DES has taken to ensure
that training is used as a resource for transformation of the
Department. We concur that the creation of the Training Leaders Council
(TLC) and the development of the initial DHS Learning and Development
Strategic Plan, "is a significant and positive step towards addressing
department wide training challenges." Further, we agree that
implementation of specific training programs, such as training for
MAX^HR is important to our efforts to meet our goal for organizational
excellence.
The Chief Human Capital Office was pleased to provide the GAO with
reports about our training organizations and to coordinate interviews
with DHS training leaders. Although, the training enterprise within the
Department is a major undertaking with limited resources, we believe
that external review and input provides us the opportunity for
improvement. We encourage GAO to use the information gathered while
creating this report as a baseline to streamline the process for future
reviews. Technical and substantive comments were provided separately
for your use in updating the final report.
We concur with the intent of the recommendations in the report and have
already taken steps to ensure that measures are appropriately used when
implementing the Learning and Development Strategic Plan. The TLC
designed the plan to link to the President's Management Agenda and to
the DES Strategic Plan, but will take further steps to make those links
more transparent to external audiences. We anticipate that the
authority and the role of the CHCO will be further clarified as we work
towards functional integration and as the TLC develops a full
accounting of current training authorities and requirements. Our plan
is to ensure that component organizations complete annual training
plans by March, 2006. We recognize that additional resources would
enhance the TLC's ability to implement training strategies but will
need to balance that issue with on-going mission requirements.
We look forward to the final report and plan to provide additional
comments at that time regarding the substance and recommendations
contained in the report.
Sincerely,
Signed by:
Steven J. Pecinovsky:
Director:
Departmental GAO Audit Liaison Office:
[End of section]
FOOTNOTES
[1] GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.:
January 2005).
[2] In previous GAO reports we have defined "training" as making
available to employees planned and coordinated educational programs of
instruction in professional, technical, or other fields that are or
will be related to their job responsibilities. Similarly, we have
defined "development" to generally include aspects of training, as well
as structured on-the-job learning experiences (such as coaching,
mentoring, or rotational assignments), and education. For the purposes
of this report, "training" will be used as a shorter substitute for
"training and development."
[3] GAO, Human Capital: A Guide for Assessing Strategic Training and
Development Efforts in the Federal Government, GAO-04-546G (Washington,
D.C.: March 2004).
[4] According to DHS, the name MAXHR was chosen to convey the intent of
the new system to foster "maximizing results, rewarding excellence."
MAXHR covers key human capital areas, including pay, performance
management, classification, labor relations, adverse actions, and
employee appeals.
[5] Homeland Security Act of 2002, Pub. L. No. 107-296 (Nov. 25, 2002).
[6] This section of the Homeland Security Act of 2002, also known as
the Chief Human Capital Officers Act of 2002, led to the creation of
chief human capital officers in each federal department. Pub. L. No.
107-296, §§ 1301-1305 (Nov. 25, 2002).
[7] GAO-04-546G.
[8] According to the Office of Personnel Management, both the survey's
governmentwide findings and DHS-specific findings are generalizable to
their respective populations. For additional findings from the 2004
Federal Human Capital Survey on these and other training and non-
training-related questions, see www.fhcs2004.opm.gov.
[9] GAO-04-546G.
[10] GAO, Executive Guide: Effectively Implementing the Government
Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June
1996).
[11] GAO, Managing for Results: Next Steps to Improve the Federal
Government's Management and Performance, GAO-02-439T (Washington, D.C.:
Feb. 15, 2002).
[12] GAO/GGD-96-118.
[13] GAO-04-546G.
[14] The TLC includes representatives from the following entities: CBP,
CIS, FLETC, ICE, TSA, Emergency Preparedness and Response/Federal
Emergency Management Agency, Information Analysis and Infrastructure
Protection, Science and Technology, the Coast Guard, the Secret
Service, and US-VISIT. The TLC also includes representatives from the
following DHS department level organizations: CHCO office, Office of
the Chief Information Officer, Office of the Inspector General, Office
of the General Counsel, Office of Civil Rights and Civil Liberties,
Office of Counter Narcotics, and Office of State and Local Coordination
and Preparedness/Center for Domestic Preparedness.
[15] The Government Employees Training Act provides that an agency can
extend its training courses to employees of other government agencies
(5 U.S.C. § 4104(2)). A Comptroller General decision reviewed the
legislative history of this provision and concluded that training can
be provided on a reimbursable or a nonreimbursable basis, at the
discretion of the agency offering the training (B-193293, Nov. 13,
1978). A DHS component that is offering training is authorized to make
its courses available to other employees of DHS or of other government
agencies.
[16] GAO, Results-Oriented Cultures: Implementation Steps to Assist
Mergers and Organizational Transformations, GAO-03-669 (Washington,
D.C.: July 2, 2003).
[17] GAO, Internal Control: Standards for Internal Control in the
Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November
1999).
[18] The DHS management directive entitled "Human Capital Line of
Business Integration and Management" specifies several roles and
responsibilities for the CHCO and component heads. Roles for the CHCO
include advising and assisting top DHS officials on training issues and
designing processes and systems to achieve departmentwide training
goals. For example, component heads have the role of recruiting,
hiring, and reviewing human capital officials, including training
staff. However, DHS has not yet specified, in detail, the
responsibility, accountability, and authority of the CHCO and the
component heads, although officials have indicated that they plan to do
so during fiscal year 2006.
[19] While several provisions in the Homeland Security Act require the
Coast Guard to be maintained as a distinct entity and would limit the
range of management initiatives regarding the Coast Guard, none of them
would appear to be applicable in this case. We find nothing in the DHS
management directive on the integration of human capital that
contravenes these limitations and nothing in the directive would
reasonably appear to threaten the status of the Coast Guard as a
distinct entity or otherwise impair its ability to perform statutory
missions. We have reported on a similar situation with respect to the
department's acquisition function. See GAO, Homeland Security:
Successes and Challenges in DHS's Efforts to Create an Effective
Acquisition Organization, GAO-05-179 (Washington, D.C.: Mar. 29, 2005).
[20] Similarly, DHS's management directive on human capital integration
also asserts that the Secret Service is exempted by statute. As with
the Coast Guard, we are unaware of any specific statutory exemption
that would prevent the application of the DHS management directive
regarding human capital, and given the nature of the management
directive, we do not see any apparent reason to exempt the Secret
Service.
[21] GAO-04-546G.
[22] We have previously reported that organizations undergoing
successful transformations look for and implement best practices
wherever they may be found. See GAO-03-669 and GAO, Highlights of a GAO
Forum, Mergers and Transformation: Lessons Learned for a Department of
Homeland Security and Other Federal Agencies, GAO-03-293SP (Washington,
D.C.: Nov. 14, 2002).
[23] GAO-04-546G.
[24] GAO-04-546G.
[25] Donald L. Kirkpatrick, author of Evaluating Training Programs: The
Four Levels (San Francisco: Berrett-Koehler Publishers, Inc., 1998),
developed a commonly recognized four-level model for evaluating
training and development efforts. The fourth level is sometimes split
into two levels with the fifth level, return on investment,
representing a comparison of costs and benefits quantified in dollars.
See Jack J. Philips, Implementing Evaluation Systems and Processes.
[26] Higher levels of evaluation, and in particular Level V, can be
challenging to conduct because of the difficulty and costs associated
with data collection and the complexity in directly linking training
programs to improved individual and organizational performance. Factors
to consider when deciding the appropriate level of evaluation include
estimated costs of the training effort, size of the training audience,
management interest, program visibility, and the anticipated "life
span" of the effort. In light of these considerations, an agency may
decide to evaluate participants' reactions (Level I) for all of its
training programs, while conducting a return on investment analysis
(Level V) for only a very few. Each agency will need to consider the
feasibility and cost-effectiveness of conducting these in-depth
evaluations, along with budgetary and staffing circumstances that may
limit the agency's ability to complete such evaluations.
[27] GAO-04-546G.
[28] See GAO, Organizational Culture: Techniques Companies Use to
Perpetuate or Change Beliefs and Values, GAO/NSIAD-92-105 (Washington,
D.C.: Feb. 27, 1992), and Organizational Culture: Use of Training to
Help Change DOD Inventory Management Culture, GAO/NSIAD-94-193
(Washington, D.C.: Aug. 30, 1994). The first of these reports examined
techniques used by nine large private sector firms to affect their
organizational cultures. We found that company officials identified two
techniques that were of very great importance to a successful culture
change: (1) total commitment of top management and (2) training that
promotes and develops skills in line with the desired culture. More
recently, we have reported on the cultural changes and key practices
necessary for successful transformation. See, GAO, 21st Century
Challenges: Transforming Government to Meet Current and Emerging
Challenges, GAO-05-830T (Washington, D.C.: July 13, 2005).
[29] GAO-04-546G.
GAO's Mission:
The Government Accountability Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. Government Accountability Office
441 G Street NW, Room LM
Washington, D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director,
NelliganJ@gao.gov
(202) 512-4800
U.S. Government Accountability Office,
441 G Street NW, Room 7149
Washington, D.C. 20548:
