Workforce Investment Act
Labor Actions Can Help States Improve Quality of Performance Outcome Data and Delivery of Youth Services Gao ID: GAO-04-308 February 23, 2004The Workforce Investment Act (WIA) has been in effect for several years and is currently undergoing reauthorization. In order to provide the Congress with information on the implementation and effectiveness of the WIA youth program, GAO was asked to explore how services have been delivered at the local level, whether the Department of Labor's guidance has addressed challenges faced by local areas, and how effective the program has been.
Local areas primarily used the WIA program for dropout prevention and other efforts to improve academic achievement for in-school youth. Nationally, about 70 percent of youth served were in school, but percentages ranged from 38 to 86 percent by state. Officials in the five states GAO visited said that they focused on in-school youth because serving out-of-school youth was much more difficult and expensive, and less effective. Local areas emphasized learning-related summer employment for in-school youth and occupational skills training and supportive services for out-of-school youth. Over half of local boards nationwide used providers that had subcontracting arrangements with others to deliver youth services. The majority of youth were served primarily from educational institutions and community organizations. Despite Labor's guidance, local areas continue to face implementation challenges in identifying and retaining out-of-school youth, providing youth with mentoring and follow-up services, and using interim measures for ongoing program assessment. While Labor supports information exchange forums, a promising practices Web site, and technical assistance, some local areas may have difficulties gaining access to and using these resources. Little is known about the effectiveness of the WIA youth program because Labor has not yet conducted an impact evaluation. In addition, while the youth program exceeded most of its performance goals, these data were questionable because of problems with state information systems and inadequate oversight of data quality. While states will be required to verify data, concerns remain about their ability to fully implement the requirement and Labor's ability to monitor implementation consistently.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-308, Workforce Investment Act: Labor Actions Can Help States Improve Quality of Performance Outcome Data and Delivery of Youth Services
This is the accessible text file for GAO report number GAO-04-308
entitled 'Workforce Investment Act: Labor Actions Can Help States
Improve Quality of Performance Outcome Data and Delivery of Youth
Services' which was released on February 23, 2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Ranking Minority Member, Committee on Health, Education,
Labor, and Pensions, U.S. Senate:
United States General Accounting Office:
GAO:
February 2004:
WORKFORCE INVESTMENT ACT:
Labor Actions Can Help States Improve Quality of Performance Outcome
Data and Delivery of Youth Services:
GAO-04-308:
GAO Highlights:
Highlights of GAO-04-308, a report to the Ranking Minority Member,
Committee on Health, Education, Labor, and Pensions, U.S. Senate
Why GAO Did This Study:
The Workforce Investment Act (WIA) has been in effect for several
years and is currently undergoing reauthorization. In order to provide
the Congress with information on the implementation and effectiveness
of the WIA youth program, GAO was asked to explore how services have
been delivered at the local level, whether the Department of Labor‘s
guidance has addressed challenges faced by local areas, and how
effective the program has been.
What GAO Found:
Local areas primarily used the WIA program for dropout prevention and
other efforts to improve academic achievement for in-school youth.
Nationally, about 70 percent of youth served were in school, but
percentages ranged from 38 to 86 percent by state. Officials in the
five states GAO visited said that they focused on in-school youth
because serving out-of-school youth was much more difficult and
expensive, and less effective. Local areas emphasized learning-related
summer employment for in-school youth and occupational skills training
and supportive services for out-of-school youth. Over half of local
boards nationwide used providers that had subcontracting arrangements
with others to deliver youth services. The majority of youth were
served primarily from educational institutions and community
organizations.
Despite Labor‘s guidance, local areas continue to face implementation
challenges in identifying and retaining out-of-school youth, providing
youth with mentoring and follow-up services, and using interim
measures for ongoing program assessment. While Labor supports
information exchange forums, a promising practices Web site, and
technical assistance, some local areas may have difficulties gaining
access to and using these resources.
Little is known about the effectiveness of the WIA youth program
because Labor has not yet conducted an impact evaluation. In addition,
while the youth program exceeded most of its performance goals, these
data were questionable because of problems with state information
systems and inadequate oversight of data quality. While states will be
required to verify data, concerns remain about their ability to fully
implement the requirement and Labor‘s ability to monitor
implementation consistently.
What GAO Recommends:
GAO is recommending that the Departments of Labor and Education
coordinate efforts to clarify how schools can work with workforce
officials to help connect school dropouts to local WIA youth programs.
GAO is also recommending that the Department of Labor provide states
and local areas with technical assistance necessary to address ongoing
implementation challenges and establish standard monitoring procedures
to improve the quality of data reported by states.
In formal comments on a draft of this report, Education concurred with
our recommendation to work with Labor to connect out-of-school youth
to local WIA youth programs. In its informal comments, Labor said that
many of the findings corroborated its own observations and that the
recommendations are consistent with Labor‘s current program direction.
www.gao.gov/cgi-bin/getrpt?GAO-04-308.
To view the full product, including the scope and methodology, click
on the link above. For more information, contact David Bellis (415)
904-2272 or bellisd@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
WIA Program Used Primarily for Dropout Prevention and Emphasized Summer
Services and Skills Training Coordinated among Multiple Service
Providers:
Labor's Guidance Has Not Fully Addressed Implementation Challenges
Faced by Local Areas:
Lack of Program Evaluations and Questionable Performance Data Prevent
Assessments of WIA Youth Program:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Comments from the Department of Education:
Appendix II: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: WIA Youth Performance Measures:
Table 2: Percentage of Local Boards Using Different Types of Service
Delivery Arrangements, by Number of Youth Served:
Figures:
Figure 1: Percentage of All WIA Youth Served Who Were in School, by
State (including the District of Columbia):
Figure 2: WIA Services Provided to In-School and Out-of-School Youth:
Figure 3: Number of WIA Youth Service Providers Used by Local Boards:
Figure 4: Primary Providers of Services to WIA Youth:
Abbreviations:
OIG: Office of Inspector General:
JTPA: Job Training Partnership Act:
WIA: Workforce Investment Act:
WIASRD: Workforce Investment Act Standardized Record Data:
United States General Accounting Office:
Washington, DC 20548:
February 23, 2004:
The Honorable Edward M. Kennedy:
Ranking Minority Member:
Committee on Health, Education, Labor, and Pensions:
United States Senate:
More than 5 million young people--15 percent of the nation's youth
population between ages 16 and 24--are out of school and out of work,
according to a recent study. Further, many teenagers and young adults
who are in school are at risk of dropping out. According to some
experts, this indicates that a considerable portion of the country's
emerging workforce may face significant difficulty making the
transition to productive adulthood.[Footnote 1] At the same time, the
Department of Labor projects that some labor demands will go unmet
because there will be too few workers in the labor market with the
necessary skills. Enacted in 1998, the Workforce Investment Act (WIA)
aims to address these issues by assisting the nation's emerging
workforce in realizing its full potential. As the administering agency,
Labor has budgeted about $1 billion annually on WIA Title I-B youth
employment and training programs to serve an estimated 721,000 of the
nation's most at-risk young people.
WIA services are based on promising practices in the fields of youth
development and employment. Research suggests, for example, that youth
can achieve positive outcomes when they interact with caring adults,
engage in hands-on education and training activities, and receive
support for personal growth. The WIA program has sought to make these
sorts of experiences available to both in-school and out-of-school
youth participants. Under WIA, local areas can tailor their approach to
the types of youth served, the services provided, and how they are
delivered. To ensure that youth programs are tailored to local areas,
WIA requires the participation of a wide variety of people--youth
policy experts, representatives from youth-serving agencies, parents,
and others with a vested interest in the local youth programs. These
participants serve on local workforce investment boards created by WIA
to establish workforce development policies and oversee implementation
at the local level, or on youth councils, subgroups of the local board
that plan and coordinate the local youth program.
We previously reported on the implementation challenges local areas
faced during the first few years of implementation.[Footnote 2] Now
that the program has been ongoing for several years and WIA is
undergoing reauthorization, you asked us to review (1) what approaches
local areas have taken to serve at-risk youth, (2) whether Labor's
youth program guidance has addressed ongoing implementation challenges,
and (3) what is known about the effectiveness of the WIA youth program.
To obtain information on what approaches local areas have taken to
deliver youth services, we administered a survey to the directors of
all 604 local workforce investment boards across the nation, including
those in the 50 states, the District of Columbia, Puerto Rico, and U.S.
territories. We received responses from 496 local workforce investment
boards (82 percent) and relied on self-reported data. To further
understand local area approaches to service delivery, we visited nine
local workforce investment areas in five states: California, Louisiana,
New Hampshire, Ohio, and Virginia. We selected these states to obtain a
mix based on their differences in geographic location, number of
workforce investment areas, amount of youth funding, and presence of a
state youth council. In addition, our selection was informed by
recommendations from Labor, youth policy experts, and state workforce
officials. We interviewed officials representing state and local
boards, youth councils, one-stop centers, youth-service providers,
business representatives, and state and local education agencies. We
reviewed Labor's program evaluation agenda and published guidance
letters from program year 2000 to the present. We also assessed the
reliability of the performance data submitted by states in their annual
reports and compiled in Labor's WIA database known as the Workforce
Investment Act Standardized Record Data (WIASRD) by performing checks
for internal consistency, reviewing Office of Inspector General reports
and other relevant documents, and speaking with the contractor for
WIASRD. We determined that the data were not sufficiently reliable to
use for the purposes of this report. We also interviewed officials from
Labor, including each of its six regional offices, as well as from the
Department of Education and national associations. Our work was
conducted between January and December 2003 in accordance with
generally accepted government auditing standards.
Results in Brief:
Local WIA youth programs primarily focused on preventive strategies to
help in-school youth avoid academic failure and dropping out of school,
emphasizing summer enrichment services that were coordinated through
individual case managers and multiple service providers. Our survey of
local boards showed that about 70 percent of WIA youth were in school,
although this percentage varied widely, from 38 percent in South Dakota
to 86 percent in Nebraska. State and local education officials said
that the WIA program allowed communities to provide complementary
services needed to support at-risk in-school youth. For example, local
officials from one urban area in Virginia said that while a local
school provided 1 counselor for as many as 300 students, the WIA
program funded 1 counselor for every 50 WIA participants. In addition,
local officials in four states we visited said they focused on serving
in-school youth because it was easier and less costly than recruiting
and retaining youth once they had dropped out of school. Local boards
reported that more than half of in-school youth received summer
employment services that were linked to classroom learning. For
example, a service provider in a rural area of California enrolled in-
school youth in a 6-week summer enrichment program where students
worked part-time while learning reading skills. In contrast, WIA youth
who were out of school were more likely to receive occupational skills
training and supportive services, such as assistance with child care,
transportation, and housing. While in-school and out-of-school youth
usually participated in programs separately, both groups worked with
case managers who helped develop individual service strategies and
coordinate delivery of services. The majority of WIA youth were served
primarily by community organizations or by educational institutions
such as high schools, colleges, and universities.
Despite guidance issued by Labor, local areas continued to face
challenges in serving out-of-school youth, providing mentoring and
follow-up services, and establishing and using optional interim
performance measures. One reason serving out-of-school youth continued
to be challenging was that such youth can be difficult to locate within
the community. Some local workforce and regional Labor officials said
that identifying youth was problematic, in part because schools did not
always ensure that dropouts were linked with the WIA program. Another
reason cited was the difficulty in retaining youth who were primarily
interested in immediate employment rather than in participating in
WIA's long-term youth development activities. Local officials added
that mentoring and follow-up services were challenging to provide, in
part because of the difficulty in finding enough qualified mentors to
work with at-risk youth and sustaining a connection with youth once
they had exited the program. Some state workforce officials said that
to more effectively serve out-of-school youth, they needed better-
targeted guidance from Labor that addressed their local areas'
particular service delivery issues. Local areas also faced challenges
establishing interim measures, such as enrollment and service
participation, to assess program performance and improve service
delivery in a timely manner. Despite Labor's encouragement, some
regional Labor officials said that interim performance measures were
not widely used by local areas. Labor has established WIA youth
learning exchanges, a Web site, and a technical assistance program to
help address WIA youth program implementation challenges, but some
states and local areas may not have access to or be aware of these
resources.
Little is known about the effectiveness of the WIA youth program
because program impact evaluations have not been performed and
performance outcome data that have been collected may not be reliable.
Impact evaluations provide information on program effectiveness by
differentiating between outcomes that result from the program itself
and those that result from other factors. Because of possible
legislative changes to the WIA youth program, Labor does not plan to
undertake an impact evaluation until 2006, with initial results
expected by 2009. While states annually report performance data to
Labor, these data are questionable because of insufficient state
monitoring of data quality and the inadequacies of some state
management information systems. Labor officials said that they will
require states to verify local area data beginning with program year
2002 data, but concerns remain about states' ability to fully implement
validation requirements and Labor's ability to monitor implementation
consistently.
We are recommending that Labor and Education coordinate efforts to
clarify how schools can work with workforce officials to connect
dropouts to WIA youth services. We are also recommending that Labor
provide guidance to address specific concerns identified by local
implementers and establish standards to monitor data quality.
Background:
Enacted in 1998, WIA replaced the fragmented and overlapping programs
under the Job Training Partnership Act (JTPA) with a system that sought
to connect employment, education, and training services to better match
job seekers to labor market needs. The youth program falls under Title
I of WIA along with programs for adults and dislocated workers; Title
II deals with adult and family literacy; Title III pertains to
employment services under the Wagner-Peyser Act; and Title IV addresses
vocational rehabilitation. To better prepare low-income youth who face
barriers to employment and education, WIA requires youth programs to
focus on long-term, comprehensive services delivered year-round through
a coordinated network of service providers. In addition to meeting the
low-income requirement, to be eligible for WIA services youth must be
between the ages of 14 and 21 and faced with one or more of six
barriers to employment.[Footnote 3] WIA serves both in-school and out-
of-school youth. A significant portion of out-of-school youth are high
school dropouts and unemployed youth who are struggling to succeed in
the public education system and lack financial, family, or social
support. Thus, at least 30 percent of local WIA youth funds must be
spent on out-of-school youth.[Footnote 4]
WIA Youth Funding:
Since WIA's full implementation in 2000, funding for the youth program
has ranged between approximately $1 billion and $1.4 billion a year.
Labor follows a formula to allocate WIA funds to states, which in turn
distribute money to their local workforce investment areas.[Footnote 5]
At the state level, the governor can reserve up to 15 percent of the
annual WIA allotment for such statewide activities as developing the
WIA strategic plan for the state. States allocate the remaining 85
percent of funds to local boards that use the funds to develop the
local strategic plan, establish a youth council, and award competitive
contracts to youth service providers that have been recommended by the
youth council. In addition to recommending eligible providers, the
youth council coordinates youth activities in the local area, oversees
providers, and carries out other duties authorized by the local board
such as forging linkages with educational agencies. Youth councils may
also leverage additional public and private funds to supplement their
WIA funding in order to provide comprehensive youth services.
Youth Services:
Once they are determined to be WIA eligible, youth receive an objective
assessment of their academic level, skills, and service needs. Local
youth programs then use the assessment to create each participant's
individual service strategy, which lays out employment goals,
educational objectives, and necessary services. Every local youth
program must offer the following 10 services, known as program
elements, to eligible youth, though participants may receive different
combinations of these elements depending on their service strategy.
Labor groups the 10 required program elements around four major themes:
Improving educational achievement:
1. tutoring, study skills training, and instruction leading to
completion of secondary school, including dropout prevention,
2. alternative school services,
Preparing for and succeeding in employment:
3. summer employment linked to academic and occupational learning,
4. paid and unpaid work experiences, including internships and job
shadowing,
5. occupational skills training,
Developing the potential of young people as citizens and leaders:
6. leadership development, which may include community service and
peer-centered activities encouraging responsibility,
Supporting youth:
7. supportive services (such as child care and housing assistance),
8. adult mentoring for at least 12 months that may occur both during
and after program participation,
9. follow-up upon program completion for at least 12 months, and:
10. comprehensive guidance and counseling.
Eligible youth may obtain these services directly from approved youth
service providers or through WIA's one-stop system, which serves as a
gateway to a variety of employment and training services.[Footnote 6]
In addition to helping WIA youth gain access to the 10 program
elements, the one-stop system also provides all youth with basic
services, whether or not they are eligible for WIA. Any young person
may walk into a one-stop center and make use of the center's career
resources and obtain information on and referrals to other providers.
While one-stop centers are designed primarily to serve those 18 and
older, some states have established one-stops that serve only youth, as
we reported previously.[Footnote 7]
Youth Program Guidance:
Labor provides guidance to help states and local areas implement WIA.
Labor's guidance includes issuing annual Training and Employment
Guidance Letters on the youth program in general as well as on specific
topics, sponsoring WIA Learning Exchanges[Footnote 8] in every region,
and maintaining a Web site for promising and effective practices. Labor
responds to requests for clarification or additional information by
phone or e-mail or at conferences. Labor's guidance often includes
promising and effective practices to reinforce state and local
flexibility under WIA to tailor programs to best meet youths' needs.
Performance Measures:
The law requires that states and local areas collect performance
information on seven youth measures, which are separated for younger
and older youth (see table 1). All seven youth measures apply to both
statewide and local performance.
Table 1: WIA Youth Performance Measures:
Younger youth measures (ages 14-18):
1. Skill attainment rate;
2. Diploma or equivalent attainment rate;
3. Placement and retention in postsecondary education, advanced
training, or employment;
Older youth measures (ages 19-21):
1. Entered into employment rate;
2. The employment retention rate at six months;
3. Increase in average earnings;
4. The credential rate.
Source: GAO analysis of the Workforce Investment Act.
[End of table]
WIA performance information is collected from service providers and
flows upward to the local boards, then to state boards or agencies, and
finally to Labor. Local performance data are aggregated and entered
into a state's own automated data system that tracks the activities of
individual WIA participants. States use two mechanisms to report
performance to Labor: WIASRD, a database of individual records
containing activity and outcome information for each registered
participant that has exited WIA, and state annual reports. Labor uses
the state annual reports to track states' progress in meeting
negotiated performance goals. Labor then awards monetary incentives to
states that meet or exceed their performance goals and sanctions states
that fail to meet at least 80 percent of each goal in two consecutive
years.
WIA Program Used Primarily for Dropout Prevention and Emphasized Summer
Services and Skills Training Coordinated among Multiple Service
Providers:
Most local areas used WIA youth program services for a range of
approaches to prevent academic failure and school dropouts, emphasizing
learning-related summer services that were coordinated through case
managers and multiple service providers. Nationally, about 70 percent
of youth served were in school, according to local boards responding to
our survey. For in-school youth, local boards provided summer
employment services linked to classroom learning more often than for
out-of-school youth, while services for this latter population more
often included occupational skills training and supportive services.
Local boards reported that most youth received services primarily from
community organizations and educational institutions. These providers
were most likely to subcontract or make informal arrangements with
other organizations to deliver the full range of WIA services to youth
participants.
Local Areas Focused on Dropout Prevention for In-School Youth:
Overall, local areas' approach to serving youth was to supplement
schools' dropout prevention efforts to keep youth connected to an
educational system, according to state and local workforce and
education officials. WIA allows states and local areas to determine the
proportion of in-school and out-of school youth to be served, requiring
only that they spend at least 30 percent of funds for out-of-school
youth. Labor reported that with some exceptions, states chose to focus
the majority of resources on in-school youth,[Footnote 9] and our
survey showed that about 81 percent of local areas served more in-
school youth than out-of-school youth. Nationally, about 70 percent of
youth served were in school, according to local boards we surveyed.
However, this percentage varied across the nation, as shown in figure
1. For example, 38 percent of youth served in South Dakota were in
school, compared with 86 percent in Nebraska.
Figure 1: Percentage of All WIA Youth Served Who Were in School, by
State (including the District of Columbia):
[See PDF for image]
[A] Includes states where less than 65 percent of boards responded.
[End of figure]
According to officials in four states that we visited, WIA youth
programs primarily targeted in-school youth because recruiting and
retaining out-of-school youth for the WIA program was much more
difficult and expensive. For example, officials from a rural area in
Ohio had difficulty identifying and retaining out-of-school youth whose
contact information changes frequently, and Labor reports that the
average cost of serving an out-of-school youth under the WIA program is
about $4,000 a year, twice as much as for an in-school youth. Officials
from other local areas we visited considered other factors. A local
official in Louisiana said that in-school youth are interested in the
WIA program, unlike out-of-school youth, who are difficult to engage.
In contrast, a rural area in Virginia chose to focus primarily on out-
of-school youth because so few services were available for this
population.
The Administration has proposed amending the WIA youth program to focus
more resources on out-of-school youth. Department of Education
officials said that WIA's services to in-school youth were not unique,
since schools already offer various services to their students to deter
them from dropping out. While Education officials said that the
department's grants geared exclusively toward dropout prevention would
be insufficient to address the national dropout problem, they also
stated that funding is available under many Education programs that
could be used for dropout prevention activities.[Footnote 10] However,
local workforce and education officials we spoke with in three states
we visited said that they were either unaware of or unable to gain
access to other available federal resources that could be used to
provide intensive services to at-risk youth. In New Hampshire, for
example, officials told us that WIA provides the only dropout
prevention program, and that they were unaware of other available
education dropout prevention resources.
State and local workforce and education officials we spoke with in the
five states we visited said that WIA funding complemented rather than
duplicated education services and was critical in preventing in-school
youth from dropping out of school. For example, in a rural area in
Ohio, workforce officials stated that the WIA program was the only
dropout prevention program and that WIA provided students with their
only chance at academic and career success. In addition, they said that
WIA's services were more intensive and comprehensive, and were
delivered in a one-on-one setting where each student received
individualized attention. In one urban school in Virginia, there was 1
WIA counselor for 50 eligible in-school youth, compared to 1 school
counselor for as many as 300 students.
In-School Youth Received More Summer Employment; Out-of-School Youth
Received More Occupational Skills Training:
Local areas tailored services provided to youth based on their needs,
emphasizing different services, depending on whether youth were in or
out of school. To meet youth's many developmental needs, the WIA youth
program requires that local areas offer the same menu of 10 academic,
employment, and support services to all eligible youth, which WIA
providers choose from when tailoring services to an individual's
service strategy based on an assessment of needs.[Footnote 11] As shown
in figure 2, services such as work experience and leadership
development were provided fairly equally to both youth populations, but
there were differences in other areas. For example, in-school youth
were more likely to receive tutoring services and summer employment
linked to classroom learning, while out-of-school youth were more
likely to receive occupational skills training and supportive services
to help prepare them for employment.
Figure 2: WIA Services Provided to In-School and Out-of-School Youth:
[See PDF for image]
Note: Follow-up services are not depicted in the figure because they
are mandatory for all participants when they exit the WIA youth
program.
[End of figure]
WIA requires that summer employment programs be linked to academic
learning, and Labor guidance promotes meaningful summer enrichment
experiences to help youth attain skills throughout the school
year.[Footnote 12] Local youth programs in the five states we visited
were providing youth with various types of summer experiences. For
example, a service provider in a rural area of California enrolled in-
school youth in a 6-week summer enrichment program where students
worked part-time while learning reading skills. Another service
provider in a rural area in Louisiana offered in-school youth summer
services that included academic enrichment, community service, and
exposure to different career options.
Local youth programs provided out-of-school youth training in
occupational skills and job readiness, as well as offered them
supportive services. Labor's guidance states that enrolling youth in
occupational skills training and retaining them until program
completion leads to better outcomes.[Footnote 13] Local areas provided
several different types of occupational skills training, depending on
local labor market needs. For example, one urban local area in Ohio
trained out-of-school youth in nursing, welding, and computer repair,
while another urban local area in Virginia offered technology
certification training. Local areas also provided job readiness skills
cited by employers as lacking in many youth seeking employment. For
example, a program for out-of-school youth in an urban area of Ohio
taught participants such job readiness skills as customer service,
conflict management, and other interpersonal skills. Supportive
services were also important for out-of-school youth who needed
additional assistance to help them overcome their multiple barriers to
employment. A service provider in New Hampshire provided out-of-school
participants with support services such as child care, transportation,
and housing assistance.
Case Managers Developed Service Strategies and Providers Coordinated to
Deliver Services:
Almost all local boards we surveyed reported using case managers to
coordinate services for youth in and out of school. Because at-risk
youth often require services from a host of departments, including
Education, Health and Human Services, Housing and Urban Development,
Justice, and others, Labor's guidance promotes the use of a case
manager to coordinate services among all these youth-serving partners
to provide seamless access to and delivery of services.[Footnote 14]
Nationwide, all but 3 percent of local boards responding to our survey
reported that their youth program uses case managers. Local boards
reported that, in addition to performing other duties, case managers
assessed youth upon enrollment in the WIA program. These assessments
typically included a review of educational attainment, work readiness,
work experience, and career interests. The assessments are then used to
develop an individual service strategy for each youth participant.
Labor's guidance requires that each strategy identify employment goals
and educational objectives, and prescribe appropriate services for each
participant.[Footnote 15] About three-fourths of local boards reported
that youth were greatly or very greatly involved in the development of
their individual service strategies.
Most local areas used multiple service providers to deliver the youth
services spelled out in each participant's individual service strategy.
WIA does not specify how services must be provided to youth, allowing
local areas to determine how many providers they will fund and hold
responsible for delivering the services outlined in each youth's
individual service strategy. Over half of local boards responding to
our survey reported using 4 or fewer service providers, but 15 percent
used more than 10 providers, as shown in figure 3.
Figure 3: Number of WIA Youth Service Providers Used by Local Boards:
[See PDF for image]
Note: Some local board directors reported using zero providers because
the board itself delivered youth services. WIA permits local boards to
provide training services if there is an insufficient number of
eligible providers to meet local demand and if the board has
demonstrated that it meets the requirements for an eligible service
provider.
[End of figure]
Over half of local boards nationwide used providers that did not
deliver all services themselves, using formal or informal
subcontracting arrangements to provide the range of services needed.
The extent that providers coordinated with others to deliver services
was related to the number of youth served in the local area. In local
areas with 250 or fewer youth, providers delivered all services
themselves more often than in areas with more than 250 youth, as
indicated in table 2.
Table 2: Percentage of Local Boards Using Different Types of Service
Delivery Arrangements, by Number of Youth Served:
Providers deliver all services without using subcontractors;
0-250 youth: 44;
251-750 youth: 34;
Over 750 youth: 30.
Providers are responsible for delivering all services but use
subcontractors;
0-250 youth: 23;
251-750 youth: 31;
Over 750 youth: 39.
Providers do not deliver all services themselves, but make
noncontractual arrangements with other organizations to make sure all
services are delivered;
0-250 youth: 22;
251-750 youth: 25;
Over 750 youth: 17.
Other arrangement;
0-250 youth: 10;
251-750 youth: 10;
Over 750 youth: 14.
Source: GAO survey of local board directors.
Note: Columns may not add to 100 because of rounding.
[End of table]
Regardless of the level of coordination used to provide youth services,
workforce officials told us that providers often informally
collaborated with one another through regular meetings to discuss
problems, train one another, share best practices, or share other
resources.
Local boards responding to our survey reported using a range of public
and private entities to deliver youth services. Over half of all youth
received services primarily from educational institutions or community
organizations, while less than 1 percent of youth received services
primarily from faith-based organizations, as shown in figure 4.
Figure 4: Primary Providers of Services to WIA Youth:
[See PDF for image]
[A] Youth opportunity centers were established under the Youth
Opportunity Grant program to enable youth living in high-poverty areas
to gain access to a wide range of services.
[End of figure]
Local boards reported that about one-fourth of youth received services
primarily from WIA one-stops--whether the one-stops focused on serving
adults or youth. However, among the one-stops we visited, these
entities were frequently used for supplemental services such as
information and referrals. Officials we spoke with in New Hampshire and
Ohio said that service providers or schools usually took youth on a
field trip to the local one-stop for basic orientation to expose them
to employment services they could use in the future. Although most
youth were served by the WIA program through other providers, one-stop
usage varied considerably by local area. For example, the majority of
WIA youth living in an urban area of California were served by youth-
exclusive one-stops, while youth in a rural area of the state rarely
interacted with the local adult one-stop.
In-school and out-of-school youth usually did not participate in WIA
programs together. Nationwide, 43 percent of local boards reported that
these two types of youth received services from different providers. In
another 27 percent of cases, the two groups received services from the
same providers but participated separately. Another 24 percent of local
boards reported that all youth were served by the same service
providers and participated together.[Footnote 16]
Local Areas Developed Partnerships with Business and Education to
Deliver Youth Services:
Local areas developed partnerships with the local business and
education communities to identify employer needs and provide
comprehensive youth services related to academic and employment
preparation. In many cases, youth councils helped facilitate these
partnerships.
Local areas partnered with businesses to identify and provide
employment and training services for WIA youth needed to fill high-
demand, high-growth occupations.[Footnote 17] Over 85 percent of local
boards reported using each of the following methods to develop and
maintain relationships with local businesses: consulting with
businesses about their job needs, training participants in skills
needed by local businesses, training program participants to understand
the values and attitudes local businesses look for in employees, and
providing employment experience to participants that suits the jobs
available in local businesses. Officials we spoke with at the local
youth programs in all five states we visited provided such work
readiness training as punctuality, teamwork, respect for others, and
appropriate dress. In addition to identifying needed skills, businesses
also provided employment opportunities. Nationally, 34 percent of local
boards reported that businesses subsidized work experience for WIA
youth. In New Hampshire, for example, a financial services firm
employed out-of-school youth and taught them about personal financial
management. In addition, an appliance-store owner, once an at-risk
youth himself, hired WIA youth and hoped to groom one of them to take
over the business. Finally, officials in three states we visited noted
that businesses donated in-kind assistance such as building materials,
work clothes, work readiness training workshops, and a financial
management curriculum. For example, in an urban area in Virginia,
businesses donated and remodeled the space for the one-stop center.
Local areas also partnered with schools to provide academic preparation
services to WIA youth. In a local area of Louisiana, for example, a
service provider helped eighth graders explore classes leading to high
school industry-based certifications. In all five states we visited,
representatives from local postsecondary institutions made
presentations to WIA youth, informing them about higher education
opportunities. In some instances, WIA youth were allowed to take
advanced level courses at their institutions and even earn college
credit. In addition, a community college in California provided a
pathway to facilitate the transition from high school to higher
education by hosting youth for a one-day college experience of classes
and conducting workshops on financial aid. Officials in four states we
visited also said that schools provided in-kind assistance such as
office space and tutoring.
Local boards reported that youth councils performed a number of
important functions that facilitated partnerships between local boards
and the community. Over 70 percent of local boards reported that the
youth council served as a forum to bring together key community
partners who may never have collaborated with each other. Almost two-
thirds of local boards reported that they would keep their youth
council even if it became optional.[Footnote 18] In addition, over half
of local boards reported that their youth councils elevated the
importance and visibility of local youth issues and programs and added
value to the youth program.
Labor's Guidance Has Not Fully Addressed Implementation Challenges
Faced by Local Areas:
Local areas continue to face challenges in implementing aspects of the
WIA youth program despite guidance issued by Labor. Some of these
ongoing challenges include recruiting and retaining out-of-school
youth, providing mentoring and follow-up services, and designing
interim performance measures that can be used to continually improve
aspects of program performance before youth exit the program. Labor
supplements guidance on these issues through information forums, a Web
site, and a technical assistance program. However, regional Labor
officials said that state and local areas' access to these resources
had been limited.
Out-of-School Youth Were Hard to Recruit and Retain:
Labor has issued guidance on recruiting and retaining out-of-school
youth for the WIA program during the last two years, but local areas
continue to face challenges in serving this population. Labor issued a
guidance letter to states in April 2001 outlining strategies for
recruiting youth to the program, suggesting such methods as engaging
youth to recruit their peers, collaborating with community
organizations that already work with disadvantaged youth, and offering
incentives for recruiting new participants.[Footnote 19] In subsequent
guidance issued in May 2002, Labor described ways to keep out-of-school
youth engaged in the program, such as by helping youth make the link
between career potential and education and skills training as well as
by providing support services like child care.[Footnote 20] Despite
this guidance, local officials from most of the states we visited said
that recruiting out-of-school youth continues to be a problem.
According to data from Labor, the District of Columbia, Delaware, and
New Mexico had not met WIA's requirement to spend at least 30 percent
of WIA funds on out-of-school youth for program year 2001 as of
September 30, 2003.[Footnote 21]
Some local workforce officials indicated that closer coordination with
schools to immediately connect dropouts to the WIA program would help
them identify and recruit more out-of-school youth. Some local
workforce and regional Labor officials said that one reason schools may
not share information on dropouts directly with the WIA program was
because of concerns about student privacy restrictions. However,
schools we visited in four states shared dropout information by
referring students who had been expelled or had dropped out directly to
the WIA service provider, notified or provided contact information to
the WIA provider when a student had dropped out, or worked with the WIA
program to develop a list of dropouts. None of the schools, however,
had procedures in place to routinely connect all dropouts with local
WIA programs. Labor and Education officials agreed that schools could
do more to work closely with local workforce officials to help connect
dropouts to local WIA youth programs.
Local officials from most of the states we visited said that retaining
out-of-school youth in the WIA program was also difficult. According to
some local officials, one reason was that out-of-school youth tend to
prefer immediate employment instead of training and academic learning.
Other officials said that both in-school and out-of-school youth face
difficulties with transportation--particularly in rural areas---that
limit their involvement in WIA programs. For example, local officials
in a rural area in Ohio said that transportation was their biggest
issue. To mitigate this challenge, the local area offered services
through a mobile one-stop unit that traveled to WIA clients throughout
the county. Some state workforce officials said that to be more
effective in serving this population, they needed guidance and
technical assistance that was more focused on the specific service
delivery issues within their local areas.
Mentoring and Follow-up Were among the Most Difficult Services to
Provide:
Nearly all local areas we visited indicated that providing mentoring
services continued to be a problem for the WIA youth program, but Labor
has not addressed mentoring in its annual youth program guidance or
shared best practices on the provision of high-quality mentoring
services. Local areas identified several reasons why mentoring has been
difficult. In one instance, a Virginia official in a rural area said
that it was difficult to identify sufficient numbers of qualified
mentors to work with eligible youth. In another instance, a local area
official in California noted that geographical distances within the
county resulted in long commutes and discouraged adults from mentoring
youth. A service provider in New Hampshire said that finding mentors
was especially difficult for out-of-school youth, as some adults are
uncomfortable working with school dropouts. Labor youth program
officials said that mentoring is an important way of connecting youth
to caring adults and said they would take action to provide guidance on
this topic.
Another service element that remains challenging for local areas is
providing complete and thorough follow-up services to help youth
succeed after they have exited the program. WIA regulations require
that follow-up services last for at least 12 months, and Labor's
guidance states that follow-up may include regular contact with a
youth's employer to track progress made, assistance in addressing work-
related problems, and help in securing better-paying jobs and further
education. Labor's policy guidance for program year 2001 provided some
principles from best practices in the field of youth development to
help local areas develop strategies for follow-up. The principles
included developing a systematic approach for maintaining contact and
interaction with the young person; evening and weekend social
activities for informal support; helping youth access services to
fulfill physical, emotional, and vocational needs; and visiting the job
site soon after the youth has started employment. Nevertheless, several
local officials cited continued difficulties in sustaining a connection
with youth and identifying outcomes for them once they exited the WIA
program.
Few Local Areas Used Interim Measures to Gauge Program Success:
Labor's guidance underscored the importance of establishing and using
optional interim performance measures to monitor the success of
delivering WIA youth services, but some regional Labor officials said
that such measures were not widely used. Six of the seven required WIA
youth measures are collected only after youth exit the program.
However, because youth may remain in the program for many years, local
areas with long-term youth retention strategies may have limited means
of gauging progress without interim measures. Labor issued guidance in
May 2002, encouraging the use of interim measures, which may be tracked
and reported separately from the required annual performance measures,
to provide ongoing feedback on which aspects of the youth program were
working well or needed modification.[Footnote 22] The guidance
identified the following interim measures that local boards could use
to monitor progress as youth move through the WIA program:
* intake, including development of a comprehensive individual service
strategy with short-term and long-term goals;
* participation in program elements to help ensure youth meet short-
term skill attainment goals;
* skill attainment to help ensure successful completion of short-and
long-term goals;
* exit data, including number of participants exiting the program; and:
* follow-up services received to help measure youth performance
outcomes.
Some local areas we visited were using interim measures to hold
providers accountable for delivering services. For example, New
Hampshire workforce officials said that they tracked enrollment and
expenditure levels to monitor activity levels across their contracted
service providers. In a local area in California, workforce officials
monitored the progress of youth by tracking enrollment, participation,
work readiness skills, and the rate at which youth made a successful
transition to other activities once they exited the WIA program.
Despite the potential usefulness of these data, interim measures were
not widely used, according to Labor officials in two regions. These
officials said that states lacked the resources to properly track them.
Additionally, the guidance did not explain how states and local areas
could collect, analyze, and use the data to assess progress and make
needed adjustments. For example, officials in one state we visited said
that they were unclear about how to apply the interim measures.
Access to and Awareness of Labor's Assistance Has Been Limited:
Labor has disseminated information related to these and other issues
through forums that allow local areas to exchange information with one
another, but access to these forums has been limited. From December
2002 to April 2003, for example, Labor sponsored peer-to-peer WIA
Learning Exchanges with two national youth organizations that provided
venues around the country for local areas to share information and
observe an exemplary program firsthand. However, state budgetary cuts
prevented many local areas from attending some of these forums in
person, according to an official from the sponsoring organization.
Although information and ideas shared at the Learning Exchanges were
later summarized and made available to all local areas, those that were
unable to attend missed the opportunity to network with their
colleagues and develop an action plan to take back and apply to their
program.[Footnote 23]
Labor has also supported the development of an online resource to
facilitate information sharing among local areas. The agency's
Promising Practices Web site was intended to provide a mechanism for
local areas to post and share promising workforce development
practices, including those pertaining to WIA youth programs. However,
some state workforce and regional Labor officials said that states and
local areas may not even have been aware of the site, that it has been
difficult to enter practices for inclusion, and that some users found
it difficult to access the documents described in the
narrative.[Footnote 24] We also had difficulty accessing and using the
Web site to find information. For example, when we conducted a search
on the key word "mentoring," the five results did not satisfactorily
address the topic.
Labor guidance encourages local areas to tap into other resources that
can help them develop quality youth programs.[Footnote 25] The
Promising and Effective Practices Network, for example, provides a
useful listing of promising practices categorized by the specific
strategy and the 10 required WIA youth services that local areas can
use for improving their WIA youth programs.[Footnote 26]
Labor has provided technical assistance to states through its
Performance Enhancement Project, initiated in program year 2002. Under
this initiative, Labor grouped into three tiers states that failed or
were at risk of failing to meet their performance goals, based on their
reported outcomes, according to Labor officials.[Footnote 27] These
officials also said that states in the first tier received priority for
targeted technical assistance to improve their youth programs, with the
goal of improving performance outcomes. Labor officials said that
states could use this technical assistance to address any of the
challenges they faced in implementing their WIA youth program. However,
unless the state falls into one of these three tiers, most local areas
may not receive such assistance to help them increase the proportion of
out-of-school youth served, improve mentoring and follow-up, and use
interim measures.
Lack of Program Evaluations and Questionable Performance Data Prevent
Assessments of WIA Youth Program:
Little is known about the effectiveness of the WIA youth program
because Labor has not yet initiated an impact evaluation, and results
from a planned evaluation will likely not be available until 2009,
according to Labor officials. While Labor's performance data for
program year 2001 indicate that five of seven youth measures were
exceeded, these results cannot be used to infer program outcomes
because of insufficient monitoring of state data quality and
inadequacies of some state management information systems. Labor
officials said that they will require states to validate local area
data beginning with program year 2002 data, but some implementation
concerns remain.
Labor Plans to Initiate First Impact Evaluation in 2006, and Results
Are At Least Five Years Away:
According to department officials, Labor intends to initiate an impact
evaluation[Footnote 28] of the WIA youth program in 2006, and
preliminary results will not be available until about 2009. While Labor
officials said that the youth program's 3 years of full implementation
was sufficient time to initiate a comprehensive evaluation, they were
anticipating significant changes to the WIA youth program as a result
of reauthorization. They did not plan to begin an impact evaluation of
the program until after these changes had been implemented.
If reauthorization is completed by spring 2004, Labor officials said
they anticipate that the process of awarding the contract for the study
will be completed by 2006 when the impact evaluation is scheduled to
begin. The evaluation will proceed with 5 or 6 years of data collection
with an additional 3 or 4 years of follow-up activity. Labor officials
said they expect to issue a series of interim reports before the final
product. If the project begins in 2006, Labor expects to issue the
first report on short-term impacts in 2009, with a final report on
long-term impacts available in 2011. According to officials, this
evaluation is part of Labor's proposed research plan to study all of
its major employment and training programs, including the WIA youth
program, on a regular cycle.[Footnote 29]
State Performance Outcome Data Were Questionable, Partly because of
Inconsistent Data Monitoring:
Performance data submitted by states to Labor in quarterly and annual
reports were not sufficiently reliable to determine outcomes for the
WIA youth program. Labor's national aggregation of WIA performance data
for program year 2001 indicated that the program exceeded its goals for
five of the seven youth measures.[Footnote 30] According to Labor's
Office of Inspector General (OIG), however, there is little assurance
that the states' performance data for all WIA programs, including the
youth program, are either accurate or complete because of inadequate
oversight of data collection and management at the local, state, and
federal levels. The OIG also found that just 2 of 12 local areas and
none of four states it reviewed had formal policies on documentation
requirements for participant activities and outcomes.[Footnote 31] At
the local level, for example, another OIG report found that WIA youth
program outcomes were adequately documented only 37 percent of the time
for a sample of 420 participants across 14 local areas.[Footnote 32] An
official in one of Labor's regional offices added that documentation
requirements are inconsistent among states and local areas.
At the state level, the OIG reported that two of four states it
reviewed had not monitored local areas' performance data at the case
file level, and that none of the four states had adequate procedures in
place to ensure the accuracy of their performance data.[Footnote 33]
According to regional Labor officials, some states had insufficient
procedures in place for verifying the accuracy of the data collected by
their local areas. For example, officials in one region said that state
monitoring rarely addressed data accuracy or included the verification
of a sample of data items against original records. In addition, some
state information systems had significant flaws that caused them to
produce incorrect information, casting further doubt on the quality of
WIA performance data. For example, Ohio state officials said they were
in the process of replacing their old information system that had
caused such errors as data changing or disappearing entirely after it
had been entered in the system.
At the federal level, Labor did not have a standard data-monitoring
guide in place, and officials in some regional offices--who, according
to agency officials, are responsible for overseeing the quality of
states' reported data---said they followed their own oversight
procedures. These procedures did not usually include verifying the
accuracy of a sample of the data submitted by states. For example, an
OIG report stated that while regional offices conducted some data
accuracy reviews, such as computer edit checks, they did not verify the
data's accuracy with such tests as comparing the data with participant
case files. Consistent with the OIG's findings, officials in all six of
Labor's regional offices said that they examined state data submissions
through desk reviews, which included checking for errors such as
incorrect calculation of performance measures, extreme outliers, and
miscoding of data. However, only the Atlanta regional office checked a
sample of data records against source documentation. In its review of
data records from six of the states it oversees, the Atlanta office
examined a sample of participant records across all WIA programs from
each state and found errors that affected the calculation of one or
more performance measures. In one state, for example, 17 percent of
participant records had at least one error, compared with 83 percent of
participant records in another state. The regional office also found
that two of six states it reviewed computed the younger youth skill
attainment measure incorrectly.
Labor recognizes these problems with data reliability, but in
accordance with WIA regulations, uses states' annual performance
reports---the only reports that depict states' progress in achieving or
exceeding negotiated performance levels---to make incentive grant and
sanction decisions. States that exceed the WIA performance goals
negotiated between Labor and the states are eligible to receive
incentive grants that generally range from $750,000 to $3
million.[Footnote 34] States that fail to meet 80 percent of their WIA
performance goals for 2 years in a row are subject to sanction of up to
a 5 percent reduction in their annual WIA formula grant. However,
Labor's use of questionable performance data reported by the states may
hinder its ability to negotiate realistic performance goals and make
appropriate incentive grant and sanction decisions. Ohio state
officials questioned the appropriateness of being sanctioned 2 percent
of its program year 2002 WIA youth allotment. They believed that poor
performance levels were due to problems with its information system,
not its workforce development system.
Data Validation Initiative May Improve Reliability but Concerns Remain:
To address the data issues described above, Labor is implementing a new
data validation policy requiring states to ensure the accuracy of their
annual reports and verify a sample of the data they submit. According
to Labor's policy, these requirements will be phased in over a 3-year
period. In the first year, states will be required to validate their
annual reports and data submissions for the program year 2002
period.[Footnote 35] According to agency officials, Labor does not plan
to publicly release these initial reports but will use their findings
to work with the states to correct their data accuracy problems. In the
second year, Labor will use validation reports covering program year
2003 data to establish acceptable error rates. In the third year, Labor
will require states to meet the acceptable error rates for their
program year 2004 data submissions. Labor will consider failure to meet
the standard a violation equivalent to failing to submit a report, for
which states may be subject to corrective action or financial sanction,
as appropriate. In addition, Labor's data validation guidance indicates
that states that do not meet data accuracy standards will receive
technical assistance.
While the data validation initiative may improve the reliability of WIA
performance data, several implementation concerns remain. First,
officials in some of Labor's regional offices said that the states they
oversee will have difficulty implementing the data element verification
requirement because of limited staff resources. Further, these
officials said that in states where local providers keep the original
documentation on-site, retrieving the documentation to check it against
records in their information system will be time-consuming and
expensive. Second, a Labor official said that the agency does not plan
to issue a program-monitoring guide to standardize procedures across
regional offices for several more years.
Conclusions:
Many youth are struggling to be successful in the public education
system and often face substantial obstacles to obtaining a high school
diploma and going on to college or getting jobs with career advancement
possibilities. Many of these youth may lack meaningful social and
family supports and may require comprehensive, intensive services to
remain engaged in society and avoid risky behaviors that can lead to
chronic unemployment, criminal activity, and other adverse outcomes.
WIA currently provides both the education and workforce systems with
strategies and resources to engage youth in academic and job training.
The opinion held by some federal education officials that WIA in-school
services overlap with existing education programs is not necessarily
seconded by officials in local areas. State and local workforce and
education officials believe WIA's educational, occupational, and
support services provide critical support and services to at-risk youth
and that without WIA's comprehensive services, schools may face an
increasing burden to keep these youth in school and ensure their
academic success. The connection between WIA youth services and schools
could be made more effective if Labor worked with Education to find
ways to connect school dropouts with local WIA youth programs.
Since WIA's passage, Labor has provided general guidance and promising
practices in addressing implementation issues, but increased
availability of technical assistance may be necessary to overcome some
of the more difficult challenges some states and local areas face in
providing youth services. This will be especially critical for states
and local areas that are reportedly meeting performance goals but still
need assistance in improving delivery of youth services and shifting
program resources to target different populations such as out-of-school
youth. In addition, local areas will need guidance, including specific
strategies, to help them provide effective mentoring and follow-up
services and use interim measures to track program performance.
Robust research and reliable performance data are needed to obtain a
complete picture of the WIA youth program's effectiveness and outcomes
and to make quality decisions about managing the program. However, none
currently exist. Labor has not yet initiated an impact evaluation and
does not expect to report on program impacts from a planned evaluation
until 2009. In addition, states and local areas continue to struggle to
collect and document accurate and complete participant and performance
data and maintain data systems that can yield reasonably reliable
outcome information. For its part, Labor is taking action to improve
data integrity by requiring states to validate WIA performance data,
beginning with data from program year 2002. However, Labor's
inconsistent monitoring processes across regions will continue to
challenge Labor's capability to ensure that all states validate and
report data consistently and effectively. In the short term, the lack
of accurate outcome data will continue to hinder Labor's ability to
negotiate realistic state performance goals and use data to make sound
decisions about financial incentives or sanctions. In the long term,
lack of accurate and complete information will keep Labor from
obtaining a true picture of how effectively the youth program is
working.
Recommendations for Executive Action:
To promote information sharing that improves local WIA youth programs'
ability to identify and serve out-of-school youth, we recommend that
the Secretaries of Labor and Education coordinate efforts to clarify
how schools can work with workforce officials to connect school
dropouts with local WIA youth programs.
To assist state and local WIA youth programs address ongoing
implementation challenges, we recommend that the Secretary of Labor:
* increase availability of guidance and technical assistance to local
areas that continue to face challenges in serving out-of-school youth;
* disseminate guidance, including specific strategies, to help local
areas provide effective mentoring services; and:
* develop additional guidance on providing follow-up services and using
interim measures to track program performance.
To obtain an accurate and complete gauge of WIA outcomes and determine
whether local programs are operating successfully, we recommend that
the Secretary of Labor establish standard monitoring procedures that
Labor's regional offices could use to oversee state data validation
efforts.
Agency Comments and Our Evaluation:
We provided a draft of this report to Labor and Education officials for
their review and comment. Education's comments are reprinted in
appendix I. In its formal comments, Education concurred with our
recommendation to work with Labor to connect out-of-school youth to
local WIA youth programs. Labor responded informally, and said that
many of the findings corroborated its own observations and that the
recommendations are consistent with Labor's current program direction.
Both agencies also provided technical comments, which we incorporated
as appropriate.
We will send copies of this report to the Secretaries of Labor and
Education, relevant congressional committees, and other interested
parties. Copies will be made available to others upon request. In
addition, the report will be available at no charge on GAO's Web site
at http://www.gao.gov.
Please contact me at (415) 904-2272 if you or your staff have any
questions about this report. Other major contributors to this report
are listed in appendix II.
Sincerely yours,
Signed by:
David D. Bellis:
Director, Education, Workforce, and Income Security Issues:
[End of section]
Appendix I: Comments from the Department of Education:
UNITED STATES DEPARTMENT OF EDUCATION:
OFFICE OF VOCATIONAL AND ADULT EDUCATION:
THE ASSISTANT SECRETARY:
JAN 2 8 2004:
Mr. David D. Bellis
Director:
Education, Workforce, and Income Security Issues
United States General Accounting Office:
441 G Street, N.W. Washington, DC 20548:
Dear Mr. Bellis:
Thank you for the opportunity to review and comment on the draft
report, Workforce Investment Act: Labor Actions Can Help States Improve
Quality of Performance Outcome Data and Delivery of Youth Services
(GAO-04-308). We appreciate your interest in improving the
effectiveness of Federal programs that serve disadvantaged youth.
In 2000, nearly 11 percent of young adults ages 16 through 24 were not
enrolled in school and had not earned a high school diploma or
equivalent credential. Without further education and training, these
3.8 million young people face a grim future in an economy that places a
premium on education. Young adults who lack a high school credential
are more likely to be unemployed and to be out of work for longer
periods of time than their more educated peers. When they do secure
employment, they earn significantly less and are more likely to be
dependent on public assistance. Today, most jobs that pay family-
supporting wages require not only a high school credential but the
completion of further education and/or training.
We concur with your recommendation that the Departments of Education
and Labor work together to find new ways to connect more of these young
people with the education and training they need to achieve
independence and succeed in the workforce. During Fiscal Year 2001,
local programs funded by the Adult Education and Family Literacy Act
(AEFLA) served more than 1.1 million young adults between the ages of
16 and 24 who were not enrolled in school and had not earned a high
school diploma or equivalent credential. However, AEFLA supports
academic instruction only. The President's proposals for the
reauthorization of AEFLA and Title I of the Workforce Investment Act
will expand access by these young adults to occupational skills
training, job placement, and other services available through the
Department of Labor's One-Stop Career Center System, as well as make it
easier for adult education participants to transition to postsecondary
education upon earning a high school credential.
The two Departments worked closely together in developing the
President's proposals and we expect to continue our collaboration in
implementing the new legislation upon its enactment. One of the
Department's chief implementation priorities is developing a
coordinated strategy for serving out-of-school youth and strengthening
the connections between adult education programs and the One-Stop
Career Center System. By working together, the two Departments can help
communities use Federal resources more effectively to improve outcomes
for out-of-school youth and ensure that no child is left behind.
Thank you for the opportunity to address this important issue.
Sincerely,
Signed by:
Susan Sclafani:
Assistant Secretary for Vocational and Adult Education:
[End of section]
Appendix II: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Lacinda Ayers, Assistant Director (206) 654-5591 Meeta Sharma, Analyst-
in-Charge (206) 287-4806:
Staff Acknowledgments:
In addition to the individuals mentioned above, the following staff
made key contributions to this report: Karyn Angulo, Susan Baker,
Andrew Bauck, Paula Bonin, Maya Chakko, Keira Dembowski, Joel Grossman,
Corinna Nicolaou, Rebecca Woiwode, and Monica Wolford.
[End of section]
Related GAO Products:
Transportation-Disadvantaged Populations: Some Coordination Efforts
Among Programs Providing Transportation Services, but Obstacles
Persist. GAO-03-697. Washington, D.C.: June 30, 2003.
Workforce Investment Act: One-Stop Centers Implemented Strategies to
Strengthen Services and Partnerships, but More Research and Information
Sharing is Needed. GAO-03-725. Washington D.C.: June 18, 2003.
Workforce Investment Act: Issues Related to Allocation Formulas for
Youth, Adults, and Dislocated Workers. GAO-03-636. Washington, D.C.:
April 25, 2003.
Workforce Investment Act: States' Spending Is on Track, but Better
Guidance Would Improve Financial Reporting. GAO-03-239. Washington,
D.C.: November 22, 2002.
Workforce Investment Act: Youth Provisions Promote New Service
Strategies, but Additional Guidance Would Enhance Program Development.
GAO-02-413. Washington, D.C.: April 5, 2002.
Workforce Investment Act: Improvements Needed in Performance Measures
to Provide a More Accurate Picture of WIA's Effectiveness. GAO-02-275.
Washington, D.C.: February 1, 2002.
School Dropouts: Education Could Play a Stronger Role in Identifying
and Disseminating Promising Prevention Strategies. GAO-02-240.
Washington, D.C.: February 1, 2002.
FOOTNOTES
[1] Center for Law and Social Policy, Leave No Youth Behind:
Opportunities for Congress to Reach Disconnected Youth, (Washington,
D.C.: July 2003).
[2] U.S. General Accounting Office, Workforce Investment Act: Youth
Provisions Promote New Service Strategies, but Additional Guidance
Would Enhance Program Development, GAO-02-413 (Washington, D.C.: April
5, 2002).
[3] A youth is considered to face employment barriers if he or she is
(1) deficient in basic literacy skills; (2) a school dropout; (3)
homeless, a runaway, or a foster child; (4) pregnant or a parent; (5)
an offender; or (6) an individual who requires additional assistance to
complete an educational program or to secure and hold employment. Up to
5 percent of WIA youth participants are not required to meet the income
eligibility requirements, but all youth served by WIA must meet barrier
requirements.
[4] Out-of-school youth do not include youth enrolled in alternative
schools at the time of WIA registration.
[5] For a discussion of WIA formula funding, see U.S. General
Accounting Office, Workforce Investment Act: Issues Related to
Allocation Formulas for Youth, Adults, and Dislocated Workers,
GAO-03-636 (Washington, D.C.: April 25, 2003).
[6] To create a more comprehensive workforce investment system, WIA
requires states and localities to bring together 17 federally funded
employment and training services into a single system, called the one-
stop center system. Locally, the one-stop system must include at least
one physical site offering a comprehensive array of WIA services as
well as those of the other partners, and may be supplemented by
satellite sites that provide one or more WIA services. For more
information on the one-stop system, see U.S. General Accounting Office,
Workforce Investment Act: One-Stop Centers Implemented Strategies to
Strengthen Services and Partnerships, but More Research and Information
Sharing Is Needed, GAO-03-725 (Washington, D.C.: June 18, 2003).
[7] Local boards in Alaska, Arizona, Arkansas, California, Colorado,
the Federated States of Micronesia, Florida, Idaho, Louisiana,
Missouri, New Jersey, New York, Puerto Rico, Tennessee, Texas, and
Wisconsin reported having one or more youth-only one-stop centers. See
GAO-02-413.
[8] WIA Learning Exchanges for Youth Systems are two-day regional
meetings designed to highlight promising practices at exemplary WIA
sites, foster peer-to-peer information sharing, and develop team-driven
action plans in three areas: (1) recruiting and retaining out-of-school
youth; (2) building and sustaining partnerships, especially with the
education community; and (3) defining and aligning assessments, skill
achievement, and credentials. Along with the National Youth Employment
Coalition and the American Youth Policy Forum, Labor conducted seven
learning exchanges around the country, one of which was specifically
for rural local areas.
[9] Excluding the territories, Labor's data show that only eight states
had spent 50 percent or more of their program year 2001 WIA youth
allotment on out-of-school youth as of September 30, 2003: Iowa, Idaho,
Illinois, Maryland, Maine, New Hampshire, North Carolina, and South
Dakota.
[10] For a discussion of Education and other federal programs to assist
youth at risk of dropping out of school, see U.S. General Accounting
Office, School Dropouts: Education Could Play a Stronger Role in
Identifying and Disseminating Promising Prevention Strategies,
GAO-02-240 (Washington, D.C.: February 1, 2002).
[11] One of the 10 required services, follow-up, is required for all
youth for at least 12 months after they exit the WIA program.
[12] Research has shown that engaging youth during the summer provides
them with an opportunity to learn new skills that they can apply in
school and also learn the value of work. See Westat, The 1993 Summer
Youth Employment and Training Program, (Rockville, MD, April 1994). In
addition, Labor's guidance highlights work-based learning during summer
employment as a strategy to help out-of-school youth acquire work
experience while making the connection between staying in school and
pursuing a career. See U.S. Department of Labor, Training and
Employment Guidance Letter No. 3-99, (Washington, D.C., 2000).
[13] See U.S. Department of Labor, Training and Employment Guidance
Letter No. 18-00, (Washington, D.C., 2001).
[14] See Training and Employment Guidance Letter No. 18-00.
[15] See U.S. Department of Labor, Training and Employment Guidance
Letter No. 9-00 (Washington, D.C., 2001).
[16] The remaining 6 percent of local boards reported their arrangement
for service delivery to in-school and out-of-school youth as "other."
[17] Over 70 percent of local boards reported that their programs
encouraged youth to seek employment in each of the following sectors:
retail, hospitality and food service, health care, and information
technology.
[18] Of the local boards that reported that they would not keep their
youth council, 73 percent reported that they would use a youth
committee of the local board to perform the functions of a youth
council.
[19] See Training and Employment Guidance Letter No. 18-00.
[20] See U.S. Department of Labor, Training and Employment Guidance
Letter No. 28-01, (Washington, D.C., 2002).
[21] Under WIA, states have three years to spend their annual
allotment, therefore program year 2001 funds must be expended by June
30, 2004.
[22] See Training and Employment Guidance Letter No. 28-01.
[23] Labor officials told us that their regional offices have also
sponsored youth conferences and other forums where guidance and
technical assistance have been provided.
[24] We previously reported on problems with the Promising Practices
Web site. See GAO-03-725. This Web site can be found at http://
www.promising-practices.org.
[25] See Training and Employment Guidance Letter No. 18-00.
[26] The National Youth Employment Coalition's Promising and Effective
Practices Network Web site can be found at http://www.nyec.org/pepnet.
[27] According to Labor, the first tier includes those states that were
financially sanctioned. The second tier includes those states that
failed one or more performance measures but were not sanctioned. The
third tier includes those states that did not fail any measures but
were deemed at risk of failing them.
[28] By isolating a program's effects, impact evaluations provide
policymakers with key information for determining program
effectiveness. To isolate a program's effect, impact evaluations divide
participants into two groups: one that receives program services and a
similar group that does not. Some impact evaluations assign
participants randomly to one or the other group; the group that does
not receive services is called the control group. The use of random
assignment allows researchers to compare outcomes for the two groups
and attribute any differences to the program services rather than other
factors.
[29] Labor anticipates that the WIA youth program will be evaluated on
a 10-year cycle.
[30] States did not meet goals for the earnings change measure and
credential rate for older youth. States reported actual earnings change
in 6 months as $3,109, compared with their negotiated goal of $3,396.
For the credential rate, about 40 percent of older youth earned
credentials, compared with the goal of 44 percent.
[31] U.S. Department of Labor, Office of Inspector General, Workforce
Investment Act Performance Outcomes Reporting Oversight, 06-02-006-03-
390 (Washington, D.C., Sept. 30, 2002).
[32] U.S. Department of Labor, Office of Inspector General, Workforce
Investment Act: Evaluation of Youth Program Enrollments, Services, and
Recorded Outcomes, 06-03-006-03-390 (Washington, D.C., September 30,
2003).
[33] U.S. Department of Labor, Office of Inspector General, 06-02-006-
03-390.
[34] To be eligible for incentive grants, states must also meet
performance goals for the Adult Education and Literacy programs under
Title II of WIA and programs authorized under the Carl D. Perkins
Vocational and Applied Technology Education Act.
[35] Program year 2002 started on July 1, 2002, and ended on June 30,
2003.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
