Juvenile Justice
DOJ Is Enhancing Information on Effective Programs, but Could Better Assess the Utility of This Information Gao ID: GAO-10-125 December 17, 2009State juvenile justice systems face critical problems when it comes to juvenile delinquency issues such as reentry--when offenders return home from incarceration--and substance abuse. GAO was asked to review juvenile reentry and substance abuse program research and efforts by the Department of Justice's (DOJ) Office of Juvenile Justice and Delinquency Prevention (OJJDP) to provide information on effective programs (i.e., whether a program achieves its intended goal) and cost-beneficial programs (i.e., whether the benefits of programs exceeded their costs). This report addresses (1) expert opinion and available research on these types of reentry and substance abuse programs, (2) the extent to which OJJDP assesses its efforts to disseminate information on effective programs, and (3) OJJDP's plans to accomplish its research and evaluation goals. GAO, among other things, reviewed academic literature, and OJJDP's dissemination efforts and research goals. GAO also interviewed OJJDP officials and a nonprobability sample of 26 juvenile justice experts selected based on their experience with juvenile reentry and substance abuse issues.
The majority of the juvenile justice reentry and substance abuse experts GAO interviewed cited evidence that shows cognitive behavioral therapy--programs that help individuals change their beliefs in order to change their behavior--and family therapy--programs that treat juveniles by focusing on improving communication with family members--are effective and cost beneficial when addressing reentry and substance abuse issues. For example, two juvenile reentry experts cited studies showing that 1 year after participating in a cognitive behavioral therapy program, participants were less likely to commit another offense than nonparticipants. Additionally, experts cited a study that reported that a family therapy program provides about $80,000 in savings per participant when accounting for savings from a decline in crime, such as the cost the police would have incurred. Most experts indicated that there was limited evidence on the effectiveness and cost benefits of reentry programs, such as aftercare--programs that assist juvenile offenders in returning to their communities during the reentry process--and substance abuse programs, such as drug courts--specialized courts that provide programs for substance-abusing juveniles and their families. GAO reviewed two OJJDP efforts that provide information on effective programs across the range of juvenile justice issues, the National Training and Technical Assistance Center (NTTAC) and the Model Programs Guide. OJJDP has mechanisms in place to regularly assess the utility of the information provided by NTTAC, but does not have such a mechanism for the guide. OJJDP ensures the utility of NTTAC's information through evaluations in accordance with federal guidelines that highlight the importance of regularly soliciting feedback from users. However, OJJDP could better ensure the utility of the information disseminated by the Model Programs Guide by having a mechanism in place to solicit regular feedback from members of the juvenile justice field--for example, program practitioners--that is specifically related to the guide. OJJDP has articulated research and evaluation goals to support its mission of improving the juvenile justice system and is developing plans to assist in meeting these goals. OJJDP is required under the Juvenile Justice and Delinquency Prevention Act, as amended, to publish an annual program plan that describes planned activities under accounts authorized for research and evaluation activities, among other things. Additionally, the Office of Management and Budget (OMB) recommended that OJJDP develop a comprehensive evaluation plan for juvenile justice programs. While OJJDP has not published an annual program plan since 2002, in December of 2009, it issued a proposed plan for public comment and aims to publish the final program plan once public comments are incorporated. Additionally, although the office has considered developing a comprehensive evaluation plan to address OMB recommendations, it had not previously done so because of a lack of resources. However, OJJDP is committed to developing a comprehensive evaluation plan once the program plan is finalized.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-10-125, Juvenile Justice: DOJ Is Enhancing Information on Effective Programs, but Could Better Assess the Utility of This Information
This is the accessible text file for GAO report number GAO-10-125
entitled 'Juvenile Justice: DOJ Is Enhancing Information on Effective
Programs, but Could Better Assess the Utility of This Information'
which was released on December 17, 2009.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as part
of a longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Subcommittee on Crime, Terrorism, and Homeland
Security, Committee on the Judiciary, House of Representatives:
United States Government Accountability Office:
GAO:
December 2009:
Juvenile Justice:
DOJ Is Enhancing Information on Effective Programs, but Could Better
Assess the Utility of This Information:
GAO-10-125:
GAO Highlights:
Highlights of GAO-10-125, a report to the Chairman, Subcommittee on
Crime, Terrorism, and Homeland Security, Committee on the Judiciary,
House of Representatives.
Why GAO Did This Study:
State juvenile justice systems face critical problems when it comes to
juvenile delinquency issues such as reentry”when offenders return home
from incarceration”and substance abuse. GAO was asked to review
juvenile reentry and substance abuse program research and efforts by
the Department of Justice‘s (DOJ) Office of Juvenile Justice and
Delinquency Prevention (OJJDP) to provide information on effective
programs (i.e., whether a program achieves its intended goal) and cost-
beneficial programs (i.e., whether the benefits of programs exceeded
their costs). This report addresses (1) expert opinion and available
research on these types of reentry and substance abuse programs, (2)
the extent to which OJJDP assesses its efforts to disseminate
information on effective programs, and (3) OJJDP‘s plans to accomplish
its research and evaluation goals. GAO, among other things, reviewed
academic literature, and OJJDP‘s dissemination efforts and research
goals. GAO also interviewed OJJDP officials and a nonprobability sample
of 26 juvenile justice experts selected based on their experience with
juvenile reentry and substance abuse issues.
What GAO Found:
The majority of the juvenile justice reentry and substance abuse
experts GAO interviewed cited evidence that shows cognitive behavioral
therapy”programs that help individuals change their beliefs in order to
change their behavior”and family therapy”programs that treat juveniles
by focusing on improving communication with family members”are
effective and cost beneficial when addressing reentry and substance
abuse issues. For example, two juvenile reentry experts cited studies
showing that 1 year after participating in a cognitive behavioral
therapy program, participants were less likely to commit another
offense than nonparticipants. Additionally, experts cited a study that
reported that a family therapy program provides about $80,000 in
savings per participant when accounting for savings from a decline in
crime, such as the cost the police would have incurred. Most experts
indicated that there was limited evidence on the effectiveness and cost
benefits of reentry programs, such as aftercare”programs that assist
juvenile offenders in returning to their communities during the reentry
process”and substance abuse programs, such as drug courts”specialized
courts that provide programs for substance-abusing juveniles and their
families.
GAO reviewed two OJJDP efforts that provide information on effective
programs across the range of juvenile justice issues, the National
Training and Technical Assistance Center (NTTAC) and the Model Programs
Guide. OJJDP has mechanisms in place to regularly assess the utility of
the information provided by NTTAC, but does not have such a mechanism
for the guide. OJJDP ensures the utility of NTTAC‘s information through
evaluations in accordance with federal guidelines that highlight the
importance of regularly soliciting feedback from users. However, OJJDP
could better ensure the utility of the information disseminated by the
Model Programs Guide by having a mechanism in place to solicit regular
feedback from members of the juvenile justice field”for example,
program practitioners”that is specifically related to the guide.
OJJDP has articulated research and evaluation goals to support its
mission of improving the juvenile justice system and is developing
plans to assist in meeting these goals. OJJDP is required under the
Juvenile Justice and Delinquency Prevention Act, as amended, to publish
an annual program plan that describes planned activities under accounts
authorized for research and evaluation activities, among other things.
Additionally, the Office of Management and Budget (OMB) recommended
that OJJDP develop a comprehensive evaluation plan for juvenile justice
programs. While OJJDP has not published an annual program plan since
2002, in December of 2009, it issued a proposed plan for public comment
and aims to publish the final program plan once public comments are
incorporated. Additionally, although the office has considered
developing a comprehensive evaluation plan to address OMB
recommendations, it had not previously done so because of a lack of
resources. However, OJJDP is committed to developing a comprehensive
evaluation plan once the program plan is finalized.
What GAO Recommends:
GAO recommends that OJJDP develop a cost-effective mechanism to
regularly solicit and incorporate feedback from the juvenile justice
field on the usefulness of information in the Model Programs Guide. DOJ
concurred with GAO‘s recommendation.
View [hyperlink, http://www.gao.gov/products/GAO-10-125] or key
components. For more information, contact Eileen Larence at (202) 512-
8777 or larencee@gao.gov.
[End of section]
Contents:
Letter:
Scope and Methodology:
Background:
Experts Cite Evidence from Available Research Indicating That Cognitive
Behavioral Therapy and Family Therapy Programs Are Effective and Cost
Beneficial When Addressing Reentry and Substance Abuse Issues:
OJJDP Has Mechanisms in Place to Ensure Training and Technical
Assistance Meet Users' Needs, but Regular Feedback on the Model
Programs Guide Would Help OJJDP Better Assess Information Utility:
Finalizing a Program Plan and Developing a Comprehensive Evaluation
Plan Would Help OJJDP Achieve Its Research and Evaluation Goals and Use
Its Limited Resources Effectively:
Conclusions:
Recommendation for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Juvenile Justice Reentry and Substance Abuse Experts We
Interviewed:
Appendix II: Juvenile Justice Experts' Views on Factors That Can Help
Programs Achieve Intended Outcomes:
Appendix III: OJJDP's Enacted Appropriations for Fiscal Years 2007
through 2009:
Appendix IV: Juvenile Reentry and Substance Abuse Programs OJJDP Funded
through Discretionary Grants:
Appendix V: Experts' Opinions of Reentry Programs That Lack Conclusive
Evidence of Effectiveness:
Appendix VI: Experts' Opinions of Substance Abuse Programs That Lack
Conclusive Evidence of Effectiveness:
Appendix VII: Additional OJJDP Efforts to Disseminate Information about
Effective Juvenile Justice Programs:
Appendix VIII: Comments from the Department of Justice:
Appendix IX: GAO Contact and Staff Acknowledgments:
Tables:
Table 1: Types of Juvenile Reentry Programs:
Table 2: Types of Juvenile Substance Abuse Programs:
Table 3: Examples of Net Benefits of Program Interventions within
Reentry and Substance Abuse Program Types We Reviewed from Four Studies
Cited by Experts:
Table 4: OJJDP Funds Authorized and Used for Research and Evaluation,
Fiscal Year 2008:
Table 5: Juvenile Justice Experts We Interviewed about Juvenile Reentry
or Substance Abuse Programs:
Table 6: Juvenile Justice Enacted Appropriations, Fiscal Years 2007
through 2009:
Table 7: OJJDP Juvenile Reentry and Substance Abuse Programs Funded
through Discretionary Grants, Fiscal Years 2007 through 2009:
Table 8: Additional OJJDP Efforts to Disseminate Information on
Effective Programs:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
December 17, 2009:
The Honorable Robert C. Scott:
Chairman:
Subcommittee on Crime, Terrorism, and Homeland Security:
Committee on the Judiciary:
House of Representatives:
Dear Mr. Chairman:
Although the juvenile arrest rate is near its lowest point in two
decades, critical problems affecting states' juvenile justice systems
remain, such as recidivism--the act of committing new offenses after
having been arrested or convicted of a crime--and substance abuse--a
pattern of use of illegal, prescription, or nonprescription drugs
leading to significant impairment in functioning. Reentry, the return
of juvenile offenders[Footnote 1] from residential facilities back into
their communities, aims to reduce recidivism by using programs that
promote the effective reintegration of juvenile offenders and assist
them in acquiring the life skills needed to succeed and become law-
abiding citizens.[Footnote 2] According to the Department of Justice
(DOJ), approximately 100,000 juvenile offenders are held in residential
facilities in the United States on a given day, and most of these
juveniles will likely leave these facilities and return home to their
communities each year.[Footnote 3] Further, DOJ has estimated that 62
percent of juveniles in residential facilities self-reported having had
at least one prior commitment in the juvenile justice system.[Footnote
4] Additionally, substance abuse plays a significant role in juvenile
crime.
In addition, in a 2008 annual survey of members conducted by the
Federal Advisory Council on Juvenile Justice,[Footnote 5] reentry of
offenders into communities and schools (28 states) and substance abuse
(21 states) were two of the top three programmatic issues most
frequently reported as affecting these states' juvenile justice
systems.[Footnote 6] Specifically, concerns were noted about the
minimal number of programs available to help juveniles reenter
communities, the large number of inexperienced practitioners operating
programs, and the limited substance abuse treatment options. Related to
research and policy, the most common issue state officials cited was
the need to identify effective practices for juvenile programs (23
states) followed by the need to enhance the states' capacities to
conduct juvenile research and collect data about juvenile programs (20
states). Respondents also cited the need for federal agencies and state
governments to collaborate on conducting research and evaluations to
provide a better explanation of (1) practices that have been evaluated
and have been shown to be effective, (2) the need for more training
from federal agencies on how to conduct evaluations, and (3) increased
funding to establish evaluation procedures (23 states).
The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is
DOJ's office charged with providing national leadership, coordination,
and resources to help prevent and respond to juvenile delinquency and
victimization. OJJDP supports states and communities in their efforts
to develop and implement effective programs to prevent delinquency and
intervene after a juvenile has offended, among other things. For
example, from fiscal years 2007 through 2009, Congress provided OJJDP
almost $1.1 billion to use for grants to states, localities, and
organizations for a variety of juvenile justice programs. In support of
its mission, the office also disseminates information to states and
communities related to a number of juvenile justice issues, including
effective reentry and substance abuse programs, and funds research and
evaluations of juvenile justice programs.
States and localities develop and implement programs that specifically
target facilitating reentry and reducing substance abuse; therefore, it
is important that federal, state, and local agencies that provide
grants for programs, as well as practitioners operating programs, have
information about which ones are effective and cost beneficial. Having
such information will better position these agencies to help ensure
that federal, state, and local funds are well spent. In general,
effectiveness is determined through program evaluations, which are
systematic studies conducted to assess how well a program is working--
that is, whether a program produced its intended results or effects.
Additionally, cost-benefit analyses can help determine if the dollar
value of a program's success--such as a reduction in recidivism--
exceeds the cost of the program. To help ensure the effective use of
grant funds for juvenile reentry and juvenile substance abuse programs,
you asked us to review the available research as well as OJJDP's
efforts to provide information about effective programs to the juvenile
justice field, that is, program practitioners and communities. In
addition, you asked us to provide information on OJJDP's research and
evaluation planning. Specifically, this report addresses the following
questions:
* What do expert opinion and available research indicate about the
types of juvenile reentry programs and juvenile substance abuse
programs that are effective or cost beneficial?
* To what extent does OJJDP have efforts under way to disseminate
information about effective juvenile justice programs and assess the
utility of the information it is providing through these efforts?
* To what extent does OJJDP have plans in place to accomplish its
juvenile justice research and evaluation goals?
Scope and Methodology:
To determine what experts and the available research indicate about the
types of reentry programs and substance abuse programs that are
effective or cost beneficial for juvenile offenders, we reviewed
relevant literature, studies, and federal resources for juvenile
justice programs, and interviewed federal officials and 26 juvenile
justice experts.[Footnote 7] Specifically, to identify the types of
programs to review, we conducted a literature search for studies and
articles, including evaluations of juvenile reentry and juvenile
substance abuse programs in the United States that were published from
May 30, 1999, through May 30, 2009. We chose this time frame, the past
10 years, because it provided us with an overview of the available
research, including unpublished and ongoing studies, which assesses the
effectiveness of reentry and substance abuse programs. We also
consulted with OJJDP officials who coordinate research on juvenile
justice programs and Department of Health and Human Services officials
who oversee substance abuse and adolescent programs to obtain their
recommendations for repositories--online databases that contain
information on effective programs--and research studies and relevant
Web sites for identifying types of reentry and substance abuse
programs.[Footnote 8] Using these recommendations, information from
relevant literature, and categories of program types used by OJJDP's
Model Programs Guide, we identified five types of juvenile justice
programs that are used to address reentry issues and five types of
programs that are used to address substance abuse issues for juvenile
offenders.[Footnote 9]
Specifically related to substance abuse, we focused on substance abuse
programs that involved relapse prevention treatment for juvenile
offenders with substance abuse histories.[Footnote 10] After consulting
with experts and reviewing the literature, we excluded juvenile alcohol
abuse programs and substance abuse programs for the general juvenile
population as well as at-risk juveniles who are prone to, but have not
yet developed, substance abuse problems. For instance, we excluded
after school or recreation programs, conflict resolution programs, and
school or classroom programs. While all of these programs may have a
substance abuse component, this component is not designed to address
juvenile offenders' actual substance abuse problems.
After identifying the types of programs to be reviewed, we looked at
online databases, academic research, and professional organizations to
select subject matter experts--researchers and practitioners--to obtain
their views on the types of programs that have been shown to be
effective or cost beneficial and the basis they used for making such
determinations. We specifically identified researchers who focus on
juvenile reentry issues or substance abuse issues and practitioners who
operate programs that address these issues. We chose 26 experts to
interview as a result of this process. Specifically, we selected 13
individuals with expertise related to juvenile reentry programs, 7
individuals with expertise related to juvenile substance abuse
programs, and 6 individuals with both juvenile reentry and substance
abuse program expertise. We selected these experts based on several
criteria, including their employment histories related to juvenile
reentry and substance abuse programs and the number of years they spent
studying, evaluating, or managing programs addressing juvenile reentry
or substance abuse issues. We evaluated their experience by reviewing
the studies the researchers had completed and determining the
experience the practitioners had managing the types of juvenile reentry
and substance abuse programs selected for our review. See appendix I
for the list of experts we interviewed.
We asked these experts to provide their views about the effectiveness
of program types (e.g., drug courts), rather than about the
effectiveness of individual intervention programs (e.g., a specific
drug court program that was implemented in one county).[Footnote 11]
Because the Model Programs Guide, like other online repositories,
contains information about the effectiveness of individual intervention
programs, it does not provide information about the effectiveness of
program types. As a result, we were interested in obtaining the
experts' consolidated views of the effectiveness of program types. We
also asked the experts to identify other program types--in addition to
those that we explicitly asked about--that they considered to be
effective or cost beneficial, but no additional program types were
mentioned. In addition, we asked the experts to identify factors that
in their view could help programs to achieve intended outcomes, such as
reducing participants' recidivism, which are summarized in appendix II.
While the results of these interviews cannot be generalized to reflect
the views of all experts knowledgeable about juvenile reentry or
substance abuse programs, we believe the interviews provided us with a
good overview of the available research and valuable information about
what program types are considered to be effective by subject matter
experts. In addition, while we did not assess the methodological rigor
of studies and evaluations in our review, we corroborated expert
testimony by reviewing and summarizing the studies or evaluations that
experts cited as the basis for their opinions. We also provided the
experts with a summary of their opinions to review in order to ensure
that we correctly captured their views.
To identify the extent to which OJJDP has efforts under way to
disseminate information about effective juvenile justice programs and
assess the extent to which OJJDP ensures the utility of the information
provided, we reviewed documentation, such as OJJDP's annual reports
outlining information dissemination efforts, OJJDP publications, and a
contract related to disseminating training information on effective
programs. We interviewed knowledgeable OJJDP officials, such as the
Training Coordinator and communications policy personnel, about OJJDP's
efforts to disseminate information about effective programs. We
selected two of OJJDP's efforts through which it disseminates
information about effective programs--the Model Programs Guide and the
National Training and Technical Assistance Center (NTTAC), which
provides training and support to the juvenile justice field in
identifying and implementing effective programs--because they provide
information about effective programs across the range of issue areas in
which OJJDP is involved, including reentry and substance abuse
programs. We then compared these efforts to guidance articulated by the
Office of Justice Programs (OJP), which oversees OJJDP, and in prior
GAO reports that stresses the importance of assessing whether the
information disseminated is meeting the needs of its users.[Footnote
12] We also interviewed representatives from the two organizations that
manage these two information dissemination efforts. Additionally, we
asked the 26 juvenile reentry and substance abuse experts we
interviewed about their views regarding OJJDP's information
dissemination efforts and their opinions about the effectiveness of
these efforts. Although their views cannot be generalized to the entire
juvenile justice field, we believe that the experts provided us with a
good overview of the utility of the information disseminated by OJJDP.
We did not contact recipients of the information OJJDP disseminates for
their views on the usefulness of the information provided because of
the large volume of recipients and the resulting cost that would be
incurred to obtain this input.
To assess the extent to which OJJDP has plans in place for its research
and evaluation efforts, we reviewed relevant laws related to the
office's role in supporting research and evaluations of juvenile
justice programs. We also reviewed relevant DOJ and OJJDP
documentation, such as annual reports and strategic plans that contain
information on OJJDP's research and evaluation goals and plans. We
interviewed cognizant OJJDP officials about the office's planning
efforts related to research and evaluation. We also reviewed criteria
found in standard practices for program management and our prior
products that highlight the importance of developing plans to meet
goals and help ensure that resources are used effectively, and then
compared these criteria to OJJDP's stated plans.[Footnote 13]
Additionally, we analyzed OJJDP funding and staff data for fiscal years
2005 through 2009 to better understand the resources the office has had
available to support its evaluation activities. We chose these years
because they provide the most recent overview of OJJDP's research and
evaluation funding.
We conducted this performance audit from April 2009 through December
2009 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit
to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives.
We believe that the evidence obtained provides a reasonable basis for
our findings and conclusions based on our audit objectives.
Background:
Within states' juvenile justice systems, reentry aims to promote the
effective reintegration of juvenile offenders back into communities
upon release from residential facilities. Reentry is a process that
incorporates a variety of programs to assist juvenile offenders in the
transition from residential facilities to communities. In addition,
reentry is intended to assist juvenile offenders in acquiring the life
skills needed to succeed in the community and become law-abiding
citizens and can incorporate the use of education, mental health, drug
rehabilitation, or vocational training programs. While reentry begins
after a juvenile is released back into the community, to help ensure a
seamless transition, a reentry process begins after sentencing, then
continues through incarceration, and into the period of release back
into the community. According to OJJDP, juvenile justice practitioners
and researchers believe that providing supervision and services to
juvenile offenders returning to the community will reduce the high rate
of recidivism among these juveniles. Several types of programs address
juvenile reentry issues, as described in table 1.
Table 1: Types of Juvenile Reentry Programs:
Aftercare: programs that focus on the delivery of services and
supervision that start while a juvenile is incarcerated to assist
juvenile offenders in returning to their communities during the reentry
process. These programs prepare juvenile offenders to return to the
community by establishing collaboration with the community and
marshaling its resources to help ensure that juvenile offenders receive
services that address their individual needs, such as treatment for a
substance abuse problem.
Cognitive behavioral therapy: programs that are designed to identify
and provide juveniles the skills to change thoughts and behavior that
contribute to their problems.
Reentry courts: specialized courts that manage the return of juvenile
offenders to the community after they are released from residential
facilities. A court manages reentry by using its authority to direct
resources to support the offender's return to the community and promote
positive behavior, among other things. For example, the court may
oversee a juvenile's release into the community by assigning a judge to
meet with the juvenile once a month.
Vocational/job training: programs that provide juveniles with
employment opportunities and are intended to improve juveniles' social
and educational functioning by, for example, increasing earnings,
raising self-esteem, and instilling a positive work ethic.
Wraparound/case management: a strategy designed to keep delinquent
youth at home and out of institutions whenever possible. This strategy
involves making an array of individualized services and support
networks available to juveniles, rather than requiring them to enroll
in structured treatment programs, which may not address individual
needs.
Source: OJJDP‘s Model Programs Guide.
[End of table]
Substance abuse includes, but is not limited to, the use or abuse of
illegal drugs (e.g., heroin), prescription drugs, and nonprescription
drugs (e.g., over-the-counter medications available without a
prescription, such as cough suppressant). Treatment of substance abuse
may occur in a variety of different settings, such as in clinics on an
outpatient basis or at a hospital. Treatment can also occur in short-
and long-term residential facilities that range from secure
environments where juveniles' activities are physically restricted, to
group homes, which are nonsecure settings where juveniles live and
receive services in a homelike environment. According to the Center for
Substance Abuse Treatment, juveniles with addictions to substances can
be helped through programs that specifically target the factors
associated with substance abuse--such as a family history of such
abuse.[Footnote 14] For example, substance abuse intervention programs,
such as cognitive behavioral therapy and family therapy, aim to change
a juvenile's behavior by focusing on improving a juvenile's response to
situations that contributed to prior substance abuse. Substance abuse
intervention programs can be provided to juvenile offenders throughout
the juvenile justice system: after sentencing, during incarceration,
and after release back into the community. Whether treatment occurs
while a juvenile is incarcerated or after the juvenile is released into
the community, according to OJJDP, effective intervention programs can
help addicted juveniles to overcome their substance abuse, lead crime-
free lives, and become productive citizens. Table 2 describes types of
programs--in addition to cognitive behavioral therapy and wraparound/
case management, which are discussed in table 1--that address juvenile
substance abuse issues.
Table 2: Types of Juvenile Substance Abuse Programs:
Drug courts: specialized courts established within and supervised by
juvenile courts to provide intervention programs, such as cognitive
behavioral therapy or family therapy, for substance-abusing juveniles
and their families.
Family therapy: programs using trained therapists to treat juvenile
offenders with substance abuse problems by including families of
juveniles in the treatment, focusing on improving communication and
interactions among family members, as well as improving overall
relationships between juveniles and their families.
Mentoring: programs that establish a relationship between two or more
people over a prolonged period of time, where an older, more
experienced individual provides support and guidance to a juvenile. The
goal of mentoring is for the juvenile to develop positive adult
contact, thereby reducing risk factors, such as exposure to juveniles
who use substances, while increasing positive factors, such as
encouragement for abstaining from substance use.
Source: OJJDP‘s Model Programs Guide.
[End of table]
The Juvenile Justice and Delinquency Prevention Act (JJDPA) established
OJJDP in 1974.[Footnote 15] As the only federal office charged
exclusively with preventing and responding to juvenile delinquency and
with helping states improve their juvenile justice systems, OJJDP
supports its mission through a variety of activities. For example,
OJJDP administers a wide variety of grants to states, territories,
localities, and public and private organizations through formula,
block, and discretionary grant programs; provides training and
technical assistance; produces and distributes publications and other
products containing information about juvenile justice topics; and
funds research and evaluation efforts.[Footnote 16] In fiscal year
2009, the total appropriation for juvenile justice programs was about
$374 million. See appendix III for more detailed information on OJJDP's
enacted appropriations for fiscal years 2007 through 2009.
OJJDP, through its various grant programs, has provided funding to
states and organizations to support juvenile reentry and substance
abuse programs, although the JJDPA does not specifically require OJJDP
to fund them. States generally have the authority to determine how
formula and block grants are allocated and may use these funds to
support a range of program areas, including programs specifically for
reentry or substance abuse. For example, from fiscal years 2007 through
2008, OJJDP reported that states used approximately $7.1 million in
applicable formula and block grant funds for programs that target
reentry and $19 million in formula and block grant funds for programs
that target substance abuse, representing approximately 1.8 percent and
4.5 percent, respectively, of such funding for those years.
Additionally, from fiscal years 2007 through 2009, OJJDP awarded a
total of approximately $33 million in discretionary grants through four
juvenile reentry grant programs and three substance abuse grant
programs. Specifically, in the area of reentry, OJJDP awarded a total
of $25.4 million to 38 grantees under 4 programs, and in the area of
substance abuse, OJJDP awarded a total of $7.6 million to 15 grantees
under 3 programs. See appendix IV for more detail on funding for these
reentry and substance abuse programs.
Experts Cite Evidence from Available Research Indicating That Cognitive
Behavioral Therapy and Family Therapy Programs Are Effective and Cost
Beneficial When Addressing Reentry and Substance Abuse Issues:
Of the five reentry program types we reviewed, reentry experts reported
that there is evidence from available research that cognitive
behavioral therapy reduces recidivism. While experts cited a lack of
evidence demonstrating that wraparound/case management, aftercare, and
vocational/job training were effective in achieving results, such as a
reduction in recidivism, they generally provided positive views on the
potential results of these three types of programs, based on their own
experience or knowledge of them.[Footnote 17] Similarly, of the five
substance abuse program types we reviewed, juvenile substance abuse
experts reported that there is evidence from available research that
cognitive behavioral therapy along with family therapy are effective at
reducing recidivism and show successful results at reducing substance
abuse. However, expert opinions regarding other substance abuse program
types, such as drug courts, mentoring, and wraparound/case management,
were mixed, with experts stating that these program types could be
effective, they were ineffective, or there was not enough evidence to
determine effectiveness. Furthermore, both reentry and substance abuse
experts cited studies indicating that cognitive behavioral therapy and
family therapy programs are cost beneficial; however, the experts cited
limited evidence for determining the costs and benefits of the other
programs we reviewed.
Reentry Experts Cited Evidence That Cognitive Behavioral Therapy Is
Effective at Reducing Recidivism but Concluded That Other Programs Lack
the Evidence Necessary to Determine Their Effectiveness:
Experts Cited Evidence from Available Research Demonstrating That
Cognitive Behavioral Therapy Is Effective at Reducing Recidivism:
Eleven of the 12 experts we interviewed who provided comments based on
their knowledge and experience with cognitive behavioral therapy stated
that evidence from available research shows that these programs can be
effective at reducing recidivism.[Footnote 18] Cognitive behavioral
therapy intervention programs are designed to identify and provide
juveniles with the skills to change thoughts and behaviors that
contribute to their problems. The underlying principle of these
programs is that thoughts affect emotions, which then influence
behaviors. These intervention programs combine two kinds of
psychotherapy--cognitive therapy[Footnote 19] and behavioral therapy.
[Footnote 20] The strategies of cognitive behavioral therapy have been
used to, among other things, prevent the start of a problem behavior--
such as violence and criminal activity--or stop the problem behavior
from continuing. A juvenile offender can receive this type of
intervention program after sentencing, throughout incarceration, or
after returning to the community. For example, a cognitive behavioral
therapy intervention program may provide individual and family services
to treat a juvenile offender who has mental health and substance abuse
issues. The treatment can occur during the juvenile's transition from
incarceration back into the community and help the juvenile lower the
risk of recidivism, connect the family with appropriate community
support, assist the juvenile in abstaining from drugs, and improve the
mental health of the juvenile.
Based on their assessment of the available research, these 11 experts
stated that cognitive behavioral therapy programs have been shown to be
effective. Experts identified two meta-analyses[Footnote 21] of
cognitive behavioral therapy programs that demonstrated effectiveness.
[Footnote 22] One such study concluded that effective cognitive
behavioral therapy programs are characterized by the low proportion of
juveniles who dropped out of the program, as well as the close
monitoring of the quality of the treatment and adequate training for
the providers. In addition, this same study also found that 12 months
after treatment, the likelihood of a juvenile who received cognitive
behavioral therapy not recidivating was about one and a half times
greater than for a juvenile who did not receive the therapy. This study
also reported that the effects of cognitive behavioral therapy were
greater for offenders who had a higher risk of recidivism than those
with a lower risk. Specifically, the best results, in terms of
recidivism reductions, occurred when high-risk offenders received more
intensive treatment that targeted criminal thinking patterns. A second
study also reported that among therapeutic interventions, such as skill
building, cognitive behavioral therapy was most effective at reducing
recidivism.[Footnote 23] The 12th expert stated that the particular
cognitive behavioral therapy intervention program he was using--
aggression replacement training®--had not been evaluated at his
particular program site, so he could not draw conclusions as to its
effectiveness.[Footnote 24]
Reentry Experts Indicated Positive Views of Wraparound/Case Management,
Aftercare, and Vocational/Job Training Programs, but Stated That These
Programs Lacked Evidence to Demonstrate Effectiveness:
Despite having generally positive views on the results of wraparound/
case management, aftercare, and vocational/job training programs based
on their experience or knowledge of these programs, reentry experts
reported a lack of evaluations that show conclusive evidence about the
effectiveness of these programs. Specifically, of the nine experts who
provided comments on wraparound/case management programs, eight offered
positive opinions about these programs. For example, two of these
experts commented that wraparound/case management can be successful at
reducing recidivism, depending on the quality and availability of
services provided to juveniles. However, two of these eight experts
also stated that there was a lack of evaluations demonstrating the
effectiveness of wraparound/case management programs. One of these
experts pointed us to a study on a specific wraparound/case management
intervention program, Wraparound Milwaukee, that showed potentially
promising results related to a reduction in recidivism rates for
juvenile offenders.[Footnote 25] However, another expert cautioned that
initial evaluations of wraparound/case management programs did not
conclusively demonstrate the effectiveness of wraparound/case
management programs.[Footnote 26] Finally, the ninth expert stated that
in her experience, wraparound/case management interventions are not
effective because, for example, juveniles are placed into these
interventions based on the availability of program staff and resources
rather than program services being tailored to the individual needs of
each juvenile.
In addition, 7 of the 15 experts who commented about aftercare programs
opined that aftercare interventions are important reentry programs, in
part, because they link a juvenile with the community and provide
regular contact with a caseworker. However, 6 other experts said there
was inconclusive evidence to determine whether these programs can be
effective in achieving results. Three of these experts based their
opinions on an evaluation of the Intensive Aftercare Program[Footnote
27] that showed inconclusive results about program effectiveness.
[Footnote 28] Specifically, the study found no evidence that the
program had its intended impact of reducing recidivism among juveniles
who were released back into the community under supervision in the
three states that piloted the program. However, the evaluation did find
that the three states that implemented the Intensive Aftercare Program
model successfully incorporated most of its core features, which
prepared juveniles to transition back into the community. For instance,
these states created new Intensive Aftercare Programs--specific
treatment programs that among other things, prepared juveniles for
increased responsibility in the community, facilitated interaction with
the community, and worked with the juveniles' schools and families. The
state programs had a large percentage of juveniles involved in various
treatment services. Despite the inconclusive results of the study, 1
expert credited the aftercare program model with addressing the issue
of juveniles interacting with multiple probation officers throughout
the entire reentry process because aftercare programs, in general,
assign one probation officer to a juvenile as a consistent point of
contact. The remaining 2 of 15 experts opined that aftercare
intervention programs had not been shown to be effective at achieving
desired results because, for example, the treatment a juvenile receives
depends on the services available in the community.
With respect to vocational/job training programs, 10 of the 11 reentry
experts who commented on these programs expressed positive opinions
about the programs' potential outcomes but noted that there had been
limited research conducted to demonstrate their effectiveness.
Specifically, experts noted that vocational/job training programs could
be beneficial if they were applied to older juveniles and if they led
to those juveniles getting jobs. The remaining expert said there is
little evidence to demonstrate the effectiveness of these intervention
programs. For a more detailed description of reentry experts' opinions
about these program types, see appendix V.
Substance Abuse Experts Cited Evidence That Cognitive Behavioral
Therapy and Family Therapy Are Effective at Reducing Recidivism and Can
Help to Reduce Substance Abuse, but Said That Evidence of Effectiveness
Was Lacking for the Other Programs:
Experts Cited Evidence from Available Research Showing Cognitive
Behavioral and Family Therapies Effectively Reduce Recidivism and
Demonstrate Success at Reducing Substance Abuse:
All of the 13 substance abuse experts we interviewed stated that based
on available research, cognitive behavioral therapy effectively reduces
recidivism and has demonstrated success at reducing substance abuse.
Experts cited six studies to support their opinions, two of which were
the same studies cited by reentry experts that demonstrate that
cognitive behavioral therapy is effective at reducing
recidivism.[Footnote 29] Two of the substance abuse experts noted that
few studies have been conducted to determine whether an intervention
program is effective at specifically reducing substance abuse. However,
3 experts also noted that within the last decade, newly emerging
research has shown promising results with respect to cognitive
behavioral therapy program types reducing substance abuse. For example,
these experts pointed us to three studies that report that juveniles
who participated in these programs showed reductions in marijuana
use.[Footnote 30]
Twelve of the 13 substance abuse experts we interviewed who provided
comments based on their knowledge and experience stated that family
therapy programs are effective at reducing recidivism or decreasing
substance use.[Footnote 31] Family therapy uses trained therapists to
treat juvenile offenders with substance abuse problems by including
families of juveniles in the treatment and focusing on improving
communication and interactions among family members and improving
overall relationships between juveniles and their families. This type
of therapy focuses on the family as it is the primary and sometimes
only source for emotional support, moral guidance, and self-esteem for
juveniles. Family habits, such as failing to set clear expectations for
children's behavior, poor monitoring and supervision, and severe and
inconsistent discipline can often lead to juveniles engaging in
delinquency and substance abuse, according to OJJDP. For example,
family drug use often results in adolescent drug use.
Based on their assessment of the available research, these 12 experts
provided positive opinions about the effectiveness of family therapy,
and 7 of these experts cited 9 studies that support their opinions.
[Footnote 32] These studies demonstrated, for example, that
multisystemic therapy--a family therapy intervention program that helps
parents identify strengths and develop natural support systems (e.g.,
extended family, neighbors, friends, and church members)--is an
effective intervention program for reducing recidivism and substance
use because, for example, juveniles who participated in multisystemic
therapy programs engaged in significantly less criminal activity than
did nonparticipants. Specifically, multisystemic therapy participants
had fewer average convictions per year for violent crimes than those
juveniles who did not participate in the program. Additionally,
analyses of drug tests demonstrated significantly higher rates of drug
abstinence for program participants than for nonparticipants. One study
also showed that participants in functional family therapy, another
family therapy intervention program, had 50 percent reductions in
substance use as compared to juveniles who did not participate in the
program.[Footnote 33]
Experts Had Mixed Views on Effectiveness of Drug Courts, Mentoring, and
Wraparound/Case Management Types of Programs:
According to the 10 experts who commented on drug courts, 5 stated that
there is a lack of evidence to determine program effectiveness, while
another expert stated that drug courts are ineffective types of
programs because they expose first-time offenders to more serious drug
users. The remaining 4 experts stated that drug courts can be effective
if, for example, they are combined with other effective intervention
programs, such as multisystemic therapy. Similarly, according to the 8
experts who commented on mentoring, 1 stated that there are too few
evaluations to determine effectiveness, while 4 stated that mentoring
programs alone are ineffective or unsuccessful at achieving desired
results and that mentoring intervention programs are more effective at
preventing at-risk juveniles from engaging in delinquent behavior.
[Footnote 34] However, 3 experts thought mentoring intervention
programs could be effective if the programs adhere to certain factors
that have been evaluated and shown to be effective, such as the mentor
being properly trained.[Footnote 35] Finally, experts also had mixed
views on the effect of wraparound/case management types of programs. Of
the 11 experts who commented on these programs, 7 experts stated that
wraparound/case management is effective or can be effective if, for
example, wraparound/case management is combined with another
intervention program that has been evaluated and has shown to be
effective, such as cognitive behavior therapy. Conversely, 4 experts
either stated that these programs are ineffective because, for example,
the intervention programs lack follow-through as there are no
consequences if a juvenile does not show up for treatment, or there is
not sufficient evidence to determine effectiveness. For a more detailed
description of substance abuse experts' opinions about these program
types, see appendix VI.
Reentry and Substance Abuse Experts Cited Available Research That
Indicates Cognitive Behavioral and Family Therapies Are Cost
Beneficial, but Provided Limited Evidence of the Costs and Benefits of
the Other Program Types:
While program evaluations establish if a program is effective in
producing its intended results or effects--such as a reduction in
recidivism--cost-benefit analyses use program evaluation to determine
if the dollar value of a program's benefits exceeds the costs to
deliver the program. For example, if a program evaluation shows that an
intervention program reduces the number of offenses committed by
juveniles from three to one, a cost-benefit analysis would first
determine a dollar value for each of the offenses. Then, the cost-
benefit analysis would estimate whether the savings of going from three
offenses to one offense is more or less costly than the amount of money
required to deliver the intervention program, as compared to an
alternative program the juvenile would have received. The intervention
may not always be more expensive than the alternative. For example, if
the alternative is incarceration, the intervention program may be less
expensive--meaning that the intervention program can be cost beneficial
even if it does not result in a reduction of offenses. By applying the
same cost-benefit analysis techniques to evaluations of different
program types, decision makers can make comparisons among alternatives
and determine which program types offer the greatest benefits for the
least cost. The results of a cost-benefit analysis are often
represented as a net benefit, meaning total benefits minus total cost.
Of the 26 reentry and substance abuse experts we interviewed, 19
provided information related to the cost benefits of the reentry and
substance abuse program types in our review. These 19 experts
identified five cost-benefit analyses of juvenile justice programs
consisting of four meta-analyses[Footnote 36] and one systematic
review.[Footnote 37] The studies demonstrate that various cognitive
behavioral therapy and family therapy intervention programs are cost
beneficial because they are effective at reducing crime and are
expected to produce more benefits than costs compared to the
alternative. For example, in one study, the authors reviewed several
program interventions that fall into the family therapy program type,
such as multisystemic therapy and multidimensional treatment foster
care.[Footnote 38] The authors analyzed three program evaluations of
multidimensional treatment foster care and found that this intervention
can be expected to reduce crime outcomes by 22 percent. Based on this
reduction in crime, the authors of the study predict that the
intervention provides about $80,000 worth of benefits per participant.
This dollar value reflects the savings per participant that result from
a decrease in criminal activity, including savings to crime victims,
police and sheriff's office costs, and juvenile detention costs, among
others.
The four studies cited by the experts show mixed or inconclusive
results for drug courts, vocational/job training, and mentoring program
types. For example, one study found that juvenile drug courts are cost
beneficial because they are expected to have a net benefit of $4,622
per program participant. The other studies could not determine drug
courts' cost-effectiveness because they either did not include program
evaluations of drug court programs or they found mixed results in the
program evaluations analyzed and therefore could not determine the net
benefits. In addition, two studies found that there are too few
evaluations of vocational/job training or mentoring in juvenile justice
programs to calculate if the benefits of these program types outweigh
the costs. The remaining program types in our review--wraparound/case
management, aftercare, and reentry courts--were not analyzed in these
studies. Table 3 presents a summary of these studies.
Table 3: Examples of Net Benefits of Program Interventions within
Reentry and Substance Abuse Program Types We Reviewed from Four Studies
Cited by Experts:
Examples of the net benefits for interventions within program types:
Studies cited by experts: Evidence-Based Public Policy Options to
Reduce Crime and Criminal Justice Costs: Implications in Washington
State (in 2007 dollars);
Cognitive behavioral therapy[A]: $23,015;
Family therapy[B]: $88,953;
Drug courts: Findings are mixed;
Vocational/job training: Too few recent evaluations;
Mentoring: Too few evaluations to date.
Studies cited by experts: Benefits and Costs of Prevention and Early
Intervention Programs for Youth (in 2003 dollars);
Cognitive behavioral therapy[A]: $8,805;
Family therapy[B]: $24,290;
Drug courts: Not included in study;
Vocational/job training: Not included in study;
Mentoring: $5,073.
Studies cited by experts: Evidence-Based Public Policy Options to
Reduce Future Prison Construction, Criminal Justice Costs, and Crime
Rates (in 2006 dollars);
Cognitive behavioral therapy[A]: $14,660;
Family therapy[B]: $77,798;
Drug courts: $4,622;
Vocational/job training: Too few recent evaluations;
Mentoring: Too few evaluations to date.
Studies cited by experts: The Economic Return on Pennsylvania's
Investment in Research-Based Programs: A Cost-Benefit Assessment of
Delinquency Prevention in Pennsylvania (in 2007 dollars);
Cognitive behavioral therapy[A]: Not included in study;
Family therapy[B]: $79,331;
Drug courts: Not included in study;
Vocational/job training: Not included in study;
Mentoring: Not included in study.
Source: GAO analysis of four studies cited by experts.
Notes: Dollar values indicate the highest net benefits for
interventions for which net benefits were calculated (i.e., for which
both cost and benefit data were available) and that experts categorized
within the program types analyzed in this report.
Net benefits are discounted, or adjusted, to reflect that costs are
incurred when a juvenile initially receives the intervention, but
savings may not result for many years.
Net benefits estimates for (1) Evidence-Based Public Policy Options to
Reduce Crime and Criminal Justice Costs: Implications in Washington
State, (2) Benefits and Costs of Prevention and Early Intervention
Programs for Youth, and (3) Evidence-Based Public Policy Options to
Reduce Future Prison Construction, Criminal Justice Costs, and Crime
Rates are for expected savings per program participant in Washington
state.
The Economic Return on Pennsylvania‘s Investment in Research-Based
Programs: A Cost-Benefit Assessment of Delinquency Prevention in
Pennsylvania shows estimated net benefits per program participant in
Pennsylvania.
In cases where there are sizable differences in methodologies between
studies, the benefits and costs reported cannot be directly compared.
For example, there can be differences in the methodological criteria
for inclusion, the surveys of literature can cover different time
periods or differing program types, or values for benefits and costs
can either come from external literature or can be drawn from the
studies themselves.
[A] Net benefits are for aggression replacement training®, which is an
intervention program categorized as cognitive behavioral therapy by the
experts.
[B] Net benefits are for multidimensional treatment foster care, which
is an intervention program categorized as family therapy by the
experts.
[End of table]
In addition, seven experts also commented on reentry and substance
abuse programs that were not included in the cited studies. For
example, three experts opined that wraparound/case management programs
may eventually be proven to be cost beneficial, based on preliminary
research and evaluations. For example, one expert cited an unpublished
study of a wraparound program pilot project that showed that recidivism
of program participants was low, and that program costs were
approximately 60 percent of the costs of incarcerating juveniles.
Additionally, although experts did not cite cost-benefit analyses of
aftercare program types, four reentry experts stated that such programs
could be cost beneficial if the intervention program being delivered is
effective because the cost of incarceration is so high. Three experts
we interviewed stressed that even though some intervention programs
that have been shown to be effective are expensive, if they reduce
recidivism, they might be cost beneficial because of the high cost of
incarcerating juveniles.
OJJDP Has Mechanisms in Place to Ensure Training and Technical
Assistance Meet Users' Needs, but Regular Feedback on the Model
Programs Guide Would Help OJJDP Better Assess Information Utility:
Consistent with the JJDPA, OJJDP has several efforts under way to
disseminate information about effective juvenile justice programs. Two
of these efforts--NTTAC and the Model Programs Guide--provide
information about effective programs for a range of juvenile justice
issues, including reentry and substance abuse issues. Consistent with
federal guidelines for ensuring the utility of information, OJJDP has
established mechanisms to ensure that the information provided through
its training and technical assistance efforts meets the needs of the
juvenile justice field. However, OJJDP could better ensure the
usefulness of the information it disseminates through the Model
Programs Guide by having a mechanism in place to solicit regular
feedback specifically related to the guide from the juvenile justice
field.
OJJDP Disseminates Information about Effective Juvenile Justice
Programs through Several Efforts:
According to the JJDPA, OJJDP is authorized, but is not required, to
provide information about juvenile justice issues and programs and to
provide training and technical assistance to help the juvenile justice
field implement and replicate such programs.[Footnote 39] In accordance
with this authority and its mission to support states and communities
in their efforts to develop and implement effective juvenile justice
programs, OJJDP disseminates information related to these programs
through a range of efforts, from those designed to meet the needs of
the juvenile justice field as a whole to those that focus on effective
programs in a specific issue area, such as gang prevention or girls'
delinquency. OJJDP distributes the broadest range of information on
juvenile justice topics through the Juvenile Justice Clearinghouse
(Clearinghouse).[Footnote 40] Through its services, the Clearinghouse
offers, among other things, the latest research findings and
statistics, publications on juvenile justice issues and programs,
announcements of funding opportunities, and other resources prepared by
a variety of researchers in juvenile justice. As part of its efforts,
the Clearinghouse responds to requests for information about effective
programs by directing users to OJJDP efforts that develop and
disseminate information about effective programs, such as NTTAC and the
Model Programs Guide. Thus, we focused on NTTAC and the Model Programs
Guide because they provide information about effective programs across
the range of issue areas in which OJJDP is involved, including reentry
and substance abuse programs. OJJDP also disseminates information about
effective programs in specific issue areas through various centers,
such as the National Youth Gang Center and the Underage Drinking
Enforcement Center. For a more detailed discussion of these centers and
other information dissemination efforts that focus on specific issues,
see appendix VII.
NTTAC was established in 1995, in part to provide information about
effective juvenile justice programs--such as programs that address
issues related to reentry and substance abuse--through its training and
technical assistance efforts.[Footnote 41] According to OJJDP, NTTAC
works to promote the use of effective programs in the field through
training and technical assistance programs. Additionally, NTTAC
develops training materials and resources, and customizes the
information included in its curricula in an effort to best meet the
needs of its training and technical assistance recipients. In terms of
its efforts specifically related to program effectiveness, NTTAC
provides training and technical assistance for members of the juvenile
justice field on how to develop and sustain effective programs, and to
help the field understand programs that are effective for various
juvenile populations, such as juveniles with mental health issues or
female offenders.
The Model Programs Guide is an online database that contains summary
information about approximately 200 juvenile justice programs, from
prevention programs to reentry programs. It is designed to help
practitioners and communities identify and implement prevention and
intervention programs that have been evaluated and have been shown to
be effective.[Footnote 42] Programs in the Model Programs Guide may
focus on a range of issues, including delinquency, violence, youth gang
involvement, substance abuse, or academic issues, and can include, but
are not limited to, delinquency prevention, community service, drug
courts, or family therapy. To be included in the Model Programs Guide,
programs are reviewed and rated along several dimensions, including
such factors as whether an evaluation of the program established a
causal association between the treatment and the outcome. Users can
search the Model Programs Guide to find programs that meet their
specific needs. For example, users can look for a program that has been
shown to be effective for juveniles with substance abuse problems who
are first-time offenders, or they can search for a program that has
been shown to be effective for juveniles involved in gang activities
who are reentering the community.
Evaluations and Needs Assessment Help OJJDP Ensure Usefulness of
Information Provided by Training and Technical Assistance Efforts, but
OJJDP Could Better Ensure the Utility of the Model Programs Guide's
Information through Regular Feedback:
In accordance with federal guidelines from OJP and prior GAO work,
OJJDP has mechanisms in place to regularly conduct evaluations and is
currently conducting a needs assessment to ensure the usefulness of the
information provided by its training and technical assistance efforts.
However, OJJDP could better ensure the utility of the information
provided by the Model Programs Guide by establishing a mechanism to
solicit regular feedback from the juvenile justice field. We have
previously reported on the importance of regularly soliciting feedback
to assess user needs and satisfaction.[Footnote 43] Specifically, we
have reported that without feedback, an agency lacks valuable
information from its users and is hindered in its ability to make
improvements to information products that are relevant to users.
Additionally, OJP has published Information Quality Guidelines for its
bureaus, including OJJDP, that highlight the importance of ensuring the
utility of information to be disseminated to the public by continuously
monitoring information needs, among other things.[Footnote 44]
OJJDP has mechanisms in place to regularly assess the usefulness of the
information disseminated by NTTAC to ensure that it meets the needs of
the juvenile justice field. Specifically, OJJDP has established an
evaluation process for NTTAC that is designed to collect the data
necessary to regularly assess the outcome and impact of the training
and technical assistance NTTAC provides to improve the quality of the
information it disseminates. Officials at NTTAC explained that after
every training or technical assistance event, all participants are
given an evaluation form to complete. This form is intended to capture
feedback from participants about the quality of the event, as well as
feedback about the referrals and resources NTTAC provides. Other
evaluation forms are also available on NTTAC's Web site so that users
can provide feedback about NTTAC's services, as well as feedback about
the utility of the Web site. NTTAC then follows up with a sample of
these respondents for more in-depth feedback. According to NTTAC
officials, NTTAC analyzes the data collected from these forms and then
provides them to OJJDP. These officials stated that OJJDP receives this
information on at least a quarterly basis, and uses the information to
make changes to existing curricula and guide future curriculum
development, among other things. In accordance with OJP guidelines and
prior GAO work that highlights the importance of assessing user needs,
these evaluation efforts allow OJJDP to regularly monitor the
usefulness of the information it disseminates in order to develop or
modify its information products.
In addition, OJJDP is conducting a needs assessment to solicit
additional information about the utility of the information it
disseminates through NTTAC's training and technical assistance efforts.
NTTAC is administering the needs assessment and, according to NTTAC
officials, it is designed to determine the training and technical
assistance that would be most helpful to the field. Specifically, the
needs assessment is soliciting feedback from members of the juvenile
justice field about OJJDP's existing efforts. It is also requesting
information regarding issues of interest to the field, any current
training or technical assistance needs, and the specific challenges
that the juvenile justice field is facing in its work. OJJDP officials
stated that they intend to use the results of the needs assessment to
influence the development of training and technical assistance
activities and curricula and the content of national conferences and
workshops.
OJJDP's efforts to conduct evaluations and a needs assessment are
consistent with comments we received from our expert interviews. We
asked all 26 of the juvenile reentry and substance abuse experts we
interviewed to comment on OJJDP's overall efforts to disseminate
information about effective programs to the juvenile justice field.
Thirteen experts provided responses, and while they did not comment
specifically on NTTAC or the Model Programs Guide, they commented on
the utility of the information OJJDP provides in general about
effective programs. Ten of 13 experts had negative opinions of how
useful the information OJJDP disseminates is to members of the juvenile
justice field. For example, 1 expert stated that practitioners often do
not have the time to read research data disseminated by OJJDP, which
prevents them from being able to effectively use it in their work. The
expert added that it would be more useful if OJJDP disseminated
information that was practical and could be applied in the field. In
addition, 2 of these 10 experts suggested that it would be helpful for
OJJDP to obtain feedback from members of the juvenile justice field
about what types of information they would find useful. Thus, OJJDP's
needs assessment should help to address this concern. The remaining 3
experts who commented on OJJDP's information dissemination efforts had
generally positive opinions, stating that the information is useful to
researchers and practitioners.
With respect to the Model Programs Guide, although OJJDP has ad hoc
mechanisms in place to solicit feedback about the information it
provides, it does not solicit this feedback on a regular basis or use
feedback to help ensure that the information disseminated by the Model
Programs Guide is useful to the field, in accordance with federal
guidelines. For example, the Model Programs Guide's Program Director
gives several presentations about the guide each year at juvenile
justice conferences. Officials who operate the Model Programs Guide
stated that following these presentations, they request verbal feedback
from participants. Officials also stated that they regularly receive
unsolicited feedback through the e-mail address that is listed on the
Model Programs Guide's Web site, which they respond to on a case-by-
case basis. Additionally, officials said that they collect feedback
about the Model Programs Guide through an annual e-mail survey that is
sent to the program points of contact listed on the guide to obtain
updated program information. Although these efforts to solicit feedback
about the Model Programs Guide provide OJJDP with some information from
its users, according to OJJDP officials, because the guide does not
have a systematic feedback mechanism, information received cannot be
analyzed on an aggregate level in order to regularly assess how the
juvenile justice field views the utility of the information provided by
the Model Programs Guide. Further, while the annual e-mail survey can
help OJJDP confirm that the program information featured in the Model
Programs Guide is accurate, it does not provide information about
whether the guide is useful to the field as a whole since OJJDP sends
the survey's request for comments about the Model Programs Guide to a
portion of the juvenile justice field whose programs are already
published in the guide, which means that the comments it receives about
the Model Programs Guide do not necessarily reflect the opinions of the
juvenile justice field as a whole.
OJJDP officials agreed that they had not established a systematic
mechanism to obtain feedback from the field regarding the usefulness of
the Model Programs Guide and recognized that such a mechanism would be
useful to have in place. Officials also stated that NTTAC's needs
assessment might be used as a model to build in more consistent
mechanisms for feedback for the office's broader efforts. Because NTTAC
uses evaluations and is taking steps to conduct a needs assessment to
monitor the information needs of the juvenile justice field, OJJDP is
in a better position to help ensure that the information provided by
NTTAC is useful to the juvenile justice field. Recognizing that,
although there is a cost associated with gathering and analyzing
feedback data, establishing a cost-effective mechanism to regularly
solicit feedback about the Model Programs Guide should provide OJJDP
with the information necessary to assess whether the information
provided by this tool is useful to the juvenile justice field.
Finalizing a Program Plan and Developing a Comprehensive Evaluation
Plan Would Help OJJDP Achieve Its Research and Evaluation Goals and Use
Its Limited Resources Effectively:
OJJDP has articulated research and evaluation goals to support its
mission of promoting effective programs and improving the juvenile
justice system. According to OJJDP, one of its three main goals is to
promote improvements in juvenile justice and facilitate the most
effective allocation of resources by conducting research to understand
how the juvenile justice system works in serving children and
families.[Footnote 45] Under the JJDPA, OJJDP is required to publish an
annual program plan that describes planned activities that are under
accounts authorized for research and evaluation activities and that
demonstrate promising initiatives, among other things.[Footnote 46]
This plan is required to be published annually in the Federal Register
for public comment, and is to describe the activities the Administrator
intends to carry out under parts D and E, the appropriations accounts
that in general are available for research and the development of new
programs and initiatives, respectively.[Footnote 47] Specifically,
according to the JJDPA, the Administrator must take into account the
public comments received during the 45-day period and develop and
publish a final plan before December 31 of each fiscal year, describing
the particular activities that the Administrator intends to carry out
under parts D and E. While OJJDP has not published an annual program
plan since 2002, it issued a proposed plan in the Federal Register to
solicit public comment in December 2009.[Footnote 48] OJJDP aims to
publish the final version once public comments are incorporated, in
accordance with the JJDPA's requirements.[Footnote 49] Although the
annual program plan is required to describe the particular activities
the Administrator intends to carry out under parts D and E of the
JJDPA, the proposed program plan includes the office's priorities with
respect to all discretionary funding, including its research and
evaluation efforts. According to the Acting Administrator, this will,
in part, provide complete transparency for all such funding. According
to OJJDP, the development and publication of the annual program plan is
a first step that will lead to a comprehensive evaluation plan as the
annual program plan outlines the agency's overall research and
evaluation goals.
Additionally, the Office of Management and Budget's (OMB) fiscal year
2006 Program Assessment Rating Tool found that juvenile justice
programs would benefit from evaluations of their effectiveness but
noted that such evaluations are difficult and expensive to do.[Footnote
50] As a result, OMB recommended that OJP develop a comprehensive
evaluation plan for juvenile justice programs to obtain better
information about the programs' impacts.[Footnote 51] Although OMB's
recommendation was directed at OJP, OJP and OJJDP officials stated that
because OJJDP is the office within OJP required to conduct juvenile
justice evaluations, it is that office's responsibility to develop this
evaluation plan.
In addition to the above requirement and recommendation, federal
guidelines stipulate the importance of developing a plan to achieve
agency goals. As established in the standard practices for program
management, specific goals of an agency must be conceptualized and
defined in a plan.[Footnote 52] Specifically, this plan is to contain a
description or road map of how the goals and objectives are to be
achieved, including identifying the needed resources and target
milestones or time frames for achieving desired results. We have also
reported on the importance of planning research and evaluation efforts,
in part to ensure that goals are met and resources are used
effectively.[Footnote 53]
OJJDP's Research Coordinator stated that such a road map or plan for
conducting research and evaluation would help better target the
agency's research and evaluation efforts toward achieving their goals.
However, from 2006 to 2009, OJJDP had not developed such a plan,
primarily because of resource constraints. According to this official,
in lieu of having a comprehensive evaluation plan in place to guide its
research and evaluation efforts, the office's efforts are influenced by
a number of factors, including whether Congress directs the agency to
conduct research in a particular area or whether ideas are generated
internally by staff or externally by members of the juvenile justice
field. For example, OJJDP staff responsible for the mentoring area may
generate ideas about how available research funds could be used, for
example, by evaluating a particular type of mentoring program. In
addition, the office may receive recommendations from the Federal
Advisory Council on Juvenile Justice or feedback from others in the
juvenile justice field. While these factors have influenced OJJDP's
research and evaluation efforts, they have not provided a framework for
helping the office meet its research and evaluation goals. Therefore,
once the program plan is finalized, OJJDP intends to develop a
comprehensive evaluation plan in accordance with OMB recommendations to
provide direction and priorities for its research and evaluation
efforts. According to the Acting Administrator, OJJDP intends to use
this comprehensive evaluation plan to better align and target available
discretionary funds toward achieving its research and evaluation goals.
In addition to having a road map to help ensure it meets its goals, it
is important for OJJDP to have a comprehensive plan that lays out how
the office will evaluate its juvenile justice programs. Such a plan
would help to ensure that its limited resources are being used
effectively. This is important because OJJDP does not currently receive
dedicated funding for research and instead must make trade-off
decisions to balance funding to implement programs with funding to
evaluate which programs are effective. The office has not received
dedicated research funding since fiscal year 2005 when it received $10
million for its part D appropriations account--the appropriations
account specifically available for research and evaluation efforts.
[Footnote 54] Without part D funding, OJJDP has relied on funds it has
set aside from its other appropriation accounts to fund its research
and evaluation activities.[Footnote 55] Specifically, as shown in table
4, OJJDP is authorized by the appropriations act to set aside up to 10
percent of certain appropriations accounts for its research and
evaluation efforts. In fiscal year 2008, the last year for which set-
aside funding data are available, the appropriations act authorized
OJJDP to set aside over $23 million for research and evaluation.
However, according to OJJDP, the office set aside approximately $11
million. OJJDP officials stated that this was, in part, because the
JJDPA requires and the agency wants to ensure that sufficient funds are
available to the states for grant programs.[Footnote 56] In addition,
officials explained that some of OJJDP's accounts are transferred to
other program offices, such as the Office of Community Oriented
Policing Services, so research funds are not deducted from those
accounts.[Footnote 57] Of the over $11 million that OJJDP did set
aside, officials reported that the office used nearly $8 million (or 70
percent) for research and evaluation. Table 4 shows the amounts
authorized to be set aside by the annual appropriations act, as well as
the amounts actually set aside and used by OJJDP.
Table 4: OJJDP Funds Authorized and Used for Research and Evaluation,
Fiscal Year 2008:
OJJDP appropriations account: Part A - Concentration of Federal
Efforts[C];
Issue area funded by the account[A]: Cooperation and coordination
between federal agencies involved in juvenile justice issues;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $65,800;
Funds set aside by OJJDP: 0;
Funds used by OJJDP: 0.
OJJDP appropriations account: Part B - State Formula Grants[D];
Issue area funded by the account[A]: Grants to state and local
governments for juvenile delinquency programs;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $7,426,000;
Funds set aside by OJJDP: $5,940,800;
Funds used by OJJDP: v3,170,011.
OJJDP appropriations account: Part D - Research, Evaluation, Technical
Assistance and Training[E];
Issue area funded by the account[A]: Research and evaluation;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: 0;
Funds set aside by OJJDP: 0;
Funds used by OJJDP: 0.
OJJDP appropriations account: Part E - Developing, Testing, and
Demonstrating Promising New Initiatives and Projects[F];
Issue area funded by the account[A]: Development and demonstration of
new projects and initiatives;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: N/A;
Funds set aside by OJJDP: N/A;
Funds used by OJJDP: N/A.
OJJDP appropriations account: Youth Mentoring Grants;
Issue area funded by the account[A]: Mentoring programs;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $7,000,000;
Funds set aside by OJJDP: $907,620;
Funds used by OJJDP: v907,479.
OJJDP appropriations account: Title V - Local Delinquency Prevention
Incentive Grants[G];
Issue area funded by the account[A]: Tribal programs, gang prevention,
alcohol prevention, other local delinquency programs;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $320,000;
Funds set aside by OJJDP: $320,000;
Funds used by OJJDP: $124,623.
OJJDP appropriations account: Secure Our Schools[H];
Issue area funded by the account[A]: Improved security at schools;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $1,504,000;
Funds set aside by OJJDP: 0;
Funds used by OJJDP: 0.
OJJDP appropriations account: Victims of Crime Act - Improving the
Investigation and Prosecution of Child Abuse Programs;
Issue area funded by the account[A]: Training and technical assistance
for professionals involved in investigating, prosecuting, and treating
issues related to child abuse;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $1,692,000;
Funds set aside by OJJDP: 0;
Funds used by OJJDP: 0.
OJJDP appropriations account: Juvenile Accountability Block Grant;
Issue area funded by the account[A]: Juveniles already involved in the
juvenile justice system;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $5,170,000;
Funds set aside by OJJDP: $4,136,000;
Funds used by OJJDP: $3,626,917.
OJJDP appropriations account: Project Childsafe[I];
Issue area funded by the account[A]: Safe firearm handling and storage
practices;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: 0;
Funds set aside by OJJDP: 0;
Funds used by OJJDP: 0.
OJJDP appropriations account: Total;
Maximum funds authorized by the appropriations act for research and
evaluation 10 percent of amount appropriated)[B]: $23,177,800;
Funds set aside by OJJDP: $11,304,420;
Funds used by OJJDP: $7,829,030.
Sources: Consolidated Appropriations Act, 2008, Pub. L. No. 110-161,
121 Stat. 1844, 1911-12 (2007); the JJDPA; and GAO analysis of OJJDP
data.
[A] The appropriation act allows OJJDP to set aside funds for research,
evaluation, and statistics activities designed to benefit the programs
or activities authorized. Consolidated Appropriations Act, 2008, Pub.
L. No. 110-161, 121 Stat. 1844, 1911-12 (2007). As such, the accounts
from which OJJDP sets aside funds for research and evaluation are tied
to specific issue areas, and therefore all of the set-asides must be
used for research, evaluation, and statistics activities designed to
benefit these specific issue areas. For example, the mentoring
appropriation account can only be used for research and evaluation
activities related to mentoring programs.
[B] The appropriations statute for fiscal year 2008 provides that not
more than 10 percent of each amount appropriated may be used for
research, evaluation, and statistics activities that benefit the
programs or activities authorized, and not more than 2 percent for
training and technical assistance. Consolidated Appropriations Act,
2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007). These
provisions applied to appropriation accounts for juvenile justice
programs, but did not apply to amounts appropriated for grants and
projects authorized by sections 261 and 262, part E, of the JJDPA”which
is the account available for developing, testing, and demonstrating
promising new initiatives and projects. See id.
[C] According to OJJDP officials, historically, OJJDP has elected not
to set aside funds from part A as the appropriation supports the
Coordinating Council on Juvenile Justice and Delinquency Prevention”a
council created, in part, to coordinate all federal juvenile offender
programs in cooperation with state and local juvenile justice programs.
[D] Under the formula grants program, funds are to be allocated among
the states on the basis of relative population of people under age 18;
however, the JJDPA also sets base amounts for awards to the states,
depending on the aggregate amount appropriated for the program each
year and taking into account amounts allocated to the states for fiscal
year 2000. 42 U.S.C. § 5632.
[E] Funds for part D have not been appropriated since fiscal year 2005.
[F] The provision in the appropriations act allowing the office to set
aside up to 10 percent of each amount appropriated is not applicable to
grants and projects authorized by sections 261 and 262, part E, of the
JJDPA and is authorized for developing, testing, and demonstrating
promising new initiatives and projects. Consolidated Appropriations
Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12 (2007).
[G] As described in app. III, the appropriation account for Title V
programs includes funds that are carved out for specific purposes.
Pursuant to the appropriations act, these amounts are not available for
set-asides, as they are under sections 261 and 262, part E, of the
JJDPA, whereas funds may be set aside from the remaining amounts
available for Title V delinquency prevention programs. Consolidated
Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844, 1911-12
(2007).
[H] According to OJJDP officials, the office does not set aside funds
from this account because the entire appropriation is transferred to
the Office of Community Oriented Policing Services”a component of DOJ
responsible for helping state and local law enforcement agencies
prevent crime.
[I] In fiscal year 2008, Project Childsafe was not appropriated funds
as part of the appropriations for juvenile justice programs.
[End of table]
Additionally, all of the set-asides from these four accounts must be
used for research, evaluation, and statistics activities designed to
benefit the juvenile justice issues that the accounts specify. For
example, set-aside funds from the youth mentoring grant appropriation
account must be used to research or evaluate mentoring programs. For
other accounts, OJJDP can elect to fund research and evaluation efforts
in a number of different areas. For example, under Juvenile
Accountability Block Grants, OJJDP provides funds to states and units
of local government to strengthen the juvenile justice system. The
states can use these funds for 17 different purpose areas, including
establishing programs to help the successful reentry of juvenile
offenders from state and local custody in the community or for hiring
staff or developing training programs for detention and corrections.
Consequently, there are limits on the amount of funds OJJDP can divert
to research and evaluation and on its discretion over how to use of
some of these funds. In fiscal year 2008, the appropriation act allowed
OJJDP to set aside more than $23 million that could be dedicated to
research and evaluation efforts on numerous eligible programs. Because
OJJDP has to decide how to split set aside funds between supporting
state and local program implementation and program evaluation, in
accordance with federal guidelines, a comprehensive evaluation plan
that in part identifies its funding resources could help OJJDP make
this determination.
According to OJJDP, the office has spent several years considering
developing a plan to provide a road map for how it would meet its
research and evaluation goals. However, officials stated that it has
been difficult to complete a comprehensive evaluation plan to fulfill
OMB's Program Assessment Rating Tool recommendation because they have
not had the resources available--that is, funding and staffing--to
develop the plan. Specifically, because funds have not been
appropriated for part D since fiscal year 2005, OJJDP has not had a
dedicated source of funding that could be used to develop a
comprehensive evaluation plan or to fund the research identified by
such a plan. Additionally, in 2003, OJJDP reorganized its divisions
and, as part of this, dissolved its research division, as well as the
training and information dissemination units. According to OJJDP, the
intention of the former Administrator who implemented this
reorganization was to better integrate these functions throughout the
agency. OJJDP officials stated that those staff who were dedicated to
research and evaluation work were reassigned to other divisions.
Although some of these staff retained the research projects they had at
the time, they also assumed new grant management duties. Also, over the
past 8 fiscal years, OJJDP's overall authorized staffing level has
decreased from 95 to 76. Specifically, those staff dedicated to
research and evaluation decreased from 10 in fiscal year 2002 to 3.5 in
fiscal year 2009. According to OJJDP officials, the reduction in staff
who were dedicated to research and evaluation has strained the staffing
resources that could be used for developing a comprehensive evaluation
plan.
Although OJJDP cited funding and staffing constraints, the Acting
Administrator has made developing a comprehensive evaluation plan a
priority and the office is committed to moving forward with developing
this plan. Following through with its planning efforts will help OJJDP
to meet its research and evaluation goals and better ensure that its
resources are being used effectively as stipulated by federal
guidelines.
Conclusions:
As the juvenile justice field--including states and local communities-
-works to implement programs to lower juvenile recidivism rates and
address juvenile substance abuse, it is important that the field has
information about which programs have been shown to be effective
through program evaluations. The importance of OJJDP's goal to research
and evaluate programs to improve juvenile delinquency underscores the
need for a comprehensive plan to evaluate juvenile justice programs,
one that identifies resources to be committed to its research and
evaluation efforts and outlines the details of how OJJDP will
accomplish its research and evaluation goals. OJJDP efforts to publish
a fiscal year 2010 program plan in December are positive steps in
developing the comprehensive evaluation plan that officials have said
they are committed to developing. Having such a plan will provide OJJDP
with a road map to help ensure that it meets its research and
evaluation goals, uses its limited resources effectively, and
contributes to identifying effective programs to help support states
and localities. With respect to OJJDP's efforts to disseminate
information about effective programs, NTTAC's efforts to regularly
assess the needs for the information it is disseminating through
training and technical assistance are important to helping OJJDP assess
the utility of its efforts and make appropriate improvements. We also
recognize that OJJDP's efforts to conduct a needs assessment could help
provide important information to NTTAC that can be used in conjunction
with its evaluation efforts. Consistent with federal guidelines from
OJP and prior GAO reports, assessing the utility of the information
disseminated through OJJDP's Model Programs Guide is also critical to
ensuring that such information meets the needs of the juvenile justice
field so the field can better implement effective programs. Having a
mechanism in place to regularly solicit feedback from the field about
the usefulness of the Model Programs Guide would better position OJJDP
to assess whether the information it is disseminating through the guide
on effective programs regularly meets the needs of its users.
Recommendation for Executive Action:
To help ensure that OJJDP's Model Programs Guide is regularly meeting
user needs and providing the most helpful information on effective
programs, consistent with federal guidelines, we recommend the
Administrator of OJJDP develop a cost-effective mechanism for regularly
soliciting and incorporating feedback from the juvenile justice field
on the usefulness of the information provided in its Model Programs
Guide.
Agency Comments and Our Evaluation:
We provided a copy of this report to the Attorney General for review
and comment. On December 3, 2009, OJP provided written comments, which
are reprinted in appendix VIII. OJP stated that it agreed with our
recommendation and intends to develop a mechanism for regularly
soliciting and incorporating feedback from the juvenile justice field
on the usefulness of the information provided in its Model Programs
Guide by March 31, 2010. OJP also provided technical comments, which we
incorporated as appropriate.
We are sending copies of this report to the Attorney General, selected
congressional committees, and other interested parties. The report also
is available at no charge on the GAO Web site at [hyperlink,
http://www.gao.gov].
If you or your staff have any questions concerning this report, please
contact Eileen Larence at (202) 512-8777 or larencee@gao.gov. Contact
points for our Offices of Congressional Relations and Public Affairs
may be found on the last page of this report. Key contributors to this
report are listed in appendix IX.
Sincerely yours,
Signed by:
Eileen R. Larence:
Director, Homeland Security and Justice Issues:
[End of section]
Appendix I: Juvenile Justice Reentry and Substance Abuse Experts We
Interviewed:
For the purposes of our review, we selected a total of 26 experts to
interview--13 of whom had expertise related to juvenile reentry
programs, 7 of whom had expertise related to juvenile substance abuse
programs, and 6 of whom had both juvenile reentry and substance abuse
program expertise. See table 5 for a list of these experts.
Table 5: Juvenile Justice Experts We Interviewed about Juvenile Reentry
or Substance Abuse Programs:
Experts on reentry programs:
* David Altschuler, John Hopkins University:
* Troy Armstrong, California State University:
* Richard Dembo, University of South Florida:
* Lynn Ellsworth, Creative Strategy Group and Columbia University:
* Jim Heafner, McLaughlin Youth Center:
* Peter Jones, Temple University:
* Barry Krisberg, National Council on Crime and Delinquency:
* Kevin Minor, Eastern Kentucky University:
* Shelli Rossman, Urban Institute:
* Richard Steele, Juvenile Court Judges' Commission, State of
Pennsylvania:
* Mercer Sullivan, Rutgers University:
* Christy Visher, University of Delaware:
* Richard Wiebush, National Council on Crime and Delinquency:
Experts on substance abuse programs:
* Paul Boxer, Rutgers University:
* Laurie Chassin, Arizona State University:
* Michael Dennis, Chestnut Health Systems:
* Scott W. Henggeler, Medical University of South Carolina:
* Randolph Muck, Substance Abuse and Mental Health Services
Administration[A]:
* Lawrence Murray, Columbia University:
* John Roman, The Urban Institute:
Experts on both reentry programs and substance abuse programs:
* Steve Aos, Washington State Institute for Public Policy:
* Peter Greenwood, Association for the Advancement of Evidence-Based
Practice:
* Nancy Jainchill, National Development and Research Institutes, Inc.:
* Bruce Kamradt, Milwaukee County:
* Doug Kopp, Washington State Juvenile Rehabilitation Administration:
* Mark Lipsey, Vanderbilt University:
Source: GAO.
[A] During our interview with Randolph Muck, other researchers were
present, including H. Westley Clark, Director, Substance Abuse and
Mental Health Services Administration.
[End of table]
[End of section]
Appendix II: Juvenile Justice Experts' Views on Factors That Can Help
Programs Achieve Intended Outcomes:
Of the 26 reentry and substance abuse experts we interviewed, 22
experts--as well as research we reviewed--identified several factors
that can help programs achieve intended outcomes, that is, be
effective.[Footnote 58] The following factors, while not an exhaustive
list of items for programs to consider when implementing juvenile
justice intervention programs, were the most frequently cited by the
experts we interviewed:
* maintaining fidelity to the program;
* selecting, training, and retaining qualified providers;
* conducting needs-based assessments to provide individualized
treatment; and:
* improving juvenile program participation by engaging and motivating
juvenile and family involvement.
While incorporating these factors into reentry or substance abuse
programs does not guarantee that any particular intervention program
will be successful, existing programs that have been evaluated and
found to be effective have generally included these factors in their
designs or implementation.
Maintaining Fidelity to the Program:
According to 17 of the 22 experts, maintaining fidelity to the program
as it was intended to be implemented can help programs achieve their
intended objectives. This factor focuses on ensuring that core program
services or intervention components are delivered as they were
designed, that is, with fidelity. For example, for cognitive behavioral
therapy, this would entail that core intervention components, such as
cognitive and social skills training, were provided exactly as they
were designed to each participant.[Footnote 59] According to one
expert's research, the degree to which an intervention program is
delivered with fidelity is closely related to its effects on
recidivism. Another expert concurred, stating that the more closely
core program services or intervention components are implemented as
they were designed, the more the intervention program will reduce
recidivism rates. For example, one expert emphasized the importance of
maintaining fidelity to the program when replicating the model within a
specific community. In particular, another expert explained that some
therapists tend to substitute their own preferred treatment techniques
instead of using those prescribed by the intervention program, which
can affect how effective a program is at reducing recidivism. This is
particularly true if the intervention program being delivered is a
program that has been evaluated and found to be effective. Furthermore,
one of these experts stated that the specific model chosen has less of
an effect on intended outcomes than the manner in which it is
delivered. As another expert explained it, a weaker intervention--one
that has not been evaluated and proven to be effective--may result in
decreased recidivism rates, for example, if it is implemented as
designed, while an effective intervention program that is implemented
poorly may have little or no effect on intended outcomes.
Selecting, Training, and Retaining Qualified Providers:
According to 19 of the 22 experts, selecting, training, and retaining
qualified providers can help intervention programs achieve intended
outcomes. For example, the quality of the services that cognitive
behavioral therapy delivers depends, in part, on the provider's ability
and whether the provider has been trained on the specific therapies and
components of the intervention program. Three of these experts noted
that if providers are not appropriately trained in the therapy or
intervention being implemented, they may not provide the program as it
was intended, or as one of them noted, may substitute their own
preferred treatment techniques for those prescribed by the intervention
program. As a result, the providers' failure to deliver the
intervention program as it was designed reduces the ability of the
program to achieve intended outcomes. Furthermore, many intervention
programs utilize providers who have certain educational or clinical
experience, such as having a background in mental health or being a
licensed practitioner for the specific therapy being implemented. One
of the 19 experts we interviewed also mentioned the importance of
gaining the support of the juvenile justice community, as well as
agencies' program management, in the selection and training of
providers.
Conducting Needs-Based Assessments to Provide Individualized Treatment:
According to 18 of the 22 experts, by assessing a juvenile's specific
treatment needs, program providers can better design intervention
programs that will be targeted to a juvenile's individual situation.
For example, 4 experts noted that this can help intervention programs
achieve intended outcomes because individualized treatment is more
likely to affect participants' individual outcomes since it takes into
account differences such as age, gender, culture, environment, and
problem severity. One expert noted that individualized treatment
ensures that juveniles do not receive unnecessary treatment, which in
some instances may produce harmful results. According to this expert,
providing juveniles who do not have substance abuse problems some
programs, such as cognitive behavioral therapy, may lead to harmful
results because these juveniles are exposed to others who have more
serious addictions. Four experts also noted that using needs-based
assessments to develop individualized treatment plans can be more cost
beneficial than using standard treatment plans. Specifically, as one of
these experts noted, this is because individualized treatment plans can
help ensure that costly interventions are not provided to juveniles who
do not need extensive services. In addition, 5 experts stated that
conducting a risk-based assessment is important to determining which
juveniles are at higher risk of reoffending in order to focus
programming efforts on them.[Footnote 60] One of these experts cited a
study that shows that targeting specific treatment needs of offenders
is correlated with recidivism outcomes, that is, providing targeted
treatment needs is generally related to lower recidivism.[Footnote 61]
Improving Juvenile Participation by Engaging and Motivating Juvenile
and Family Involvement:
According to 16 of the 22 experts, engaging and motivating juvenile and
family involvement can help to improve a juvenile's program
participation, thereby helping intervention programs to achieve
intended outcomes. For example, 1 expert noted that successful programs
rely on staff members to gain the trust of juvenile offenders. These
programs also recognize that juveniles may experience program fatigue
because they are participating in numerous programs and that motivation
may become an issue. In addition, this expert noted that after being
released into the community, juveniles and their families may not be
motivated to participate in intervention programs. Additionally,
research has shown that encouraging families to participate in the
juvenile's treatment program can reduce family risk factors for
delinquency.[Footnote 62] Eleven experts also mentioned that motivating
juvenile offenders and their families to participate can assist
juveniles in successfully completing an intervention program. Two of
these experts noted that by involving family members in treatment, some
issues that may contribute to juvenile dropout rates, such as a history
of traumatic stress and family members who also abuse substances, can
be addressed within an intervention program.[Footnote 63]
[End of section]
Appendix III: OJJDP's Enacted Appropriations for Fiscal Years 2007
through 2009:
To prevent and respond to juvenile delinquency and help states improve
their juvenile justice systems, the Office of Juvenile Justice and
Delinquency Prevention (OJJDP) administers a wide variety of grants to
states, territories, localities, and public and private organizations
through formula, block, and discretionary grant programs.[Footnote 64]
The office also provides training and technical assistance, produces
and distributes publications and other products containing information
about juvenile justice topics, and funds research and evaluation
efforts. Table 6 shows funding by fiscal year from 2007 through 2009
for the appropriation accounts for juvenile justice programs.
Table 6: Juvenile Justice Enacted Appropriations, Fiscal Years 2007
through 2009 (Dollars in thousands):
Funding by fiscal year:
Line item: Part A - Concentration of Federal Efforts[B];
2007: $703;
2008: $658;
2009[A]: 0.
Line item: Part B - State Formula Grants;
2007: $78,978;
2008: $74,260;
2009[A]: $75,000.
Line item: Part D--Research, Evaluation, Technical Assistance, and
Training;
2007: 0;
2008: 0;
2009[A]: 0.
Line item: Part E - Developing, Testing, and Demonstrating Promising
New Initiatives and Projects;
2007: $104,674;
2008: $93,835;
2009[A]: $82,000.
Youth Mentoring Grants;
2007: $9,872;
2008: $70,000;
2009[A]: $80,000.
Line item: Title V - Local Delinquency Prevention Incentive Grants[C];
2007: $64,171;
2008: $61,100;
2009[A]: $62,000;
Line item: Title V - Local Delinquency Prevention Incentive Grants[C];
Tribal Youth;
2007: ($9,872);
2008: ($24,680);
2009[A]: ($24,680);
Line item: Title V - Local Delinquency Prevention Incentive Grants[C];
Gang Prevention;
2007: (v14,100);
2008: ($18,800);
2009[A]: ($25,000);
Line item: Title V - Local Delinquency Prevention Incentive Grants[C];
Alcohol Prevention;
2007: ($25,000);
2008: ($10,000);
2009[A]: ($25,000).
Line item: Project Childsafe[D];
2007: $987;
2008: 0;
2009[A]: 0.
Line item: Secure Our Schools[E];
2007: $14,808;
2008: $15,040;
2009[A]: 0.
Line item: Victims of Crime Act--Improving Investigation and
Prosecution of Child Abuse Program[F];
2007: $14,808;
2008: $16,920;
2009[A]: $20,000.
Line item: Juvenile Accountability Block Grant Program[G];
2007: $49,361;
2008: $51,700;
2009[A]: $55,000.
Total:
2007: $338,362;
2008: $383,513;
2009[A]: $374,000.
Source: OJJDP funding data.
Note: According to OJJDP, the office‘s overall budget also includes
funds that are transferred from other appropriations accounts; for
example, Juvenile Drug Court programs are administered by OJJDP, but
the funding for these programs is provided by the Bureau of Justice
Assistance”a component of the Department of Justice that in part
provides funding, training, and technical assistance in support of
national, state, and local efforts to prevent crime, drug abuse, and
violence.
[A] Not included in this table is the American Recovery and
Reinvestment Act of 2009, Pub. L. No. 111-5, 123 Stat. 115, 130, which
provided an additional $225 million under the State and Local Law
Enforcement Assistance account that the conference report directed be
used over 2 years for competitive, peer-reviewed grants to units of
state, local, and tribal governments and to national, regional, and
local nonprofit organizations to support critical nurturing and
mentoring of juveniles, among other things. H.R. Rep. No. 111-16, at
419.
[B] According to the Office of Justice Program‘s fiscal year 2010
congressional budget submission, the Concentration of Federal Efforts
program promotes interagency cooperation and coordination among federal
agencies with responsibilities in the area of juvenile justice, as
authorized by part A of the Juvenile Justice and Delinquency Prevention
Act, as amended.
[C] These three amounts are carved out from the Title V appropriations
account in accordance with the annual appropriations acts. The
remaining funds are used for Title V delinquency prevention programs
for juveniles who have had contact with the juvenile justice system, as
well as nonoffenders”juveniles who have not yet had contact with the
juvenile justice system but are in need of preventive services.
[D] Project Childsafe is a nationwide program to promote safe firearms
handling and storage practices through the distribution of safety
education messages and free gun-locking devices.
[E] The Secure Our Schools program provides discretionary grants to
states, local governments, and Indian tribes to provide improved
security, such as placement and use of metal detectors, at schools and
on school grounds.
[F] The Victims of Crime Act”Improving Investigation and Prosecution of
Child Abuse program provides training and technical assistance to
professionals involved in investigating, prosecuting, and treating
issues related to child abuse.
[G] Under the Juvenile Accountability Block Grant Program, OJJDP
provides funds to states and units of local government to strengthen
the juvenile justice system. These funds can be used for 17 different
purpose areas, including establishing programs to help the successful
reentry of juvenile offenders from state and local custody in the
community or for hiring or training programs for detention and
corrections personnel.
[End of table]
[End of section]
Appendix IV: Juvenile Reentry and Substance Abuse Programs OJJDP Funded
through Discretionary Grants:
From fiscal years 2007 through 2009, OJJDP allocated approximately $33
million through discretionary grants to four juvenile reentry grant
programs and three juvenile substance abuse programs. See table 7 for a
description of these reentry and substance abuse programs and the
amounts OJJDP awarded to grantees.
Table 7: OJJDP Juvenile Reentry and Substance Abuse Programs Funded
through Discretionary Grants, Fiscal Years 2007 through 2009:
Reentry programs:
Grant program: High-Risk Youth Offender Reentry and Family
Strengthening Initiatives;
Description: The purpose of this program is to facilitate the
successful transition of juvenile offenders to their families and
communities following confinement in a juvenile residential facility.
In 2007, OJJDP awarded a total of $15 million to 19 grantees to develop
programs and strategies aimed at helping juvenile offenders who pose
significant public safety risks stay out of the juvenile justice
system, for example, by improving their family relationships.
Grant program: Second Chance Juvenile Mentoring Initiative;
Description: The purpose of this program is to support the successful
and safe transition of juvenile offenders from residential facilities
to their communities. In 2009, OJJDP awarded a total of $4.7 million to
11 grantees to develop, implement, and expand mentoring programs and
transitional services that specifically match juveniles with mentors
during the juveniles' confinement to facilitate successful community
reintegration and reduce recidivism.
Grant program: Second Chance Act Youth Offender Reentry Initiative;
Description: The purpose of this program is to assist jurisdictions
characterized by large numbers of juvenile offenders returning to their
communities after release from residential facilities and reduce the
rate of recidivism for these juvenile offenders. In 2009, OJJDP awarded
a total of nearly $3.7 million to five grantees to develop projects
aimed at providing juvenile offenders with services, including
vocational and job placement services, substance abuse treatment,
family strengthening practices, and mentors who work with juvenile
offenders during confinement and after reentry into the community.
Grant program: Tribal Juvenile Detention and Reentry Green
Demonstration Program;
Description: The purpose of this program is to provide federally
recognized tribes with the funds to support program services for tribal
juveniles residing in, or soon to be released from, tribal juvenile
detention centers. In 2009, OJJDP awarded a total of $2 million to
three grantees to develop programs that provide services, such as needs
assessments, vocational training, substance abuse treatment, family
strengthening, and aftercare to help successfully reintegrate the
juveniles into the tribal community.
Substance abuse programs:
Grant program: Juvenile Drug Court/Reclaiming Futures Program[A];
Description: The purpose of this program is to build the capacity of
states, state courts, local courts, units of local government, and
Indian tribal governments to develop and establish juvenile drug courts
adopting the Reclaiming Futures model. This model embodies three
essential elements for juvenile offenders who are abusing substances:
(1) designing a system to coordinate services, (2) involving the
community in creating new opportunities, and (3) improving treatment
services for drug and alcohol abuse. In 2007, OJJDP awarded nearly $1.3
million to three grantees for programs that applied the Reclaiming
Futures model to their juvenile drug courts by helping youth meet
educational goals, identifying juveniles requiring substance abuse
treatment, and effectively engaging youth in treatment by increasing
the number and range of available options.
Grant program: Substance Abuse Prevention and Intervention Programs;
Description: The purpose of these programs is to enhance juvenile
justice, child protection, and delinquency prevention by funding
substance abuse programs or strategies that focus on change at the
individual, family, and community levels. In 2007, OJJDP awarded $4.7
million to nine grantees for programs that aim to prevent or reduce
juvenile delinquency associated with substance abuse problems by
offering a range of services, from community-based interventions, such
as academic assistance and job skills training, to individual treatment
services, such as counseling or therapy. These programs incorporate the
use of prevention and intervention programs that use innovative
approaches or that have been evaluated and have been shown to be
effective, such as wraparound/case management or family therapy, as
well as those that target both substance-abusing juveniles and their
families.
Grant program: Brief Interventions and Referrals to Treatment for
Juvenile Courts and Juvenile Drug Courts[A];
Description: The purpose of this program is to increase the provision
and effectiveness of outpatient treatment for adolescents with
substance abuse disorders by funding juvenile courts and juvenile drug
courts to adopt and expand a combination of two interventions for
adolescents in outpatient treatment settings. These interventions are
(1) motivational enhancement therapy, an intervention program designed
to change behavior by helping clients understand their ambivalence and
achieve lasting changes for a range of problematic behaviors, and (2)
cognitive behavioral therapy. In 2008, OJJDP awarded nearly $1.6
million to three grantees for juvenile drug court programs that adopt
and expand the use of these two therapies.
Source: GAO analysis of OJJDP grant programs and funding data.
[A] According to OJJDP, funding for the Juvenile Drug Court/Reclaiming
Futures Program and Brief Interventions and Referrals to Treatment for
Courts and Juvenile Drug Courts is provided by the Bureau of Justice
Assistance but administered by OJJDP. In addition, OJJDP has funded
family drug court programs and training and technical assistance
efforts for drug courts.
[End of table]
[End of section]
Appendix V: Experts' Opinions of Reentry Programs That Lack Conclusive
Evidence of Effectiveness:
Experts Provided Positive Views about Wraparound/Case Management
Programs, but Could Not Cite Evidence from Available Research
Indicating That They Are Effective at Reducing Recidivism:
Of the 19 reentry experts we interviewed, 9 had specific experience or
knowledge related to wraparound/case management and 8 experts had
positive comments about the effectiveness of this program.[Footnote 65]
In general, wraparound/case management interventions involve making an
array of individualized services and support networks available to
juveniles, rather than requiring them to enroll in treatment programs
that may not address individual needs. According to OJJDP, the goal of
wraparound/case management programs is to keep delinquent juveniles at
home and out of institutions whenever possible. The basic elements that
constitute a wraparound program include, among other things, (1) a
collaborative, community-based interagency team responsible for
designing, implementing, and overseeing the intervention program in a
given jurisdiction; (2) care coordinators who are responsible for
helping juveniles create customized treatment programs, among other
things; (3) juvenile and family teams consisting of family members and
community members who work together to ensure the juvenile's needs are
met at home, at school, and in the community; and (4) a plan of care
developed and updated by all members of the juvenile and family teams
that identifies the juvenile's strengths and weaknesses, targets
specific goals such as improved performance in school, and outlines how
to achieve them.
Of these nine experts, eight provided positive opinions of the results
of wraparound/case management intervention programs. For example, an
expert commented about how in one specific wraparound intervention
program, a single case manager is assigned to a juvenile and is
responsible for determining the services the juvenile is to receive
based on his or her specific needs, instead of enrolling the juvenile
into a treatment program that may not be as beneficial for the
juvenile. Two of these eight experts noted that there was a lack of
evaluations demonstrating effectiveness of these intervention programs
but pointed us to a study[Footnote 66] on a specific wraparound/case
management intervention program, Wraparound Milwaukee, that showed
potentially promising results related to a reduction in recidivism
rates for juvenile offenders.[Footnote 67] However, these experts
stressed that this study alone did not conclusively demonstrate the
effectiveness of wraparound/case management programs. The ninth expert
stated that in her experience, wraparound/case management interventions
are not effective because, for example, a juvenile is placed into this
intervention program based on the availability of program staff and
resources rather than program services being tailored to the individual
needs of the juvenile. Additionally, three of the nine experts
cautioned that these intervention programs are difficult to implement
because of such issues as a lack of quality services or low retention
of juveniles and their families in the intervention being provided.
Specifically, one of these experts noted that the quality of wraparound
services can vary depending on a community's resources. In addition,
another expert emphasized the importance of obtaining buy-in from
diverse service providers who may be used to working on their own, such
as within the welfare, foster care, and public school systems.
Reentry Experts Cited Lack of Conclusive Evidence Based on Available
Research and Mixed Views of the Effectiveness of Aftercare Programs at
Reducing Recidivism:
Of 19 reentry experts we interviewed, 15 had specific experience or
knowledge related to aftercare programs and 6 cited a lack of
conclusive evidence of effectiveness of the program type. Aftercare
intervention programs are intended to prepare juvenile offenders to
return to the community during the reentry process by focusing on the
delivery of services and supervision that start while juveniles are
incarcerated and continue after they return to their communities.
Specifically, aftercare programs collaborate with the community and
marshal its resources to help ensure that juvenile offenders receive
services that address their individual needs, such as treatment for a
substance abuse problem. These intervention programs focus on changing
individual behavior thereby preventing further delinquency. For
example, an aftercare program might incorporate the use of techniques
from an intervention therapy, such as motivational enhancement therapy,
to engage juvenile offenders in treatment and increase their commitment
to change.
Of these 15 experts, 7 offered positive opinions regarding aftercare
intervention programs, based on their own experience or knowledge of
the intervention programs. For example, 1 expert noted that if
aftercare included intervention programs that were proven to be
effective, used assessment tools that identified the individual needs
of the juvenile, and implemented the therapies as they were designed,
then the aftercare intervention program should be effective at reducing
recidivism rates. Although these experts could not provide examples of
studies that had been conducted to show evidence of the effectiveness
of the intervention programs, all 7 of them agreed based on their own
experience or knowledge that aftercare interventions are important
reentry programs, in part, because they link the juvenile with his or
her community and provide regular contact with a caseworker.
Additionally, 3 of these 7 experts stated that aftercare could be
effective depending on the intervention programs used and if they were
delivered as intended. For example, they said that if aftercare
includes intervention programs that have proven to be effective, such
as cognitive behavioral therapy, and identifies the individual needs of
the juvenile, the programs can reduce recidivism. However, 6 of the 15
reentry experts said there was inconclusive evidence to determine
whether these programs can be effective in achieving results. Three of
these experts based their opinions on an evaluation of the Intensive
Aftercare Program that showed inconclusive results about program
effectiveness. Specifically, the study[Footnote 68] found no evidence
that the program had its intended impact of reducing recidivism among
juveniles who were released back into the community under supervision
in the three states that piloted the program.[Footnote 69] However, the
evaluation did find that the three states that implemented the
Intensive Aftercare Program model did successfully incorporate most of
its core features, which prepared juveniles to transition back into the
community. For instance, these states created new Intensive Aftercare
Program-specific treatment programs that among other things, prepared
juveniles for increased responsibility in the community, facilitated
interaction with the community, and worked with the juveniles' schools
and families. The state programs had a large percentage of juveniles
involved in various treatment services. Despite the inconclusive
results of the study, one expert credited aftercare programs with
addressing the issue of juveniles having to deal with different
probation officers throughout the reentry process because, in general,
aftercare programs assign one probation officer to a juvenile as a
consistent point of contact. The evaluation also stated that in order
for the general aftercare model to be effective, it must not only
provide supervision and services after a juvenile's release into the
community, but also focus on preparing a juvenile for release. The
remaining 2 experts opined that aftercare intervention programs had not
been shown to be effective at achieving desired results because, for
example, the treatment a juvenile receives depends on what services are
actually available in the community.
Experts Indicated Potential Positive Outcomes for Vocational or Job
Training Programs:
Of the 19 reentry experts we interviewed, 11 had specific experience or
knowledge related to vocational or job training programs and indicated
potential positive outcomes for these programs. According to OJJDP,
providing juveniles with employment opportunities during reentry is a
common strategy used to try to reduce future criminal behavior.
Vocational or job training intervention programs are intended to
improve juveniles' social and educational functioning by, for example,
increasing earnings, raising self-esteem, and instilling a positive
work ethic. Juveniles can participate in vocational/job training
intervention programs while they are incarcerated and after they return
to the community.
Of the 11 reentry experts, 10 of them had positive comments based on
their experience or knowledge of the program type. Specifically, they
said that vocational/job training programs were potentially beneficial,
in part, if they were applied to older juveniles and if they led to
those juveniles getting jobs. The remaining expert said that there is
little evidence to demonstrate the effectiveness of these intervention
programs.
Experts Cited No Evidence for Effectiveness of Reentry Courts:
Reentry courts are specialized courts that manage the return of
juvenile offenders to the community after they are released from
residential facilities.[Footnote 70] The court manages reentry by using
its authority to direct resources to support the offender's return to
the community and promote positive behavior, among other things. For
example, a reentry court would oversee a juvenile's release into the
community by assigning a judge to meet with the juvenile once a month.
The judge would actively engage the supervising authority, such as a
parole officer, in assessing the juvenile's progress. The judge would
also oversee sanctions for violations as well as rewards, like early
release from parole, for successful achievement of goals, such as
successfully completing a cognitive behavioral therapy intervention
program.
Of the 19 reentry experts we interviewed, 2 provided comments related
to reentry courts and had differing opinions on their effectiveness.
One had a negative impression of the courts, stating that the reentry
courts do not provide more to a juvenile than a probation officer
would. The other commented that he considers concepts encompassed in
reentry courts, such as intensity of supervision, to be a best practice
when it comes to reentry programs. However, neither was aware of any
evaluations of these types of courts.[Footnote 71]
[End of section]
Appendix VI: Experts' Opinions of Substance Abuse Programs That Lack
Conclusive Evidence of Effectiveness:
Experts Views Were Divided on the Effectiveness of Drug Courts, with
Half Indicating That Additional Evidence Is Needed to Determine
Effectiveness:
Of the 13 substance abuse experts we interviewed, 10 had specific
experience or knowledge related to drug courts that resulted in mixed
views of the effectiveness of this program type.[Footnote 72] Juvenile
drug courts are specialized courts established within and supervised by
juvenile courts to provide intervention programs, such as cognitive
behavioral therapy or family therapy, for substance-abusing juveniles
and their families. Juvenile offenders assigned to drug courts are
identified by a juvenile court as having problems with alcohol or
drugs. The drug court judge maintains close oversight of each case
through frequent--often weekly--status hearings with the individuals
involved. The judge both leads and works as a member of a team that can
comprise representatives from juvenile justice, social services, school
and vocational training programs, law enforcement, probation, the
prosecution, and the defense. Together, the team determines how best to
address the substance abuse and related problems of the juvenile and
his or her family.
Specifically, of these 10 experts, 5 experts described drug courts as
having insufficient evidence to determine program effectiveness. For
example, 2 experts mentioned that while some studies show drug courts
reducing substance abuse while juveniles were under court supervision,
the results did not last after juveniles were no longer being
supervised by the courts. Another expert stated that since drug courts
tend to be used for juveniles who have their first or second contact
with the juvenile justice system, they are ineffective at achieving
desired results because they expose these first-time offenders to peers
who have more serious substance abuse addictions and therefore might
influence them to continue to abuse substances. By contrast, the
remaining 4 experts stated that drug courts can be effective at
achieving desired results such as reducing substance abuse if, for
example, the juvenile is sent to a community where there are
intervention programs offered that have been evaluated and have been
shown to be effective, such as cognitive behavioral therapy or family
therapy intervention programs. One expert cited a study to support the
opinion that drug courts supplemented with multisystemic therapy
[Footnote 73] resulted in a decrease in substance abuse by juvenile
offenders.[Footnote 74]
Experts Report Mixed Views on the Effectiveness of Mentoring Programs:
Of the 13 substance abuse experts we interviewed, 8 had specific
experience or knowledge related to mentoring intervention programs that
resulted in mixed views of their effectiveness. Mentoring programs
consist of a relationship between two or more people over a prolonged
period of time, where an older, more experienced individual provides
support and guidance to a juvenile. The goal of mentoring is for the
juvenile to develop positive adult contact, thereby reducing risk
factors, such as exposure to juveniles who use substances, while
increasing positive factors, such as encouragement for abstaining from
substance use. In the substance abuse field, juveniles in need of
sobriety are teamed with older sponsors to serve as positive role
models in helping them become sober.
Of these eight experts, four stated that mentoring programs are
ineffective or unsuccessful at achieving desired results, such as
reducing substance abuse, and that these intervention programs are more
effective at preventing at-risk juveniles from engaging in delinquent
behavior.[Footnote 75] Also, one expert stated that there have been too
few evaluations conducted on mentoring programs to make a general
statement about the relative benefits of mentoring. Conversely, three
experts stated that mentoring programs are effective or can be
effective if, for example, mentors are trained or if mentoring is
combined with another intervention program that has been evaluated and
has been shown to be effective, such as multisystemic therapy.
Experts Provided Positive Views about the Potential Results of
Wraparound/Case Management, but Cited Limited Evidence of Its
Effectiveness:
Of the 13 substance abuse experts we interviewed, 11 had specific
experience or knowledge related to wraparound/case management
intervention programs that resulted in mixed views of the program type.
Of these 11 experts, 7 stated that wraparound/case management is
effective or can be effective if, for example, it is combined with
another intervention program that has been evaluated and has shown to
be effective, such as multisystemic therapy. Although these experts had
limited evidence demonstrating the effectiveness of wraparound/case
management, 2 experts cited two studies that show potentially promising
results related to a reduction in recidivism.[Footnote 76] For example,
one study showed that juveniles in wraparound/case management receive a
number of individualized services, such as mental health treatment for
those juveniles who struggle with emotional issues. However, this study
stressed that it is difficult to evaluate wraparound/case management in
a controlled way since treatment plans are individualized for each
juvenile. The other 4 experts stated that wraparound/case management
intervention programs are ineffective because, for example, the
intervention programs lack follow-through as there are no consequences
if a juvenile does not show up for treatment, or there is not yet
sufficient evidence to determine their effectiveness.
[End of section]
Appendix VII: Additional OJJDP Efforts to Disseminate Information about
Effective Juvenile Justice Programs:
In addition to the National Training and Technical Assistance Center
and the Model Programs Guide, OJJDP disseminates information about
effective programs through a variety of other efforts. Specifically,
the office has developed mechanisms to disseminate information related
to effective programs in specific issue areas, such as youth gang
activity, disproportionate minority contact, and girls' delinquency, as
described in table 8.
Table 8: Additional OJJDP Efforts to Disseminate Information on
Effective Programs:
Effort: National Youth Gang Center;
Description: OJJDP established the National Youth Gang Center in 1995
to assist policymakers, practitioners, and researchers in their efforts
to reduce youth gang involvement and crime by contributing information,
resources, practical tools, and expertise toward the development and
implementation of effective gang prevention, intervention, and
suppression strategies. For example, the center continually identifies
and reviews current gang literature and uses an automated database to
compile gang-related legislation. The center also conducts assessments
of the scope and characteristics of youth gang activity in the United
States; develops and disseminates resources for practitioners and
communities; and provides training and technical assistance in support
of community-based prevention, intervention, and suppression efforts.
For example, the center annually collects and analyzes gang-related
data from law enforcement agencies across the nation.
Effort: Disproportionate Minority Contact Reduction Database;
Description: OJJDP established the Disproportionate Minority Contact
Reduction Database in 2006 to address the issue of a disproportionate
number of minorities coming into contact with the juvenile justice
system. The database is a searchable database that assists
jurisdictions in identifying effective programs that may prove useful
to address disproportionate minority contact. The database provides
guidelines for the juvenile justice field to assist practitioners in
the field in choosing strategies and developing intervention plans.
Effort: Girls Study Group;
Description: OJJDP established the Girls Study Group in 2004 to further
the juvenile justice field's understanding of female juvenile offending
and to identify effective strategies for preventing and reducing female
juvenile involvement in delinquency and violence. The Girls Study Group
conducted a literature search and reviewed 61 girls' delinquency
programs to identify risk and protective factors for girls' delinquency
and to identify effective programs or strategies for preventing and
reducing girls' delinquency. In July 2009, we reported on the findings
of the Girls Study Group and OJJDP's efforts related to addressing the
group's findings. Among other things, we reported that the study group
found no effective programs for preventing or reducing girls'
delinquency, and we recommended that OJJDP develop and implement a plan
for responding to the findings of the Study Group.[A].
Effort: Underage Drinking Enforcement Training Center;
Description: OJJDP established the Underage Drinking Enforcement
Training Center in 1999 to build leadership capacity and increase the
effectiveness of states and local communities in their efforts to
enforce underage drinking laws, prevent underage drinking, and
eliminate the consequences associated with alcohol use by underage
youth. The center works to achieve this goal by providing a wide
variety of practical training and technical assistance services, such
as electronic seminars where national experts, researchers, and
representatives from OJJDP give presentations on a specific areas of
interest, for example, school substance abuse policies or the effects
of alcohol on adolescent brain development.
Source: GAO analysis of OJJDP information.
[A] GAO, Juvenile Justice: Technical Assistance and Better Defined
Evaluation Plans Will Help to Improve Girls‘ Delinquency Programs,
[hyperlink, http://www.gao.gov/products/GAO-09-721R] (Washington, D.C.:
July 24, 2009).
[End of table]
[End of section]
Appendix VIII: Comments from the Department of Justice:
U.S Department of Justice:
Office of Justice Programs:
Office of the Assistant Attorney General:
Washington, D.C. 20531:
December 3, 2009:
Ms. Eileen R. Larence:
Director, Homeland Security and Justice Issues:
Government Accountability Office:
441 G Street, NW:
Washington, DC 20548:
Dear Ms. Larence:
Thank you for the opportunity to comment on the draft Government
Accountability Office (GAO) report entitled "Juvenile Justice: DOJ Is
Enhancing Information on Effective Programs, but Could Better Assess
the Utility of This Information" (GAO-10-125). The Office of Justice
Programs agrees with the Recommendation for Executive Action, which is
restated in bold text below and is followed by our response.
To help ensure that OJJDP's Model Programs Guide is regularly meeting
user needs and providing the most helpful information on effective
programs, consistent with federal guidelines, we recommend the
Administrator of OJJDP develop a cost-effective mechanism for regularly
soliciting and incorporating feedback from the juvenile justice field
on the usefulness of the information provided in its Model Programs
Guide.
By March 31, 2010, the Office of Juvenile Justice and Delinquency
Prevention (OJJDP) will develop and implement an ongoing mechanism to
solicit regular feedback from the juvenile justice field on the
usefulness of the information provided in its Model Programs Guide.
OJJDP's goal will be to implement a process which ensures that its
Model Programs Guide is responsive to the needs of users, accurately
represents the featured programs, and promotes the use of evidence-
based program of practices in juvenile justice.
If you have any questions regarding this response, you or your staff
may contact Maureen A. Henneberg, Director, Office of Audit,
Assessment, and Management, on (202) 616-3282.
Sincerely,
Signed by:
Laurie O. Robinson:
Assistant Attorney General:
cc:
Beth McGarry:
Deputy Assistant Attorney for Operations and Management:
Jeffrey Slowikowski:
Acting Administrator Office of Juvenile Justice and Delinquency
Prevention:
Maureen Henneberg:
Director:
Office of Audit, Assessment, and Management:
Richard P. Theis:
Audit Liaison:
Department of Justice:
[End of section]
Appendix IX: GAO Contact and Staff Acknowledgments:
GAO Contact:
Eileen R. Larence, (202) 512-8777 or larencee@gao.gov:
Acknowledgments:
In addition to the contact named above, Mary Catherine Hult, Assistant
Director; David Alexander; Elizabeth Blair; Ben Bolitzer; Carissa
Bryant; Katherine Davis; Sean DeBlieck; Allyson Goldstein; Rebecca
Guerrero; Jared Hermalin; Dawn Locke; Lisa Shibata; Janet Temko; and
Delia Zee made key contributions to this report.
[End of section]
Footnotes:
[1] A juvenile offender is a youth under the age of 18 who is too young
to be tried as an adult, and who has been found to have committed an
offense that would be criminal if committed by an adult.
[2] Residential facilities house juveniles who are awaiting
adjudication (i.e., resolution of a case by a judge) or have been
adjudicated for an offense, or juveniles who are removed from their
homes because of nondelinquency issues, such as being victims of child
abuse.
[3] This number is based on data from the Census of Juveniles in
Residential Placement (the Census), which takes 1-day population counts
of juveniles placed in residential facilities across states. The number
of juveniles placed in residential facilities ranged from about 108,000
in 1999 to around 93,000 in 2006, based on the most recently conducted
Census. The Census does not capture data on juveniles held in adult
prisons or jails nor does it include facilities exclusively intended
for drug or mental health treatment, even though such facilities may
house juvenile offenders.
[4] This estimate, the most recent available as reported in the Office
of Juvenile Justice and Delinquency Prevention's (OJJDP) 2006 National
Report, is derived from the 2003 Survey of Youth in Residential
Placement, which consists of anonymous interviews with a large,
nationally representative sample of juvenile offenders placed, or
committed, in residential facilities. The universe for this survey is
the population of juvenile offenders ages 10 to 20 in all facilities
included in the 2003 Census of Juveniles in Residential Placement.
There is no national recidivism rate for juveniles since juvenile
justice systems vary across states. Department of Justice, Office of
Juvenile Justice and Delinquency Prevention, Juvenile Offenders and
Victims: 2006 National Report (Washington, D.C., 2006).
[5] The Federal Advisory Committee on Juvenile Justice is an advisory
body established by the Juvenile Justice and Delinquency Prevention Act
of 1974, as amended, to, among other things, advise the President and
Congress on state perspectives regarding the operation of OJJDP and on
federal legislation pertaining to juvenile justice and delinquency
prevention. 42 U.S.C. § 5633(f). The committee consists of appointed
representatives from advisory groups in each of the 50 states, the
District of Columbia, and the 5 U.S. territories. As part of its
mandated responsibilities, the committee submits two annual reports--
one to the President and Congress and the second to the OJJDP
Administrator. These reports are informed by data gathered through an
annual request for information from the individual state and territory
advisory groups. Specifically, 47 of 55 states and territories
(responses were only requested from 4 of the 5 U.S. territories)
responded to the 2008 request.
[6] The most frequently cited issue was mental health assessment and
treatment (38 states). Respondents cited concerns about the lack of
resources and funding available for mental health programs, the minimal
number of treatment services available, the minimal number of trained
staff operating programs, and the limited mental health treatment
options.
[7] GAO defines an expert as a person who is recognized by others who
work in the same subject matter area as having knowledge that is
greater in scope or depth than that of most people working in that
area. The expert's knowledge can come from education, experience, or
both.
[8] Repositories and Web sites we identified include OJJDP's Model
Programs Guide, Substance Abuse and Mental Health Service
Administration's National Registry of Evidence-based Programs and
Practices, University of Colorado's Blueprints for Violence Prevention,
and the National Institute of Justice.
[9] We used program types outlined in the Model Programs Guide because
it describes a range of potential program types that juvenile offenders
may encounter in the juvenile justice system. Specifically, the Model
Programs Guide is a database that in part is designed to provide
information on individual intervention programs within these program
types. The Model Programs Guide categorizes programs by the type of
program being administered (e.g., vocational training or family
therapy), the stage at which the program is administered (e.g.,
prevention, during incarceration, or after return to the community),
and the issue area the program addresses (e.g., gangs or substance
abuse).
[10] According to the Substance Abuse and Mental Health Services
Administration, relapse prevention is a variety of interventions
designed to teach individuals who are trying to maintain behavior
changes how to anticipate and cope with the problem of relapse.
[11] For the purposes of this review, we define an intervention program
as a specific activity, treatment, therapy, or service funded at the
local, state, or federal level that is intended to address the reasons
behind a juvenile's delinquent behavior and prevent the juvenile from
committing increasingly serious offenses.
[12] Department of Justice, Office of Justice Programs, OJP Information
Quality Guidelines (Washington, D.C.: 2007). GAO, U.S. Public
Diplomacy: Actions Needed to Improve Strategic Use and Coordination of
Research, [hyperlink, http://www.gao.gov/products/GAO-07-904]
(Washington, D.C.: July 18, 2007), and Transportation Research:
Opportunities for Improving the Oversight of DOT's Research Programs
and User Satisfaction with Transportation Statistics, [hyperlink,
http://www.gao.gov/products/GAO-06-917] (Washington, D.C.: Aug. 15,
2006).
[13] Program management standards we reviewed are reflected in the
Project Management Institute's The Standard for Program Management ©
(2006). Also, see GAO, South Florida Ecosystem Restoration: A Strategic
Plan and a Process to Resolve Conflicts Are Needed to Keep the Effort
on Track, [hyperlink, http://www.gao.gov/products/GAO/T-RCED-99-170]
(Washington, D.C.: Apr. 29, 1999); South Florida Ecosystem Restoration:
Substantial Progress Made in Developing a Strategic Plan, but Actions
Still Needed, [hyperlink, http://www.gao.gov/products/GAO-01-361]
(Washington, D.C.: Mar. 27, 2001); Great Lakes: A Coordinated Strategic
Plan and Monitoring System Are Needed to Achieve Restoration Goals,
[hyperlink, http://www.gao.gov/products/GAO-03-999T] (Washington, D.C.:
July 16, 2003); and [hyperlink,
http://www.gao.gov/products/GAO-06-917], 7, 11.
[14] The mission of the Center for Substance Abuse Treatment of the
Substance Abuse and Mental Health Services Administration, within the
Department of Health and Human Services, is to promote the quality and
availability of community-based substance abuse treatment services for
individuals and families who need them. The center works with states
and community-based groups to improve and expand existing substance
abuse treatment services.
[15] 42 U.S.C. § 5611.
[16] In general, formula and block grant awards provide funds to states
in accordance with statutory requirements. OJJDP allocates some formula
and block grants to states on the basis of states' juvenile
populations, while others may be awarded at a fixed level to all
states. Discretionary grants provide funds to states, units of local
government, and organizations to administer programs. OJJDP awards
discretionary grants to recipients through an application process or
based on congressional direction. The term "state" means any state of
the United States, the District of Columbia, the Commonwealth of Puerto
Rico, the Virgin Islands, Guam, American Samoa, and the Commonwealth of
the Northern Mariana Islands. 42 U.S.C. § 5603.
[17] In addition to the four program types discussed, we also reviewed
reentry courts. However, because only 2 of 19 experts provided comments
related to reentry courts, we included their comments in app. V.
[18] Twelve out of 19 reentry experts provided specific comments on
cognitive behavioral therapy as a program type. Not all of the reentry
experts provided comments on each program type as their comments were
based on their particular area(s) of expertise.
[19] Cognitive therapy concentrates on thoughts, assumptions, and
beliefs and encourages the recognition and change of thoughts that
contribute to problem behaviors, such as violence, criminal activity,
substance use, or school-based problem behaviors. For instance when a
student has trouble completing a math problem and automatically thinks,
"I'm stupid, I'm not a good student, I can't do math," the student is
encouraged to replace the negative thoughts with more realistic
thoughts, such as "this problem is difficult, I'll ask for help."
[20] Behavioral therapy concentrates on specific actions and
environments that either change or maintain behaviors. For instance,
when people try to stop smoking they are often encouraged to change
their daily habits--for example, instead of having their daily coffee
upon waking, which may trigger the urge to have a cigarette, they are
encouraged to take a morning walk.
[21] A meta-analysis is a study that is a systematic synthesis of
quantitative research results. In other words, a meta-analysis of a
substance abuse or reentry intervention program involves conducting a
literature search for all studies that have been conducted on that
program, evaluating the methodological quality of those studies, and
then systematically combining the findings of the studies using
statistical procedures to calculate the overall effect the program has
on the various outcomes, such as recidivism and substance abuse.
[22] (1) N.A. Landenberger and M. W. Lipsey, "The Positive Effects of
Cognitive-Behavioral Programs for Offenders: A Meta-Analysis of Factors
Associated with Effective Treatment," Journal of Experimental
Criminology, vol. 1 (2005). (2) M.W. Lipsey, "The Primary Factors that
Characterize Effective Interventions with Juvenile Offenders: A Meta-
Analytic Overview," Victims and Offenders, vol. 4 (2009).
[23] Skill-building programs assist juveniles in developing skills to
control their behavior or enhance their ability to participate in
society in a positive manner.
[24] Aggression replacement training® is an intervention program
designed to alter the behavior of chronically aggressive juveniles,
such as those who have continual problems managing their anger. The
goal of the program is to improve skill competence, anger control, and
moral reasoning.
[25] Bruce Kamradt, Stephen Gilbertson, and Nancy Lynn, "Wraparound
Milwaukee: Program Description and Evaluation," Outcomes for Children
and Youth with Emotional and Behavioral Disorders and Their Families,
ed. Michael H. Epstein et al. (Austin: ProEd, 2005), 307.
[26] Wraparound Milwaukee is an intervention program for juveniles with
serious emotional, behavioral, and mental health needs and for their
families. This intervention program attempts to meet the mental health,
substance abuse, social service, and other supportive needs of
juveniles in the Milwaukee community by identifying the specific
personal, community, and professional services each family needs to
care for a juvenile with special needs.
[27] The Intensive Aftercare Program was a program initiative funded by
OJJDP from 1987 until 2000 that was intended to reduce recidivism among
juveniles who are released back into the community under supervision.
[28] Department of Justice, Office of Juvenile Justice and Delinquency
Prevention, Implementation and Outcome Evaluation of the Intensive
Aftercare Program (Washington, D.C.: March 2005).
[29] (1) Steve Aos, Marna Miller, and Elizabeth Drake, Evidence-Based
Public Policy Options to Reduce Future Prison Construction, Criminal
Justice Costs, and Crime Rates (Olympia, Wash.: Washington State
Institute for Public Policy, October 2006). (2) M. Dennis, S. Godley,
G. Diamond, F. Tims, T. Babor, J. Donaldson, H. Liddle, J. Titus, Y.
Kaminer, C. Webb, N. Hamilton, and R. Funk, "The Cannabis Youth
Treatment Study: Main findings from two randomized trials," Journal of
Substance Abuse Treatment, vol. 27 (2004). (3) Landenberger and Lipsey,
"Positive Effects of Cognitive-Behavioral Programs," 451. (4) Lipsey,
"Primary Factors," 124. (5) H. B. Waldron and C. W. Turner, "Evidence-
Based Psychosocial Treatments for Adolescent Substance Abuse," Journal
of Clinical Child & Adolescent Psychology, vol. 37, no. 1 (2008). (6)
H. B. Waldron and Y. Kaminer, "On the Learning Curve: The Emerging
Evidence Supporting Cognitive-Behavioral Therapies for Adolescent
Substance Abuse," Addiction, vol. 99 (2004).
[30] (1) Waldron and Kaminer, "On the Learning Curve: The Emerging
Evidence Supporting Cognitive-Behavioral Therapies for Adolescent
Substance Abuse," 93. (2) Waldron and Turner, "Evidence-Based
Psychosocial Treatments for Adolescent Substance Abuse," 238. (3)
Dennis et al., "The Cannabis Youth Treatment Study: Main findings from
two randomized trials," 197.
[31] Not all 13 of the substance abuse experts provided comments on
each program type as their comments were based on their particular
area(s) of expertise.
[32] (1) Aos et al., Benefits and Costs. (2) Dennis et al., "The
Cannabis Youth Treatment Study: Main findings from two randomized
trials," 197. (3) P. Greenwood, "Prevention and Intervention Programs
for Juvenile Offenders," The Future of Children, vol. 18, no. 2 (2008).
(4) A. Sheidow and S. Henggeler, "Multisystemic Therapy with Substance
Using Adolescents: A Synthesis of the Research," Crossing Frontiers
(Brighton: Pavilion Publishing, 2008). (5) S. Schoenwald, D. Ward, S.
Henggeler, S. Pickrel, and H. Patel, "Multisystemic Therapy Treatment
of Substance Abusing or Dependent Adolescent Offenders: Costs of
Reducing Incarceration, Inpatient, and Residential Placement," Journal
of Child and Family Studies, vol. 5, no. 4 (1996). (6) L. Stambaugh, S.
Mustillo, B. Burns, R. Stephens, B. Baxter, D. Edwards, and M. DeKraai,
"Outcomes From Wraparound and Multisystemic Therapy in a Center for
Mental Health Services System-of-Care Demonstration Site," Journal of
Emotional and Behavioral Disorders, vol. 15, no. 3 (2007). (7) S.
Henggeler, W. Clingempeel, M. Brondino, and S. Pickrel, "Four-Year
Follow-up of Multisystemic Therapy with Substance-Abusing and Substance-
Dependent Juvenile Offenders," Journal of the American Academy of Child
and Adolescent Psychiatry, vol. 41, no. 7 (2002). (8) M. Lipsey and F.
Cullen, "The Effectiveness of Correctional Rehabilitation: A Review of
Systematic Reviews," The Annual Review of Law and Social Science, vol.
3 (2007). (9) Waldron and Turner, "Evidence-Based Psychosocial
Treatments for Adolescent Substance Abuse," 238.
[33] Functional family therapy is a family therapy intervention program
that among other things, attempts to reduce and eliminate problem
behaviors, such as substance abuse, through improving communication
skills, parenting, problem solving, and conflict management while
increasing the family's exposure to community resources to prevent a
reoccurrence of substance use.
[34] At-risk juveniles are youths who, because of certain
characteristics or experiences, are statistically more likely than
other youths to encounter certain problems, such as legal, social,
financial, educational, emotional, and health problems.
[35] Experts we interviewed noted that several factors could help a
program be effective at achieving its goals, such as ensuring that
juveniles receive programs based on their individual needs and ensuring
that staff are well trained to correctly implement a specific program.
See app. II for a summary of these most frequently cited factors.
[36] The four meta-analyses cited by experts were (1) Steve Aos, Marna
Miller, and Elizabeth Drake, Evidence-Based Public Policy Options to
Reduce Future Prison Construction, Criminal Justice Costs, and Crime
Rates (Olympia, Wash.: Washington State Institute for Public Policy,
October 2006); (2) Aos et al., Benefits and Costs; (3) Elizabeth Drake,
Steve Aos, and Marna Miller, "Evidence-Based Public Policy Options to
Reduce Crime and Criminal Justice Costs: Implication in Washington
State," Victims and Offenders, vol. 4 (2009); and (4) Damon Jones,
Brian K. Bumbarger, Mark T. Greenberg, Peter Greenwood, and Sandee
Kyler, The Economic Return on PCCD's Investment in Research-Based
Programs: A Cost-Benefit Assessment of Delinquency Prevention in
Pennsylvania (University Park, Pa.: The Prevention Research Center for
the Promotion of Human Development, Pennsylvania State University,
2008).
[37] C. McDougall, M. Cohen, R. Swaray, and A. Perry, "Benefit-Cost
Analyses of Sentencing," Campbell Systematic Reviews (March 2008). A
systematic review is a summary of the results of available research on
a given topic. The systematic review examined available literature
about sentencing of both adult and juvenile offenders to determine how
many cost-benefit analyses of these programs exist and the quality of
the studies that had been done. Because the systematic review did not
examine the specific program types in this report, we did not include
the review's assessments of the costs and benefits of sentencing
options in this report.
[38] Multidimensional treatment foster care is an intervention that
places juveniles with specially trained foster parents instead of in
residential facilities.
[39] 42 U.S.C. § 5661(e).
[40] The Clearinghouse was established in 1979 to provide individuals
and organizations with access to a comprehensive collection of
information and resources on juvenile justice topics in a centralized
location. The Clearinghouse is responsible for coordinating OJJDP's
mailing list, newsletter, publications and editorial services, library
and reference services, and Web site maintenance. It also coordinates
such efforts as the Juvenile Justice Listserv--a listserv that provides
information on juvenile justice and other youth service-related
publications, funding opportunities, and events; News-at-a-Glance--
OJJDP's bimonthly newsletter; as well as OJJDP's library services,
among other things. The Clearinghouse is a component of the National
Criminal Justice Research Service, which offers justice and substance
abuse information to support research, policy, and program development
by providing access to published reports, research findings, and other
information products.
[41] According to OJJDP, NTTAC is one of many providers that OJJDP uses
for its training and technical assistance efforts. Whereas NTTAC
provides information on a wide range of juvenile justice issues,
OJJDP's other providers deliver specialized training and technical
assistance on specific topics, such as gang prevention through the
National Youth Gang Center.
[42] The Model Programs Guide is one of several online databases that
provide information about programs that have been evaluated and may
have been shown to be effective. In general, these online databases
provide information about specific intervention programs (e.g.,
multisystemic therapy) rather than types of programs (e.g., drug courts
or mentoring). At the time this report was issued, GAO was in the
process of issuing another report that indicated that these online
databases use different criteria for determining whether a program is
effective. See GAO, Program Evaluation: a Variety of Rigorous Methods
Can Help Identify Effective Interventions, [hyperlink,
http://www.gao.gov/products/GAO-10-30] (Washington, D.C.: Nov. 23,
2009).
[43] [hyperlink, http://www.gao.gov/products/GAO-07-904] and
[hyperlink, http://www.gao.gov/products/GAO-06-917].
[44] OJP developed the Information Quality Guidelines in accordance
with Office of Management and Budget (OMB) guidance, which is found in
Office of Management and Budget, Guidelines for Ensuring and Maximizing
the Quality, Objectivity, Utility, and Integrity of Information
Disseminated by Federal Agencies, 67. Fed. Reg. 8,452 (Feb. 22, 2002).
Utility, as defined by OMB, refers to the usefulness of the information
to the intended users.
[45] OJJDP has established three goals that it states constitute the
major elements of a sound policy for juvenile justice and delinquency
prevention. In addition to its goal for research and evaluation,
OJJDP's remaining two goals are (1) to promote delinquency prevention
and early intervention efforts and (2) to foster the use of community-
based programs and services for juvenile offenders.
[46] 42 U.S.C. § 5614(b)(5).
[47] Under part D, OJJDP is authorized to conduct research, evaluation,
and technical assistance, among other things. 42 U.S.C. §§ 5661-62.
Under part E, OJJDP is authorized to make grants for developing,
testing, and demonstrating promising new initiatives and programs. 42
U.S.C. §§5665-66. OJJDP has awarded funds under part E pursuant to
congressional direction for a number of years. For example, the
explanatory statement accompanying the fiscal year 2008 appropriations
act directed that funds appropriated for part E programs be provided to
specific grantees. Explanatory statement accompanying the Consolidated
Appropriations Act, 2008, Pub. L. No. 110-161, 121 Stat. 1844 (2007).
For fiscal year 2009, the appropriations act directed that funds
appropriated under part E be provided to specific grantees, as
specified in the explanatory statement. Omnibus Appropriations Act,
2009, Pub. L. No. 111-8, 123 Stat. 524, 581.
[48] 74 Fed. Reg. 62,821 (Dec. 1, 2009). OJJDP officials did not
provide a reason why the office has not published annual program plans
for fiscal years 2003 through 2009.
[49] See 42 U.S.C. § 5614(b)(5).
[50] The Program Assessment Rating Tools aimed to assess and improve
program performance so that the federal government could achieve better
results. Office of Management and Budget, "Juvenile Justice Programs,"
2006 Performance and Accountability Report (2006), [hyperlink,
http://www.whitehouse.gov/omb/expectmore/detail/10003813.2006.html]
(accessed May 19, 2008).
[51] The other two recommendations included in OMB's review were to (1)
make juvenile justice programs' performance results available to the
public through program publications and the Internet, and (2) include
performance information in budget submissions to better link resources
requested to program performance goals.
[52] Project Management Institute, The Standard for Program Management
© (2006).
[53] [hyperlink, http://www.gao.gov/products/GAO-03-999T], [hyperlink,
http://www.gao.gov/products/GAO-01-361], and [hyperlink,
http://www.gao.gov/products/GAO/T-RCED-99-170].
[54] OJP's Performance Budget shows that OJP has not requested part D
funding for OJJDP since fiscal year 2007.
[55] Appropriations statutes for fiscal years 2007 through 2009
provided that OJJDP may use not more than 10 percent of each amount
appropriated for research, evaluation, and statistics activities that
benefit the programs or activities authorized, and not more than 2
percent of each appropriated amount for training and technical
assistance. See, e.g., Consolidated Appropriations Act, 2008, Pub. L.
No. 110-161, 121 Stat. 1844, 1911-12 (2007). This provision applied to
appropriation accounts under juvenile justice programs, but did not
apply to amounts appropriated for grants and projects authorized by
sections 261 and 262, part E, of the JJDPA--which is the account
available for developing, testing, and demonstrating promising new
initiatives and projects. See, e.g., id.
[56] Under the formula grants program, funds are to be allocated among
the states on the basis of relative population of people under age 18;
however, the JJDPA also sets base amounts for awards to the states,
depending on the aggregate amount appropriated for the program each
year and taking into account amounts allocated to the states for fiscal
year 2000. 42 U.S.C. § 5632.
[57] The Office of Community Oriented Policing Services is a component
of DOJ responsible for helping state and local law enforcement agencies
prevent crime.
[58] Four experts did not provide comments on any factors.
[59] Cognitive skills include training on general thinking and decision-
making skills, such as to stop and think before acting, generate
alternative solutions, evaluate consequences, and make decisions about
appropriate behavior. Social skills include training in pro-social
behaviors, interpreting social cues, and taking other persons' feelings
into account, among others.
[60] Risk factors for reoffending can include antisocial attitudes and
peer associations, self-control and self-management skills, and drug
dependencies.
[61] Four additional experts commented on assessing an individual
juvenile's treatment needs, but did not expand on its significance.
[62] For example, family risk factors for delinquency can include
patterns of high family conflict, having a parent who has been involved
in the criminal justice system, and sibling delinquent behavior.
[63] Five additional experts commented on engaging and motivating
juvenile and family involvement, but did not expand on its
significance.
[64] OJJDP formula and block grant awards provide funds to states in
accordance with statutory requirements. OJJDP allocates some formula
and block grants to states on the basis of states' juvenile
populations, while others may be awarded at a fixed level to all
states. OJJDP discretionary grants provide funds to states, units of
local government, and organizations to administer programs. OJJDP
awards discretionary grants to recipients through an application
process or based on congressional direction. The term "state" means any
state of the United States, the District of Columbia, the Commonwealth
of Puerto Rico, the Virgin Islands, Guam, American Samoa, and the
Commonwealth of the Northern Mariana Islands. 42 U.S.C. § 5603.
[65] Not all of the reentry experts provided comments on each program
type as their comments were based on their particular area(s) of
expertise.
[66] Kamradt et al., "Wraparound Milwaukee: Program Description and
Evaluation," Outcomes for Children and Youth with Emotional and
Behavioral Disorders and Their Families, 307.
[67] Wraparound Milwaukee is an intervention program for juveniles with
serious emotional, behavioral, and mental health needs and for their
families. This intervention program attempts to meet the mental health,
substance abuse, social service, and other supportive needs of
juveniles in the Milwaukee community by identifying the specific
personal, community, and professional services each family needs to
care for a juvenile with special needs.
[68] Wiebush et al., "Implementation and Outcome Evaluation of the
Intensive Aftercare Program."
[69] The Intensive Aftercare Program was implemented in three states:
Nevada, Colorado, and Virginia. In Nevada, evaluators found no
significant or substantive differences in recidivism between youth
involved in the Intensive Aftercare Program and youth assigned to a
control group receiving traditional services--such as education and
individual and group counseling--except that youth in the Intensive
Aftercare Program group were more likely to be charged with violating
parole because of increased monitoring. Evaluators could not draw
conclusions about Colorado and Virginia because of implementation
issues, such as small samples of available participants in those
states.
[70] Residential facilities are correctional facilities that house
juveniles who are awaiting adjudication (i.e., a trial in a juvenile
court) or have been adjudicated for an offense, or juveniles who are
removed from their homes because of nondelinquency issues, such as
being victims of child abuse.
[71] None of the remaining 17 experts we interviewed could cite studies
demonstrating the effectiveness of reentry courts or had opinions on
this program type.
[72] Not all of the substance abuse experts provided comments on each
program type as their comments were based on their particular area(s)
of expertise.
[73] Multisystemic therapy is a type of family therapy that helps
parents identify strengths and develop natural support systems (e.g.,
extended family, neighbors, friends, and church members).
[74] Sheidow and Henggeler, "Multisystemic Therapy with Substance Using
Adolescents: A Synthesis of the Research."
[75] At-risk juveniles are youth who, because of certain
characteristics or experiences, are statistically more likely than
other youths to encounter certain problems, such as legal, social,
financial, educational, emotional, and health problems.
[76] (1) Stambaugh et al., "Outcomes From Wraparound and Multisystemic
Therapy in a Center for Mental Health Services System-of-Care
Demonstration Site." (2) Bruce Kamradt, Stephen A. Gilbertson, and
Nancy Lynn, "Wraparound Milwaukee Program Description and Evaluation,"
Outcomes for Children and Youth with Emotional and Behavioral Disorders
and Their Families: Programs and Evaluation Best Practices, 2nd ed.
(Austin: ProEd, 2005).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO‘s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO‘s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548:
