Adult Drug Courts

Studies Show Courts Reduce Recidivism, but DOJ Could Enhance Future Performance Measure Revision Efforts

Gao ID: GAO-12-53

December 9, 2011

A drug court is a specialized court that targets criminal offenders who have drug addiction and dependency problems. These programs provide offenders with intensive court supervision, mandatory drug testing, substance-abuse treatment, and other social services as an alternative to adjudication or incarceration. As of June 2010, there were over 2,500 drug courts operating nationwide, of which about 1,400 target adult offenders. The Department of Justice's (DOJ) Bureau of Justice Assistance (BJA) administers the Adult Drug Court Discretionary Grant Program, which provides financial and technical assistance to develop and implement adult drug-court programs. DOJ requires grantees that receive funding to provide data that measure their performance. In response to the Fair Sentencing Act of 2010, this report assesses (1) data DOJ collected on the performance of federally funded adult drug courts and to what extent DOJ used these data in making grant- related decisions, and (2) what is known about the effectiveness of drug courts. GAO assessed performance data DOJ collected in fiscal year 2010 and reviewed evaluations of 32 drug- court programs and 11 cost-benefit studies issued from February 2004 through March 2011.

BJA collects an array of data on adult drug-court grantees, such as drug-court completion rates, and during the course of GAO's review, began expanding its use of this performance data to inform grant-related decisions, such as allocating resources and setting program priorities. For example, during September 2011, BJA assessed a sample of adult drug-court grantees' performance across a range of variables, using a new process it calls GrantStat. BJA developed recommendations following this assessment and is determining their feasibility. In addition, in October 2011, BJA finalized revisions to the performance measures on which grantees report. BJA's process of revising its performance measures generally adhered to key practices, such as obtaining stakeholder involvement; however, BJA could improve upon two practices as it continues to assess and revise measures in the future. First, while BJA plans to assess the reliability of the new measures after the first quarter of grantees' reporting, officials have not documented, as suggested by best practices, how it will determine if the measures were successful or whether changes would be needed. Second, should future changes to the measures be warranted, BJA could improve the way it documents its decisions and incorporates feedback from stakeholders, including grantees, by recording key methods and assumptions used to guide its revision efforts. By better adhering to best practices identified by GAO and academic literature, BJA could better ensure that its future revision efforts result in successful and reliable metrics--and that the revision steps it has taken are transparent. In the evaluations that GAO reviewed, drug-court program participation was generally associated with lower recidivism. GAO's analysis of evaluations reporting recidivism data for 32 programs showed that drug-court program participants were generally less likely to be re-arrested than comparison group members drawn from criminal court, with differences in likelihood reported to be statistically significant for 18 of the programs. Cost-benefit analyses showed mixed results. For example: (1) Across studies showing re-arrest differences, the percentages of drug- court program participants re-arrested were lower than for comparison group members by 6 to 26 percentage points. Drug court participants who completed their program had re-arrest rates 12 to 58 percentage points below those of the comparison group. (2) GAO's analysis of evaluations reporting relapse data for eight programs showed that drug-court program participants were less likely than comparison group members to use drugs, based on drug tests or self- reported drug use, although the difference was not always significant. (3) Of the studies assessing drug-court costs and benefits, the net benefit ranged from positive $47,852 to negative $7,108 per participant. GAO recommends that BJA document key methods used to guide future revisions of its performance measures for the adult drug-court program. DOJ concurred with GAO's recommendation.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Director: David C. Maurer Team: Government Accountability Office: Homeland Security and Justice Phone: (202) 512-9627


GAO-12-53, Adult Drug Courts: Studies Show Courts Reduce Recidivism, but DOJ Could Enhance Future Performance Measure Revision Efforts This is the accessible text file for GAO report number GAO-12-53 entitled 'Adult Drug Courts: Studies Show Courts Reduce Recidivism, but DOJ Could Enhance Future Performance Measure Revision Efforts' which was released on December 9, 2011. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States Government Accountability Office: GAO: Report to Congressional Committees: December 2011: Adult Drug Courts: Studies Show Courts Reduce Recidivism, but DOJ Could Enhance Future Performance Measure Revision Efforts: GAO-12-53: GAO Highlights: Highlights of GAO-12-53, a report to congressional committees. Why GAO Did This Study: A drug court is a specialized court that targets criminal offenders who have drug addiction and dependency problems. These programs provide offenders with intensive court supervision, mandatory drug testing, substance-abuse treatment, and other social services as an alternative to adjudication or incarceration. As of June 2010, there were over 2,500 drug courts operating nationwide, of which about 1,400 target adult offenders. The Department of Justice‘s (DOJ) Bureau of Justice Assistance (BJA) administers the Adult Drug Court Discretionary Grant Program, which provides financial and technical assistance to develop and implement adult drug-court programs. DOJ requires grantees that receive funding to provide data that measure their performance. In response to the Fair Sentencing Act of 2010, this report assesses (1) data DOJ collected on the performance of federally funded adult drug courts and to what extent DOJ used these data in making grant-related decisions, and (2) what is known about the effectiveness of drug courts. GAO assessed performance data DOJ collected in fiscal year 2010 and reviewed evaluations of 32 drug- court programs and 11 cost-benefit studies issued from February 2004 through March 2011. What GAO Found: BJA collects an array of data on adult drug-court grantees, such as drug-court completion rates, and during the course of GAO‘s review, began expanding its use of this performance data to inform grant- related decisions, such as allocating resources and setting program priorities. For example, during September 2011, BJA assessed a sample of adult drug-court grantees‘ performance across a range of variables, using a new process it calls GrantStat. BJA developed recommendations following this assessment and is determining their feasibility. In addition, in October 2011, BJA finalized revisions to the performance measures on which grantees report. BJA‘s process of revising its performance measures generally adhered to key practices, such as obtaining stakeholder involvement; however, BJA could improve upon two practices as it continues to assess and revise measures in the future. First, while BJA plans to assess the reliability of the new measures after the first quarter of grantees‘ reporting, officials have not documented, as suggested by best practices, how it will determine if the measures were successful or whether changes would be needed. Second, should future changes to the measures be warranted, BJA could improve the way it documents its decisions and incorporates feedback from stakeholders, including grantees, by recording key methods and assumptions used to guide its revision efforts. By better adhering to best practices identified by GAO and academic literature, BJA could better ensure that its future revision efforts result in successful and reliable metrics-”and that the revision steps it has taken are transparent. In the evaluations that GAO reviewed, drug-court program participation was generally associated with lower recidivism. GAO‘s analysis of evaluations reporting recidivism data for 32 programs showed that drug- court program participants were generally less likely to be re- arrested than comparison group members drawn from criminal court, with differences in likelihood reported to be statistically significant for 18 of the programs. Cost-benefit analyses showed mixed results. For example: * Across studies showing re-arrest differences, the percentages of drug-court program participants re-arrested were lower than for comparison group members by 6 to 26 percentage points. Drug court participants who completed their program had re-arrest rates 12 to 58 percentage points below those of the comparison group. * GAO‘s analysis of evaluations reporting relapse data for eight programs showed that drug-court program participants were less likely than comparison group members to use drugs, based on drug tests or self-reported drug use, although the difference was not always significant. * Of the studies assessing drug-court costs and benefits, the net benefit ranged from positive $47,852 to negative $7,108 per participant. What GAO Recommends: GAO recommends that BJA document key methods used to guide future revisions of its performance measures for the adult drug-court program. DOJ concurred with GAO‘s recommendation. View [hyperlink, http://www.gao.gov/products/GAO-12-53]. For more information, contact David C. Maurer at (202) 512-9627 or maurerd@gao.gov. [End of section] Contents: Letter: Background: BJA Is Expanding Use of Grantee Performance Data but Could Enhance Processes as It Continues to Refine Performance Measures: Drug Courts Were Associated with Lower Recidivism and Relapse Rates for Program Participants Than Criminal Courts: Conclusions: Recommendation for Execution Action: Agency Comments: Appendix I: DOJ Has Fully Implemented Most of Our 2002 Recommendations and Plans to Address the Remaining One: Appendix II: MADCE Is the Most Comprehensive Study of Drug Courts to Date, but Generalizability of Findings May Be Limited: Appendix III: Objectives, Scope, and Methodology: Appendix IV: Overview of Drug Court Program Characteristics: Appendix V: Ten Key Components of a Drug Court--Developed by BJA in Collaboration with The National Association of Drug Court Professionals: Appendix VI: BJA Offers Solicitations in Four Broad Drug-Court Grant Categories--Implementation, Enhancement, Statewide, and Joint: Appendix VII: Key Management Activities Identified for Which Performance Information Can Be Most Useful: Appendix VIII: Comments from the Department of Justice, Bureau of Justice Assistance: Appendix IXGAO Contacts and Staff Acknowledgments: Bibliography: Tables: Table 1: List of the Seven Questions to Which Adult Drug Court Grantees Must Submit Narrative Responses: Table 2: Types of Information BJA Officials Reported Using or Planning to Use When Performing Key Management Activities for the Adult Drug Court Grant Program: Table 3: Differences in Reported Rearrest Rates between Drug Court Program Participants and Comparison Group Members: Table 4: Drug Use Relapse Results of Evaluations GAO Reviewed: Table 5: Cost Conclusions of the 11 Drug Court Program Evaluations in Our Cost-Benefit Review: Table 6: Status of DOJ's Efforts to Address Recommendations We Made in 2002 on DOJ's Collection of Performance Data to Measure the Impact of Federally Funded Drug Court Programs: Table 7: Methodological Quality Categories for Evaluations of a Drug Court Program: Table 8: Five Criteria for Assessing a Cost-Benefit Analysis of a Drug Court Program: Table 9: General Description of Drug Court Program Components: Table 10: Ten Key Components of a Drug Court: Table 11: Adult Drug-Court Discretionary Grant Program--Grant Type and Description: Table 12: Definitions: Key Management Activities Identified for which Performance Information Can Be Most Useful: Figures: Figure 1: Number of Adult Drug Court Discretionary Grant Program Awards Increased 588 Percent from Fiscal Year 2006 Through 2010: Abbreviations: BJA: Bureau of Justice Assistance: DOJ: Department of Justice: GPRA: Government Performance and Results Act: GMS: Grants Management System: MADCE: Multi-Site Adult Drug Court Evaluation: NADCP: National Association of Drug Court Professionals: NIJ: National Institute of Justice: OJP: Office of Justice Programs: PMT: Performance Measurement Tool: SAMHSA: Substance Abuse and Mental Health Services Administration: [End of section] United States Government Accountability Office: Washington, DC 20548: December 9, 2011: The Honorable Patrick Leahy: Chairman: The Honorable Chuck Grassley: Ranking Member: Committee on the Judiciary: United States Senate: The Honorable Lamar Smith: Chairman: The Honorable John Conyers, Jr. Ranking Member: Committee on the Judiciary: House of Representatives: Drug court programs were established beginning in the late 1980s as a local response to increasing numbers of drug-related cases and expanding jail and prison populations nationwide. A drug court is a specialized court-based program that targets criminal offenders who have alcohol and other drug addiction and dependency problems. Drug courts have implemented deferred prosecution or post-adjudication case- processing approaches, or have blended both in their organizational structures. In drug courts using deferred prosecution, defendants waive rights to a trial and enter a treatment program shortly after being charged; those who subsequently fail to complete the treatment program have their charges adjudicated, while those who complete the program are not prosecuted further, or have their charges dismissed. In post- adjudication case processing, defendants are tried and convicted, but either have deferred sentences or suspensions of incarceration until they complete or withdraw from the treatment program. The first approach offers individuals the opportunity to obtain treatment and avoid the possibility of a felony conviction, while the second provides a rehabilitation incentive because treatment progress is factored into the sentencing determination. As of June 2010, there were over 2,500 drug courts operating throughout the United States, of which about 1,400 of these target adult offenders.[Footnote 1] Drug courts are generally based on a comprehensive model involving: * offender assessment; * judicial interaction; * monitoring (e.g., drug testing) and supervision; * graduated sanctions and incentives; and: * treatment services. The Department of Justice (DOJ), through its Office of Justice Programs' (OJP) Bureau of Justice Assistance (BJA), administers the Adult Drug Court Discretionary Grant Program, which provides financial and technical assistance to states, state courts, local courts, units of local government, and Indian tribal governments to develop and implement drug treatment courts.[Footnote 2] The total amount BJA has awarded in grants through the program increased from about $2 million in fiscal year 2006 to $29 million in fiscal year 2010, and the number of grants it has awarded during the same period increased 588 percent. Pursuant to the Government Performance and Results Act (GPRA), DOJ requires applicants that receive funding through the program to provide data that measure the results of their work.[Footnote 3] In April 2002, we reported that DOJ had not sufficiently managed its efforts to collect performance measurement and outcome data from federally funded drug courts.[Footnote 4] We recommended that DOJ take actions to address these concerns, and DOJ agreed with our recommendations and took actions in response. Appendix I provides information on the status of these recommendations. In February 2005, we studied drug courts again and reported that in most of the 27 drug- court program evaluations we reviewed, adult drug-court programs led to recidivism reductions--that is, reductions in new criminal offenses--during periods of time that generally corresponded to the length of the drug court program.[Footnote 5] We also reported that the evidence about the effectiveness of drug court programs in reducing participants' substance-use relapse was limited and mixed. [Footnote 6] This report responds to the Fair Sentencing Act of 2010, which directed GAO to report on drug court programs.[Footnote 7] We briefed your offices on our preliminary results on July 18, 2011. This report includes our final results related to the following questions: (1) What data does DOJ collect on the performance of federally funded adult drug courts, and to what extent has it used these data in making grant related decisions? And (2) What is known about the effectiveness of adult drug courts in reducing recidivism and substance-abuse relapse rates, and what are the costs and benefits of adult drug courts? In addition, appendix I of this report provides information on the extent to which DOJ has addressed the recommendations that we made in 2002 regarding drug court programs. To address the first question, we analyzed: the reporting guidance and requirements that BJA provided in fiscal years 2007 through 2011 to grantees applying for Adult Drug Court Discretionary Grant Program funds;[Footnote 8] BJA-generated grantee performance data reports from October to December 2010; and BJA's guides for managing grants and enforcing grantee compliance that were issued in fiscal year 2011. We selected 2007 as the starting point for our review because BJA implemented its Performance Measurement Tool (PMT)--an online reporting tool that supports BJA grantees' ability to collect, identify, and report performance measurement data activities funded by the award--in fiscal year 2007. We also reviewed our prior reports and internal control standards as well as other academic literature regarding effective performance management practices.[Footnote 9] Further, we interviewed cognizant BJA officials about the extent to which they use grantees' performance data when engaging in these management activities, any challenges faced with ensuring grantee compliance, ongoing efforts to revise program performance metrics, and the extent to which BJA's revisions incorporate best practices we previously identified.[Footnote 10] To address the second question, we conducted a systematic review of evaluations of drug court program effectiveness issued from February 2004 through March 2011 to identify what is known about the effect of drug court programs on the recidivism of and relapse of drug involved individuals as well as the costs and benefits of drug courts.[Footnote 11] We also reviewed DOJ's National Institute of Justice (NIJ)-funded Multi-Site Adult Drug Court Evaluation (MADCE), a 5-year longitudinal process, impact, and cost evaluation of adult drug courts that was issued in June 2011, a summary of which we provide in appendix II.[Footnote 12] We identified the universe of evaluations to include in our review using a three-stage process. First, we identified evaluations by searching databases and Web sites. Second, we selected evaluations of adult drug court programs in the United States that report recidivism, substance use relapse, and/or costs and benefits. Third, we screened the selected studies to determine whether each met criteria for methodological soundness based on generally accepted social science principles or cost-benefit analysis criteria. From more than 260 studies in our initial group, we assessed the findings of 44 studies that met our criteria and reported on the effectiveness of 32 drug court programs or sets of programs. See appendix III for additional details on our scope and methodology. We conducted this performance audit from November 2010 through December 2011 in accordance with generally accepted government- auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our objectives. Background: Drug court programs are designed to address the underlying cause of an offender's behavior--alcohol, drug addiction, and dependency problems. Drug court programs share several general characteristics but vary in their specific policies and procedures because of, among other things, differences in local jurisdictions and criminal justice system practices. In general, judges preside over drug court proceedings, which are called status hearings; monitor offenders' progress with mandatory drug testing; and prescribe sanctions and incentives as appropriate in collaboration with prosecutors, defense attorneys, treatment providers, and others. Drug court programs vary in terms of the substance-abuse treatment required. However, most programs offer a range of treatment options and generally require a minimum of 1 year of participation before an offender completes the program. Practices for determining defendants' eligibility for drug court participation vary across drug court programs, but typically involve screening defendants for their criminal history, current case information, whether they are on probation, and their substance use, which can include the frequency and type of use, prior treatment experiences, and motivation to seek treatment. In 2005, we reported that based on literature reviewed, eligible drug-court program participants ranged from nonviolent offenders charged with drug- related offenses who had substance addictions, to relatively medium risk defendants with fairly extensive criminal histories and who had failed prior substance-abuse-treatment experiences. Appendix IV presents additional information about the general characteristics of drug court programs. As shown in appendix V, BJA, in collaboration with the National Association of Drug Court Professionals (NADCP), identified The Key Components, which describes the basic elements that define drug courts and offers performance benchmarks to guide implementation.[Footnote 13] BJA administers the Adult Drug Court Discretionary Grant Program to provide financial and technical assistance to states, state courts, local courts, units of local government, and Indian tribal governments to develop and implement drug treatment courts.[Footnote 14] Through the Adult Drug Court Discretionary Grant Program, BJA offers funding in four broad drug-court grant categories. See appendix VI for a more detailed discussion on each of the following grant categories. * Implementation grants: Available to jurisdictions that have completed a substantial amount of planning and are ready to implement an adult drug court. * Enhancement grants: Available to jurisdictions with a fully operational (at least 1-year) adult drug court. * Statewide grants: Available for two purposes: (1) To improve, enhance, or expand drug court services statewide through activities such as training and/or technical assistance programs for drug court teams and (2) To financially support drug courts in local or regional jurisdictions that do not currently operate with BJA Adult Drug Court Discretionary Grant Program funding. * Joint grants: In fiscal year 2010, BJA, in collaboration with the Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (SAMHSA), offered a joint grant program for the enhancement of adult drug court services, coordination, and substance-abuse treatment capacity.[Footnote 15] From fiscal years 2006 through 2010, Congress appropriated about $120 million for DOJ's administration of all drug court programs.[Footnote 16] Of this amount, $76 million was used for the Adult Drug Court Discretionary Grant Program, which includes funding provided to grantees through the previously mentioned grant categories. The grant award totals for the Adult Drug Court Discretionary Grant Program increased from $2 million in fiscal year 2006 to $29 million in fiscal year 2010.[Footnote 17] Correspondingly, the number of Adult Drug Court Discretionary Grant Program awards increased from 16 in fiscal year 2006 to 110 in fiscal year 2010--an increase of 588 percent, as shown in figure 1.[Footnote 18] Figure 1: Number of Adult Drug Court Discretionary Grant Program Awards Increased 588 Percent from Fiscal Year 2006 through 2010: [Refer to PDF for image: vertical bar graph] Fiscal year: 2006; Grant awards: 16. Fiscal year: 2007; Grant awards: 19. Fiscal year: 2008; Grant awards: 42. Fiscal year: 2009; Grant awards: 95. Fiscal year: 2010; Grant awards: 110. Source: GAO analysis of Bureau of Justice Assistance data. [End of figure] With regard to drug courts' effectiveness, however, drug courts have been difficult to evaluate because they are so varied, and the resources required to conduct a study that would allow conclusions about the effectiveness of drug courts can be substantial. In particular, while drug courts generally adhere to certain key program components, drug courts can differ in factors including admission criteria, type and duration of drug treatment, degree of judicial monitoring and intervention, and application of sanctions for noncompliance. In February 2005, we studied drug courts and reported that in most of the 27 drug-court program evaluations we reviewed, adult drug court programs led to recidivism reductions during periods of time that generally corresponded to the length of the drug court program.[Footnote 19] Several syntheses of multiple drug court program evaluations, conducted in 2005 and 2006, also concluded that drug courts are associated with reduced recidivism rates, compared to traditional correctional options. However, the studies included in these syntheses often had methodological limitations, such as the lack of equivalent comparison groups and the lack of appropriate statistical controls.[Footnote 20] BJA Is Expanding Use of Grantee Performance Data but Could Enhance Processes as It Continues to Refine Performance Measures: BJA's Ongoing Data Collection Efforts: BJA collects an array of performance data from its adult drug court grantees through its Performance Measurement Tool (PMT) and OJP's Grants Management System (GMS). [Side bar: The Performance Measurement Tool: (PMT) is an online reporting tool that supports BJA grantees' ability to collect, identify, and report performance-measurement data activities funded by the award. OJP's Grants Management System (GMS) is an online system designed to make the grant application process easier and more efficient for grantees. GMS allows grantees to fill out forms and submit application materials online. Source: BJA. End of side bar] Since fiscal year 2008, BJA has required grantees to submit quantitative performance data on a quarterly basis and qualitative performance information on a semi-annual basis. The quantitative information grantees submit to BJA varies depending on the type of grant awarded. For example, information that BJA can calculate based on what Implementation grantees have been required to submit quarterly includes "the percent of drug court participants who exhibit a reduction in substance use during the reporting period," "the percent of program participants who re-offended while in the drug court program," and "the number and percent of drug court graduates." Information that BJA can calculate based on what Enhancement grantees have been required to submit includes "the increase in units of substance-abuse treatment services" and "the percent increase in services provided to participants." In addition to the quarterly reporting of quantitative performance data, all adult drug court grantees must submit progress reports semi-annually. As part of these progress reports, grantees provide qualitative or narrative responses to seven questions. Table 1 shows the seven questions to which grantees must submit narrative responses when completing their semi- annual reports. Table 1: List of the Seven Questions to Which Adult Drug Court Grantees Must Submit Narrative Responses: 1. What goals were accomplished, as they relate to your grant application? 2. What problems/barriers did you encounter, if any, within the reporting period that prevented you from reaching your goals or milestones? 3. Is there any assistance that BJA can provide to address any problems/barriers identified in question number three above? 4. Are you on track to fiscally and programmatically complete your program as outlined in your grant application? 5. What major activities are planned for the next 6 months? 6. What were your accomplishments within this reporting period? 7. Based on your knowledge of the criminal justice field, are there any innovative programs/accomplishments that you would like to share with BJA? Source: BJA. [End of table] Recent Steps to Improve Use of Performance Data: BJA officials told us that grant managers regularly review individual grantees' quarterly performance data and semi-annual progress reports and use this information to determine whether additional training or technical assistance could improve their performance. However, according to BJA officials, resource constraints in the past had prevented staff from fully analyzing the performance data BJA collects from all adult drug court grantees--specifically the analysis of grantees' answers to the seven narrative questions--to identify more effective program approaches and processes to share with the drug court community. In early fiscal year 2011, BJA officials initiated a new process called GrantStat to maximize the use of performance information by leveraging the resources of other BJA divisions, [Footnote 21] BJA's training and technical assistance partners, its contractor, and other key stakeholders.[Footnote 22] GrantStat provides an analytical framework to assess grantee performance data and other relevant information on a semi-annual basis to determine the effectiveness of the grant programs in BJA's portfolio. In September 2011, BJA officials applied GrantStat to a review of the Adult Drug Court Discretionary Grant Program. As part of the process, they collected, reviewed, and analyzed performance data and other relevant information from a cohort of Implementation grantees to determine the overall effectiveness of the adult drug court program and to identify grantees that might need additional technical assistance to improve their outcomes. BJA officials told us that as part of the GrantStat review, they and their technical-assistance provider's staff reviewed selected Implementation grantees' responses to the seven narrative questions and discussed common issues they each identified. For example, BJA identified that a number of grantees had lower-than-expected capacity because drug court stakeholders (e.g., district attorneys) were referring fewer drug-involved defendants to these drug courts. BJA also reported reviewing and discussing other qualitative information, such as the training and technical assistance provider's site-visit reports, to determine grantees' fidelity to the 10 key components.[Footnote 23] BJA officials acknowledged that prior to GrantStat, they had not leveraged the summary data that its technical assistance providers had previously compiled from grantees' narrative responses to these seven questions and indicated that future iterations of GrantStat would continue to include both qualitative and quantitative performance data reviews. Our prior work has emphasized the importance of using performance data to inform key decisions[Footnote 24] and underscored that performance measures can be used to demonstrate the benefits of a program or identify ways to improve it.[Footnote 25] In addition, we also have reported that effective performance measurement systems include steps to use performance information to make decisions. In doing so, program managers can improve their programs and results.[Footnote 26] Recognizing that BJA is working through GrantStat to improve its use of performance data in managing the drug court program, we identified six management activities for which performance information can be most useful to decision makers and benchmarked BJA's practices against them.[Footnote 27] The six activities are: (1) setting program priorities, (2) allocating resources, (3) adopting new program approaches, (4) identifying and sharing with stakeholders more effective program processes and approaches, (5) setting expectations for grantees, and (6) monitoring grantee performance. See appendix VII for the definition of the six management activities. As illustrated in table 2, BJA has current and planned efforts underway across all six activities. Table 2: Types of Information BJA Officials Reported Using or Planning to Use When Performing Key Management Activities for the Adult Drug Court Grant Program: Key management activities GAO identified: 1. Setting program priorities; Current or prior use of performance grantee data: BJA officials reported using a range of information when setting program priorities including NIJ-sponsored research, other drug court evaluations, NADCP's annual problem-solving court census, input from state drug court coordinators, and grantee quantitative performance data to set program priorities; Planned or proposed use of grantee data: BJA officials stated that they have finalized grantees' quantitative performance measures and plan to use GrantStat to identify the most effective grantees and their common characteristics. They told us that through GrantStat they plan to systematically assess performance information to prioritize which types of drug courts BJA should fund in future grant solicitations. Key management activities GAO identified: 2. Allocating resources; Current or prior use of performance grantee data: BJA officials reported regularly using grantees' quantitative performance data when deciding the level of funding to be allocated toward technical assistance annually and the types of technical assistance grantees need to improve their performance; Planned or proposed use of grantee data: BJA officials reported the revised quantitative performance measures will allow BJA to analyze information across all grant categories (e.g., Enhancement, Implementation, and Statewide) to determine how grantees are performing relative to one another and then allocate funding and other resources accordingly. Key management activities GAO identified: 3. Adopting new program approaches or changing work processes; Current or prior use of performance grantee data: BJA officials reported that they use grantees' quantitative data to revise training courses for the program and in drafting the program's grant solicitations; Planned or proposed use of grantee data: BJA officials stated the revised quantitative measures will allow them to conduct more sophisticated analyses through GrantStat. As a result, BJA officials expect to be able to identify not only the grantees that are underperforming, but also the reasons why, and then target the appropriate technical assistance to those in the most need. For example, BJA officials reported that the revised measures will help determine the extent to which grantees have adopted evidence-based practices, such as the seven design features highlighted in the MADCE study.[A]. Key management activities GAO identified: 4. Identifying and sharing with stakeholders more effective program processes and approaches; Current or prior use of performance grantee data: According to BJA officials, because of resource constraints, BJA had been unable to conduct analyses across all grantees' responses to the seven narrative questions in their semi-annual progress reports. As a result, the officials had not used this qualitative data when carrying out this activity. Instead, they reported using information gathered in site visits, desk reviews, and technical assistance assessments, as well as MADCE and other NIJ-sponsored research and drug court evaluations to identify effective drug court processes and procedures. BJA officials stated that information from NIJ-sponsored research and drug court evaluations is disseminated to stakeholders through the BJA-NIJ Research to Practice initiative[B]; Planned or proposed use of grantee data: BJA officials stated that GrantStat will address BJA's difficulties with collectively analyzing grantee performance data on a regular basis by leveraging internal and external resources. They also stated that future GrantStat reviews will allow BJA to identify high-performing grantees and share their success stories with other grantees. Key management activities GAO identified: 5. Setting expectations for grantees; Current or prior use of performance grantee data: BJA officials said that they have been unable to utilize adult drug-court program grantees' quantitative performance data to set grantees' expectations because the measures lacked benchmarks against which to gauge grantee performance; Planned or proposed use of grantee data: BJA's revised quantitative performance measures include benchmarks and other performance indicators allowing BJA to use grantees' data to establish the targets and goals that grantees are expected to achieve. According to BJA officials, some of these performance indicators were established as part of GrantStat's first review of the program and will be communicated in the 2012 Adult Drug Court Discretionary Grant Program solicitation announcement. These indicators are based on grantee cohort averages, individual grantees' 4-year data averages, and adult drug court averages obtained from adult drug court research. Key management activities GAO identified: 6. Monitoring grantee performance; Current or prior use of performance grantee data: According to BJA officials, they have analyzed individual grantees' performance data on a regular basis and provide training and technical assistance as warranted. However, according to BJA officials, because of resource constraints, BJA had been unable to conduct analyses across all grantees' responses to the seven narrative questions in their semi- annual progress reports. As a result, they had not used this qualitative information when carrying out this activity. They also acknowledged that prior to GrantStat, they had not leveraged the summary data that its technical assistance providers prepared based on grantees' responses, despite recognizing its utility; Planned or proposed use of grantee data: BJA officials reported the revised quantitative measures will improve BJA's ability to compare grantees' performance results with established targets and goals to determine the extent to which grantees have met them and, if necessary, to target program resources (e.g., technical assistance) to improve underperforming grantees' performance. BJA officials also told us that GrantStat's review included an assessment of the narrative responses and would continue to include it in the future. Source: GAO analysis of types of information BJA officials reported using when performing management activities. [A] The seven design features include: (1) screening and assessment, (2) target population, (3) procedural and distributive justice behavior, (4) judicial interaction, (5) monitoring, (6) treatment and other services, and (7) relapse prevention and community integration. According to DOJ officials, the seven principles were developed with NIJ on the basis of MADCE, and other rigorous research studies. The language used to describe the seven principles was determined in consultation with BJA's drug court training and technical assistance providers. [B] The Adult Drug Court Research to Practice Initiative is a joint partnership between the National Center for State Courts and the Justice Programs Office of the School of Public Affairs at American University, with the purpose of disseminating information to drug court practitioners about current research relevant to the operations and services of adult drugs. The initiative was co-funded by BJA and NIJ. [End of table] According to BJA officials, after the GrantStat review, they identified trends and developed several potential findings and action items for program design changes. However, BJA officials added that since the action items originated from GrantStat's first review, they are not implementing them immediately. Instead, BJA plans to evaluate the action items over the next 6 months to ensure they are feasible and effective alternatives for improving grantee outcomes. We are encouraged by BJA's recent efforts to regularly analyze grantee performance data to determine whether the program is meeting its goals. We also are encouraged that BJA is using this information to better inform its grant-related management activities, such as setting program priorities, identifying and sharing effective processes and approaches, and setting expectations for grantees. BJA Recently Revised Its Drug Court Performance Measures: During the course of our review, BJA revised its adult drug court program performance measures to improve their reliability and usefulness. BJA provided us with the revised measures on October 28, 2011. According to BJA officials, unclear definitions of some of the previous measures confused grantees about what data elements they were expected to collect. For example, officials told us that grantees may have been confused with how to measure "the number of participants admitted" and "the number of drug court participants." Specifically, BJA officials added that their analysis of several years of data shows that some grantees reported the same number for these two measures, some grantees reported a higher number than were admitted, a few grantees reported a lesser number for the number of participants than the number admitted, and some grantees reported these two measures in each of these three ways over multiple reporting periods. According to BJA officials, such a wide degree of variability made these measures unreliable, and BJA was thus hindered from comparing grantee performance data across grantee cohorts. BJA's performance measure revisions resulted in the following: * All grantees are required to report on "participant level" measures. Examples of these measures include the demographic make-up of their drug court participant populations, the amount of service provided to their participants, and the geographic location of their drug courts; * Enhancement, Joint, and Statewide grantees are required to report on participant level outcomes, such as graduation rates, to ensure consistency with measures BJA collects from Implementation grantees; * Measures previously excluded from the PMT, such as retention rates and outcomes of participants once they complete the drug court program, are now included; * BJA has established two sets of benchmarks as points of reference against which to gauge grantees' performance. The first set of benchmarks requires a comparison of grantees' performance against averages of drug court performance derived from research. The second set of benchmarks requires a comparison of grantees' performance to historical performance data reported to BJA by adult drug court grantees; and: * BJA revised the descriptions and the definitions of the measures to help ensure their clarity. To revise the performance measures, BJA officials consulted with technical assistance providers and a drug court researcher to discuss possible improvements to the performance measures, reviewed drug court literature, and reviewed and analyzed BJA grantees' clarification and information requests to identify the most common problems adult drug court grantees historically experienced submitting performance information to BJA.[Footnote 28] In addition, BJA obtained comments on the proposed measures from BJA staff and other DOJ stakeholders, as well as Enhancement, Implementation, Joint, and Statewide grantees. [Footnote 29] BJA officials also invited all current grantees to participate in four teleconferences to obtain their feedback on the feasibility of collecting and reporting the new measures and their suggestions to improve the clarity of the measures' definitions and descriptions. BJA officials finalized the new measures in October 2011 and plan to closely monitor grantees' performance data submissions to ensure the reliability and usefulness of the measures and then revise as necessary after the first reporting period. BJA officials also stated that they expected to review the measures' overall reliability and validity after the first reporting period--October 1, 2011, through December 30, 2011. BJA officials reported that the revised measures will strengthen the reliability and improve the usefulness of grantee performance data in making grant-related decisions. For example, BJA officials stated that reliable and useful data would help them to identify the most effective grantees and common characteristics these courts share to inform the types of drug courts the officials choose to fund in future grant solicitations. BJA officials also reported that as a result of the revision, they expect to be able to conduct more sophisticated analyses using GrantStat that are needed to inform grant-related decisions. For example, BJA officials told us that implementing benchmarks and participant level measures will enable the agency to compare similar drug courts (e.g., large-urban jurisdictions of similar size, demographic make-up, and geographic context) to one another and across jurisdictions, thereby improving BJA's understanding of grantees' impact on the populations they serve. BJA Could Enhance Two Key Practices as It Continues to Review and Revise Its Adult Drug Court Performance Measures: BJA's process to revise its performance measures generally adhered to some of the key practices that we have identified as important to ensuring that measures are relevant and useful to decision-making. These key practices included obtaining stakeholder involvement [Footnote 30] and ensuring that the measures have certain key attributes, such as clarity.[Footnote 31] The key practices also describe the value of testing the measures to ensure that they are credible, reliable and valid[Footnote 32] and documenting key steps throughout the revision process.[Footnote 33] However, BJA could take actions to improve its efforts in these two areas. For instance, BJA officials told us that after the grantees' first reporting period concludes, they plan to assess the data that grantees submitted to ensure that the measures produce reliable and useful data over at least the first quarter of fiscal year 2012. They stated that if necessary, at that point they will then further revise the measures. Nevertheless, BJA officials have not documented how they will determine if the measures were successful or whether changes would be needed. In addition, BJA officials did not record key methods and assumptions used to guide their revision efforts, such as the feedback stakeholders provided and BJA's disposition of these comments. For example, BJA officials provided a document generally showing the original performance measure; whether it was removed, revised or replaced; and BJA's justification for the action, but this document did not demonstrate how BJA had incorporated the stakeholder feedback it considered when making its decisions. The document also did not include a link to a new performance measure in instances where an older one was being replaced. Further, BJA's justification did not include the rationale for the changes it made to 22 of the 51 performance measures. According to BJA officials, they did not document their decisions in this way because of the rapid nature of the revision process and limited staff resources. They also told us that maintaining such documentation and providing it to stakeholders held little value. Our previous work has shown the importance of documentation to the successful development of effective performance measures.[Footnote 34] In the past, we have reported that revising performance measures involves a number of aspects needing to be carefully planned and carried out and that by documenting the steps undertaken in developing and implementing the revised measures, agencies can be better assured their revisions result in effective performance measures.[Footnote 35] In addition, academic literature on the best practices for developing effective performance measures states that agencies should develop products to document and guide their revision efforts. These products, among other things, can include plans for ensuring the quality and integrity of the data for full-scale implementation of the measures. [Footnote 36] Further, Standards for Internal Control in the Federal Government call for clear documentation of significant events, which can include assumptions and methods surrounding key decisions, and this documentation should be readily available for examination. [Footnote 37] As BJA moves forward in assessing the revised measures and implementing additional changes, if it deems necessary, BJA could better ensure that its efforts result in successful and reliable metrics and are transparent by documenting key methods used to guide revision efforts and an assessment of its measures. This would also help bolster the integrity of its decisions. Drug Courts Were Associated with Lower Recidivism and Relapse Rates for Program Participants Than Criminal Courts: In the evaluations we reviewed, adult drug-court program participation was generally associated with lower recidivism. Our analysis of evaluations reporting recidivism data for 32 programs showed that drug court program participants were generally less likely to be re- arrested than comparison group members drawn from the criminal court system, although the differences in likelihood were reported to be statistically significant in 18 programs.[Footnote 38] Across studies showing re-arrest differences, the percentages of drug court program participants rearrested were lower than for comparison group members by 6 to 26 percentage points. One program did not show a lower re- arrest rate for all drug-court program participants relative to the comparison group within 3 years of entry into the program, although that study did show a lower re-arrest rate for drug court participants who had completed the program than for members of the comparison group. In general, the evaluations we reviewed found larger differences in re-arrest rates between drug-court program completers and members of the comparison group than between all drug-court program participants and the comparison group members. The rearrest rates for program completers ranged from 12 to 58 percentage points below those of the comparison group.[Footnote 39] The completion rates reported in the evaluations we reviewed ranged from 15 percent to 89 percent. Included among the evaluations we reviewed was the MADCE, a 5-year longitudinal process, impact, and cost evaluation of adult drug courts. The MADCE reported a re-arrest rate for drug court participants that was 10 percentage points below that of the comparison group; specifically, 52 percent of drug court participants were re-arrested after the initiation of the drug court program, while 62 percent of the comparison group members were re-arrested.[Footnote 40] However, the 10 percentage point difference between these rearrest rates for the samples of drug court participants and comparison group members was not statistically significant. The MADCE study also reported that drug court participants were significantly less likely than the comparison group to self-report having committed crimes when they were interviewed 18 months after the baseline (40 percent vs. 53 percent), and drug court participants who did report committing crimes committed fewer than comparison group members. We assigned a numerical rating to each evaluation to reflect the quality of its design and the rigor of the analyses conducted. Our methodology for rating the evaluation studies is detailed in appendix III. After assigning the rating, we grouped the studies into two tiers. Tier 1 studies were the most carefully designed and incorporated substantial statistical rigor in their analyses. Tier 2 studies, while still meeting our basic criteria for methodological soundness, were relatively less rigorous in their design and analyses. Both tier 1 and tier 2 studies reported differences between drug court participants and comparison group members and both sets of studies found that some but not all differences were statistically significant.[Footnote 41] Table 3 shows whether a difference in recidivism rates was reported for each program--expressed as the difference in the rate of re-arrest between all drug court program participants and the comparison group. In some cases the difference in recidivism was reported as something other than a difference in the re-arrest rate, such as a difference in the number of arrests or the relative odds of an arrest. In those cases, table 3 notes that a difference was reported, but does not include the difference in re-arrest rates. For example, the evaluation of the Queens Misdemeanor Treatment Court reported that the re-arrest rate for program participants was 14 percentage points lower than the re-arrest rate of comparison group members up to 2 years after participants entered into the program, and 10 percentage points lower at 3 or more years after entry. Similarly, the evaluation of the Hillsborough County Adult Drug Court reported a statistically significant difference in the relative odds of an arrest after drug court program enrollment but did not report the difference in rearrest rates, therefore table 3 indicates a statistically significant reduction in rearrest rates but does not show the difference in rates. Table 3: Differences in Reported Rearrest Rates between Drug Court Program Participants and Comparison Group Members: Tier 1 Evaluations: Drug court program (state): Breaking the Cycle Program (Florida); Reduction reported? Yes; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -10%[B]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Hillsborough County Adult Drug Court (Florida); Reduction reported? Yes[C]; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Baltimore City Drug Treatment Court (Maryland); Reduction reported? Yes; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: -9%; -8%[D]. Drug court program (state): Queens Misdemeanor Treatment Court (New York); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -14%*; 3 or more years after entry: -10%*. Drug court program (state): Multnomah County Drug Court (Oregon); Reduction reported? Yes[E]; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): MADCE (Multiple States); Reduction reported? Yes; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -10%; 3 or more years after entry: [Empty]. Drug court program (state): Breaking the Cycle Program (Washington); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -10%*[F]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Tier 2 Evaluations: Drug court program (state): Multiple Drug Courts (California); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: -12%. Drug court program (state): Sacramento Drug Court (California); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -20%; 3 or more years after entry: [Empty]. Drug court program (state): Guam Adult Drug Court (Guam); Reduction reported? Yes; Reduction statistically significant?[A]: Yes[G]; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: -20% . Drug court program (state): Ada County Drug Court (Idaho); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: -25%*[H]. Drug court program (state): Multiple Drug Courts (Idaho); Reduction reported? Yes[I]; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Monroe County Drug Treatment Court (Indiana); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -16%; 3 or more years after entry: [Empty]. Drug court program (state): St. Joseph County Drug Court (Indiana); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 2 years after entry: -16%; Percentage point difference in rate of re-arrest where reported, time frame covered: 3 or more years after entry: [Empty]. Drug court program (state): Vanderburgh County Day Reporting Drug Court (Indiana); Reduction reported? Yes; Reduction statistically significant?[A]: Yes[J]; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -9%; 3 or more years after entry: [Empty]. Drug court program (state): Baltimore City Circuit Court Adult Drug Treatment Court and Felony Diversion Initiative (Maryland); Reduction reported? Yes; Reduction statistically significant?[A]: Yes[J]; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Baltimore City District Court Adult Drug Treatment Court (Maryland); Reduction reported? Yes[K]; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Harford County District Court Adult Drug Court (Maryland); Reduction reported? Yes; Reduction statistically significant?[A]: Yes[J]; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Howard County District Court Drug Treatment Court (Maryland); Reduction reported? Yes; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -13%; Up to 2 years after entry: -10%; 3 or more years after entry: [Empty]. Drug court program (state): Montgomery County Adult Drug Court (Maryland); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -26%*; Up to 2 years after entry: -19%*; 3 or more years after entry: [Empty]. Drug court program (state): Prince George's County Circuit Court Adult Drug Court (Maryland); Reduction reported? No; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: 0%; 3 or more years after entry: +1%. Drug court program (state): Wicomico County Circuit Court Adult Drug Treatment Court (Maryland); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -24%*; Up to 2 years after entry: -18%; 3 or more years after entry: [Empty]. Drug court program (state): Suffolk County Drug Court (Massachusetts); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -6%*; 3 or more years after entry: [Empty]. Drug court program (state): Barry County Adult Drug Court (Michigan); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -24%; 3 or more years after entry: [Empty]. Drug court program (state): Kalamazoo County Adult Drug Treatment Court (Michigan); Reduction reported? Yes; Reduction statistically significant?[A]: Yes[J]; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -14%; 3 or more years after entry: [Empty]. Drug court program (state): Unnamed Drug Court (Midwest); Reduction reported? Yes[L]; Reduction statistically significant?[A]: No; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Kings County District Attorney's Office Drug Treatment Alternative to Prison Program (New York); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -22%*; Up to 2 years after entry: -18%*; 3 or more years after entry: -22%* (3 yrs) -26%* (4 yrs). Drug court program (state): Multiple Drug Courts (Ohio); Reduction reported? Yes[M]; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -18%*; 3 or more years after entry: [Empty]. Drug court program (state): Multnomah County Clean Court (Oregon); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: -16%; Up to 2 years after entry: [Empty]; 3 or more years after entry: [Empty]. Drug court program (state): Marion County Adult Drug Court (Oregon); Reduction reported? Yes; Reduction statistically significant?[A]: Not reported; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: -14%; 3 or more years after entry: [Empty]. Drug court program (state): Multiple Drug Courts (Oregon); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: -12%*. Drug court program (state): Rutland County Adult Drug Court (Vermont); Reduction reported? Yes; Reduction statistically significant?[A]: Yes; Percentage point difference in rate of re-arrest where reported, time frame covered: Up to 1 year after entry: [Empty]; Up to 2 years after entry: [Empty]; 3 or more years after entry: -23%*. Source: GAO analysis of drug court program evaluations. *The difference was reported to be statistically significant. [A] Indicates whether a reduction in any measure of recidivism reported by the study was statistically significant. [B] Study reported a difference in recidivism of -9% for self-reported criminal acts. [C] Study reported a statistically significant difference in the relative odds of an arrest between 12 and 18 months after drug court program enrollment. [D] Reflects findings from 2 studies of the same drug court by the same author. See, Gottfredson et al. [E] Study reported a 17% reduction in re-arrest rates over 5 years, but the difference in rates was not reported. [F] Study reported a difference in recidivism of -15% for self- reported criminal acts. [G] Study reported a statistically significant difference in the average number of arrests resulting in court filings after program initiation between the treatment and comparison groups, but did not report the significance of the difference in rates of arrest resulting in court case filings. [H] Study reported a statistically significant difference in the rate of new court filings following program initiation. Statistical controls, if any, were not presented. [I] Study reported a statistically significant difference in the rate of new court filings following program initiation for program completers. Statistical controls, if any, were not presented. [J] While the differences in rearrest rates were not reported as significant, or not reported at all, study did report a statistically significant difference in average number of re-arrests between the treatment and comparison groups. [K] Re-arrest rates for drug court participants were lower than for comparison group members for some years over a 10-year period, but none of the differences was statistically significant. [L] Study reported a difference in re-arrest rates of -12% for program completers vs. the comparison group, which was not statistically significant. [M] Findings reflect a comparison group that combines multiple courts. [End of table] The evaluations we reviewed showed that adult drug-court program participation was also associated with reduced drug use. Our analysis of evaluations reporting relapse data for eight programs showed that drug court program participants were less likely than comparison group members to use drugs, based on drug tests or self-reported drug use, although the difference was not always significant.[Footnote 42] This was true for both within-program and post-program measures, and whether drug use was reported as the difference in the frequency of drug use or the proportion of the treatment and comparison groups who used drugs. The MADCE concluded drug courts produce significant reductions in drug relapse. Specifically, MADCE reported that "drug court participants were significantly less likely than the comparison group to report using all drugs (56 vs. 76 percent) and also less likely to report using 'serious' drugs (41 vs. 58 percent), which omit marijuana and 'light' alcohol use (fewer than four drinks per day for women or less than five drinks per day for men). On the 18-month oral fluids drug test, significantly fewer drug court participants tested positive for illegal drugs (29 vs. 46 percent). Further, among those who tested positive or self-reported using drugs, drug court participants used drugs less frequently than the comparison group." Regarding post-drug court program relapses, the MADCE concluded that participation in drug court--along with less frequent drug use among offenders prior to arrest, and the absence of mental health problems--were the strongest predictors of success against relapses. Table 4 summarizes the results of drug-use relapse reported in the evaluations we reviewed. Table 4: Drug Use Relapse Results of Evaluations GAO Reviewed: Drug tests: Drug court program (state): Barry County Adult Drug Court (Michigan); Results of drug use relapse: Reduction within-program: Drug court participants generally had fewer positive drug test results than the comparison group in the 12-month period following program initiation. Drug court program (state): St. Joseph County Drug Court (Indiana); Results of drug use relapse: Reduction within-program: At all times during a 12-month period following program initiation, the drug court participants had a lower percentage of positive urine drug screens than the comparison group sample. Drug court program (state): Vanderburgh County Day Reporting Drug Court (Indiana); Results of drug use relapse: Reduction within-program: At all times during the 12-month period following program initiation, the drug court participants had a lower percentage of positive urine drug screens than the comparison group. Drug court program (state): Methamphetamine Treatment Project (California); Results of drug use relapse: Reduction within-program: Results show that drug court participants were significantly more likely than comparison participants to provide a higher proportion of clean urine samples, 97.3 percent versus 90.5 percent, respectively. Drug court program (state): Methamphetamine Treatment Project (California); Results of drug use relapse: Reduction post-program: There were substantial reductions in methamphetamine use over time for drug court participants compared to non-drug court comparison participants at program completion, and at 6 and 12 months following program completion. Self-reported drug use: Drug court program (state): Baltimore City Drug Treatment Court (Maryland); Results of drug use relapse: Reduction post-program: Participants in the treatment group had about 27 fewer days of alcohol use, 19 fewer days of cocaine use, and 28 fewer days of heroin use on average compared to the control group. The differences were statistically significant only for the effect on cocaine. Among participants originating in the circuit court, the average number of days of cocaine use was 50 days lower in the treatment group than the control group. Drug court program (state): Breaking the Cycle (Florida); Results of drug use relapse: Reduction within-program: Approximately 9 months after the initial arrest, drug court participants were less likely to report drug use in the past 30 days (24 percent) than the comparison group (33 percent); however, this difference was not significant at the 95 percent level. Drug court program (state): Breaking the Cycle (Washington); Results of drug use relapse: Reduction within-program: Approximately 9 months after the initial arrest, there was no significant difference between the percentage of drug court participants (50 percent) and the comparison group (51percent) who self-reported drug use in the past 30 days. Both drug tests and self-reported use: Drug court program (state): MADCE (Multiple States); Results of drug use relapse: Reduction 18 months after program initiation: 56 percent of drug court participants reported using any drugs compared with 76 percent among the comparison group; 41 percent of the drug court participants reported using "serious" drugs (not marijuana or light alcohol use) compared with 58 percent among the comparison group. 29 percent of drug court participants tested positive for illegal drugs compared with 46 percent of the comparison group. Among those who tested positive or self-reported using drugs, drug court participants used drugs less frequently than the comparison group. Source: GAO analysis of adult drug court program evaluations. [End of table] Drug Court Programs Were Associated with Both Positive and Negative Net Benefits: Of the studies we reviewed, 11 included sufficient information to report a net benefit figure. Of these studies, the net benefit ranged from positive $47,852 to negative $7,108 per participant. The net benefit is the monetary benefit of reduced recidivism accrued to society from the drug court program through reduced future victimization and justice system expenditures, less the net costs of the drug court program--that is, the cost of the program less the cost of processing a case in criminal court. A negative net benefit value indicates that the costs of the drug court program outweigh its estimated benefits and that the program was not found to be cost beneficial. Eight of the studies reported positive net benefits--the benefits estimated to accrue from the drug court program exceeded the program's net costs. Three of the 11 studies reported negative net benefits. We did not attempt to determine whether the differences in the reported values were because of differences in study methodology or the attributes of the drug courts themselves. The environment in which the drug court operates may also be important. For example, the largest net benefit reported was for Kings County, in which members of the comparison group were incarcerated, in contrast to other programs in which members of the comparison group were given probation, which is less costly. The more costly the alternative, such as incarceration, the more likely a drug court will have positive net benefits. In this case, the study reported that society would accrue $47,852 in benefits relative to conventional court processing. Table 5 below shows whether, based on the available information, the study was shown to be cost beneficial. It also shows the net benefits per participant of the drug court study. For example, MADCE found that the drug court participants led to a net benefit of $6,208 per participant--within the range of the other studies.[Footnote 43] The MADCE analysis of costs and benefits is discussed further in appendix II. Table 5: Cost Conclusions of the 11 Drug Court Program Evaluations in Our Cost-Benefit Review: Drug court program (state): Kings County District Attorney's Office Drug Treatment Alternative to Prison Program (New York)[A]; Program shown to be cost beneficial? Yes; Net benefits: $47,836. Drug court program (state): Multiple Drug Courts (Maine); Program shown to be cost beneficial? Yes; Net benefits: $42,177. Drug court program (state): Douglas County Drug Court (Nebraska); Program shown to be cost beneficial? Yes; Net benefits: $11,336. Drug court program (state): Multnomah County Drug Court (Oregon); Program shown to be cost beneficial? Yes; Net benefits: $10,826. Drug court program (state): MADCE (Multiple States); Program shown to be cost beneficial? Yes[B]; Net benefits: $6,208. Drug court program (state): Multiple Drug Courts (Kentucky); Program shown to be cost beneficial? Yes; Net benefits: $5,446. Drug court program (state): St. Joseph County Drug Court (Indiana); Program shown to be cost beneficial? Yes; Net benefits: $3,148. Drug court program (state): St. Louis City Adult Felony Drug Court (Missouri); Program shown to be cost beneficial? Yes; Net benefits: $2,615. Drug court program (state): Vanderburgh County Day Reporting Drug Court (Indiana); Program shown to be cost beneficial? No; Net benefits: ($1,640). Drug court program (state): Barry County Adult Drug Court (Michigan); Program shown to be cost beneficial? No; Net benefits: ($3,552). Drug court program (state): Monroe County Drug Treatment Court (Indiana); Program shown to be cost beneficial? No; Net benefits: ($7,108). Source: GAO of drug court program evaluations. [A] Comparison was to prison population. [B] Because of the variability in the estimate, the MADCE study could not determine that the net benefits were statistically significant. Most other studies did not report on whether differences in cost were statistically significant. [End of table] Conclusions: During the course of our review, BJA made strides in managing its adult drug court program, including implementation of the GrantStat process and recent revisions to the grantee performance measures. Given that BJA has committed to testing its new measures during this first grantees' reporting period, enhancements could be made to facilitate this assessment. By documenting how it plans to assess the measures and determine any changes that may be needed and providing the rationale for future revisions, BJA could bolster the transparency and integrity of its decisions. Doing so could also improve the reliability of the data it collects, its usefulness to managers in guiding the program, and the success of its measures. Recommendation for Execution Action: Recognizing that BJA has recently revised the adult drug-court performance measures and has plans to assess their utility, we recommend that BJA's Director take the following action to ensure that its revision process is transparent and results in quality and successful metrics to inform management's key decisions on program operations: * Document key methods used to guide future revisions of its adult drug-court program performance measures. This documentation should include both a plan for how BJA will assess the measures after conclusion of the grantees' first reporting period and a rationale for why each measure was refined, including a discussion of the scope and nature of any relevant stakeholder comments. Agency Comments: We provided a draft of this report to DOJ for review and comment. On December 1, 2011, we received written comments on the draft report from DOJ, which are reproduced in full in appendix VIII. DOJ concurred with our recommendation and described actions under way or planned to address the recommendation. DOJ also provided technical comments, which we incorporated as appropriate. DOJ stated that BJA will continue to document grantee feedback and will ensure that revisions to the measures are documented in accordance with GAO's best practices standards. In particular, DOJ stated that BJA will document (1) whether the name and definition of the measure is consistent with the methodology used to calculate it; (2) whether the measure is reasonably free from bias; (3) whether the measure meets the expectation of the program; and (4) its rationale for why each performance measure was refined, including the scope and nature of any relevant stakeholder comments. We believe that such actions would improve the reliability of the information collected, its usefulness to managers in making key decisions on program operations, and the success of its measures. We are sending copies of this report to the Attorney General and interested congressional committees. In addition, this report will be available at no charge on the GAO Web site at [hyperlink, http://www.gao.gov]. Should you or your staff have any questions concerning this report, please contact me at (202) 512-8777 or by e-mail at maurerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IX. Signed by: David C. Maurer: Director, Homeland Security and Justice Issues: [End of section] Appendix I: DOJ Has Fully Implemented Most of Our 2002 Recommendations and Plans to Address the Remaining One: The following provides the current status of the seven recommendations we made in 2002--which have since been closed--on DOJ's collection of performance data.[Footnote 44] Specifically, DOJ has fully implemented six of them and partially implemented one.[Footnote 45] DOJ has plans to fully address the remaining recommendation related to analyzing performance and outcome data collected from grantees and reporting annually on the results. Table 6 reflects this status. Table 6: Status of DOJ's Efforts to Address Recommendations We Made in 2002 on DOJ's Collection of Performance Data to Measure the Impact of Federally Funded Drug Court Programs: Recommendation: 1. Develop and implement a management information system that is able to track and readily identify the universe of drug court programs DOJ funds; Actions to address recommendation: BJA is currently using OJP's GMS to track and identify the universe of BJA funded drug court programs.[B] In 2009, BJA also developed the Enterprise Reporting Tool, an internal system, which allows BJA to query most, if not all, BJA databases, including GMS. BJA officials said this tool allows them to run reports on the universe of drug court programs funded by grant type, amount awarded, status, year awarded, and jurisdiction; Status[A]: Fully implemented. Recommendation: 2. Reinstate the collection of post-program data, selectively spot checking grantee responses to ensure accurate reporting; Actions to address recommendation: On October 28, 2011, BJA provided us with the revised performance measures for the Adult Drug Court Discretionary Grant Program--which includes the data requirement for grantees to track drug-court program participants following their program completion. According to BJA officials, data collection on the revised measures will take effect with grant activities on October 1, 2011, through December 30, 2011. Data entry and reporting by grantees in the Performance Management Tool (PMT) will begin on January 1, 2012.[C] BJA officials reported concerns regarding the usefulness and reliability of post-program data, emphasizing that once the drug court grants expire, drug courts are no longer required to track participants or report their status to BJA. BJA reported that it will test this new measure (post-program data), along with all other revised measures, and monitor for reliability in the data it receives from grantees. Having these new measures in place could help ensure BJA has the program management data it needs to make informed grantee decisions; Status[A]: Fully implemented. Recommendation: 3. Take immediate steps to accelerate the funding and implementation of a methodologically sound national impact evaluation and consider ways to reduce the time needed to provide information on the overall impact of federally funded drug court programs; Actions to address recommendation: In 2002, NIJ commissioned the adult drug court evaluation (MADCE) that selected multiple sites from across the country for review. In June 2011, MADCE was issued, and the main objectives were to: (1) test whether drug courts reduce drug use, crime, and multiple other problems associated with drug abuse, in comparison with similar offenders not exposed to drug courts; (2) address how drug courts work and for whom by isolating key individual and program factors that make drug courts more or less effective in achieving their desired outcomes; (3) explain how offender attitudes and behaviors change when they are exposed to drug courts and how these changes help explain the effectiveness of drug court programs; and (4) examine whether drug courts generate cost savings. The evaluation found that drug courts prevent crime and substance use and work equally well for most participant subgroups. See appendix II for a summary of the study; Status[A]: Fully implemented. Recommendation: 4. Take steps to ensure and sustain an adequate grantee response rate by improving efforts to notify and remind grantees of their reporting requirements; Actions to address recommendation: In fiscal year 2007, BJA began using GMS to send notifications to remind grantees of upcoming due dates for progress reports. If a progress report is more than 15 days late, GMS automatically freezes the grantee's available funding until the report is submitted. Similarly, the grantee is unable to draw down funds on a grant if a financial report is more than one day late. BJA officials said that these procedures provide an additional tool to assist grant managers in providing adequate oversight of grantees' reporting activities to ensure compliance with reporting requirements; Status[A]: Fully implemented. Recommendation: 5. Take corrective action toward grantees that do not comply with data collection reporting requirements; Actions to address recommendation: In fiscal year 2007, OJP implemented a policy whereby available grant funds are frozen for noncompliant grantees that are delinquent in submitting semi-annual progress reports or quarterly financial reports. In addition, BJA has the ability to designate a grantee as high risk if the grantee continues to be noncompliant in reporting requirements. Once grantees are notified of their high-risk designation, all new awards to the grantee include high-risk special conditions that provide for additional oversight, as necessary, and restrict the grantee from obligating, expending, or drawing down funds under the new awards from DOJ; Status[A]: Fully implemented. Recommendation: 6. Analyze performance and outcome data collected from grantees and report annually on the results; Actions to address recommendation: Since 2007, using PMT, BJA has collected quarterly quantitative performance data from federally funded drug court grantees. Semi-annually, BJA also collects responses to seven narrative questions that grantees provide using PMT. BJA officials said they regularly analyze the numeric data and publish the results of the performance measure reporting on BJA's Web site. BJA does not fully analyze and report on the grantees' responses to the narrative questions. As mentioned, on October 28, 2011, BJA provided us with the revised adult drug-court program performance measures, which include measures previously excluded from PMT, such as retention rates and outcomes of participants once they complete the program. As noted, BJA plans to reassess the reliability of the measures after the initial grantee reporting period concludes. After this period, BJA officials explained that they will make any necessary revisions or changes to the measures--then analyze and report on the results. As mentioned previously, BJA initiated a new process called GrantStat to maximize the use of performance information--providing an analytical framework to assess grantee performance data and other relevant information on a semi-annual basis to determine the effectiveness of the grant programs in BJA's portfolio; Status[A]: Partially implemented; BJA plans to fully implement. Recommendation: 7. Consolidate the multiple DOJ-funded drug-court- program-related data collection efforts; Actions to address recommendation: BJA has been using PMT to consolidate data collection efforts since 2007. According to officials, PMT allows for grantees' online performance measurement data submission and enhanced capacity for BJA to (1) aggregate grantee data across performance measures, (2) distill performance by the type of adult drug court grant, and (3) more quickly "error check" the reliability of grantees' data submissions. BJA officials said PMT allows them to query results and assess performance outcomes, which helps them make decisions when designing future grant solicitations. According to BJA officials, using PMT to consolidate the federally funded drug court program data collection efforts enables DOJ to better manage the programs; Status[A]: Fully implemented. Source: BJA. [A] The following explains the definitions we used in assessing DOJ status in addressing the recommendations. Fully implemented--DOJ provided evidence that satisfies the entire recommendation. Partially implemented--DOJ provided evidence that satisfies about half of the recommendation. Not implemented--DOJ provided no evidence that satisfies any of the recommendation. [B] GMS is an online system designed to make the grant application process easier and more efficient for grantees. GMS allows grantees to fill out forms and submit application materials online. [C] PMT is an online reporting tool that supports BJA grantees' ability to collect, identify, and report performance measurement data activities funded by their award. [End of table] [End of section] Appendix II: MADCE Is the Most Comprehensive Study of Drug Courts to Date, but Generalizability of Findings May Be Limited: NIJ's MADCE was conducted by the Urban Institute, Center for Court Innovation, and Research Triangle Institute.[Footnote 46] Data were collected from 1156 drug court participants in 23 different drug courts in 7 geographic clusters and from a comparison group of 625 drug-involved offenders in six different sites in four geographic clusters. Data collected included: three waves of interviews; drug tests; administrative records on treatment, arrests, and incarceration; court observation and interviews with staff and other stakeholders; and budget and other cost information. The evaluation was designed to address the following four questions: (1) Do drug courts reduce drug use, criminal behavior, and other associated offender problems? (2 )Do drug courts generate cost savings for the criminal justice system and other public institutions? (3) Are drug courts especially effective or less effective for certain categories of offenders or program characteristics? (4) Which drug court policies and offender perceptions explain their overall impact? MADCE Found Reductions in Recidivism and Relapse, but Generalizability May Be Limited: The MADCE's major findings can be summarized as follows: * Drug courts produce statistically significant reductions in self- reported crime. While both the drug court participants and comparison group participants reported large numbers of crimes in the year preceding the 18-month follow-up, drug court participants reported statistically significantly fewer than the comparison group members. Drug court participants were less likely than members of the comparison group to report committing any crimes (40 percent vs. 53 percent) and drug court participants reported committing fewer crimes in the preceding 12 months than comparison group members (43 criminal acts vs. 88 criminal acts). The difference between the two groups in the probability of an official re-arrest over 24 months was not statistically significant, though the percentage of individuals rearrested was lower for the drug court group than the comparison group (52 percent vs. 62 percent), as was the average number of re- arrests (1.24 vs. 1.64).[Footnote 47] * Drug courts produce statistically significant reductions in drug use. Drug court participants were less likely than members of the comparison group to report using any drugs (56 percent vs. 76 percent) and any serious drugs (41 percent vs. 58 percent), and less likely to test positively for drugs at the 18-month follow-up (29 percent vs. 46 percent). Furthermore, the large difference in self-reported relapse rates is evident at 6 months (40 percent vs. 59 percent), so the impact of drug courts on alcohol and other drug use is sustained. The interview data also indicate that among the drug court participants and comparison group members that were using drugs, the drug court participants, on average, were using them less frequently. * Drug court participants reported some benefits, relative to comparison group members, in other areas of their lives. At 18 months, drug court participants were statistically significantly less likely than comparison group members to report a need for employment, educational, and financial services, and reported statistically significantly less family conflict. However, there were modest, non- significant differences in employment rates, income, and family emotional support, and no differences found in experiencing homelessness or depression. * Regardless of background, most offenders who participated in drug courts had better outcomes than offenders who were in the comparison programs. However, the impact of drug courts was greater for participants with more serious prior drug use and criminal histories, and the impact was smaller for participants who were younger, male, African-American, or who had mental health problems. * While the treatment and service costs were higher for drug court participants than treatment and service costs associated with the alternative "business-as-usual" comparison programs, drug courts save money through improved outcomes, according to the researchers, primarily through savings to victims resulting from fewer crimes and savings resulting from fewer re-arrests and incarcerations. The authors of the study assert that their findings have strong internal validity--that is, that the findings were actually produced by the drug court programs--and external validity--that is, that the findings can be generalized to the population of all drug court participants and potential comparison group members. The claim to strong internal validity is not without merit, given the high response rates, low attrition, propensity score adjustments, and conservative estimates produced by the hierarchical models used.[Footnote 48] The claim of high internal validity is also supported by the sensitivity analyses undertaken for several outcomes using other models and methods of adjustments that produced little or no change in conclusions. The claim to strong external validity, which relates to the generalizability of the results beyond the sample of courts and comparison sites and specific offenders considered, may be somewhat overstated. The authors note that the 23 drug courts included in the study represent "a broad mix of urban, suburban, and rural courts from 7 geographic clusters nationwide," but that doesn't assure that, collectively, the drug courts that were included resemble the hundreds of drug courts that were not included, especially since they were not chosen at random. It also seems unlikely that the six comparison sites from four states are representative of all potential controls, or all alternative programs in all states, and it is potentially problematic that all of the selected sites, including drug court and comparison sites, were alike in their willingness and interest in participating. Those concerns notwithstanding, this is the broadest and most ambitious study of drug courts to date; it is well done analytically; and the results, as they relate to the impact of drug courts, are transparent and well described. MADCE's Cost Benefit Analysis Focused on Individuals: The MADCE cost benefit analysis approach differed from most of the other studies we reviewed. In most of the other studies, the average cost and benefit of a drug court participant was compared to the average cost and benefit of normal court processing. In contrast, the MADCE obtained a separate net benefit figure for each individual. The net benefit was obtained by tracking each individual's use of resources, such as hearings or meetings with case managers, and program outcomes like use of public assistance. The MADCE also tracked each individual's rates of re-arrest, number of crimes, and time of incarceration. The crimes are multiplied by cost to victims per crime to obtain the cost to society. The difference between the net benefits of the drug court participants and the comparison group were obtained using a hierarchical model similar to the one used for program outcomes. After applying the method, the MADCE found that the drug court participants led to a net benefit of $6,208[Footnote 49] to society per participant, as compared to the comparison group. However, due to the variability in the estimate, the study did not find that the net benefits were statistically significant. The lack of a statistically significant difference may be because of greater variability in the MADCE approach than the approach used in other studies. Specifically, the MADCE did not assume identical costs for each participant. As a result, costs may be higher for individuals who have lower rates of re-arrest, perhaps because those individuals received more treatment. According to the study's authors, by assuming identical costs for each participant, the standard approach understates the variance in the computed net benefit figure by not including the variability in cost. However, the MADCE authors assumed that the prices of services were consistent across sites by using a weighted average. In contrast, some studies generate site-specific cost figures. In this way, the MADCE approach did exclude one source of variation that is present in some other studies. In addition to tracking costs and benefits at the individual level, the MADCE also included some effects of drug court participation that some other studies omit. This is consistent with OMB guidance that states that studies should be comprehensive in the benefits and costs to society considered.[Footnote 50] One of the benefits considered by the MADCE, sometimes omitted elsewhere, is the estimated earnings of the drug court participant. However, it is unclear that the full value of earnings should have been considered a net benefit to society. For example, to be comprehensive, a study should also consider the cost to society of providing that benefit. The net benefit would account for the value of production from this employment less the wages paid. Although in this case, it is unlikely that this would affect the result of the analysis, as the earnings are similar for drug court participants and the comparison group. [End of section] Appendix III: Objectives, Scope, and Methodology: To determine what data DOJ collects on the performance of federally funded adult drug courts and to what extent DOJ has used this data in making grant-related decisions, we analyzed the reporting guidance and requirements that BJA provided in fiscal years 2007 through 2011 to grantees applying for Adult Drug Court Discretionary Grant Program funds;[Footnote 51] BJA-generated grantee performance data reports from October to December 2010; and BJA's guides for managing grants and enforcing grantee compliance that were issued in fiscal year 2011. We selected 2007 as the starting point for our review because BJA implemented its Performance Measurement Tool (PMT)--an online reporting tool that supports BJA grantees' ability to collect, identify, and report performance-measurement data activities funded by the grantees' awards--in fiscal year 2007. We also reviewed our prior reports and internal control standards as well as other academic literature regarding effective performance-management practices. [Footnote 52] We then used this information and BJA officials' statements to identify and define six management activities for which performance information can be most useful in making grant-related decisions.[Footnote 53] Further, we interviewed cognizant BJA officials about the extent to which they use grantees' performance data when engaging in these management activities, any challenges faced with ensuring grantee compliance, ongoing efforts to revise program performance metrics, and the extent to which BJA's revisions incorporate best practices we previously identified.[Footnote 54] To determine what is known about the effectiveness of adult drug courts in reducing recidivism and substance-abuse relapse rates and what the costs and benefits of adult drug courts are, we conducted a systematic review of evaluations of drug-court program effectiveness issued from February 2004 through March 2011 to identify what is known about the effect of drug court programs on the recidivism of and relapse of drug-involved individuals as well as the costs and benefits of drug courts.[Footnote 55] We also reviewed DOJ's NIJ-funded MADCE, a 5-year longitudinal process, impact, and cost evaluation of adult drug courts that was issued in June 2011. We identified the universe of evaluations to include in our review using a three-stage process. First, we (1) conducted key-word searches of criminal justice and social science research databases;[Footnote 56] (2) searched drug court program-related Web sites, such as those of BJA and NADCP; (3) reviewed bibliographies, meta-analyses of drug court evaluations, and our prior reports on drug court programs;[Footnote 57] and (4) asked drug court researchers and DOJ officials to identify evaluations. Our literature search identified 260 documents, which consisted of published and unpublished outcome evaluations, process evaluations, commentary about drug court programs, and summaries of multiple program evaluations.[Footnote 58] Second, we reviewed the 260 documents our search yielded and identified 44 evaluations that reported recidivism or substance use relapse rates using either an experimental or quasi-experimental design, or analyzed program costs and benefits.[Footnote 59] Third, we used generally accepted social science and cost benefit criteria to review the 44 evaluations. To assess the methodological quality of evaluations that reported on recidivism or relapse rates, we placed each evaluation into one of five categories, with category 1 evaluations being the most rigorous and category 5 the least, as outlined in table 7. Table 7: Methodological Quality Categories for Evaluations of a Drug Court Program: Category: 1; Required methodological elements: Random assignment to drug court or control group, drawn from local offenders eligible for the program. The sample: (a) is of sufficient size to ensure that randomization balances all covariates (e.g., potential predictors of outcomes, such as past criminal history); or (b) has constrained key covariates to balance in small samples through stratification (e.g., sampling subpopulations independently to improve representativeness) or through other adjustments. Randomization occurs after eligible offense, not after screening or other self-selection. Drug court is compulsory if assigned, and rates of attrition from the program are low. Category: 2; Required methodological elements: Either; Random assignment with several factors in group 1 missing, such as small sample sizes or some amount of pre-screening of eligible participants; or: Nonrandom assignment, but analysis models or controls for the specific process used to assign participants to the drug court when constructing a longitudinal comparison group. Alternatively, assignment to the program was nonrandom but clearly exogenous to the outcomes. The comparison group is similar to the treatment group on any variables the program explicitly uses to screen participants, such as "readiness for treatment" or "likely response to treatment." Comparison group is matched on multiple years of pre-treatment data. Category: 3; Required methodological elements: Either; Problematic random assignment: extremely small sample sizes, many large differences between treatment and control groups, randomization that occurs after all important forms of self-selection or screening; or: Nonrandom assignment, and the analysis controls for pre-treatment outcomes and participant demographics without considering the specific process used to assign participants to the program being evaluated. Comparison group is used, but has limited pre-treatment covariate data for construction. Comparison or treatment groups are constructed in ways that could have a clear impact on the outcomes (e.g., truncating the sample). Category: 4; Required methodological elements: Nonrandom assignment. Comparison group constructed with few controls for pre-treatment outcomes or shows covariate differences with the treatment group. Several plausible sources of selection bias, such as preexisting differences between the two groups in the degree of substance use. Category: 5; Required methodological elements: Nonrandom assignment. Cross- sectional design with few controls, pre-post design with no comparison group and few controls, or treatment group that includes only program graduates. Source: GAO. [End of table] We excluded studies that were placed in category 5 in the table above or studies in which the comparison group was not drawn from a criminal court. We were left with 33 studies, plus the MADCE, that reported on the effectiveness of 32 drug court programs or sets of programs.[Footnote 60] As noted in our report, we then grouped the 34 studies, including the MADCE, into two tiers according to their quality category, Tier 1 studies were those that fell into categories 1 or 2, Tier 2 studies were those that fell into categories 3 or 4. Observed differences in recidivism could arise from measured and unmeasured sources of variation between drug court participants and comparison group members. If comparison group members differed systematically from drug court participants on variables that are also associated with recidivism, such as the degree of their substance- abuse addiction problem and these variables were not accounted for by the design or analysis used in the evaluation, then the study could suffer from selection bias wherein observed differences in recidivism could be because of these sources of variation rather than participation in the drug court program. As indicated in table 7, our evaluation of the methods used to deal with selection bias was reflected in the quality categorization of each study.[Footnote 61] To assess the methodological quality of evaluations that reported on drug court program costs and benefits, we assessed them according to the five criteria we developed and outlined in table 8 below.[Footnote 62] Table 8: Five Criteria for Assessing a Cost-Benefit Analysis of a Drug Court Program: Criterion: 1. States the program's purpose; Description: In general, the purpose of a drug court program is to reduce repeated criminal behavior--to reduce recidivism--by reducing offenders' substance-using behavior. Criterion: 2. Identifies the baseline; Description: The baseline, or alternative, is what would happen to an offender if the drug court program did not exist. Criterion: 3. Assesses all relevant costs; Description: The costs involved in a drug court program are those associated with the program's operation and those associated with the baseline. Criterion: 4. Assesses all relevant benefits; Description: Benefits usually attributed to drug court programs are costs avoided because of reduced recidivism; they accrue to the criminal justice system and potential victims of crime. Other benefits an analysis could consider include reduced medical costs and successful program participants' increased productivity. Criterion: 5. Assesses uncertainty in cost and benefit estimates; Description: Most cost and benefit estimates entail uncertainty from imprecision in the data underlying the analysis and the assumptions built into the analysis. Assessing uncertainty enhances confidence in the estimates used in evaluation. Source: GAO-05-219. [End of table] We determined that an essential criterion for reporting a net benefit of drug courts was that the costs of the drug court were assessed against a baseline (i.e., "business-as-usual" or traditional court processing). Eleven studies met this essential standard and were used to report on program costs and benefits. We excluded other studies not meeting this standard even though they may have met others. To obtain information on our outcomes of interest--that is, recidivism, substance use relapse, and costs and benefits--we used data collection instruments to systematically collect information about the methodological characteristics of each evaluation, the drug court participants and comparison group members studied, and the outcomes of the participants and other comparable groups reported. Each evaluation was read and coded by a senior social scientist, statistician, or economist with training and experience in evaluation research methods. A second senior social scientist, statistician, or economist then reviewed each completed data collection instrument to verify the accuracy of the information included. Part of our assessment also focused on the quality of the data used in the evaluations as reported by the researchers and our observations of any problems with missing data, any limitations of data sources for the purposes for which they were used, and inconsistencies in reporting data. We incorporated any data problems that we noted in our quality assessments. We selected the evaluations in our review based on their methodological strength; therefore, our results cannot be generalized to all drug court programs or their evaluations. Although the findings of the evaluations we reviewed are not representative of the findings of all evaluations of drug court programs, the evaluations consist of those evaluations we could identify that used the strongest designs to assess drug-court program effectiveness. To identify the extent to which DOJ has addressed the recommendations that we made in 2002 regarding drug court programs, we interviewed cognizant DOJ officials and obtained and reviewed documentation (e.g., drug-court program grant solicitations and grantee-performance reporting guidance) on the actions taken to address and implement each of our prior recommendations. We conducted this performance audit from November 2010 through December 2011 in accordance with generally accepted government-auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our objectives. [End of section] Appendix IV: Overview of Drug Court Program Characteristics: This appendix provides a general description of drug court program components (see table 9). Drug court programs rely on a combination of judicial supervision and substance-abuse treatment to motivate defendants' recovery.[Footnote 63] Judges preside over drug court proceedings, which are called status hearings; monitor defendants' progress with mandatory drug testing; and prescribe sanctions and incentives, as appropriate in collaboration with prosecutors, defense attorneys, treatment providers, and others. Drug court programs can vary in terms of the substance-abuse treatment required. However, most programs offer a range of treatment options and generally require a minimum of about 1 year of participation before a defendant completes the program. Table 9: General Description of Drug Court Program Components: Drug court elements: Drug court program approaches; Description: Drug court programs generally have taken two approaches to processing cases: (1) deferred prosecution (diversion); and (2) post-adjudication. In the diversion model, the courts defer prosecution dependent on the defendant's agreement to participate in the drug court program. Deferred adjudication models do not require the defendant to plead guilty. Instead the defendant enters the drug court before pleading to a charge. Defendants who complete the treatment program are not prosecuted further and their charges are dismissed. Failure to complete the program results in prosecution for the original offense. This approach is intended to capitalize on the trauma of arrest and offers defendants the opportunity to obtain treatment and avoid the possibility of a felony conviction. In contrast, defendants participating in a post-adjudication (post-plea) drug court program plead guilty to the charge(s) and their sentences are suspended or deferred. Upon successful completion of the program, sentences are waived and in many cases records are expunged. This approach provides an incentive for the defendant to rehabilitate because progress toward rehabilitation is factored into the sentencing determination. Both of these approaches provide the defendant with a powerful incentive to complete the requirements of the drug court program. Some drug court programs use both deferred prosecution and post-adjudication approaches and assign defendants to an approach depending on the severity of the charge. Additionally, drug court programs may also combine aspects of these models into a hybrid or combined approach. Drug court elements: Screening process and participant eligibility criteria; Description: Defendants reach the drug court program from different sources and at varying points in case processing. Screening defendants to determine eligibility for a drug court program generally includes assessing their criminal history and current case information (e.g., charging offense, prior convictions, pending cases, and probation status). Depending on the program, an assistant district or prosecuting attorney, court clerk, or drug court coordinator typically conducts the review. Drug courts generally accept defendants charged with drug possession or other nonviolent offenses such as property crimes. Some drug court programs allow defendants who have prior convictions to participate, and others do not. Federal grants administered by BJA may not be awarded to any drug court program that allows either current or past violent offenders to participate in its program.[A] After defendants are determined to be legally eligible for the program, treatment providers or case managers will typically determine defendants' clinical eligibility. This can be determined through structured assessment tests, interviews, or even preliminary drug test results. While drug courts generally only accept defendants with substance-abuse problems, they vary in the level of addiction or type of drug to which defendants are addicted. For example, some programs do not accept defendants who only have addictions to marijuana or alcohol, while others do. Clinical eligibility can also include factors such as medical or mental health barriers and motivation or treatment readiness. In several drug court programs in our review, the drug court judge's satisfaction with or assessment of an offender's motivation and ability to complete the program was a factor used to screen defendants. Drug court elements: Program completion requirements; Description: Drug court programs typically require defendants to complete a 1-year treatment program in order to graduate from or complete the program. Some programs impose other conditions that participants must meet in addition to treatment. These conditions could include remaining drug-free for a minimum amount of time, not being arrested for a specified period of time, maintaining employment or obtaining an educational degree or certification, or performing community service. Drug court elements: Judicial supervision and status hearings; The central element of all drug court programs is attendance at the regularly scheduled status hearings, at which the drug court judge monitors the progress of participants. Monitoring is based on treatment-provider reports on such matters as drug testing and attendance at counseling sessions. The judge is to reinforce progress and address noncompliance with program requirements. The primary objectives of the status hearing are to keep the defendant in treatment and to provide continuing court supervision. More broadly, judicial supervision includes regular court appearances and direct in- court interaction with the judge, as well as scheduled case manager visits. Drug court elements: Drug-testing requirements; Monitoring participants' substance use through mandatory and frequent testing is a core component of drug court programs. Programs vary in the specific policies and procedures regarding the nature and frequency of testing. For example, in some programs in our review participants were required to call to find out whether they are required to be tested in a given period or on a randomly selected day of the week. The frequency of testing generally varied depending on the stage or phase of the program that participants were in. Drug court elements: Treatment components; Description: In most drug court programs, treatment is designed to last at least 1 year and is generally administered on an outpatient basis with limited inpatient treatment, as needed, to address special detoxification or relapse situations. Many of the programs operate with the philosophy that because drug addiction is a disease, relapses can occur and that the court must respond with progressive sanctions or enhanced treatment, rather than immediate termination. Treatment services are generally divided into three phases. Detoxification, stabilization, counseling, drug education, and therapy are commonly provided during phases I and II, and in some instances, throughout the program. Other services relating to personal and educational development, job skills, and employment services are provided during phases II and III, after participants have responded to initial detoxification and stabilization. Housing, family, and medical services are frequently available throughout the program. In some instances, a fourth phase consisting primarily of aftercare- related services is provided. The objectives of drug-court program treatment are generally to (1) eliminate the program participants' physical dependence on drugs through detoxification; (2) treat the defendant's craving for drugs through stabilization (referred to as rehabilitation stage) during which frequent group or individual counseling sessions are generally employed; and (3) focus on helping the defendant obtain education or job training, find a job, and remain drug free. Drug court programs can also either directly provide or refer participants to a variety of other services and support, and they may include medical or health care, mentoring, and educational or vocational programs. The use of community-based treatment self-help groups, such as Alcoholics Anonymous and Narcotics Anonymous, and aftercare programs also varies across drug court programs. Drug court elements: Sanctions for noncompliance; Description: Judges generally prescribe sanctions and incentives as appropriate in collaboration with prosecutors, defense attorneys, treatment providers, and others. Typical sanctions for program noncompliance include oral warnings from the judge; transfer to an earlier stage of the program; attendance at more frequent status hearings, treatment sessions, or drug tests; and serving jail time for several days or weeks. The approach or philosophy for how a drug court judge prescribes sanctions can vary. For example, some judges use a graduated sanctions approach, where sanctions are applied in increasing severity. Other judges may use discretion in prescribing sanctions, assessing participants' noncompliance on a case-by-case basis. Drug court elements: Termination criteria; Description: Drug court programs typically use various criteria for ending a defendant's participation in the program before completion. These criteria may include a new felony offense, multiple failures to comply with program requirements such as not attending status hearings or treatment sessions, and a pattern of positive drug tests. Before terminating a defendant for continuing to use drugs, drug court programs generally will use an array of treatment services and available sanctions. There are no uniform standards among all programs on the number of failed drug tests and failures to attend treatment sessions that lead to a participant's termination. Drug court program judges generally make decisions to terminate a program participant on a case-by-case basis, taking into account the recommendations of others, including the treatment provider, prosecutor, and defense counsel. Relapses are expected, and the extent to which noncompliance results in terminations varies from program to program. Once a defendant is terminated, he or she is usually referred for adjudication or sentencing. Source: GAO-05-219. [A] 42 U.S.C. § 3797u-1-u-2. Violent offenders generally include those who have been charged with or convicted of an offense that is punishable by a term of imprisonment of greater than one year, and the offense involved a firearm or dangerous weapon; death or serious bodily injury; or the use of force. Past violent offenders include those who have one or more prior convictions for a felony crime of violence involving the use or attempted use of force against a person with the intent to cause death or serious bodily harm. § 3797u-2. [End of table] [End of section] Appendix V: Ten Key Components of a Drug Court--Developed by BJA in Collaboration with the National Association of Drug Court Professionals: Table 10: Ten Key Components of a Drug Court: 1. Integration of substance-abuse treatment with justice system case processing. 2. Use of a non-adversarial approach, in which prosecution and defense promote public safety while protecting the right of the participant to due process. 3. Early identification and prompt placement of eligible participants. 4. Access to continuum of treatment, rehabilitation, and related services. 5. Frequent testing for alcohol and illicit drugs. 6. A coordinated strategy governs drug court responses to participants' compliance. 7. Ongoing judicial interaction with each participant. 8. Monitoring and evaluation to measure achievement of program goals and gauge effectiveness. 9. Continuing interdisciplinary education to promote effective planning, implementation, and operation. 10. Forging partnerships among drug courts, public agencies, and community-based organizations generates local support and enhances drug court program effectiveness. Source: BJA. [End of table] [End of section] Appendix VI: BJA Offers Solicitations in Four Broad Drug-Court Grant Categories--Implementation, Enhancement, Statewide, and Joint: As mentioned, the Adult Drug Court Discretionary Grant Program provides financial and technical assistance to states, state courts, local courts, units of local government, and Indian tribal governments to develop and implement drug treatment courts. There are four different types of awards that BJA makes to adult drug-court grantees through the program. Table 11 provides a description of the grant types. Table 11: Adult Drug-Court Discretionary Grant Program--Grant Type and Description: Grant type: Implementation grants; Description: Available to jurisdictions that have completed planning and are ready to implement an adult drug court. Grantees may use their awards to fund various court operations and services including offender supervision, management, and services; provision and coordination of non-treatment recovery support services; and childcare and other family supportive services. Grant type: Enhancement grants; Description: Available to jurisdictions with fully operational adult drug courts. Applicants may use funding to expand their target population, enhance court services, or enhance offender services. Grant type: Statewide grants; Description: Used to improve, enhance, or expand drug court services statewide by encouraging adherence to the evidence-based design features and through activities such as: training and/or technical assistance programs for drug court teams geared to improve drug court functioning and to increase drug court participation and participant outcomes; tracking, compiling, coordinating, and disseminating state drug court information and resources; increasing communication, coordination, and information sharing among drug court programs; conducting a statewide drug court evaluation; or establishing a statewide automated drug-court data collection and/or performance management system. Grant type: Joint grants; Description: BJA, in collaboration with the Department of Health and Human Services, Substance Abuse and Mental Health Services Administration (SAMHSA), offers a joint grant program for the enhancement of adult drug court services, coordination, and substance- abuse treatment capacity.[A] Successful applicants are awarded two grants: an Enhancement grant from BJA and a Behavioral Health Court Collaboration Grant from SAMHSA. This joint program offers grantees the opportunity to design a comprehensive strategy for enhancing drug court capacity while accessing both criminal justice and substance- abuse treatment funds under a single grant application. Source: BJA. [A] SAMHSA is authorized under section 509 of the Public Health Service Act, as amended (42 U.S.C. § 290bb-2) to provide Adult Treatment Drug Court grants. [End of table] [End of section] Appendix VII: Key Management Activities Identified for Which Performance Information Can Be Most Useful: Table 12: Definitions: Key Management Activities Identified for Which Performance Information Can Be Most Useful: Key management activities: a. Setting program priorities; How performance information may be used to support the activity: Performance information is used to set priorities in budgeting and to target resources. Agencies can also use this information to identify priorities on which to focus their efforts. For example, targeting grants to address "underserved" client groups. Key management activities: b. Allocating resources; How performance information may be used to support the activity: Performance information is used to compare results of agencies' programs with goals and to identify where program resources should be targeted to improve performance and achieve goals. When faced with reduced resources, such analyses can assist agencies' efforts to minimize the impact on program results. Key management activities: c. Adopting new program approaches or changing work processes; How performance information may be used to support the activity: Performance information is used to assess the way a program is conducted and the extent to which a program's practices and policies have or have not led to improvements in outcomes. Such information is used to identify problems and consider alternative approaches and processes in areas where goals are not being met and to enhance the use of program approaches and processes that are working well. Key management activities: d. Identifying and sharing with stakeholders more effective processes and approaches to program implementation; How performance information may be used to support the activity: Performance information is used to identify and increase the use of program approaches that are working well and share these effective processes and approaches with stakeholders. Key management activities: e. Setting expectations for grantees; How performance information may be used to support the activity: Performance information is used to establish the targets and goals that grantees are expected to achieve. These targets and goals can be used as the basis for corrective action (e.g., technical assistance, freezing of funds) or to reward high performing grantees. Key management activities: f. Monitoring grantee performance; How performance information may be used to support the activity: Performance information is used to compare grantees' performance results with established targets and goals to determine the extent to which grantees have met them and, if necessary, target program resources (e.g., technical assistance) to improve grantees' performance. Source: GAO analyses. Note: We identified the first four management activities above as relevant from governmentwide surveys of federal managers conducted in 1997, 2000, and 2003. See GAO-05-927. The remaining two activities we identified by reviewing performance management literature. In defining the management activities, we reviewed the literature identified and met with BJA officials to determine the extent to which they agreed with our definitions. BJA staff confirmed each of these six to be relevant to managing the drug court program. [End of table] [End of section] Appendix VIII: Comments from the Department of Justice, Bureau of Justice Assistance: U.S. Department of Justice: Office of Justice Programs: Washington, D.C. 20531: December 1, 2011: Mr. David C. Maurer: Director: Homeland Security and Justice Issues: Government Accountability Office: 441 G Street, NW: Washington, DC 20548: Dear Mr. Maurer: Thank you for the opportunity to comment on the draft Government Accountability Office (GAO) report entitled, "Adult Drug Courts: Studies Show Courts Reduce Recidivism, But DOJ Could Enhance Future Performance Measure Revision Efforts" (GAO-12-53). The draft GAO report contains one Recommendation for Executive Action to the U.S. Department of Justice (DOJ), which is restated in bold text below and is followed by our response. [Begin bold text: Recognizing that BJA has recently revised the adult drug court performance measures and has plans to assess their utility, we recommend that BJA's Director take the following action to ensure that its revision process is transparent and results in quality and successful metrics to inform management's key decisions on program operations: * Document key methods used to guide future revisions of its adult drug court program performance measures. This documentation should include both a plan for bow BJA will assess the measures after the first grantee reporting period concludes and a rationale for why each measure was refined, including the scope and nature of any relevant stakeholder comments. End bold text] The Office of Justice Programs (OJP) agrees with the Recommendation for Executive Action, and will continue to ensure that any revisions to the Drug Court measures or the process to revise those measures is transparent, and results in quality and successful metrics to inform management's key decisions on program operations. As stated in the GAO draft report, the Bureau of Justice Assistance (BJA) documented key components of the revision process, including meeting minutes, stakeholder call recordings, and email documentation; however, BJA did not consolidate all information collected into a single document. BJA will continue to document grantee feedback, and will ensure that revisions to the measures are documented in accordance with GAO Best Practices standards regarding: (1) whether the name and definition of the measure is consistent with the methodology used to calculate it; (2) whether the measure is reasonably free from bias; (3) whether the measure meets the expectation of the program; and (4) BJA's rationale for why each performance measure was refined, including the scope and nature of any relevant stakeholder comments. Beginning with the first reporting cycle, which ends on December 31, 2011, BJA will gauge the ability of Drug Court grantees to understand and accurately report on the new performance measures. By July 15, 2012, BJA plans to analyze two quarters of performance data submitted by Drug Court grantees for such inaccuracies, including, but not limited to missing data, outliers, and duplicate counts. This will enable BJA to identify performance measures that may potentially produce unreliable results. BJA anticipates that the assessment of the quality of the data and refinement of performance measures will be an ongoing process. If you have any questions regarding this response, you or your staff may contact Maureen Henneberg, Director, Office of Audit, Assessment, and Management, at (202) 616-3282. Sincerely, Signed by: Laurie O. Robinson: Assistant Attorney General: cc: Mary Lou Leary: Principal Deputy Assistant Attorney General: James H. Burch, II: Deputy Assistant Attorney General for Operations and Management: Denise O'Donnell: Director: Bureau of Justice Assistance: Leigh Benda: Chief Financial Officer: Maureen Henneberg: Director: Office of Audit, Assessment, and Management: Louise Duhamel, Ph.D. Acting Director, Audit Liaison Group: Internal Review and Evaluation Office: Justice Management Division: OJP Executive Secretariat: Control Number 20111887: [End of section] Appendix IX: GAO Contacts and Staff Acknowledgments: GAO Contacts: David C. Maurer, (202) 512-8777 or maurerd@gao.gov: Staff Acknowledgments: In addition to the contact named above, Joy Booth, Assistant Director and Frederick Lyles, Jr., Analyst-in-Charge, managed this assignment. Christoph Hoashi-Erhardt, Michael Lenington, and Jerry Seigler, Jr., made significant contributions to the work. David Alexander, Benjamin Bolitzer, Michele Fejfar, and Doug Sloane assisted with design and methodology. Pedro Almoguera, Carl Barden, Harold Brumm, Jr., Jean McSween, Cynthia Saunders, Jeff Tessin, Susan B. Wallace, and Monique Williams assisted with evaluation review. Janet Temko provided legal support, and Katherine Davis provided assistance in report preparation. Bibliography: Bouffard, Jeffrey A., and Katie A. Richardson. "The Effectiveness of Drug Court Programming for Specific Kinds of Offenders: Methamphetamine and DWI Offenders Versus Other Drug-Involved Offenders." Criminal Justice Policy Review, 18(3) (September 2007): 274-293. Carey, Shannon M., and Michael W. Finigan. Indiana Drug Courts: St. Joseph County Drug Court Program Process, Outcome and Cost Evaluation- Final Report. Portland, OR: NPC Research, 2007. Carey, Shannon M., and Michael W. Finigan. Indiana Drug Courts: Vanderburgh County Day Reporting Drug Court Process, Outcome and Cost Evaluation-Final Report. Portland, OR: NPC Research, 2007. Carey, Shannon M., and Michael W. Finigan. "A Detailed Cost Analysis in a Mature Drug Court Setting: A Cost-Benefit Evaluation of the Multnomah County Drug Court." Journal of Contemporary Criminal Justice, 20(3) (August 2004): 315-338. Carey, Shannon M., Michael W. Finigan, et. al. Indiana Drug Courts: Monroe County Drug Treatment Court Process, Outcome and Cost Evaluation-Final Report. Portland, OR: NPC Research, 2007. Carey, Shannon M., Michael Finigan, Dave Crumpton, and Mark S. Waller. "California Drug Courts: Outcomes, Costs and Promising Practices: An Overview of Phase II in a Statewide Study." Journal of Psychoactive Drugs (November 2006). Carey, Shannon M., Lisa M. Lucas, Mark S. Waller, Callie H. Lambarth, Robert Linhares, Judy M. Weller, and Michael W. Finigan. Vermont Drug Courts: Rutland County Adult Drug Court Process, Outcome, and Cost Evaluation-Final Report. Portland, OR: NPC Research, 2009. Carey, Shannon, and Gwen Marchand. Marion County Adult Drug Court Outcome Evaluation-Final Report. Portland, OR: NPC Research, 2005. Carey, Shannon M. and Mark S. Waller. Oregon Drug Court Cost Study: Statewide Costs and Promising Practices-Final Report. Portland, OR: NPC Research, 2010. Carey, Shannon M., and Mark Waller. California Drug Courts: Costs and Benefits-Phase III. Portland, OR: NPC Research, 2008. Carey, Shannon M., and Mark S. Waller. Guam Adult Drug Court Outcome Evaluation-Final Report. Portland, OR: NPC Research, 2007. Dandan, Doria Nour. Sex, Drug Courts, and Recidivism. University of Nevada, Las Vegas: 2010. Ferguson, Andrew, Birch McCole, and Jody Raio. A Process and Site- Specific Outcome Evaluation of Maine's Adult Drug Treatment Court Programs. Augusta, ME: University of Southern Maine, 2006. Finigan, Michael W., Shannon M. Carey, and Anton Cox. Impact of a Mature Drug Court Over 10 Years of Operation: Recidivism and Costs (Final Report). Portland, OR: NPC Research, 2007. Gottfredson, Denice C., Brook W. Kearley, Stacy S. Najaka, and Carlos M. Rocha. "How Drug Treatment Courts Work: An Analysis of Mediators." Journal of Research in Crime and Delinquency, 44(1) (February 2007): 3- 35. Gottfredson, Denice C., Brook W. Kearley, Stacy S. Najaka, and Carlos M. Rocha. "Long-term effects of participation in the Baltimore City drug treatment court: Results from an experimental study." Journal of Experimental Criminology, 2(1) (January 2006): 67-98. Gottfredson, Denice C., Brook W. Kearley, Stacy S. Najaka, and Carlos M. Rocha. "The Baltimore City Drug Treatment Court: 3-Year Self-Report Outcome Study." Evaluation Review, 29(1) (February 2005): 42-64. Krebs, C.P., C.H. Lindquist, W. Koetse, and P.K. Lattimore. "Assessing the long-term impact of drug court participation on recidivism with generalized estimating equations." Drug and Alcohol Dependence, 91(1) (November 2007): 57-68. Labriola, Melissa M. The Drug Court Model and Chronic Misdemeanants: Impact Evaluation of the Queens Misdemeanor Treatment Court. New York, NY: Center for Court Innovation, 2009. Latimer, Jeff, Kelly Morton-Bourgon, and Jo-Anne Chrétien. A Meta- Analytic Examination of Drug Treatment Courts: Do They Reduce Recidivism? Ottawa, Ontario: Department of Justice Canada, 2006. Listwan, Shelley Johnson, James Borowiak, and Edward J. Latessa. An Examination of Idaho's Felony Drug Courts: Findings and Recommendations-Final Report. Kent State University and University of Cincinnati: 2008. Logan, T. K., William H. Hoyt, Kathryn E. McCollister, Michael T. French, Carl Leukefeld, and Lisa Minton. "Economic evaluation of drug court: methodology, results, and policy Implications." Evaluation and Program Planning, 27 (2004):381-396. Loman, Anthony L. A Cost-Benefit Analysis of the St. Louis City Adult Felony Drug Court. Institute of Applied Research. St. Louis, MO: 2004. Lowenkamp, Christopher T., Alexander M. Holsinger, Edward J. Latessa. "Are Drug Courts Effective: A Meta-Analytic Review." Journal of Community Corrections. (Fall 2005): 5-28. Mackin, Juliette R., Shannon M. Carey, and Michael W. Finigan. Harford County District Court Adult Drug Court: Outcome and Cost Evaluation. Portland, OR: NPC Research, 2008. Mackin, Juliette R., Shannon M. Carey, and Michael W. Finigan. Prince George's County Circuit Court Adult Drug Court: Outcome and Cost Evaluation. Portland, OR: NPC Research, 2008. Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller, Shannon M. Carey, and Michael W. Finigan. Baltimore City Circuit Court Adult Drug Treatment Court and Felony Diversion Initiative: Outcome and Cost Evaluation-Final Report. Portland, OR: NPC Research, 2009. Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller, Theresa Allen Herrera, Shannon M. Carey, and Michael W. Finigan. Howard County District Court Drug Treatment Court Program Outcome and Cost Evaluation. Portland, OR: NPC Research, 2010. Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller, Theresa Allen Herrera, Shannon M. Carey, and Michael W. Finigan. Montgomery County Adult Drug Court Program Outcome and Cost Evaluation. Portland, OR: NPC Research, 2010. Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller, Theresa Allen Herrera, Shannon M. Carey, and Michael W. Finigan. Wicomico County Circuit Court Adult Drug Treatment Court Program Outcome and Cost Evaluation. Portland, OR: NPC Research, 2009. Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller, Judy M. Weller, Jennifer A. Aborn, Robert Linhares, Theresa L. Allen, Shannon M. Carey, and Michael W. Finigan. Baltimore City District Court Adult Drug Treatment Court: 10-Year Outcome and Cost Evaluation. Portland, OR: NPC Research, 2009. Marchand, Gwen, Mark Waller, and Shannon M. Carey. Barry County Adult Drug Court Outcome and Cost Evaluation-Final Report. Portland, OR: NPC Research, 2006. Marchand, Gwen, Mark Waller, and Shannon M. Carey. Kalamazoo County Adult Drug Treatment Court Outcome and Cost Evaluation-Final Report. Portland, OR: NPC Research, 2006. Marinelli-Casey, Patricia, Rachel Gonzales, Maureen Hillhouse, Alfonso Ang, Joan Zweben, Judith Cohen, Peggy Fulton Hora, and Richard A. Rawson. "Drug court treatment for methamphetamine dependence: Treatment response and posttreatment outcomes." Journal of Substance Abuse Treatment, 34(2) (March 2008): 242-248. Mitchell, Ojmarrh, and Adele Harrell. "Evaluation of the Breaking the Cycle Demonstration Project: Jacksonville, FL and Tacoma, WA." Journal of Drug Issues, 36(1) (Winter 2006): 97-118. Piper, R. K., and Cassia Spohn. Cost/Benefit Analysis of the Douglas County Drug Court. Omaha, NE: University of Nebraska at Omaha, 2004. Rhodes, William, Ryan Kling, and Michael Shively. Suffolk County Drug Court Evaluation. Abt. Associates, Inc., 2006. Rhyne, Charlene. Clean Court Outcome Study. Portland, OR: Multnomah County Department of Community Justice, 2004. Rossman, S., M. Rempel, J. Roman, et.al. The Multi-Site Adult Drug Court Evaluation: The Impact of Drug Courts. Washington, D.C.: Urban Institute, 2011. Shaffer, Deborah K., Kristin Bechtel, and Edward J. Latessa. Evaluation of Ohio's Drug Courts: A Cost Benefit Analysis. Cincinnati, OH: Center for Criminal Justice Research, University of Cincinnati, 2005. Wilson, David B., Ojmarrh Mitchell, and Doris L. Mackenzie. "A systematic review of drug court effects on recidivism." Journal of Experimental Criminology, 2(4) (2006): 459-487. Zarkin, Gary A., Lara J. Dunlap, Steven Belenko, and Paul A. Dynia. "A Benefit-Cost Analysis of the Kings County District Attorney's Office Drug Treatment Alternative to Prison (DTAP) Program." Justice Research and Policy, 7(1) (2005). [End of section] Footnotes: [1] The types of drug courts include adult drug courts, juvenile drug courts, family drug courts, tribal drug courts, designated Driving Under the Influence (DUI) courts, campus drug courts, reentry drug courts, federal reentry drug courts, veterans drug courts, and co- occurring disorder courts--for offenders with mental health and substance addiction issues. [2] The Adult Drug Court Discretionary Grant Program was originally authorized under Title V of the Violent Crime Control and Law Enforcement Act of 1994, Pub. L. No. 103-322, 108 Stat. 1796,1955-59, and subsequently reauthorized by Title II of the 21st Century Department of Justice Appropriations Authorization Act, Pub L. No. 107- 273, § 2301, 116 Stat. 1758, 1794-99 (2002) (codified at 42 U.S.C. §§ 3797u-u-8). Drug court programs have also received funding from other federal sources, and state and local governments. [3] BJA's grant solicitation states that to assist DOJ in fulfilling its obligation under GPRA, grantees must provide certain requested data. GPRA was intended to address several broad purposes, including, among other things, improving federal program effectiveness, accountability, and service delivery; and enhancing congressional decision making by providing more objective information on program performance. [4] GAO, Drug Courts: Better DOJ Data Collection and Evaluation Efforts Needed to Measure Impact of Drug Court Programs, [hyperlink, http://www.gao.gov/products/GAO-02-434] (Washington, D.C.: Apr. 18, 2002). [5] We use the term recidivism to refer generally to the act of committing new criminal offenses after having been arrested or convicted of a crime. [6] GAO, Drug Courts: Evidence Indicates Recidivism Reductions and Mixed Results for Other Outcomes, [hyperlink, http://www.gao.gov/products/GAO-05-219] (Washington, D.C.: Feb. 25, 2005). [7] Fair Sentencing Act of 2010, Pub. L. No. 111-220, § 9, 124 Stat. 2372, 2374-75. [8] Grantees are defined as states, state courts, local courts, units of local government, and Indian tribal governments acting directly or through an agreement with other public or private entities that receive funding under the drug court program. 42 U.S.C. § 3797u(a). [9] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-AIMD-00-21.3.1] (Washington, D.C.: November 1999). [10] GAO, Tax Administration: IRS Needs to Further Refine Its Tax Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143], (Washington, D.C.: November 2002); and GAO, Recovery Act: Department of Justice Could Better Assess Justice Assistance Grant Program Impact, [hyperlink, http://www.gao.gov/products/GAO-11-87] (Washington, D.C.: October 2010). [11] In February 2005, we studied evaluations of drug court programs that were published from May 1997 through January 2004. [12] NIJ is the research, development, and evaluation agency of DOJ. [13] NADCP is a national membership and advocacy organization of drug court professionals that provides for the collection and dissemination of information, technical assistance, and mutual support to association members. [14] 42 U.S.C. §§ 3797u-u-8. Drug courts funded by BJA are required to involve mandatory periodic drug testing, graduated sanctions for participants who fail drug tests, and continuing judicial supervision over offenders, among other requirements. Id. Federal drug court grants have a matching requirement. Drug court grants are not permitted to cover more than 75 percent of the total costs of the project being funded. Grant applicants are required to identify a nonfederal source of 25 percent of the program's cost with cash or in- kind services, or some combination of both. 42 U.S.C. § 3797u-5. [15] This joint program offers grantees the opportunity to design a comprehensive strategy for enhancing drug court capacity while accessing both criminal justice and substance-abuse treatment funds under a single grant application. These grants are authorized under section 509 of the Public Health Service Act, as amended (42 U.S.C. § 290bb-2) to provide Adult Treatment Drug Court grants. [16] The appropriation amounts include adult drug courts, juvenile drug court programs, training and technical assistance, and other related expenses, among other things. [17] For fiscal year 2011, the number of Adult Drug Court Discretionary Grant Program grantee awards and award amounts were not available at the time of our review. [18] The average Adult Drug Court Discretionary Grant Program award amount totals ranged from $122,000 in fiscal year 2006 to $267,000 in fiscal year 2010. [19] [hyperlink, http://www.gao.gov/products/GAO-05-219]. [20] See Jeff Latimer, Jeff, Kelly Morton-Bourgon, and Jo-Anne Chrétien. A Meta-Analytic Examination of Drug Treatment Courts: Do they Reduce Recidivism? (Ottawa, Ontario: Department of Justice Canada, 2006), 12. Christopher T. Lowenkamp, Alexander M. Holsinger, Edward J. Latessa. "Are drug courts effective: A meta-analytic review," Journal of Community Corrections. (Fall 2005), 8, 9, 28. David B. Wilson, Ojmarrh Mitchell, and Doris L. Mackenzie. "A systematic review of drug court effects on recidivism." Journal of Experimental Criminology, 2(4) (2006), 468-469. [21] BJA officials stated that its Policy, Programs, and Planning Offices participate in the GrantStat reviews. [22] According to BJA officials, the contactor provides a range of data collection, technical assistance, analytical, and research services to BJA and its grantees. This includes developing and maintaining the PMT and providing a user support help desk and formal training to grantees regarding their reporting requirements. In addition, contractor analysts review, analyze, and report on BJA grantees' performance data to BJA. [23] See appendix VI for more information regarding the 10 key components. [24] GAO, Managing For Results: Enhancing Agency Use of Performance Information for Management Decision Making, [hyperlink, http://www.gao.gov/products/GAO-05-927] (Washington, D.C.: Sept. 9, 2005). [25] GAO, Justice: A Time Frame for Enhancing Grant Monitoring Documentation and Verification of Data Quality Would Help Improve Accountability and Resource Allocation Decisions, [hyperlink, http://www.gao.gov/products/GAO-09-850R] (Washington, DC: September 2009) and GAO, Performance Measurement and Evaluation: Definitions and Relationships, [hyperlink, http://www.gao.gov/products/GAO-05-739SP] (Washington, DC: May 2005). [26] GAO, Drug Control: DOD Needs to Improve Its Performance Measurement System to Better Manage and Oversee Its Counternarcotics Activities, [hyperlink, http://www.gao.gov/products/GAO-10-835] (Washington, D.C.: July 2010). [27] We identified the first four management activities in table 1 as relevant from governmentwide surveys of federal managers that GAO conducted in 1997, 2000, and 2003. See [hyperlink, http://www.gao.gov/products/GAO-05-927]. The remaining two activities we identified, in part, by reviewing performance management literature. BJA staff confirmed each of these six to be relevant to managing the drug court program. [28] The technical assistance providers included: American University, Tribal Law and Policy Institute, Center for Court Innovation, the National Association of Drug Court Professionals, the Office of Management and Budget, and the National Center for State Courts. [29] BJA reported that DOJ stakeholders consulted included staff from NIJ, OJP, DOJ's Policy, Management, and Planning Branch, DOJ's Chief Financial Officer, the Office of Juvenile Justice and Delinquency Programs, and BJA's Director. [30] GAO, Information Security: Concerted Effort needed to Improve Federal Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-09-617] (Washington, D.C.: September 2009); GAO, Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success, [hyperlink, http://www.gao.gov/products/GAO-03-488] (Washington, D.C.: March 2003); and GAO, Managing for Results: Measuring Program Results That Are Under Limited Federal Control [hyperlink, http://www.gao.gov/products/GAO/GGD-99-16], (Washington, D.C.: December 1998). [31] These attributes are clarity, reliability, linkage to strategic goals, objectivity, and measurable targets. See GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143], (Washington, D.C.: November 2002); and GAO, Recovery Act: Department of Justice Could Better Assess Justice Assistance Grant Program Impact, [hyperlink, http://www.gao.gov/products/GAO-11-87] (Washington, D.C.: October 2010). [32] GAO, Grants Management: Enhancing Performance Accountability Provisions Could Lead to Better Results, [hyperlink, http://www.gao.gov/products/GAO-06-1046] (Washington, D.C.: September 2006). [33] [hyperlink, http://www.gao.gov/products/GAO-09-850R]; GAO, Performance Plans: Selected Approaches for Verification and Validation of Agency Performance Information, [hyperlink, http://www.gao.gov/products/GAO/GGD-99-139] (Washington, D.C.: July, 1999). [34] [hyperlink, http://www.gao.gov/products/GAO-05-927]. [35] [hyperlink, http://www.gao.gov/products/GAO/GGD-99-139]. [36] Theodore H. Poister, Measuring Performance in Public and Nonprofit Organizations. The Jossey-Bass Non-Profit and Public Management Series (San Francisco: Jossey-Bass, 2003). [37] [hyperlink, http://www.gao.gov/products/GAO-AIMD-00-21.3.1]. [38] We report findings to be statistically significant only if they were significant at the 95-percent, or greater, level of statistical significance, even though some studies reported findings to be statistically significant at the 90-percent level. In general, the evaluations we reviewed reported differences in overall rearrest rates--that is, the percentage of a group rearrested for any new offense in a given period of time--although some evaluations reported differences in the number of re-arrests or the relative odds of re- arrest. Of the 32 programs reviewed, 31 showed lower recidivism for drug court program participants, and for 18 of these programs, the differences were statistically significant. The findings for the remaining 13 programs were either not statistically significant or the significance of their findings was not reported. [39] It is important to note that the studies we reviewed did not include treatments other than drug court; for example, they did not measure the relative effectiveness of drug treatment programs administered outside of a drug court. [40] These percentages were adjusted for differences in the baseline characteristics of the individuals in the two groups compared as well as differences in the baseline characteristics of the programs they were in. [41] The range of percentage differences for re-arrest rates was narrower for higher quality studies as a group than for lower quality studies, and the differences for higher quality studies did not range as high. [42] We are reporting on the eight programs for which drug -relapse data from drug court participants were compared with a comparison group. Evaluations of other programs included information on drug- relapse only for drug court participants. [43] The estimate of $6,208 reflects the hierarchical modeling used in the MADCE study. However, according to NIJ officials, the estimated net benefits could be as low as $5,680, under different assumptions. [44] GAO tracks recommendations for implementation and has closed these as either being fully or partially implemented. [45] [hyperlink, http://www.gao.gov/products/GAO-02-434]. [46] The Urban Institute is a nonpartisan economic and social policy research organization. The Center for Court Innovation functions as the independent research and development arm for the New York State court system and provides criminal justice consulting services to jurisdictions outside New York. The Research Triangle Institute is an independent, nonprofit institute that provides research, development, and technical services to government and commercial clients. [47] Numbers, percentages, and differences in the foregoing and following bullets, are adjusted (or estimated), as opposed to raw (or observed) numbers, percentages or differences; that is, they were obtained by the MADCE researchers from statistical models that estimated them after adjusting for differences in the baseline characteristics of the individuals in the two groups compared as well as differences in the baseline characteristics of the programs they were in. [48] Propensity score adjustments are a statistical approach to control for baseline differences between the drug court and comparison groups and to correct for attrition and selection biases by effectively giving greater weight to underrepresented categories of offenders and lesser weight to overrepresented categories of offenders. Hierarchical linear models are used to take account of the nesting--or clustering--of participants within the different sites. These statistical adjustments were necessary as a result of baseline differences between specific drug court and comparison groups and the specific individuals in them, and because of the attrition that occurred in both the drug offender and comparison samples over time. [49] The estimate of $6,208 reflects the hierarchical modeling used in the MADCE study. However, according to NIJ officials, the estimated net benefits could be as low as $5,680, under different assumptions. [50] Office of Management and Budget, Circular A-94 Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs (Washington, D.C.: 1992.) p 4. [51] Grantees are defined as states, state courts, local courts, units of local government, and Indian tribal governments acting directly or through an agreement with other public or private entities that receive funding under the drug court program. 42 U.S.C. § 3797u(a). [52] GAO, Standards for Internal Control in the Federal Government, [hyperlink, http://www.gao.gov/products/GAO-AIMD-00-21.3.1] (Washington, D.C.: November, 1999). [53] The management activities include: (1) setting program priorities; (2) allocating resources; (3) adopting new program approaches or changing work processes; (4) identifying and sharing with stakeholders more effective processes and approaches to program implementation; (5) setting expectations for grantees; and (6) monitoring grantee performance. [54] GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, [hyperlink, http://www.gao.gov/products/GAO-03-143], (Washington, D.C.: November 2002); and GAO, Recovery Act: Department of Justice Could Better Assess Justice Assistance Grant Program Impact, [hyperlink, http://www.gao.gov/products/GAO-11-87] (Washington, D.C.: October 2010). [55] In February 2005, we studied evaluations of drug court programs that were published from May 1997 through January 2004. [56] We searched the ERIC, Biosis Previews, Social Scisearch, Gale Group Magazine Database, Gale Group Health & Wellness Database, Gale Group Legal Resource Index, Wilson Social Science Abstracts, and Periodical Abstracts PlusText. [57] Prior GAO reports included, GAO, Drug Courts: Information on a New Approach to Address Drug-Related Crime, [hyperlink, http://www.gao.gov/products/GAO/GGD-95-159BR] (Washington, D.C.: May 22, 1995); GAO, Drug Courts: Overview of Growth, Characteristics, and Results, [hyperlink, http://www.gao.gov/products/GAO/GGD-97-106] (Washington, D.C.: July 31, 1997); and GAO, Drug Courts: Better DOJ Data Collection and Evaluation Efforts Needed to Measure Impact of Drug Court Programs, [hyperlink, http://www.gao.gov/products/GAO-02-434] (Washington, D.C.: Apr. 18, 2002); GAO, Adult Drug Courts: Evidence Indicates Recidivism Reductions and Mixed Results for Other Outcomes, [hyperlink, http://www.gao.gov/products/GAO-05-219] (Washington, D.C.: Feb. 28, 2005). [58] A process evaluation assesses the extent to which a program is operating as it was intended. It typically assesses program activities' conformance to statutory and regulatory requirements, program design, and professional standards or customer expectations. [59] An experimental design is one in which eligible offenders were randomly assigned to different programs. A quasi-experimental design is one in which (1) all drug-court program participants were compared with an appropriate group of comparable offenders who did not participate in the drug court program, and (2) appropriate statistical methods were used to adjust, or control, for group differences. [60] Some studies reported results that were aggregated from multiple drug court programs. [61] For a summary of how drug court studies have addressed selection bias in the past, see [hyperlink, http://www.gao.gov/products/GAO-05-219], p 16-24. [62] [hyperlink, http://www.gao.gov/products/GAO-05-219], p 27. [63] Drug courts funded by BJA are required to involve mandatory periodic drug testing, graduated sanctions for participants who fail drug tests, and continuing judicial supervision over offenders, among other requirements. 42 U.S.C. §§ 3797u-u-8. [End of section] GAO‘s Mission: The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO‘s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO‘s website [hyperlink, http://www.gao.gov]. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e mail you a list of newly posted products, go to [hyperlink, http://www.gao.gov] and select ’E- mail Updates.“ Order by Phone: The price of each GAO publication reflects GAO‘s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO‘s website, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO: Connect with GAO on facebook, flickr, twitter, and YouTube. Subscribe to our RSS Feeds or E mail Updates. Listen to our Podcasts. Visit GAO on the web at [hyperlink, http://www.gao.gov]. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Website: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]; E-mail: fraudnet@gao.gov; Automated answering system: (800) 424-5454 or (202) 512-7470. Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125 Washington, DC 20548. Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548.

The Justia Government Accountability Office site republishes public reports retrieved from the U.S. GAO These reports should not be considered official, and do not necessarily reflect the views of Justia.