Grants Management
Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain Gao ID: GAO-04-245 March 24, 2004The Weed and Seed program, within the Department of Justice's Office of Justice Programs (OJP), aims to prevent and reduce violent crime in targeted neighborhoods, but it cannot optimize its effectiveness without sound management practices. In 1999, GAO made four recommendations to the Executive Office for Weed and Seed (EOWS) to improve the program's management, including (1) developing adequate internal controls to fully document decisions, (2) improving program monitoring, (3) developing criteria for determining when sites have become self-sustaining and when to reduce or withdraw program funding, and (4) developing additional performance measures. GAO did this study to assess progress in implementing these recommendations.
Despite some progress toward addressing GAO's recommendations aimed at improving program management, GAO's review shows that EOWS has not fully implemented the management improvement recommendations GAO made in 1999. First, although EOWS has revised its internal controls to require that significant qualification and funding decisions be documented and readily available in the central grant files for review, EOWS has not always ensured that its policies and procedures were followed, for the grant files GAO reviewed. Second, EOWS reported taking a number of actions intended to improve program monitoring, such as mandating the timely submission of progress reports and adequate recording of site visits as GAO recommended. Nonetheless, GAO found that while EOWS was able to provide such documentation before its review ended, documentation was not available in some of the central grant files GAO reviewed. Thus, the documentation was not readily available for external reviewers, as required by OJP policies and GAO's internal control standards. Third, GAO found that EOWS still lacks fully developed criteria to determine when sites become self-sustaining and when to reduce or withdraw Weed and Seed funds because of the level of sustainability, even though sustainability is a central goal of the program. At the time of GAO's review, no site's funding had been reduced or withdrawn because of sustainability during the 13 years of the program's existence. Fourth, EOWS has not developed outcome performance measures that can be used to adequately track progress toward program outcomes of the Weed and Seed program. While EOWS has initiated studies on how to develop performance measures, at the time of GAO's review, none of these studies had been completed. Without requirements to monitor improvements and assign accountability, progress will be difficult to achieve.
RecommendationsOur recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.
Director: Team: Phone:GAO-04-245, Grants Management: Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain
This is the accessible text file for GAO report number GAO-04-245
entitled 'Grants Management: Despite Efforts to Improve Weed and Seed
Program Management, Challenges Remain' which was released on March 24,
2004.
This text file was formatted by the U.S. General Accounting Office
(GAO) to be accessible to users with visual impairments, as part of a
longer term project to improve GAO products' accessibility. Every
attempt has been made to maintain the structural and data integrity of
the original printed product. Accessibility features, such as text
descriptions of tables, consecutively numbered footnotes placed at the
end of the file, and the text of agency comment letters, are provided
but may not exactly duplicate the presentation or format of the printed
version. The portable document format (PDF) file is an exact electronic
replica of the printed version. We welcome your feedback. Please E-mail
your comments regarding the contents or accessibility features of this
document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to the Chairman, Committee on the Judiciary, House of
Representatives:
United States General Accounting Office:
GAO:
March 2004:
GRANTS MANAGEMENT:
Despite Efforts to Improve Weed and Seed Program Management, Challenges
Remain:
GAO-04-245:
GAO Highlights:
Highlights of GAO-04-245, a report to the Chairman, Committee on the
Judiciary, House of Representatives
Why GAO Did This Study:
The Weed and Seed program, within the Department of Justice‘s Office of
Justice Programs (OJP), aims to prevent and reduce violent crime in
targeted neighborhoods, but it cannot optimize its effectiveness
without sound management practices. In 1999, GAO made four
recommendations to the Executive Office for Weed and Seed (EOWS) to
improve the program‘s management, including (1) developing adequate
internal controls to fully document decisions, (2) improving program
monitoring, (3) developing criteria for determining when sites have
become self-sustaining and when to reduce or withdraw program funding,
and (4) developing additional performance measures. GAO did this study
to assess progress in implementing these recommendations.
What GAO Found:
Despite some progress toward addressing GAO‘s recommendations aimed at
improving program management, GAO‘s review shows that EOWS has not
fully implemented the management improvement recommendations GAO made
in 1999. First, although EOWS has revised its internal controls to
require that significant qualification and funding decisions be
documented and readily available in the central grant files for review,
EOWS has not always ensured that its policies and procedures were
followed, for the grant files GAO reviewed. Second, EOWS reported
taking a number of actions intended to improve program monitoring, such
as mandating the timely submission of progress reports and adequate
recording of site visits as GAO recommended. Nonetheless, GAO found
that while EOWS was able to provide such documentation before its
review ended, documentation was not available in some of the central
grant files GAO reviewed. Thus, the documentation was not readily
available for external reviewers, as required by OJP policies and GAO‘s
internal control standards. Third, GAO found that EOWS still lacks
fully developed criteria to determine when sites become self-sustaining
and when to reduce or withdraw Weed and Seed funds because of the level
of sustainability, even though sustainability is a central goal of the
program. At the time of GAO‘s review, no site‘s funding had been
reduced or withdrawn because of sustainability during the 13 years of
the program‘s existence. Fourth, EOWS has not developed outcome
performance measures that can be used to adequately track progress
toward program outcomes of the Weed and Seed program. While EOWS has
initiated studies on how to develop performance measures, at the time
of GAO‘s review, none of these studies had been completed. Without
requirements to monitor improvements and assign accountability,
progress will be difficult to achieve.
What GAO Recommends:
GAO recommends that the Attorney General require the Assistant Attorney
General for OJP to ensure that the Executive Office for Weed and Seed
fully implement the intent of GAO‘s previous recommendations.
In commenting on a draft of this report, the Department of Justice
agreed to strengthen controls on maintaining documentation and take
further steps to define and apply criteria for self-sustainability.
Justice believes that the studies currently under way will help develop
outcome measures.
www.gao.gov/cgi-bin/getrpt?GAO-04-245.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Laurie Ekstrand at (202)
512-8777 or Ekstrandl@gao.gov.
[End of section]
Contents:
Letter:
Results in Brief:
Background:
Internal Controls Have Been Developed, but Challenges Remain:
EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but
Challenges Remain:
Despite Some Steps, Challenges Remain in Developing Criteria for Self-
Sustainability:
Performance Measures Generally Did Not Track Program Outcomes:
Conclusions:
Recommendations for Executive Action:
Agency Comments and Our Evaluation:
Appendix I: Scope and Methodology:
Appendix II: GAO Site Visit Summary:
Appendix III: Proposed Legislation Could Affect Self-Sustainability:
Appendix IV: Executive Office for Weed and Seed Selected Activities:
Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003:
Appendix VI: Description of EOWS's Ongoing and Completed Studies:
Appendix VII: Comments from the Department of Justice:
Appendix VIII: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Staff Acknowledgments:
Related GAO Products:
Tables:
Table 1: EOWS Funding History for Fiscal Years 1999-2003:
Table 2: Sites' Weed and Seed Funding History--Fiscal Years 1999-2003:
Table 3: Examples of Activities Funded by Weed and Seed in Pittsburgh,
Charleston, and North Charleston:
Table 4: Examples of Leveraging Efforts at Weed and Seed Sites in
Pittsburgh, Charleston, and North Charleston:
Abbreviations:
AAG: Assistant Attorney General:
COPS: Community Oriented Policing Services:
DFCSP: Drug-Free Communities Support Program:
EOWS: Executive Office for Weed and Seed:
FBI: Federal Bureau of Investigation:
FY: fiscal year:
GMS: grant management system:
GPRA: Government Performance and Results Act:
HUD: Housing and Urban Development:
JRSA: Justice Research and Statistics Association:
OJP: Office of Justice Programs:
OMB: Office of Management and Budget:
OR: official recognition:
United States General Accounting Office:
Washington, DC 20548:
March 24, 2004:
The Honorable F. James Sensenbrenner, Jr.:
Chairman:
Committee on the Judiciary:
House of Representatives:
Dear Mr. Chairman:
The Department of Justice's (Justice) Weed and Seed program proposes to
"weed" out crime from targeted neighborhoods and "seed" them with a
variety of programs and resources to prevent crime from recurring. Weed
and Seed sites have sponsored activities such as police bike patrols,
community cleanups, youth and recreational activities, and computer
training. According to Justice, a central tenet of the Weed and Seed
program is for local Weed and Seed sites to develop partnerships with
other federal, state, and local governments and private sector agencies
to leverage federal Weed and Seed grant funds with additional resources
from these partners to promote weeding and seeding activities. These
additional resources are intended to help the sites achieve the goal of
becoming self-sustaining after Weed and Seed funding ends and to
sustain crime reduction and community revitalization activities to
ensure stable communities.
This report responds to your request to assess the Executive Office for
Weed and Seed's (EOWS) efforts to implement the management improvement
recommendations we made in 1999.[Footnote 1] It is not intended to
evaluate the overall management or results of the program. In our
previous report on the Weed and Seed program, we recommended that EOWS
improve its management of the Weed and Seed program. Specifically, we
recommended that EOWS (1) develop adequate internal controls to ensure
that the basis and the rationale for new and existing site
qualification and funding decisions are always fully documented, (2)
improve program monitoring to ensure that sites meet the grant
requirement of submitting progress reports and that EOWS site visits
are documented, (3) develop criteria for determining when sites are
self-sustaining and when to reduce or withdraw program funding, and (4)
develop additional performance measures that track program outcomes.
This report summarizes our assessment of EOWS's efforts to address our
recommendations.
To obtain information on EOWS's efforts, we reviewed relevant
documents, including EOWS's policies and procedures, monitoring
documentation, and agency staffing and budget data. We reviewed GAO,
Congressional Research Service, and Justice's Office of Inspector
General reports. In addition, we interviewed officials from EOWS, the
Office of Justice Programs, other entities in the Justice Department,
and researchers in performance measurement and evaluation at the
Justice Research and Statistics Association and the Urban Institute in
Washington, D.C. We visited and interviewed program staff at three Weed
and Seed sites. We reviewed 30 randomly selected Weed and Seed grant
files to identify the actions taken by EOWS for those grants to
document its qualification and funding decisions and the steps EOWS
took to ensure that grant documentation requirements were met. Since
the files we reviewed were not representative of all EOWS grant files,
we cannot project the results to the larger population. However, the
information helps to identify the level of implementation of our
recommendations for those grants. Additional information about the
report's scope and methodology is presented in appendix I. We conducted
this engagement in accordance with generally accepted government
auditing standards.
Results in Brief:
Despite some progress toward addressing our recommendations aimed at
improving program management, our review shows that EOWS has not fully
implemented any of the management improvement recommendations we made
in 1999. First, EOWS has revised its internal controls to require that
significant qualification and funding decisions be documented and
readily available in two types of centralized files: official
recognition and official grant files. However, EOWS has not always
ensured that the documentation was readily available and that its
policies and procedures for internal controls were followed for the
official recognition and official grant files we reviewed. Second, EOWS
reported to us that it had taken a number of actions intended to
improve program monitoring, such as mandating the timely submission of
progress reports and adequate recording of site visits, as we
recommended. Nonetheless, we found that while EOWS was able to provide
such documentation before our review ended, documentation was not
available in some of the central grant files we reviewed. Thus, the
documentation was not readily available for EOWS management or external
reviewers, as required by the Office of Justice Programs (OJP) policies
and GAO's internal control standards. Third, we found that EOWS still
lacks fully developed criteria we recommended that they develop to
determine when sites have become self-sustaining and when to reduce or
withdraw Weed and Seed funds because of the level of sustainability,
even though sustainability is a central goal of the program. At the
time of our review, no site's funding had been reduced or withdrawn
because of sustainability during the 13 years of the program's
existence. Fourth, EOWS began developing additional performance
measures to better assess how well sites are meeting program
objectives, as we recommended in 1999. However, our work showed that
although EOWS collected data on a variety of activities taking place at
Weed and Seed sites, they generally did not measure the extent to which
grantees were weeding crime from neighborhoods and preventing it from
recurring. While EOWS has initiated studies on how to develop
additional performance measures, at the time of our review, none of
these studies had been completed.
To further improve program management, we make a recommendation to the
Attorney General for four actions to help ensure full implementation of
the recommendations we made in our 1999 report. In commenting on a
draft of this report, Justice partially agreed with our recommendations
and mentioned actions being taken or planned in response.
Background:
The Weed and Seed program is a Justice discretionary grant program
within OJP.[Footnote 2] The Assistant Attorney General for OJP is
responsible for the overall management and oversight of offices within
OJP, including EOWS. The Assistant Attorney General for OJP sets
policies, promotes coordination among OJP bureaus and offices, and
ensures that EOWS follows its policies and procedures. EOWS provides
funding to grantees to help prevent and control crime and improve the
quality of life in targeted high-crime neighborhoods across the
country.[Footnote 3] It is a joint federal, state, and local program
for coordinated law enforcement and neighborhood reinvestment. Federal
program funding is to support Weed and Seed sites and to provide
training and technical assistance.[Footnote 4]
The Weed and Seed program has grown since it began in fiscal year 1991
with three pilot sites in Kansas City, Missouri; Omaha, Nebraska; and
Trenton, New Jersey, and a relatively small investment of federal
money. From fiscal year 1999 through 2003, the number of Weed and Seed
sites increased from 163 to 221, while the total annual program budget
generally increased from about $45 million to $70 million. See table 1
for fiscal years 1999-2003 data on the Weed and Seed program, including
EOWS's funding history. In fiscal year 2003, with a budget of about $70
million, EOWS awarded grants to 221 Weed and Seed sites.
Table 1: EOWS Funding History for Fiscal Years 1999-2003:
Fiscal year: 1999;
Unobligated balance carried forward from prior year: $4,122,668;
Prior year recoveries and other reimbursable adjustments: $714,466;
EOWS appropriation: $33,500,000;
Funds from the asset forfeiture fund[A]: $6,500,000;
Funds from the Housing and Urban Development (HUD)[B]: $0;
Total funding available: $44,837,134;
Number of funded sites: 163.
Fiscal year: 2000;
Unobligated balance carried forward from prior year: $4,129,852;
Prior year recoveries and other reimbursable adjustments: $2,066,682;
EOWS appropriation: $33,500,000;
Funds from the asset forfeiture fund[A]: $6,500,000;
Funds from the Housing and Urban Development (HUD)[B]: $0;
Total funding available: $46,196,533[C];
Number of funded sites: 177.
Fiscal year: 2001;
Unobligated balance carried forward from prior year: $4,955,423;
Prior year recoveries and other reimbursable adjustments: $1,269,708;
EOWS appropriation: $33,925,200;
Funds from the asset forfeiture fund[A]: $15,500,000;
Funds from the Housing and Urban Development (HUD)[B]: $0;
Total funding available: $55,650,331;
Number of funded sites: 212.
Fiscal year: 2002;
Unobligated balance carried forward from prior year: $584,278;
Prior year recoveries and other reimbursable adjustments: $2,359,292;
EOWS appropriation: $58,918,000;
Funds from the asset forfeiture fund[A]: $0;
Funds from the Housing and Urban Development (HUD)[B]: $10,000,000;
Total funding available: $71,861,570;
Number of funded sites: 218.
Fiscal year: 2003;
Unobligated balance carried forward from prior year: $4,102,334;
Prior year recoveries and other reimbursable adjustments: $2,713,913;
EOWS appropriation: $58,542,000;
Funds from the asset forfeiture fund[A]: $0;
Funds from the Housing and Urban Development (HUD)[B]: $4,935,000;
Total funding available: $70,293,247;
Number of funded sites: 221.
Source: OJP's Office of Budget and Management Services data. GAO did
not verify the data.
Note: For EOWS funding history prior to 1999, see GAO/GGD-99-110.
[A] The proceeds from the asset forfeitures fund were used for federal
and state law enforcement purposes. EOWS received these funds from
Justice, under a reimbursable agreement, for the payment of various
costs incurred by state and local law enforcement officers that
participated in joint federal law enforcement operations with federal
agencies.
[B] HUD made $10 million in additional funding available, under a
reimbursable agreement, to help Weed and Seed sites reduce drug-related
crimes in public housing.
[C] Does not add because of rounding.
[End of table]
EOWS is responsible for the national management and administration of
the Weed and Seed program, including developing policy and providing
guidance and oversight. EOWS currently administers the Weed and Seed
program with a staff of 2 management officials, 10 grant monitors, 3
support staff, 3 detailees,[Footnote 5] and 6 contractors.
Before a community can apply for Weed and Seed grant funding and become
an eligible site, it must first apply for and gain official recognition
from EOWS. In order to obtain official recognition, a potential Weed
and Seed site must show in its application that it has a strategy for
weeding and preventing crime. Once the application has been received,
EOWS creates and maintains the official recognition files that are
intended to include documentation such as the rationale for decisions
to grant or deny official recognition to an applicant. Official
recognition requires the U.S. Attorney in the area where the Weed and
Seed site is to be located to organize a local steering committee made
up of various federal, state, and local representatives, including
residents, to be responsible for local administration of the
program.[Footnote 6] For official recognition, a site is also required
to develop a management plan, engage residents and other partners in
its activities, and develop a comprehensive program to weed out crime
and gang activity and seed the area with social services, economic
services, and economic revitalization.
Weed and Seed program guidance requires that its sites show plans for
addressing four required elements: (1) law enforcement; (2) community
policing; (3) crime prevention and intervention, and substance abuse
prevention, intervention, and treatment; and (4) neighborhood
restoration. According to EOWS, law enforcement should attempt to
eliminate the most violent offenders by coordinating and integrating
the efforts of federal, state, and local law enforcement agencies in
targeted high-crime neighborhoods. The objective of community policing
is to raise the level of citizen and community involvement in crime
prevention and intervention activities. Crime and substance abuse
prevention, intervention, and treatment should include youth services,
school programs, community and social programs, and support groups.
Finally, neighborhood restoration should focus on assistance to
distressed neighborhoods through economic and housing development. Weed
and Seed sites fund a variety of law enforcement and community
activities. For example, law enforcement-funded activities range from
participation in a multijurisdictional task force to conducting bike
patrols in the community. See appendix II for additional information on
Weed and Seed activities at the sites GAO visited.
Applicants that are officially recognized are eligible to apply for
funding for up to 5 years. OJP's Office of the Comptroller creates and
maintains official grant files for sites that are awarded funding by
EOWS. Applicants are rated based on the strategy they developed in
response to the four required elements, as stated above.
In our 1999 report, we made four recommendations to help EOWS improve
program monitoring and management. The first was to develop adequate
internal controls to fully document significant qualification and
funding decisions. We found that EOWS lacked internal controls
requiring significant program management decisions be documented.
Therefore, EOWS was not able to ensure that it was making the best
decisions about allocating available funds. Our second recommendation
was to improve program monitoring to ensure that sites met the grant
requirement of submitting progress reports and that EOWS site visits
were documented. We found that EOWS did not always ensure that local
sites submitted progress reports and that grant monitors documented the
results of site visits. Such documentation would help EOWS management
and grant monitors determine how sites are meeting program objectives
and how well sites are complying with grant requirements, and assist
them in making future grant qualification decisions. Our third
recommendation was to develop criteria for determining when sites have
become self-sustaining and when to reduce or withdraw program funding.
We found that although self-sustainability is central to the program,
no site's funding had been reduced or withdrawn as a result of its
efforts to become self-sustaining in the 9 years of the program's
existence. Establishing such criteria is important because the Weed
Seed program was founded on the premise that federal funding would
continue for a finite period, after which a Weed and Seed site would be
self-sustaining. Our fourth recommendation was to develop additional
performance measures to track program outcomes. We found that EOWS's
performance indicators generally tracked activities rather than program
results; therefore, EOWS was not able to measure the success of the
program.
EOWS generally agreed with three of the four recommendations presented
in the report and discussed future actions it planned to take. EOWS
officials disagreed with our recommendation on self-sustainability.
They stated that developing criteria to ascertain self-sustainment is
redundant since EOWS adopted a 5-year rule under which it could
discontinue awarding funding to qualifying sites unless the sites
expanded to an additional neighborhood site.
Internal Controls Have Been Developed, but Challenges Remain:
Although EOWS has developed internal controls intended to require that
significant qualification and funding decisions be documented and
readily available for review as we recommended in 1999, these policies
and procedures are generally not being followed in the files we
reviewed. In response to our 1999 recommendation on internal controls,
EOWS established policies and procedures intended to ensure that
significant qualification decisions were documented. In addition, OJP
requires that qualification and funding decisions are to be documented
in official recognition and official grant files. Our review of 20
official recognition and 10 official grant files showed that some of
EOWS's official recognition and official grant files were missing full
documentation regarding the qualification and funding
decisions.[Footnote 7] However, before our file review ended, EOWS
officials produced further documentation, which they acknowledged was
not in the official recognition and official grant files as required,
but rather in the personal working files of grant monitors and thus not
readily available to EOWS management and external reviewers. This lack
of ready availability is not in keeping with EOWS's and OJP's policies
and procedures, or with the Comptroller General's standards for
internal controls.[Footnote 8] Without having official recognition and
official grant files complete and readily available, it may delay and
complicate EOWS officials' oversight of the documentation of
qualification and funding decisions.
Policies and Procedures Have Been Developed to Document Significant
Decisions:
Both EOWS and OJP policies and procedures have been developed to help
ensure the documentation of significant decisions. In 2000, EOWS
developed a policies and procedures guide in response to the
recommendation we made in our 1999 report intended to ensure that
significant qualification and funding decisions for new and existing
sites[Footnote 9] were always fully documented, and further revised the
guide in 2003. This guide requires that all documentation pertaining to
official recognition decisions be kept in the official recognition
files. EOWS is also required to follow the policies in the OJP Grants
Management Policies and Procedures Manual, which requires that EOWS
fully document program management decisions in both official
recognition files and official grant files. The OJP manual specifically
requires that the official recognition files and official grant files
contain key documents such as ratings information, rejection letters,
and applications.
Official Recognition and Official Grant Files Were Incomplete:
Despite EOWS's and OJP's efforts to require the documentation of
qualification and funding decisions in official recognition and
official grant files, some of EOWS's official recognition and official
grant files we reviewed were incomplete at the time of our file review.
In reviewing the official recognition files, we randomly selected 10
files to review from the 94 files submitted for official recognition in
fiscal year 2002. Additional information about how we selected the
files is in appendix I. We found sufficient documentation in 7 of the
10 files to determine the basis and rationale for decisions to award
official recognition. However, in the remaining 3 files we were unable
to determine the basis and rationale for such decisions because
documentation was insufficient. For example, key documents such as the
ratings information that EOWS grant monitors use to record their
assessment of the official recognition applications did not contain the
basis and rationale for the award decision.[Footnote 10] Additionally,
OJP's grant policies and procedures require EOWS's grant monitors to
prepare a rejection letter, informing applicants of reasons for
rejection when funding is denied and place a copy of the rejection
letter in the official recognition file. We reviewed all 10 official
recognition files for the applicants that were eligible to apply for
funding in fiscal year 2002 but were rejected. We found a rejection
letter was missing in 7 of the 10 files.
We also reviewed documents in EOWS's official grant files to assess
EOWS's efforts to comply with EOWS and OJP policies for fully
documenting funding decisions. We randomly selected 10 official grant
files to review from the 31 sites that were funded in fiscal year 2002.
We found that none of the 10 official grant files fully documented
funding decisions for Weed and Seed sites. For example, the
application, a basic component of the official grant file, was missing
in 8 of the 10 files.
Challenges Persist in Making Documentation Readily Available:
Several directives require that grant documentation be readily
available. OJP's policies and procedures manual requires that
documentation be kept readily available in the official recognition and
official grant files, so that OJP and EOWS management can identify and
resolve any problems or deficiencies in grantees' compliance with
relevant policies and procedures. OJP's policies and procedures manual
requires that all pertinent information that should be in the official
recognition and the official grant files be kept in a centralized
location to facilitate reviewing for completeness. In addition, the
Comptroller General's guidance on internal controls in the federal
government, Standards for Internal Control in the Federal Government,
requires that the documentation for all transactions and significant
events be readily available for examination. The Assistant Attorney
General for OJP is responsible for holding EOWS accountable for
following its policies and procedures. In reference to the ratings
information and rejection letters missing from the official recognition
files, EOWS produced the documents after our initial review, informing
us that the documents were kept in the personal files of the grant
monitors--rather than the official recognition files--a practice not in
keeping with EOWS's and OJP's policies and procedures. In reference to
the missing applications, EOWS officials also produced them after our
initial review, but did not specify where they found them. Not having
complete and readily available official recognition and official grant
files may delay and make it more difficult for OJP and EOWS officials
to perform management functions such as overseeing and assessing the
documentation and oversight of qualification and funding decisions.
While we eventually were able to obtain documentation showing the basis
and rationale for EOWS decision making, the incomplete official
recognition and official grant files made the review difficult and
time-consuming.
In responding to a draft of this report, Justice stated that electronic
documents maintained in its grant management system (GMS) are
considered a part of the official recognition and official grant files
and are readily available to EOWS, and that they should have been
readily available to, and easily accessible by, GAO.
EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but
Challenges Remain:
In response to our 1999 recommendation, EOWS has taken steps to improve
program monitoring, as with the documentation of qualification and
funding decisions, but challenges remain in making the monitoring
documentation readily available. While OJP requires progress reports
and site visit reports to be included in the official grant files, our
review in 2003 of 10 official grant files showed that some of EOWS's
files were missing full documentation. However, before our review
ended, EOWS provided further documentation that fully documented
progress reports and site visits, which it acknowledges was not in the
official grant files, and thus not readily available. This lack of
readily available documentation is not in keeping with OJP's policies
and procedures. Not having complete and readily available official
grant files may delay and complicate EOWS officials' and external
reviewers' assessment of whether EOWS's monitoring requirements are
being followed.
Official Grant Files Were Incomplete:
Despite EOWS's efforts, some of the official grant files we reviewed
were incomplete at the time of our file review. We randomly selected 10
files from the 31 sites that were first funded in fiscal year
2002[Footnote 11] to determine whether the sites had submitted the
required progress reports from January 1, 2003, through June 30,
2003.[Footnote 12] Of the 10 files we reviewed, 6 did not contain any
of the required progress reports for the period. The remaining 4 files
included the required progress reports. OJP's policies and procedures
require EOWS to ensure that progress reports are included in the
official grant files. However, when we asked EOWS about the missing
progress reports, EOWS provided us with the requested documents. We
asked, but EOWS did not specify where it found the missing
documentation.
In addition, EOWS's grant monitors are required to conduct site visits
a minimum of every 18 months and document their visits. This
documentation is meant to convey to EOWS management officials how well
sites are complying with grant requirements and is to be used by EOWS
in making funding decisions. Grant monitors are to prepare a report of
the visits and forward a copy to the Office of the Comptroller for
inclusion in the official grant file. We reviewed the same 10 grant
files discussed above to determine whether site visits were fully
documented. We found that a site visit report was not yet due for 7 of
the 10 files. In the remaining 3 files where a site visit was due, none
of the 3 files contained documentation that the site visit had been
conducted. We requested to speak to the responsible grant monitors to
determine whether site visits had been conducted. EOWS officials told
us that the monitors were no longer involved with the program but
provided additional documentation for the 3 files we reviewed. The
documentation showed that of the 3 sites, only 1 visit had been
conducted; the other 2 were scheduled but had been delayed. While we
only reviewed 3 files where a site visit was due, the lack of
documentation in all 3 files, as well as the fact that only 1 of 3 site
visits had been conducted, added to our concerns about the completeness
of official grant files for program oversight.
Challenges Persist in Making Monitoring Documentation Readily
Available:
As we have previously mentioned, OJP's policies and procedures manual
and the Comptroller General's guidance on internal controls require
that documentation should be kept readily available for examination.
Subsequent to our finding missing progress reports and site visit
reports, EOWS provided us with additional documentation that
demonstrated that the progress reports and one of the site visit
reports had been completed. When asked about where they found the
missing progress reports and site visit reports, EOWS officials did not
specify where they found the missing documentation.
Despite Some Steps, Challenges Remain in Developing Criteria for Self-
Sustainability:
EOWS has established a rule and set some activities for sites to
complete to encourage them to become self-sustaining. However, it has
still not fully developed criteria to determine when sites have become
self-sustaining and when to reduce or withdraw Weed and Seed grant
funds, as we recommended in 1999. To become self-sustaining, sites must
leverage additional resources from sources other than EOWS to sustain
the Weed and Seed program without EOWS funds. We found that EOWS has
not reduced or withdrawn funds from any Weed and Seed sites for reasons
related to becoming self-sustaining. This is important because without
overall program funding increases, new sites cannot be funded unless
funding can be reduced or discontinued from sites that have achieved
self-sustainability.
EOWS Encourages Self-Sustainability, yet Needs Criteria to Make Funding
Decisions:
In 1999, EOWS established a rule to encourage self-sustainability that
was generally intended to limit Weed and Seed funding to a site to 5
calendar years and require grantees to shift the majority of EOWS funds
to a different site after 5 years.[Footnote 13] According to EOWS
officials, EOWS also encouraged sites to become self-sustaining by
establishing activities to be completed during each year of the 5-year
grant.[Footnote 14] Program monitors are to assess sites' completion of
these activities by recording results on a checklist.
We found that while EOWS undertook efforts to encourage self-
sustainability, it has not fully developed criteria to make funding
decisions based on whether sites had achieved self-sustainability. In
fiscal year 2002, 73 sites, which had completed 5 years of EOWS
funding, reapplied for funding. EOWS approved an additional round of 5-
year funding to 67 of these sites. EOWS denied funding to 6 sites, but
not for reasons related to self-sustainability. At the time of our
review, no site's funding had been reduced or withdrawn because the
site had achieved self-sustainability, even though EOWS has funded some
sites since the early 1990s. In relation to the activities EOWS set to
encourage sites to achieve self-sustainability, because these
activities were established in 2003, we could not assess the sites'
progress toward completing them. As we reported in 1999, without
criteria, EOWS does not have a basis for determining when sites are
self-sustaining and when to reduce or withdraw Weed and Seed funds.
EOWS officials told us that OJP is currently developing criteria for
self-sustainability for EOWS and other OJP programs and that a report
detailing the criteria may be completed in 2004. According to Justice,
EOWS developed a new criterion for self-sustainability in the FY2003
Competitive Application Kit regarding whether or not a full-time
coordinator is funded by a reallocation of resources other than the
Weed and Seed grant.
For information about proposed legislation that could affect self-
sustainability for Weed and Seed sites, similar to other Justice
programs, see appendix III.
Performance Measures Generally Did Not Track Program Outcomes:
In 1999, we reported[Footnote 15] that while EOWS had developed various
performance measures in an attempt to respond to the Government
Performance and Results Act (GPRA) of 1993,[Footnote 16] EOWS's
measures generally did not track program outcomes.[Footnote 17] GPRA
seeks to shift the focus of federal management and decision making away
from activities performed to outcomes, or the results of activities
undertaken. Since our last report, EOWS has continued to refine and
develop its performance measures, but we found that EOWS still
generally collects site activity data rather than measuring
outcomes.EOWS also generally does not use intermediate measures, which
represent conditions believed to precede or contribute to achieving the
ultimate outcomes that may be considered constructive steps toward
measuring outcomes. While assessing success using outcomes and/or
intermediate measures can be difficult, doing so is important to
program management and to policy makers for understanding whether the
program is achieving the intended results, and for identifying
opportunities for improvement.
EOWS Generally Collects Activity Data Rather Than Outcome Performance
Measures:
Our review showed that EOWS generally collects activity data, but has
not developed outcome or intermediate measures that enable EOWS
management to track the success of the Weed and Seed program in meeting
its goals.[Footnote 18] As established in prior work by GAO, outcome
measures help officials track the success of their programs in meeting
program goals.[Footnote 19] EOWS mostly collects data about site
activities, such as whether sites have foot patrols, safe havens, and
provide job training. An intermediate measure, for example in relation
to job training programs, might be the extent to which those who attend
job training obtain employment. This intermediate measure rests on the
assumption that individuals who are employed are less likely to commit
crimes. See appendix V for activity data collected by EOWS.
EOWS officials told us that to measure its success in reducing violent
crime, they collect data on the number of homicides and consider
homicides the significant indicator for measuring performance outcomes.
However, using the number of homicides as an outcome measure indicative
of program success is problematic for several reasons. First, because
homicides are relatively rare, even in high crime areas, homicide
trends may be too unstable (fluctuate too much from year to year) to
assess the success of the Weed and Seed program. Second, outcome
measures, such as homicides, can present some methodological challenges
because it is difficult to draw a direct causal link between the
homicides and a program's work. Economic trends and other law
enforcement initiatives could also be responsible for the observed
outcomes. If homicides are used as a performance measure, any analysis
should attempt to control for other factors influencing the outcomes.
Third, given the broad nature of this program goal --to reduce violent
crime, measuring outcomes in relation to only one type of violent crime
also seems problematic because it is too narrowly focused. According to
EOWS officials, homicide data was selected because of its reliability,
and because gang-related homicides are a significant indicator of the
success of a Weed and Seed strategy.
In February 2004,the Office of Management and Budget (OMB) issued a
detailed assessment of the Weed and Seed program.[Footnote 20]
According to OMB, Weed and Seed has not demonstrated results. OMB's
report states that Weed and Seed lacks clear targets or timelines for
reducing violent crime. Additionally, the report states that Weed and
Seed has difficulty collecting data and developing aggregate measures
for assessing program performance.
GAO Has Previously Reported on How Other Federal Programs Have
Developed Effective Performance Measures, Despite Challenges:
Although the Weed and Seed program faces many challenges in developing
outcome measures for its various activities, we have previously
reported that other federal programs have developed effective
performance measures even under difficult circumstances.[Footnote 21]
These reports identified a variety of strategies that other federal
programs have used to develop performance measures. For example, we
reported that some federal programs utilized a mix of outcome and
intermediate measures.[Footnote 22] This combination of measures
allowed them to minimize the risk of not showing outcomes because of
their limited control over external factors such as economic trends,
which may prevent programs from achieving intended outcomes.
Intermediate measures are also helpful to show progress when it is
expected to take many years before the desired outcome is likely to be
achieved. In such instances, progress toward program outcomes may be
demonstrated through intermediate outcomes. We also reported that where
measuring outcomes is, after careful consideration, deemed infeasible,
intermediate measures can be used to track progress toward outcomes for
programs such as Weed and Seed.[Footnote 23]
In addition, we have reported other ways federal agencies have used
intermediate outcomes.[Footnote 24] For example, the National Highway
Traffic Safety Administration uses the rate of safety belt use as an
intermediate measure of its goal to reduce motor vehicle crashes and
the resulting fatalities and injuries. As we reported, an increase in
the use of safety belts is considered an intermediate outcome--not an
end outcome--because it is desirable not in itself but because it has
been demonstrated to contribute to the ultimate goal--reducing highway-
related fatalities and injuries.[Footnote 25]
Additional Performance Measures Are Being Developed for EOWS:
The Justice Research and Statistics Association and the Urban Institute
are currently developing additional outcome performance measures under
contracts awarded by EOWS in 2002 and 2003. According to EOWS
officials, these research organizations have the expertise needed to
develop outcome performance measures for the Weed and Seed program. We
interviewed researchers from these organizations and spoke to EOWS
officials about their research plans. However, none of the studies on
developing additional performance measures had been completed at the
time of our review, and it is too soon to tell whether the studies will
produce outcome measures needed to adequately assess the Weed and Seed
program.[Footnote 26] The Justice Research Statistics Association and
other researchers were conducting studies on crime data and evaluation
during our review; one of those studies was submitted to EOWS in
November 2003. See appendix VI for a description of EOWS's ongoing and
completed studies.
Conclusions:
We believe that 5 years has been ample time for OJP and EOWS to fully
implement the recommendations we made in our 1999 report. EOWS has
developed and partially implemented policies requiring the completion
of documentation to support major decisions, such as qualification and
funding decisions, and recorded monitoring information. But by failing
to ensure that these documents are appropriately maintained in official
recognition and grant files, the documents are not readily available to
meet their purpose. That is, the intent of this type of internal
control is to ensure that both management and external reviewers such
as auditors can adequately perform their responsibilities, for example,
reviewing work and making management decisions based on complete and
accurate information. While EOWS was eventually able to produce the
documentation we requested, failure to appropriately file the
documentation made the process of using it very inefficient and, in
effect, may have defeated the purpose of having it completed.
While EOWS may have moved forward in addressing its self-sustainability
goals by recently developing an activity checklist for grantees to use
to document actions that are intended to achieve self-sustainability,
the activities do not constitute criteria for determining when sites
should be considered self-sustaining and consequently have federal
funds reduced or discontinued. One of the benefits of such criteria is
to enable EOWS to determine when current projects are likely to be able
to self-sustain so that available funding can be used to help reduce
crime and achieve other benefits in other deserving communities.
Because no sites have had their funding withdrawn because they were
deemed self-sustaining during the 13-year life of this program, EOWS
may be foregoing the opportunity to use the funds in another location
where the need for federal funding is greater or EOWS may need less
overall funding. Without criteria to determine when federal funds are
no longer needed, EOWS also runs the risk of providing funds beyond
what is needed to sustain some sites.
EOWS also needs performance measures that focus on program outcomes,
and/or achievement of intermediate goals, so that it and those that
provide oversight, such as the Congress and OMB, will be able to
adequately assess the extent to which the program is achieving its
goals. Management depends on this type of assessment to make the
strategic and operations decisions needed to achieve the program's
missions and goals. Congress and oversight agencies need this type of
assessment so they can make funding decisions and help ensure that EOWS
is in the best position and has the best tools to accomplish its
mission. While EOWS recognizes the need for outcome performance
measures and has funded studies to help develop them, it is unclear at
this time whether these studies will be able to suggest the needed
measures.
Recommendations for Executive Action:
We recommend that the Attorney General of the United States require the
Assistant Attorney General for OJP to ensure that the Executive Office
for Weed and Seed fully implement the intent of our previous
recommendations by taking the following four steps:
* maintain the documentation of the basis and rationale for
qualification and funding decisions in appropriate grant files;
* retain progress reports and site visit reports in official grant
files;
* clearly define criteria to assess when sites are self-sustaining and
apply the criteria to sites when making further funding decisions; and:
* develop outcome performance measures--or, where measuring outcome is,
after careful consideration, deemed infeasible, intermediate measures-
-that can be used to adequately track progress toward program outcomes
of the Weed and Seed program.
Agency Comments and Our Evaluation:
We provided a draft of this report to the Attorney General of the
United States for review and comment. In a March 11, 2004, letter, the
Assistant Attorney General (AAG) for OJP commented on the draft. Her
written comments are summarized below and presented in their entirety
in appendix VII. Based on Justice's comments, we modified the text of
our report where appropriate and also incorporated Justice's technical
comments into this report where appropriate.
In the AAG's response, the Justice Department generally agreed with our
recommendations on maintaining the documentation of the basis and
rationale for qualification and funding decisions in appropriate grant
files and retaining progress reports and site visit reports in official
grant files. Justice agreed in part with our recommendation on self-
sustainability criteria and fully agreed with our recommendation on
outcome performance measures, and offered additional information to
show it had moved forward in responding to our past recommendation.
Justice agreed that controls should be strengthened to ensure that the
basis and rationale for qualification and funding decisions are
documented in the centrally maintained official recognition (OR) files.
In Justice's comments, the AAG explained that electronic documents
maintained in its grant management system (GMS) are considered a part
of the official recognition file and are readily available to EOWS, and
that they should have been, but were not, readily available to, and
easily accessible by, GAO. As of the current fiscal year, OJP will not
only post all solicitations and receive all grant applications via GMS,
but also receive and maintain all grant progress reports on GMS. In
this way, there will be no opportunity to misplace official grant
documents. Justice stated that OJP's Office of the Comptroller will
implement procedures to ensure that external reviewers have ready
access to GMS information.
Justice agreed with our second recommendation on retaining progress
reports and site visit reports in official grant files. Justice noted
that EOWS recently implemented a progress reporting module in GMS that
now enables recipients of grants awarded through GMS to submit semi
annual progress reports electronically. OJP is developing a monitoring
module that will enable program monitors to record and report on site
visits directly in GMS. In the interim, EOWS and the Office of the
Comptroller will take steps to strengthen controls to ensure that
copies of completed site visit reports are included in the official
grant file.
Justice agreed in part with our third recommendation on self-
sustainability and noted that EOWS will further define the criteria to
assess when sites are self-sustaining and apply the criteria when
making funding decisions. Further, Justice said it is promoting a
"graduation" process to bring sites to sustainability and cease
providing Weed and Seed funding to those sites, thus making funding
available to assist newer developing sites. However, Justice does not
agree that EOWS has never set criteria for self-sustainability. Justice
said that EOWS set a criterion for self-sustainability in the FY2003
Weed and Seed Competitive Application Kit, which asked if the
application provided for a full-time coordinator funded by reallocation
of existing resources other than the Weed and Seed grant. According to
Justice, applications were ranked against this criterion. According to
EOWS officials, in March 2004, EOWS will announce an additional
criterion specific to measuring self-sustainability. The new criterion
will require grant applicants to identify other funding sources at a
level five times the EOWS contribution. This criterion will be
considered when making funding decisions.
We applaud EOWS's commitment to further develop criteria to assess when
sites are self-sustaining and apply the criteria to sites when making
further funding decisions. However, Justice's comments did not provide
specific information about how the Application Kit criterion has been
used to determine self-sustainability. We added Justice's belief that
this criterion relates to self-sustainability to the text, but also
note that over the 13-year history of the program, funding has never
been reduced or withdrawn from a site because a site was deemed to be
self-sustaining.
Justice agrees that developing outcome performance measures is
important and states that EOWS has been working to develop measures
that track progress toward program outcomes in the Weed and Seed
Program. In a letter commenting on this draft report, Justice provided
a summary of results from JRSA's study to support EOWS's use of the
change in homicides as a measure of program success. According to
Justice's summary, this study concluded that measuring reductions of
homicides is an adequate performance measure of the Weed and Seed
program because of its reliability and explicit selection as a goal by
over 30 percent of the Weed and Seed sites participating in the Crime
Pattern Study. In addition, Justice includes a 5 percent reduction in
homicides in Weed and Seed sites as part of its Strategic Plan. With
its comments, Justice included summaries of two recent research
reports. One of these, Analysis of Homicide in Weed and Seed Sites,
showed positive results in homicide trends in Weed and Seed sites when
compared to host jurisdictions. Another report, Crime Pattern Analysis
(Three-Top Crime Study), showed that after the third year of Weed and
Seed program implementation, sites encountered a significant decrease
in reported crimes.
Because the Crime Pattern Analysis study was not completed at the time
of our review, we could not fully assess its methodological rigor and
whether it would result in effective outcome performance measures.
Although we believe that EOWS may be moving in the right direction
regarding developing additional performance measures and we mention the
studies in the report text, we also believe that homicides have
shortcomings as described in the text when used as the only measure of
the program's performance. In addition, our review of the Analysis of
Homicide in Local Weed and Seed Sites study generated several concerns.
First, the study used changes in the raw volume of homicides, instead
of changes in homicide rates, as its indicator of success or failure of
program sites. This method does not account for changes in the
population as would be accounted for were a homicide rate measured.
Second, the study used the larger host jurisdictions for specific Weed
and Seed sites as the comparison locations to which the Weed and Seed
sites are compared. In doing so, Weed and Seed sites' homicide data are
included with host jurisdictions' homicide data, thus making the
differences between them extremely difficult to interpret. It is also
worth noting that this study evaluates Weed and Seed sites only on the
basis of decreases in homicides, while Weed and Seed was intended to
decrease crime more generally.
We are sending copies of this report to the Attorney General,
appropriate congressional committees, and other interested parties. We
will also make copies available to others upon request. In addition,
the report will be available at no charge on GAO's Web site at http://
www.gao.gov. If you or your staff has any questions on this report,
please call Weldon McPhail on (202) 512-8644 or me on (202) 512-8777.
Sincerely yours,
Signed by:
Laurie E. Ekstrand, Director:
Homeland Security and Justice Issues:
[End of section]
Appendix I: Scope and Methodology:
To determine what actions the Executive Office for Weed and Seed (EOWS)
has taken to develop an adequate internal control to ensure full
documentation of the basis and rationale for qualifying new and
existing sites for funding, we reviewed (1) the criteria used to
determine which new and existing sites should be qualified for funding,
and (2) EOWS's policies and guidance. To gather this information, we
interviewed officials from Justice and EOWS and reviewed pertinent
documents, including EOWS policies and procedures, official recognition
and grant applications, management oversight and monitoring
documentation, and budget reports. In addition, we randomly selected 10
of 94 fiscal year 2002 official recognition files submitted to EOWS for
review. These 10 files included 5 files from new applicants and 5 files
from existing sites that applied in 2002 and were funded in fiscal year
2002. We reviewed 10 of the 14 official recognition files that EOWS
decided not to fund in fiscal year 2002. We also reviewed 10 randomly
selected official grant files from the 31 new and existing sites that
were approved for funding in fiscal year 2002. Because of limited
resources, we did not attempt to review all the files. While the small
sample size prevents us from making reliable generalizations, the 30
files we reviewed represented various types of files that EOWS
maintains. We selected the files from a list that was provided to us by
EOWS. To ensure the list of files was sufficient for the purpose of our
review, we spoke to knowledgeable EOWS officials about the completeness
and accuracy of the list.
To assess what steps EOWS has taken to improve program monitoring to
ensure that sites meet the grant requirement of submitting progress
reports, and that EOWS site visits are documented, we reviewed EOWS
program grant guidance, including training offered, the EOWS monitoring
guidance used by grant monitors when conducting site visits, and the
grant files for the Weed and Seed sites that we visited: Pittsburgh,
Pennsylvania; Charleston, South Carolina; and North Charleston, South
Carolina. We selected these sites from the 221 sites funded by EOWS in
fiscal year 2003. Based on our discussions with EOWS officials about
the sites, we selected three sites, which although not representative
of all sites, had received Weed and Seed grant funds since the early
1990s, which enabled us to learn about their efforts to implement the
Weed and Seed strategy over time. We also reviewed selected monitoring
visit reports prepared by grant monitors for these sites and biannual
progress reports submitted in fiscal years 2002 and 2003. We
interviewed EOWS officials and grant coordinators at these 3 sites
regarding procedures used for monitoring Weed and Seed sites. We also
reviewed the same 10 randomly selected official grant files discussed
above from the 31 new and existing sites that were funded in fiscal
year 2002 to determine whether the progress reports and site visit
reports were in the official grant files and readily available. The
sites were funded under EOWS's competitive application and did not
include sites funded under EOWS's continuation application.[Footnote
27] While the 10 files we reviewed represented various types of files
that EOWS maintains, the small sample size prevents us from making
reliable generalizations about all official grant files. To ensure the
list of files was sufficient for the purpose of our review, we spoke to
a knowledgeable official about the completeness and accuracy of the
list.
To assess what criteria, if any, EOWS has developed when sites are
self-sustaining and when to reduce or withdraw program funding, we
asked EOWS officials for any relevant information. Following repeated
requests, EOWS officials did not provide any documentation on criteria
that they had developed. They did, however, provide information on the
five-year rule for self-sustainability and the sites' activities, which
we reviewed.
To determine what additional performance measures EOWS has developed
that track program outcomes, we interviewed officials from EOWS and the
sites we visited. We reviewed pertinent documents, including EOWS
policies and procedures, grant applications, and data collected
pursuant to the Government Performance and Results Act of 1993 (GPRA).
In relation to the studies EOWS commissioned to develop program-wide
performance measures, we interviewed the authors of the studies under
way to discuss their research plans. At the time of our review, none of
the studies on developing additional performance measures had been
completed.
We conducted our audit work between June and December 2003 in
accordance with generally accepted government auditing standards.
[End of section]
Appendix II: GAO Site Visit Summary:
Table 2: Sites' Weed and Seed Funding History--Fiscal Years 1999-2003:
Fiscal year: 1999;
Pittsburgh, Pa., grant funds: $0;
Charleston, S.C., grant funds: $175,000;
North Charleston, S.C., grant funds: $0.
Fiscal year: 2000;
Pittsburgh, Pa., grant funds: $475,000;
Charleston, S.C., grant funds: $125,000;
North Charleston, S.C., grant funds: $175,000.
Fiscal year: 2001;
Pittsburgh, Pa., grant funds: $675,000;
Charleston, S.C., grant funds: $225,000;
North Charleston, S.C., grant funds: $125,000.
Fiscal year: 2002;
Pittsburgh, Pa., grant funds: $825,000[A];
Charleston, S.C., grant funds: $225,000;
North Charleston, S.C., grant funds: $275,000.
Fiscal year: 2003;
Pittsburgh, Pa., grant funds: $435,000[B];
Charleston, S.C., grant funds: $275,000;
North Charleston, S.C., grant funds: $225,000.
Fiscal year: Total;
Pittsburgh, Pa., grant funds: $2,410,000;
Charleston, S.C., grant funds: $1,025,000;
North Charleston, S.C., grant funds: $800,000.
Source: GAO site visits and EOWS data.
[A] Pittsburgh's Weed and Seed funds for fiscal year 2002 were amended
into the 2001 grant award.
[B] Pittsburgh's Weed and Seed funds for fiscal year 2003 were amended
into the 2001 grant award.
[End of table]
Pittsburgh, Pa.
Background:
Pittsburgh established its first Weed and Seed site, the Hill District,
in 1992. In 1995, Hazelwood became the second Pittsburgh site. In 1997
and 1999, East Liberty and Homewood became the third and fourth Weed
and Seed sites in Pittsburgh. Pittsburgh currently receives funds for
three sites: East Liberty, Hazelwood, and Homewood. The sites are
located between the Allegheny and Monongahela rivers on the eastern
side of the city. The population of the target area is 10,897 in East
Liberty, 15,849 in Hazelwood, and 9,283 in Homewood. In fiscal year
2003, the Pittsburgh sites in total received $435,000 in Weed and Seed
funding. See table 2 for the Weed and Seed funding history for
Pittsburgh's three sites.
Activities:
Pittsburgh's weeding activities include funding a drug violence
initiative headed by the Federal Bureau of Investigation (FBI) and a
community-based crime prevention activity coordinated with the police.
Pittsburgh's seeding activities include funding a drug education youth
camp that provides children with leadership training and fitness tests,
and a downtown revitalization program. See table 3 for examples of the
types of activities funded by the Pittsburgh Weed and Seed program.
Leveraging Efforts:
An important goal of the Weed and Seed program is for sites to leverage
resources from sources other than EOWS in order to become self-
sustaining. During our site visit and through documentation obtained
from Pittsburgh Weed and Seed program staff, we identified several
partnerships established by the Pittsburgh Weed and Seed program to
leverage resources. These cooperative arrangements involved partners
such as the University of Pittsburgh Medical Center and the Pittsburgh
Harlequin Rugby Association. Table 4 provides specific examples of
leveraging efforts that we identified.[Footnote 28] Pittsburgh Weed and
Seed program staff estimate that they coordinated and leveraged
approximately $1.1 million in 2002.
Charleston, S.C.
Background:
Charleston became a Weed and Seed site in the spring of 1992. The
Charleston Weed and Seed target area includes approximately 8 square
miles on the Charleston Peninsula and in the city of Charleston. The
entire Weed and Seed site is located in Charleston's Renewal Community.
The population of the target area, encompassing 13 neighborhoods, is
roughly 17,000. In fiscal year 2003, the site received $275,000 in Weed
and Seed funding. See table 2 for Charleston's Weed and Seed funding
history.
Activities:
Charleston's weeding activities include supporting a youth court with a
Weed and Seed juvenile investigator and funding community policing foot
patrols. Charleston's seeding activities include supporting a police-
youth after-school boxing program, and boarding up abandoned and vacant
buildings to improve the appearance of the area. See table 3 for
examples of the types of activities funded by the Charleston Weed and
Seed program.
Leveraging Efforts:
An important goal of the Weed and Seed program is for sites to leverage
resources from sources other than EOWS in order to become self-
sustaining. During our site visit and through documentation obtained
from Charleston, we identified several partnerships established by the
Charleston Weed and Seed program to leverage additional resources.
These cooperative arrangements involved partners such as the FBI and
the College of Charleston's School of Education. Table 4 provides
specific examples of leveraging efforts that we identified.[Footnote
29] Charleston Weed and Seed program staff estimate that they were able
to coordinate and leverage approximately $1 million in 2002.
North Charleston, S.C.
Background:
North Charleston was originally funded as a Weed and Seed site in
August 1993. The city of North Charleston is located about 7 miles
north of the city of Charleston. Today, North Charleston has four
target neighborhoods: Union Heights, Chicora/Cherokee, Accabee, and
Liberty Hill. The population of the target area is 13,606. In fiscal
year 2003, the site received $225,000 in Weed and Seed funding. See
table 2 for North Charleston's Weed and Seed funding history.
Activities:
North Charleston's weeding activities include participating in
Operation Cease Fire with the U.S. Attorney's Office, to reduce gun
crime in target neighborhoods, and partnering with local law
enforcement to improve the bike patrol program. North Charleston's
seeding activities include developing the Drug Education for Youth
program and supporting an annual neighborhood cleanup. See table 3 for
examples of the types of activities funded by the North Charleston Weed
and Seed program.
Leveraging Efforts:
An important goal of the Weed and Seed program is for sites to leverage
resources from sources other than EOWS in order to become self-
sustaining. During our site visit and through documentation obtained
from North Charleston, we identified several partnerships established
by the North Charleston Weed and Seed program to leverage resources.
These cooperative arrangements involved partners such as the United
States Air Force and the Medical University of South Carolina. Table 4
provides specific examples of leveraging efforts that we
identified.[Footnote 30] North Charleston Weed and Seed program staff
estimate that they coordinated and leveraged approximately $3.3 million
in 2002.
Table 3: Examples of Activities Funded by Weed and Seed in Pittsburgh,
Charleston, and North Charleston:
Site: Pittsburgh, Pa;
Program component: Law enforcement;
Activity: Safe Streets program;
Weed and Seed partner: U.S. Attorney's Office, Federal Bureau of
Investigation (FBI);
Description: Drug violence initiative headed by the FBI.
Program component: Site: Community policing;
Activity: Site: Bike safety classes and registration;
Weed and Seed partner: Site: Community residents and police officers;
Description: Site: Community-based crime prevention activity
coordinated with the police.
Program component: Site: Prevention, intervention, and treatment;
Activity: Site: Drug Education For Youth (DEFY) Camp;
Weed and Seed partner: Site: Boy Scouts of America, U.S. Air Force;
Description: Site: About 60 children participated in leadership
training and fitness tests.
Program component: Neighborhood restoration;
Activity: Improved housing;
Weed and Seed partner: Hazelwood Initiative, Inc., and the Second
Avenue Main Street Program;
Description: A community-wide effort to reestablish the local business
district, revitalize a 200-acre area, reclaim historic buildings, and
build new housing.
Site: Charleston, S.C;
Program component: Law enforcement;
Activity: Youth Court;
Weed and Seed partner: South Carolina Bar Association, Young Lawyers
Division, Charleston Enterprise Community;
Description: First-time juvenile offenders are tried by their peers and
given an opportunity to improve their behavior.
Program component: Site: Community policing;
Activity: Site: Foot patrols;
Weed and Seed partner: Site: Charleston Weed and Seed Patrol Officers,
Charleston Police Department;
Description: Site: Weed and Seed officers patrol the community on foot.
Program component: Site: Prevention, intervention, and treatment;
Activity: Site: Police Athletic League Boxing;
Weed and Seed partner: Site: Police Athletic League;
Description: Site: Participants include junior and senior boxers and
coaches.
Program component: Neighborhood restoration;
Activity: Boarding up abandoned and vacant buildings;
Weed and Seed partner: Charleston Police Department, Maintenance
Division;
Description: The city boarded up abandoned and vacant buildings in Weed
and Seed neighborhoods.
Site: North Charleston, S.C;
Program component: Law enforcement;
Activity: Cease Fire program;
Weed and Seed partner: U.S. Attorney's Office, North Charleston Police
Department;
Description: A gun control program that began in January 2003 to reduce
gun crime in target areas.
Program component: Site: Community policing;
Activity: Site: Bike patrol;
Weed and Seed partner: Site: North Charleston Police Department;
Description: Site: Two officers on bike patrol are dedicated
exclusively to the Weed and Seed target area.
Program component: Site: Prevention, intervention, and treatment;
Activity: Site: DEFY program;
Weed and Seed partner: Site: Charleston Air Force Base;
Description: Site: Weed and Seed staff was developing a DEFY program
for participants from the Weed and Seed target area.
Program component: Neighborhood restoration;
Activity: Neighborhood cleanup;
Weed and Seed partner: Neighborhood residents;
Description: An annual neighborhood cleanup was held in April 2003.
[End of table]
Source: GAO site visits and EOWS.
Table 4: Examples of Leveraging Efforts at Weed and Seed Sites in
Pittsburgh, Charleston, and North Charleston:
Weed and Seed site: Pittsburgh, Pa;
Type of partnership or cooperative arrangement: Federal government;
Name of partner or cooperative arrangement: FBI;
Description: The FBI leads the Safe Streets program in Pittsburgh to
reduce street gang and drug related violence.
Type of partnership or cooperative arrangement: State government;
Name of partner or cooperative arrangement: State police;
Description: State Police officers are assisting the U.S. Attorney's
Office in implementing Justice's Project Safe Neighborhoods (PSN)
initiative to reduce gun violence.
Type of partnership or cooperative arrangement: Local government,
university;
Name of partner or cooperative arrangement: Allegheny County Health
Department (ACHD) and University of Pittsburgh Medical Center (UPMC);
Description: In March and April 2000, Pittsburgh began a partnership
with ACHD, UPMC, and others to conduct medical and dental screening at
community events. The first major effort of the partnership was a lead
abatement project, which identified and cleaned contaminated dwellings.
The Pittsburgh Urban Redevelopment Authority and the Housing Authority
also participated in this effort.
Type of partnership or cooperative arrangement: Local community;
Name of partner or cooperative arrangement: Pittsburgh Harlequins
Rugby Association;
Description: The partnership with the Harlequins began in March 2001.
The Harlequins established two teams of 20 children to play touch rugby
while also incorporating training in teamwork, partnership, and
discipline into the sport.
Weed and Seed site: Charleston, S.C;
Type of partnership or cooperative arrangement: Federal government;
Name of partner or cooperative arrangement: FBI;
Description: The FBI leads the Charleston Safe Streets Task Force to
reduce drug-related activities.
Type of partnership or cooperative arrangement: Local government;
Name of partner or cooperative arrangement: Charleston Police
Department;
Description: Police officers train youth in law enforcement procedures
and tactics as well as physical training. The goal of this program is
to instill responsibility in participants and to prepare them for
possible careers in law enforcement.
Type of partnership or cooperative arrangement: Local college;
Name of partner or cooperative arrangement: College of Charleston's
School of Education;
Description: Graduate interns tutor students from Safe Havens.
Weed and Seed site: North Charleston, S.C;
Type of partnership or cooperative arrangement: Federal government;
Name of partner or cooperative arrangement: U.S. Air Force;
Description: The DEFY program will be conducted year-round, through a
partnership with the Air Force.
Type of partnership or cooperative arrangement: Local government;
Name of partner or cooperative arrangement: North Charleston Police
Department;
Description: Police officers instruct youth on crime prevention during
school breaks at Safe Havens.
Type of partnership or cooperative arrangement: Local university;
Name of partner or cooperative arrangement: Medical University of South
Carolina Crime Victims Center;
Description: Through this partnership, information was distributed to
Hispanic families in the Weed and Seed target area about a program
designed to provide medical services to Hispanic families.
Source: GAO site visits and EOWS.
Note: Grantees also receive funds from other federal programs and non-
federal funds.
[End of table]
[End of section]
Appendix III: Proposed Legislation Could Affect Self-Sustainability:
Proposed legislation could affect self-sustainability for Weed and Seed
sites by mandating that EOWS impose more rigorous requirements on its
grantees, similar to some other Justice programs. Unlike some other
Justice discretionary grant programs, the law establishing the Weed and
Seed program currently does not have a matching funding
requirement,[Footnote 31] nor does it limit the maximum number of years
a grantee can receive funding.[Footnote 32] H.R. 3036, a bill to
reauthorize the Department of Justice for fiscal years 2004 through
2006, was introduced on September 9, 2003. This bill includes a
provision that would limit the federal contribution to 75 percent of
the total Weed and Seed program costs. The bill would also impose
limitations on the duration of grants funded under the Weed and Seed
program, limiting to 10 the total number of fiscal years a grantee may
receive grants. Additionally, the bill requires that in order for a
grantee to be eligible for a grant, it must agree to formulate a timely
and effective plan to independently sustain the Weed and Seed program
when federal funding ends.
Currently there are other federal grant programs that require a funding
match and a limit on the duration of grant awards.[Footnote 33] For
example, the Drug-Free Communities Support Program (DFCSP) and
Community Oriented Policing Services (COPS) both have matching
requirements and clearly specify the number of years a grantee may be
funded. DFCSP is a discretionary grant program with a budget of about
$60 million in fiscal year 2003. DFCSP requires a dollar-for-dollar
match, limits grantees to no more than 5 years of funding, and reduces
awards by 25 percent in the fourth and fifth years. The COPS program,
with a budget of about $929 million in fiscal year 2003, generally
requires grantees to contribute 25 percent of the costs of the program.
The federal share of a COPS grant that covers more than 1 year must
decline year to year.[Footnote 34]
[End of section]
Appendix IV: Executive Office for Weed and Seed Selected Activities:
Year 1: The local U.S. Attorney's Office establishes a steering
committee, which establishes clear roles and responsibilities for the
site's organizational and management structure[A];
Year 2: The steering committee continually educates, guides, and
reevaluates site management;
Year 3: [Empty];
Year 4: [Empty];
Year 5: [Empty].
Year 1: The steering committee meets monthly and develops a job
description for the site coordinator;
Year 2: Steering committee meets at least monthly;
Year 3: Steering committee and organizational structure continue to
function. Steering committee meets at least quarterly;
Year 4: [Empty];
Year 5: [Empty].
Year 1: The grantee and the site coordinator establish procedures for
tracking grant funds and expenditures;
Year 2: The grantee and/or local U.S. Attorney's Office evaluates the
site coordinator's job performance, with input from the steering
committee[B]; The site coordinator and the steering committee work
together to ensure timely expenditure of grant funds;
Year 3: [Empty];
Year 4: [Empty];
Year 5: [Empty].
Year 1: Based on the time line, goals, and objectives established by
the site's official recognition,[C] the site begins implementing
strategies and developing baseline data, addressing the four required
program elements[D];
Year 2: The site's strategy to achieve measurable outcomes in the four
required program areas (especially law enforcement/community policing)
is on track. The site continues to collect data related to goals and
objectives;
Year 3: The site continues to implement strategy for achieving goals
and objectives in all four required program elements;
Year 4: [Empty];
Year 5: The site makes plans to continue community development efforts
and strategic planning through various funding sources.
Year 1: The site begins to publicize the program, to recruit
neighborhood leaders, and to mobilize the community;
Year 2: The site continues program publicity and outreach to recruit
neighborhood leaders and mobilize community;
Year 3: The site continues program publicity and outreach efforts to
recruit neighborhood leaders and to mobilize the community while
identifying and building sustainable leadership among community
members;
Year 4: The site continues program publicity and outreach efforts to
mobilize community and recruit new community leaders;
Year 5: The site's community leaders are working to sustain efforts and
continue community mobilization.
Year 1: The site begins coordinating its efforts with related local
initiatives, particularly law enforcement resources and activities;
Year 2: The site continues to coordinate its efforts with local
initiatives, particularly programs with local parks and recreation
departments;
Year 3: The Weed and Seed effort begins to be included in city/
community plans;
Year 4: Community leaders have established working relationships with
local officials;
Year 5: The Weed and Seed community is included in city/community
planning.
Year 1: [Empty];
Year 2: The steering committee develops and implements the special
emphasis program based on community needs;
Year 3: The special emphasis program is fully operational;
Year 4: [Empty];
Year 5: [Empty].
Year 1: The site submits timely grant expenditure reports to Executive
Office for Weed and Seed and Office of Justice Programs;
Year 2: [Empty];
Year 3: [Empty];
Year 4: [Empty];
Year 5: [Empty].
Year 1: The site submits timely progress reports, providing program
outcome information related to the achievement of the goals and
objectives stated in the official recognition and grant applications;
Year 2: [Empty];
Year 3: [Empty];
Year 4: [Empty];
Year 5: The site submits a final progress report.
Year 1: [Empty];
Year 2: The site communicates regularly with all partners, including
U.S. Attorney's Office, Executive Office for Weed and Seed, and
community members, using newsletters, faxes, neighborhood visits,
e-mail, community meetings, and so forth;
Year 3: [Empty];
Year 4: [Empty];
Year 5: [Empty].
Year 1: The steering committee and the site coordinator evaluate site
progress to determine site's technical assistance needs, if any;
Year 2: [Empty];
Year 3: [Empty];
Year 4: The steering committee and site coordinator review previous
years' grant applications to determine plans for final year of official
recognition;
Year 5: The steering committee determines if there is a need to
continue the program in other neighborhoods based on the program.
Year 1: The site develops operating plans for next year;
Year 2: [Empty];
Year 3: [Empty];
Year 4: [Empty];
Year 5: [Empty].
Year 1: The site begins developing strategies for long-term self-
sustainment;
Year 2: [Empty];
Year 3: The site establishes a subcommittee to look for additional
funding or in-kind resources to sustain the program;
Year 4: Subcommittee continues to search for additional resources to
sustain the program;
Year 5: [Empty].
Year 1: [Empty];
Year 2: The site begins to develop plans for evaluating its programs;
Year 3: The site identifies an evaluation tool and begins
developing an evaluation based on Executive Office for Weed and Seed
site evaluation literature;
Year 4: The site undergoes program evaluation to assess overall
effectiveness of official recognition strategy implementation;
Year 5: By the end of the year, the site releases its full
evaluation report, assessing the overall program accomplishments and
its effectiveness.
Source: EOWS data.
Note: Arrow indicates action from the previous year continues to the
current year.
[A] A site is a geographically defined area ranging in size from
several neighborhood blocks to several square miles. With the input of
the U.S. Attorney's Office and the steering committee, each site
develops and implements its own Weed and Seed strategy.
[B] A grantee is an entity that receives funding from EOWS to implement
the Weed and Seed program. The grantee distributes the funds to sites
and is responsible for ensuring that sites comply with the terms of the
grant. A grantee may have more than one site.
[C] Official recognition is a designation given to a grantee by EOWS
that signifies that EOWS has approved the grantee's Weed and Seed
strategy in a specific site. A grantee must receive official
recognition from EOWS before a grantee can apply for funding. Official
recognition requires, among other things, that the site develop a
management plan and a comprehensive strategy for implementing the Weed
and Seed strategy.
[D] The four required program elements that sites are required to
address are (1) law enforcement; (2) community policing; (3) crime and
substance prevention, intervention, and treatment; and (4) neighborhood
restoration.
[End of table]
[End of section]
Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003:
Activity data collected by EOWS: 1. Does site include a
multijurisdictional task force.
Activity data collected by EOWS: 2. Does site participate in Project
Safe Neighborhoods.
Activity data collected by EOWS: 3. Does site have foot patrols.
Activity data collected by EOWS: 4. Does site have bike patrols.
Activity data collected by EOWS: 5. Does site have substations.
Activity data collected by EOWS: 6. Does site have crime watch.
Activity data collected by EOWS: 7. Do police participate in community
meetings.
Activity data collected by EOWS: 8. Number of homicides for 3 preceding
calendar years for Weed and Seed site.
Activity data collected by EOWS: 9. Number of homicides for 3 preceding
calendar years for entire jurisdiction.
Activity data collected by EOWS: 10. Number of total drug arrests[A].
Activity data collected by EOWS: 11. Number of heroin drug arrests.
Activity data collected by EOWS: 12. Number of cocaine (not including
crack) drug arrests.
Activity data collected by EOWS: 13. Number of crack cocaine drug
arrests.
Activity data collected by EOWS: 14. Number of marijuana drug arrests.
Activity data collected by EOWS: 15. Number of methamphetamines only
drug arrests.
Activity data collected by EOWS: 16. Number of safe haven facilities.
Activity data collected by EOWS: 17. Number of safe havens receiving
EOWS funding.
Activity data collected by EOWS: 18. Does site provide academic courses
and tutoring.
Activity data collected by EOWS: 19. Does site provide mentoring.
Activity data collected by EOWS: 20. Does site provide prevention
education.
Activity data collected by EOWS: 21. Does site provide dispute
resolution and mediation.
Activity data collected by EOWS: 22. Does site provide recreation and
athletics.
Activity data collected by EOWS: 23. Does site provide job training.
Activity data collected by EOWS: 24. Does site provide job placement.
Activity data collected by EOWS: 25. Does site provide antidrug
education.
Activity data collected by EOWS: 26. Does site provide community police
co-located in safe haven.
Activity data collected by EOWS: 27. Does site provide safe corridors
(school escorts for children).
Activity data collected by EOWS: 28. Does site provide summer day camp.
Activity data collected by EOWS: 29. Does site provide youth leadership
training.
Activity data collected by EOWS: 30. Does site provide boys and girls
club programs.
Activity data collected by EOWS: 31. Does site provide scouting
programs.
Activity data collected by EOWS: 32. Does site provide military cadet
training.
Activity data collected by EOWS: 33. Does site provide antigang
education and training.
Activity data collected by EOWS: 34. Does site provide Communities in
Schools programs.
Activity data collected by EOWS: 35. Does site provide performance or
applied arts programs.
Activity data collected by EOWS: 36. Does site provide victim
assistance programs.
Activity data collected by EOWS: 37. Does site provide community
projects, such as cleanups.
Activity data collected by EOWS: 38. Does site provide general health-
screening services.
Activity data collected by EOWS: 39. Does site provide lead-poisoning-
screening service.
Activity data collected by EOWS: 40. Number of persons receiving safe
haven services.
Activity data collected by EOWS: 41. Number of community development
corporations within site area.
Activity data collected by EOWS: 42. Is site constructing and
renovating housing developments.
Activity data collected by EOWS: 43. Is site constructing and
renovating commercial developments.
Activity data collected by EOWS: 44. Is site constructing and
renovating business and community partnerships.
Activity data collected by EOWS: 45. Number of community cleanups done
using EOWS support.
Activity data collected by EOWS: 46. Do Weed and Seed activities relate
to or involve Brownfields Assessment Demonstration Pilots.
Activity data collected by EOWS: 47. Do Weed and Seed activities relate
to or involve Brownfields Cleanup and Revolving Load Fund Pilots.
Activity data collected by EOWS: 48. Do Weed and Seed activities relate
to or involve Brownfields Showcase Communities.
Activity data collected by EOWS: 49. Do Weed and Seed activities relate
to or involve Brownfields Tax Incentive.
Activity data collected by EOWS: 50. Source of non-EOWS funding[B].
Activity data collected by EOWS: 51. Amount of non-EOWS grant funding.
Activity data collected by EOWS: 52. Source and type of in-kind
contributions.
Source: EOWS data.
[A] In addition to the number of drug arrests, EOWS also requests that
each site report the number of sale, manufacturing, and possession drug
arrests for each type of drug.
[B] Non-EOWS grant funding includes funding from other federal, state,
local, and private sources.
[End of table]
[End of section]
Appendix VI: Description of EOWS's Ongoing and Completed Studies:
Title of study and research partner: A Comparison of Homicide Trends in
Local Weed and Seed Sites Relative to Their Host Jurisdictions, 1996 to
2001;
Justice Research and Statistics Association (JRSA);
Purpose of study: Compare homicides in Weed and Seed sites with
homicides in the jurisdictions that sites are located in, to determine
whether homicides will decline faster in Weed and Seed sites than in
the remainder of the jurisdiction;
When results are expected: Report was expected in January 2004 but was
issued in November 2003.
Title of study and research partner: Performance Indicator Study;
Urban Institute;
Purpose of study: Develop additional performance indicators that will
be used by EOWS and Weed and Seed sites to evaluate program performance
with respect to crime control;
When results are expected: March 2004.
Title of study and research partner: Weed and Seed Crime Pattern Data
Collection;
Justice Research and Statistics Association;
Purpose of study: Summarize the types of crime selected as "target"
crimes by Weed and Seed sites (sites choose three crimes) to determine
crime reduction strategies. Assess the degree of success of Weed and
Seed sites by analyzing the percentage of reduction in crime compared
with the percentage in the rest of the jurisdiction that they are
located in;
When results are expected: March 2004.
Title of study and research partner: Meta-Analysis Evaluation Justice
Research and Statistics Association;
Purpose of study: Summarize the completed evaluations of individual
Weed and Seed sites. The analysis will include more than 80 studies.
JRSA's preliminary analysis (March 3, 2003) includes 36 evaluations. Of
the 36 evaluations, JRSA determined that 5 were process, 19 were
impact, and 9 were both process and impact evaluations. The remaining 3
evaluations were not identified as process or impact;
Note: A process evaluation addresses whether the program is working as
intended. An impact evaluation isolates the effects of a particular
program or factor from all other potential contributing factors that
could also effect change;
When results are expected: Mid 2004.
Title of study and research partner: Local Pilot Evaluations Local
researchers;
Purpose of study: Conduct local evaluations of Weed and Seed sites. In
fiscal year 2002, EOWS funded 13 local evaluations. Each site contracts
with its own researcher;
When results are expected: Mid 2004.
Source: EOWS and research organizations.
[End of table]
[End of section]
Appendix VII: Comments from the Department of Justice:
U.S. Department of Justice
Office of Justice Programs
Office of the Assistant Attorney General:
Washington, D.C. 20531
MAR 11 2004:
Ms. Laurie E. Ekstrand:
Director, Homeland Security and Justice Issues,
General Accounting Office:
441 G Street, N.W. Mail Stop 2440A
Washington, DC 20548:
Dear Ms. Ekstrand:
This letter responds to the General Accounting Office (GAO) draft
report entitled, "GRANTS MANAGEMENT: Despite Efforts to Improve Weed
and Seed Program Management, Challenges Remain " (GAO-04-025). We
agree, in part, with the recommendations, and we are in the process of
implementing appropriate corrective actions to address the
recommendations. However, we do not believe that GAO fully acknowledges
in the draft report the steps the Executive Office for Weed and Seed
(EOWS) has taken to address the recommendations concerning self-
sustainability criteria and outcome measures included in the prior GAO
report on EOWS. The draft report recommendations are restated below in
bold, followed by our response.
1. Maintain the documentation of the basis and rationale for
qualification and funding decisions in appropriate grant files.
We agree that controls should be strengthened to ensure that the basis
and rationale for qualification and funding decisions are documented in
the centrally maintained Official Recognition (OR) files when
applications are not funded. The Acting Director for the EOWS has
directed the Program Managers to forward a copy of all documents
supporting funding decisions to the centrally maintained OR files.
GAO also criticizes EOWS for having incomplete grant files because a
paper version of the application was not in the grant file. The
applications were submitted electronically through the Office of
Justice Programs (OJP's) Grants Management System (GMS). GMS has been a
high priority of mine since being named Assistant Attorney General for
OJP in September, 2001. As of the current fiscal year, ON will not only
post all solicitations and receive all grant applications via GMS, but
also receive and maintain all grant progress reports on GMS. In this
way, there will be no opportunity to misplace any
official document relating to any grant. Documents maintained in GMS
are considered part of the official grant file in an electronic format.
In compliance with the government's e-gov efforts, we no longer
maintain paper copies of documents retained in GMS. Those documents,
however, are readily available in electronic format and should be
available to outside reviewers as well.
For the ease of the review process, we agree that applications should
have been readily available to, and easily accessible by, GAO. In the
future, the OJP's Office of the Comptroller (OC) will implement
procedures to ensure that external reviewers are provided: 1) read-only
access to GMS information, in coordination with OJP's Office of the
Chief Information Officer; or 2) hard copies of pertinent GMS
information.
2. Retain progress reports and site visit reports in official grant
files.
We agree with the recommendation. EOWS and OC will strengthen controls
to ensure that copies of completed site visit reports are included in
the official grant file maintained in OC. Additionally, as part of GMS,
OJP is developing a monitoring module that will enable Program Managers
to record and report on site visits directly in GMS. When that module
is available, it will be the policy of ON to require that grant
managers submit the reports in this manner, thus making the entire file
fully accessible and fully contained in GMS.
Beginning with the semiannual progress reporting period that ended on
December 31, 2003, ON implemented a progress reporting module in GMS
that enables grantees to submit semiannual progress reports
electronically. As stated in response to Recommendation 1, documents
maintained in GMS are considered a part of the official grant file in
an electronic format, and all grantees are now required to submit their
semiannual progress reports electronically. Finally, OJP in 2002
adopted a policy of withholding further fund drawn-down capability from
any grantee who fails to file timely progress reports.
3. Clearly define criteria to assess when sites are self-sustaining and
apply the criteria to sites when making further funding decisions.
We agree, in part, with the recommendation. EOWS will further define
the criteria to assess when sites are self-sustaining and apply the
criteria when making funding decisions. Further, we are promoting a
"graduation" process to bring sites to sustainability and cease
providing Weed and Seed funding to those sites, thus making funding
available to assist newer developing sites.
However, we do not agree with GAO's assertion on page 12 of the draft
report that EOWS has never set criteria for self-sustainability. EOWS
set a criterion for self-sustainability in the FY2003 Weed and Seed
Competitive Application Kit: "Does the application provide for a full-
time coordinator funded by reallocation of existing resources other
than the Weed and Seed grant?" The applications were ranked against
this criterion. In March 2004, EOWS will announce an additional
criterion specific to measuring self-sustainability. This new criterion
will require grant applicants to identify other funding sources at a
level five times the EOWS contribution. This criterion will be
considered when making funding decisions.
4. Develop outcome performance measures-or, where measuring outcome is,
after careful consideration, deemed infeasible, intermediate measures-
that can be used to adequately track progress toward program outcomes
of the Weed and Seed program.
We fully agree that developing outcome performance measures is
important. This, too, is a high personal priority of mine, and OJP's
Office of Budget and Management Services is working with every OJP
grant program to develop outcome measures (and, in appropriate cases,
interim measures). EOWS has been working to develop measures that track
progress toward program outcomes in the Weed and Seed Program.
As early as 1997, EOWS enlisted the assistance of experts in the field,
such as the Justice Research and Statistics Association (JRSA), who
contracted with John O'Connell, Director, Statistical Analysis Center
in Delaware, to advise on the appropriate outcome measures.
O'Connell concluded, in a recent study, that measuring reductions of
homicides is an adequate performance measure of the Weed and Seed
program because of its reliability and explicit selection as a goal by
over 30 percent of the Weed and Seed Sites participating in the Crime
Pattern Study. In addition, the Department of Justice includes a five
percent reduction in homicides in Weed and Seed sites as part of its
Strategic Plan.
On March 2, 2004, JRSA submitted the following summaries of two studies
that were conducted recently: (1) Analysis of Homicide in Weed and Seed
Sites; and (2) Crime Patterns Analysis (Three-Top Crime Study).
Evaluation; Analysis of Homicide in Weed and Seed Sites;
Summary; One of the overriding goals of the national Weed and Seed
strategy is to reduce violence in some of the nation's most
crime-ridden neighborhoods. As part of GPRA homicide data have been
collected for Weed and Seed sites since 1996 as one measure of violence.
For this multi-year analysis, the most recent GPRA reporting is for
2001. Over this six year period 220 sites provided sufficient homicide
information for both the Weed and Seed sites and their host
jurisdictions (newer sites do not yet have enough history for
analysis). Seventy-seven percent of the Weed and Seed sites had
positive results in their homicide trends when compared to host
jurisdictions. In most of the sites (122) homicides decreased by about
50 percent (from an average of about 10 homicides per site in 1996 to 5
per site in 2001). In 31 sites (14%), the number of homicides remained
stable while the homicide statistics for their respective jurisdictions
increased and in 17 sites (8%), homicides increased at a slower rate
than their jurisdictions. These trends in the most serious of crimes
were a significant accomplishment for Weed and Seed sites considering
that homicides in the host jurisdictions (and the nation) increased in
2000 and 2001.
Evaluation; Crime Pattern Analysis; (Three-Top Crime Study);
Summary; As a voluntary effort to provide more information regarding
the extent of crime reduction in Weed and Seed sites, EOWS solicited
comparative crime information in mid-2003 from all active Weed and Seed
sites. The Weed and Seed sites were asked to submit information
regarding the top three crime issues. To be included in the analysis
crime information both prior to and following the implementation of
their Weed and Seed operations had to be provided. By the fall of 2003,
98 sites had submitted crime pattern information. Forty percent of the
sites were able to provide crime data for both pre and post time
periods. Forty-one percent of the sites had not implemented their Weed
and Seed strategy long enough to conduct a pre/post crime analysis. The
remainder of the sites had data inconsistencies that allow for only a
partial analysis of the information provided. Preliminary analysis
show, that as expected, reported crimes increase during the initial
phases of a Weed and Seed operation but that after the third year of
program implementation sites encounter a significant decrease in
reported crimes within the Weed and Seed site areas. This decrease is
apparent both in the total number of each crime reported for the Weed
and Seed sites as well as in the proportion of city-wide or
jurisdiction-wide crime accounted for by the Weed and Seed target area.
Ninety-four percent of the sites are addressing violent crimes or
homicide as one of their key crime initiatives.
[End of table]
OJP wishes to stress its full commitment to outcome measurement, even
in cases, such as a Weed and Seed strategy, which, as the GAO report
suggests, may not easily lend itself to measurement due to multiple
factors potentially affecting those outcomes. We will continue to work
towards this goal.
We would like this information and the above summaries included in
GAO's report. As always, OJP appreciates the opportunity to comment on
the draft report. Thank you for your continued cooperation and
assistance.
Sincerely,
Signed by:
Deborah J. Daniels,
Assistant Attorney General:
cc: Robert Samuels, Acting Director Executive Office for Weed and Seed:
Cynthia J. Schwimer Comptroller, OJP:
LeToya A. Johnson Audit Liaison, OJP:
Vickie L. Sloan Audit Liasion, DOJ:
OAAG Executive Secretariat
Control Number 20040254:
[End of section]
Appendix VIII: GAO Contacts and Staff Acknowledgments:
GAO Contacts:
Laurie Ekstrand (202) 512-2758 Weldon McPhail (202) 512-8644:
Staff Acknowledgments:
In addition to those named above, the following individuals made key
contributions to this report: Lisa G. Shibata, Kirstin B. L. Nelson,
Cady L. Summers, David Alexander, Sidney Schwartz, Kevin L. Jackson,
Elizabeth Curda, Tom James, Denise Fantone, Jan Montgomery, Ann H.
Finley, Katherine M. Davis, and Leo Barbour.
[End of section]
Related GAO Products:
Law Enforcement: Better Performance Measures Needed to Assess Results
of Justice's Office of Science and Technology, GAO-04-198. Washington,
D.C.: November 14, 2003.
Standards for Internal Control in the Federal Government. GAO/AIMD-00-
21.3.1. Washington, D.C.: November 1999.
Federal Grants: More Can Be Done to Improve Weed and Seed Program
Management. GAO/GGD-99-110. Washington, D.C.: July 16, 1999.
Small Business Administration: Enhancements Needed for Loan Monitoring
System Benchmark Study. GAO/AIMD-99-165. Washington, D.C.: May 14,
1999.
Managing for Results: Measuring Program Results That Are Under Limited
Federal Control. GAO/GGD-99-16. Washington, D.C.: December 11, 1998.
Managing for Results: Analytic Challenges in Measuring Performance.
GAO/HEHS/GGD-97-138. Washington, D.C.: May 30, 1997.
Federal Grants: Design Improvements Could Help Federal Resources Go
Further. GAO/AIMD-97-7. Washington, D.C.: December 18, 1996.
Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to
Society. GAO/RCED-92-106. Washington, D.C.: May 1992.
FOOTNOTES
[1] U.S. General Accounting Office, Federal Grants: More Can Be Done to
Improve Weed and Seed Program Management, GAO/GGD-99-110 (Washington,
D.C.: July 1999).
[2] Discretionary grants are awarded to eligible grantees, most often
on a competitive basis.
[3] A grantee is an entity that receives funding from EOWS to implement
the Weed and Seed program. The grantee distributes the funds to sites
and is responsible for ensuring that sites comply with the terms of the
grant. A grantee may have more than one site.
[4] A site is a geographically defined area ranging in size from
several neighborhood blocks to several square miles. With the input of
the local U.S. Attorney's Office and the site's steering committee,
each site develops and implements its own Weed and Seed program.
[5] Two detailees are from the Federal Bureau of Investigation (FBI)
and one is from the Department of the Interior. The FBI detailees work
part-time at EOWS.
[6] There are 93 U.S. Attorneys throughout the United States, Puerto
Rico, the Virgin Islands, Guam, and the Northern Mariana Islands. Each
U.S. Attorney is the chief federal law enforcement officer of the
United States within his or her particular jurisdiction. The U.S.
Attorneys serve as the nation's principal litigators under the
direction of the Attorney General.
[7] When applicants apply for funding, EOWS creates and maintains
official recognition files. If applicants receive funding, OJP creates
and maintains official grant files.
[8] U.S. General Accounting Office, Standards for Internal Control in
the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November
1999).
[9] Throughout the rest of this report, we refer to both new and
existing sites as sites.
[10] EOWS uses the information in the documents as the basis to award
official recognition.
[11] The remaining 187 of the 218 sites received funding in fiscal year
2002, but were initially funded prior to fiscal year 2002.
[12] Progress reports describe Weed and Seed activities and the
accomplishment of objectives in a site's funding application. Progress
reports help EOWS officials determine how sites are meeting program
objectives and assist them in making future grant decisions. Progress
reports are due 30 days after June 30 and December 31, respectively.
[13] Grantees may have more than one site.
[14] EOWS refers to these activities as benchmarks. For a list of
EOWS's benchmarks, see appendix IV.
[15] See GAO/GGD-99-110.
[16] P.L. 103-62.
[17] Performance measures translate program goals into concrete,
observable conditions that determine what data to collect to learn
whether progress has been made toward achieving program goals. Such
measures are meant to cover the key aspects of performance that will
enable programs to assess accomplishments, make decisions, realign
processes, and assign accountability.
[18] EOWS officials told us that the Weed and Seed program has three
main goals: (1) reduce violent crime, (2) reduce drug crime, and (3)
coordinate Weed and Seed funds with other resources. Such goals are
related to the overall intent of the program to weed out crime from
targeted neighborhoods and seed them with a variety of programs to
prevent crime from recurring.
[19] U.S. General Accounting Office, Managing for Results: Analytic
Challenges in Measuring Performance, GAO/HEHS/GGD-97-138 (Washington,
D.C.: May 30, 1997).
[20] OMB periodically assesses goals and results of federal programs,
such as Weed and Seed, and reports on its findings. The purpose is to
tie performance with the budget process.
[21] U.S. General Accounting Office, Managing for Results: Measuring
Program Results That Are Under Limited Federal Control,GAO/GGD-99-16
(Washington, D.C.: December 11, 1998) and GAO/HEHS/GGD-97-138.
[22] See GAO/GGD-99-16.
[23] U.S General Accounting Office, Law Enforcement: Better Performance
Measures Needed to Assess Results of Justice's Office of Science and
Technology, GAO-04-198 (Washington, D.C.: November 2003).
[24] See GAO/GGD-99-16.
[25] It is commonly accepted that safety belt use reduces fatality
rates and the severity of injuries. See U.S. General Accounting Office,
Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to
Society, GAO/RCED-92-106 (Washington, D.C.: May 1992)
[26] In November 2003 the Justice Research and Statistics Association
completed a study comparing homicide trends in sites to their host
jurisdictions. However, this study relied on homicide rates collected
by EOWS, rather than developing additional performance measures.
[27] Sites funded under EOWS's competitive application can be either
the first Weed and Seed site in the jurisdiction that received official
recognition or a new geographical area (that is, not contiguous with an
existing or former target area) in a jurisdiction with an existing (or
continuation) or former Weed and Seed site. Sites funded under EOWS's
continuation application are in their second, third, fourth, or fifth
year of funding and implementing the Weed and Seed program.
[28] GAO did not verify the funds leveraged by Weed and Seed sites.
[29] GAO did not verify the funds leveraged by Weed and Seed sites.
[30] GAO did not verify funds leveraged by Weed and Seed sites.
[31] A matching requirement requires grant recipients to contribute
their own funds to obtain federal grant funds. The Weed and Seed
program is authorized and funded by the Department of Justice annual
appropriation acts. These acts have not included either a requirement
for matching funds or a limitation on the maximum number of years a
grantee may receive funding. See for example, Departments of Commerce,
Justice, and State, the Judiciary, and Related Agencies Appropriations
Act, 1998, P.L. 105-119, 111 Stat. 2440, 2454.
[32] As discussed earlier, beginning in 1999, EOWS imposed a 5-year
site expansion rule, under which a site is awarded funding for 5 years
and may receive an additional 5 years of funding if the majority of
funds are shifted to a new site. This is not a requirement imposed by
the law.
[33] U.S. General Accounting Office, Federal Grants: Design
Improvements Could Help Federal Resources Go Further, GAO/AIMD-97-7
(Washington, D.C.: December 1996).
[34] Grants for hiring and rehiring may be renewed for up to 5 years.
Grants for other purposes may not cover more than 3 years.
GAO's Mission:
The General Accounting Office, the investigative arm of Congress,
exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability
of the federal government for the American people. GAO examines the use
of public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains
abstracts and full-text files of current reports and testimony and an
expanding archive of older products. The Web site features a search
engine to help you locate documents using key words and phrases. You
can print these documents in their entirety, including charts and other
graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as "Today's Reports," on its
Web site daily. The list contains links to the full-text document
files. To have GAO e-mail this list to you every afternoon, go to
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order
GAO Products" heading.
Order by Mail or Phone:
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or
more copies mailed to a single address are discounted 25 percent.
Orders should be sent to:
U.S. General Accounting Office
441 G Street NW,
Room LM Washington,
D.C. 20548:
To order by Phone:
Voice: (202) 512-6000:
TDD: (202) 512-2537:
Fax: (202) 512-6061:
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov
Automated answering system: (800) 424-5454 or (202) 512-7470:
Public Affairs:
Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.
General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.
20548:
